Tool Category Author Part of Claim to fame unit testing first unit test framework to be included in Python standard library; easy to use by people familiar with the xUnit frameworks; strong support for test organization and reuse via test suites unit testing copy and paste output from shell session;: unit tests themselves can serve as documentation when combined with epydoc; also see unit testing It used to be named py.test which was part of the. Standalone now. No API!;automatic collection of tests; simple asserts; strong support for test fixture/state management via setup/teardown hooks; strong debugging support via customized traceback unittest extensions unit test framework, used most notably by and; provides an alternate test discovery and running process for unittest, one that is intended to mimic the behavior of py.test as much as is reasonably possible without resorting to too much magic. More friendly with unittest.-based tests than py.test. There are also many available. Unittest extensions unit test framework, provides Enhanced test fixture setup, Split test suites into buckets for easy parallelization, PEP8 naming conventions & Fancy color test runner with lots of logging / reporting option.
Awesome-test-automation - A curated list of awesome test automation frameworks, tools. Autopy - A simple, cross-platform GUI automation toolkit for Python.
Unittest extensions Extension of unittest to support writing asynchronous unit tests using Deferreds and new result types ('skip' and 'todo'). Includes a command-line program that does test discovery and integrates with doctest and coverage. Unittest extensions Transparently adds support for running unittest test cases/suites in a separate process: prevents system wide changes by a test destabilising the test runner. It also allows reporting from tests in another process into the unittest framework, giving a single integrated test environment. Unittest extensions Provides a mechanism for managing 'resources' - expensive bits of infrastructure - that are needed by multiple tests. Resources are constructed and free on demand, but with an optional, the test run order is optimised to reduce the number of resource constructions and releases needed. Compatible with unittest.
Unittest extensions Useful extensions to unittest derived from custom extensions by projects such as Twisted and Bazaar. Unit testing Sancho 2.1 runs tests, and provides output for tests that fail; Sancho 2.1 does not count tests passed or failed; targets projects that do not maintain failing tests unit testing Zope3 community Powerful test runner that includes support for post-mortem debugging of test failures. Also includes profiling and coverage reporting. This is a standalone package that has no dependencies on Zope and works just fine with projects that don't use Zope. Unit testing Elegant unit testing framework with built-in coverage analysis, profiling, micro-benchmarking and a powerful command-line interface.
Unit testing Tool that will automatically, or semi-automatically, generate unit tests for legacy systems written in Python. Unittest extensions logilab-common Gives more power to standard unittest. More assert.
methods; support for module level setup/teardown; skip test feature. Tests runner logilab-common Tests finder / runner.
Selectivly run tests; Stop on first failure; Run pdb on failed tests; Colorized reports; Run tests with coverage / profile enabled. Unittest extensions (distributed separately too) An object oriented interface to retrieve unittest test cases out of doctests. Hides initialization from doctests by allowing setUp and tearDown for each interactive example. Allows control over all the options provided by doctest.
Specialized classes allow selective test discovery across a package hierarchy. The following tools are not currently being developed or maintained as far as we can see.
The cover art and liner notes are included for a CD. Nintendo game and parachute. VHS or DVD box is included. May have minor damage to jewellery case including scuffs or cracks, or to the item cover including scuffs, scratches, or cracks. Video game instructions are included.
They are here for completeness, with last activity date and an indication of what documentation there is. If you know better, please edit.
Tool Category Author Claim to fame mocks, stubs, spy, and dummies Gustavo Rezende Elegant test doubles framework in Python (mocks, stubs, spy, and dummies) mock testing Python Mock enables the easy creation of mock objects that can be used to emulate the behaviour of any class that the code under test depends on. You can set up expectations about the calls that are made to the mock object, and examine the history of calls made. This makes it easier to unit test classes in isolation.
Mock testing is based on the Java package. It uses a recording and replay model rather than using a specification language.
Easymock lives up to its name compared to other mocking packages. Takes advantage of python's dynamic nature for futher improvements.
Mock testing Michael Foord aka Provides 'action - assertion' mocking patter, instead of standard 'record - replay' pattern mock testing Graham Carlyle Inspired by the Java library, pMock makes the writing of unit tests using mock object techniques easier. Development of pmock has long since stopped and so it can be considered dead.
Mock testing Embeds mock testing constructs inside doctest tests. Mock testing enables easier testing of Python programs that make use of Python bindings mock testing Graceful platform for test doubles in Python (mocks, stubs, fakes, and dummies). Well-documented and fairly feature-complete. Stub testing Stubble allows you to write arbitrary classes for use as stubs instead of read classes while testing.
Stubble lets you link a stub class loosely to the real class which it is a stub for. This information is then used to ensure that tests will break if there is a discrepancy between the interface supported by your stub class and that of the real class it stands in for. Mock testing smiddlek, dglasser Mox is based on, a Java mock object framework. Mox will make mock objects for you, so you don't have to create your own! It mocks the public/protected interfaces of Python objects.
You set up your mock objects expected behavior using a domain specific language (DSL), which makes it easy to use, understand, and refactor! Mock testing Tim Cuthbertson (gfxmonk) Mocktest allows you to mock / stub objects and make expectations about the methods you expect to be called as well as the arguments they should be called with. Expectations are very readable and expressive, and checked automatically. Any stubbed methods are reverted after each test case. Still under development, so subject to change mock and stub testing A module for using fake objects (mocks, stubs, etc) to test real ones.
Uses a declrative syntax like jMock whereby you set up expectations for how an object should be used. An error will raise if an expectation is not met. Mock and stub testing A port of the mocking framework to Python. (Technically speaking, Mockito is a Test Spy framework.) mock testing Geoff Bache True record-replay approach to mocking. Requires no coding, just telling it which modules/attributes you want to mock. Then stores the behaviour in an external file, which can be used to test the code without those modules installed. Mock/stub/spy testing and fake objects Port of the popular Ruby mocking library to Python.
Includes automatic integration with most popular test runners. Easy and powerful stubs, spies and mocks Free and restricted doubles using hamcrest matchers for all assertions. It provides a wrapper for the pyDoubles framework. Spies and mock responses Lightweight spies and mock responses, and a capture/replay framework (via the Story/Replay context managers). Fuzz Testing Tools According to Wikipedia, is a software testing technique whose basic idea is to attach the inputs of a program to a source of random data ('fuzz').
If the program fails (for example, by crashing, or by failing built-in code assertions), then there are defects to correct. The great advantage of fuzz testing is that the test design is extremely simple, and free of preconceptions about system behavior. Tool Author Claim to fame Hypothesis combines unit testing and fuzz testing by letting you write tests parametrized by random data matching some specification. It then finds and minimizes examples that make your tests fail. Ivan Moore Tests your tests by mutating source code and finding tests that don't fail!
Michael Eddington Peach can fuzz just about anything from.NET, COM/ActiveX, SQL, shared libraries/DLL's, network applications, web, you name it. Dmckinney The purpose of antiparser is to provide an API that can be used to model network protocols and file formats by their composite data types. Once a model has been created, the antiparser has various methods for creating random sets of data that deviates in ways that will ideally trigger software bugs or security vulnerabilities. Rodrigomarcos Taof is a GUI cross-platform Python generic network protocol fuzzer. It has been designed for minimizing set-up time during fuzzing sessions and it is specially useful for fast testing of proprietary or undocumented protocols. It helps to start process with a prepared environment (limit memory, environment variables, redirect stdout, etc.), start network client or server, and create mangled files.
Fusil has many probes to detect program crash: watch process exit code, watch process stdout and syslog for text patterns (eg. 'segmentation fault'), watch session duration, watch cpu usage (process and system load), etc. Web Testing Tools First, let's define some categories of Web testing tools:.
Browser simulation tools: simulate browsers by implementing the HTTP request/response protocol and by parsing the resulting HTML. Browser automation tools: automate browsers by driving them for example via COM calls in the case of Internet Explorer, or XPCOM in the case of Mozilla. In-process or unit-test-type tools: call an application in the same process, instead of generating an HTTP request; so an exception in the application would go all the way up to the command runner (py.test, unittest, etc). Tool Category Author Part of Claim to fame Browser simulation & In-process offers simple commands for navigating Web pages, posting forms and asserting conditions; can be used as shell script or Python module; can be used for; stress-test functionality; port of; uses John J. Tool Author Claim to fame Simplest GUI automation with Python on Windows.
Lets you automate your computer with simple commands such as start, click and write. Uses the X11 accessability framework (AT-SPI) to drive applications so works well with the gnome desktop on Unixes. Has extensive tests for the evolution groupware client. Mark Simple Windows (NT/2K/XP) GUI automation with Python. There are tests included for Localization testing but there is no limitation to this. Most of the code at the moment is for recovering information from Windows windows and performing actions on those controls. The idea is to have high level methods for standard controls rather then rely on Sending keystrokes to the applications.
Team members OS X Cocoa accessibility based automation library for Mac Geoff Bache Domain-language based UI testing framework with a recorder. Generates plain-text descriptions (ASCII art) of what the GUI looks like during the test, intended to be used in conjunction with (see above). Mechanism for being able to record synchronisation points. Currently has mature support for PyGTK, beta status support for SWT/Eclipse RCP and Tkinter, and an early prototype for wxPython. Swing support is being developed. Raimund Hocke (aka ) and the open-source community Python scripts and visual technology to automate and test graphical user interfaces using screenshot images (api for java available) The following tools are not currently being developed or maintained as far as we can see. They are here for completeness, with last activity date and an indication of what documentation there is.
If you know better, please edit. Tool Last Activity Docs Author Claim to fame 2013 Good Created by Redhat engineers on linux.
Uses the X11 accessability framework (AT-SPI) to drive applications so works well with the gnome desktop on Unixes. Has 2005 Good Dr Tim Couper Windows Application Test System Using Python - another Windows GUI automation tool. 2005 Limited Python helper library for testing Python GUI applications, with being the most mature 2005 Limited pyAA is an object oriented Python wrapper around the client-side functionality in the Microsoft Active Accessibility (MSAA) library. MSAA is a library for the Windows platform that allows client applications inspect, control, and monitor events and controls in graphical user interfaces (GUIs) and server applications to expose runtime information about their user interfaces. See the tutorial for more info. 2004 No aims to be a gui unittesting library for python; initially provided solely for, but it may be extended in the future 2003 No Simon Brunning Low-level library for Windows GUI automation used by PAMIE and WATSUP.
Source Code Checking Tools. Tool Author Claim to fame measures code coverage during Python execution; uses the code analysis tools and tracing hooks provided in the Python standard library to determine which lines are executable, and which have been executed figleaf is a Python code coverage analysis tool, built somewhat on the model of Ned Batchelder's fantastic coverage module.
Support
The goals of figleaf are to be a minimal replacement of 'coverage.py' that supports more configurable coverage gathering and reporting; figleaf is useful for situations where you are recording code coverage in multiple execution runs and/or want to tweak the reporting output Olivier Grisel HTML test coverage reporting tool with white and blacklisting support The coverage langlet weaves monitoring commands, so called sensors, into source code during global source transformation. When a statement is covered the weaved sensor responds.
The coverage langlet is part of Elegant unit testing framework with built-in coverage analysis, profiling, micro-benchmarking and a powerful command-line interface. Instrumental is a Python code coverage tool that measures statement, decision, and during code execution. Instrumental works by modifying the AST on import and adding function calls that record the circumstances under which code is executed.
Continuous Integration Tools Although not properly a part of testing tools, continuous integration tools are nevertheless an important addition to a tester's arsenal. Tool Author Claim to fame buildbot is a system to automate the compile/test cycle required by most software projects to validate code changes. By automatically rebuilding and testing the tree each time something has changed, build problems are pinpointed quickly, before other developers are inconvenienced by the failure. Bitten is a Python-based framework for collecting various software metrics via continuous integration. It builds on to provide an integrated web-based user interface. Heinrich Wendel of SVNChecker is a framework for Subversion pre-commit hooks in order to implement checks of the to be commited files before they are commited. For example, you can check for the code style or unit tests.
The output of the checks can be send by mail or be written into a file or simply print to the console. An Automated Pythonic Code Tester: designed to run tests on a code repository on a daily basis.
It comes with a set of predefined test, essentially for python packages, and a set of predefined reports to display execution results. However, it has been designed to be higly extensible, so you could write your own test or report using the Python language pony-build is a simple continuous integration package that lets you run a server to display client build results. It consists of two components, a server (which is run in some central & accessible location), and one or more clients (which must be able to contact the server via HTTP). Philosophy statement: good development tools for Python should be easy to install, easy to hack, and not overly constraining.
Browse All Files
Two out of three ain't bad;). Tox is a generic virtualenv management and test command line tool. You can use it to check that your package installs correctly with different Python versions and interpreters, to configure your test tool of choice and to act as a frontend to Continuous Integration servers. Documentation and examples. Automatic Test Runners Tools that run tests automatically on file changes. Provides continuous feedback during development before continuous integration tools act on commits.
Tool Author Claim to fame Meme Dough Monitors paths and upon detecting changes runs the specified command. The command may be any test runner. Uses Linux inotify so it is fast with no disk churn. Doug Latornell Run the nose test discovery and execution tool whenever a source file is changed. Jeff Winkler & Jerome Lacoste A minimalist personal command line friendly CI server.
Automatically runs your build whenever one of the monitored files of the monitored projects has changed. Noah Kantrowitz Continuous testing for paranoid developers. Test Fixtures. Tool Author Part of Claim to fame module for loading and referencing test data Russell Keith-Magee A test case for a database-backed website isn't much use if there isn't any data in the database.
To make it easy to put test data into the database, Django provides a fixtures framework. Chris Withers A collection of helpers and mock objects for unit tests and doc tests Fred Drake Test support composition, providing for fixture-specific APIs with unittest.TestCase. Miscellaneous Python Testing Tools. Tool Author Part of Claim to fame enables tests of Python regular expressions in a web browser; it uses SimpleHTTPServer and AJAX a wxPython GUI to the re module a QT (KDE) Regular Expression Debugger A simple environment for testing command-line applications, running commands and seeing what files they write to supports profiling, debugging and optimization regarding memory related issues in Python programs Nick Smallbone a memory usage profiler for Python code Memory profiling of Python processes, memory dumps can be viewed in Reginald B. Charney Generates metrics (e.g., LoC,%comments, etc.) for Python code.
Curt Finch Analyzes cyclomatic complexity of Python code (written in Perl) test and automation framework, in python Records interactive sessions and extracts test cases when they get replayed Test combinations generator (in python) that allows to create set of tests using 'pairwise combinations' method, which reduces a number of combinations of variables into a lesser set which covers most situations. See for more info on this functional testing technique. Model-based testing framework where the models are written in Python. Supports offline and on-the-fly testing. It uses composition for scenario control.
Coverage can be guided by a programmable strategy. PythonTestingToolsTaxonomy (last edited 2017-06-01 18:49:43 by ).