[Python Std Library] unittest — Unit testing framework

Source From Here 
Preface 
The Python unit testing framework, sometimes referred to as “PyUnit,” is a Python language version of JUnit, by Kent Beck and Erich Gamma. JUnit is, in turn, a Java version of Kent’s Smalltalk testing framework. Each is the de facto standard unit testing framework for its respective language. 

Basic example 
The unittest module provides a rich set of tools for constructing and running tests. This section demonstrates that a small subset of the tools suffice to meet the needs of most users. Here is a short script to test three functions from the random module: 

view plaincopy to clipboardprint?

1. import random  
2. import unittest  
3.   
4. class TestSequenceFunctions(unittest.TestCase):  
5.   
6.     def setUp(self):  
7.         self.seq = range(10)  
8.   
9.     def test_shuffle(self):  
10.         # make sure the shuffled sequence does not lose any elements  
11.         random.shuffle(self.seq)  
12.         self.seq.sort()  
13.         self.assertEqual(self.seq, range(10))  
14.   
15.         # should raise an exception for an immutable sequence  
16.         self.assertRaises(TypeError, random.shuffle, (1,2,3))  
17.   
18.     def test_choice(self):  
19.         element = random.choice(self.seq)  
20.         self.assertTrue(element in self.seq)  
21.   
22.     def test_sample(self):  
23.         self.assertRaises(ValueError, random.sample, self.seq, 20)  
24.         for element in random.sample(self.seq, 5):  
25.             self.assertTrue(element in self.seq)  
26.   
27. if __name__ == '__main__':  
28.     unittest.main()  

A testcase is created by subclassing unittest.TestCase. The three individual tests are defined with methods whose names start with the letters test. This naming convention informs the test runner about which methods represent tests. 

The crux of each test is a call to assertEqual() to check for an expected result; assertTrue() to verify a condition; or assertRaises() to verify that an expected exception gets raised. These methods are used instead of the assert statement so the test runner can accumulate all test results and produce a report. 

When a setUp() method is defined, the test runner will run that method prior to each test. Likewise, if a tearDown() method is defined, the test runner will invoke that method after each test. In the example, setUp() was used to create a fresh sequence for each test. 

The final block shows a simple way to run the tests. unittest.main() provides a command-line interface to the test script. When run from the command line, the above script produces an output that looks like this: 

...
----------------------------------------------------------------------
Ran 3 tests in 0.000s

OK

Instead of unittest.main(), there are other ways to run the tests with a finer level of control, less terse output, and no requirement to be run from the command line. For example, the last two lines may be replaced with: 

view plaincopy to clipboardprint?

1. suite = unittest.TestLoader().loadTestsFromTestCase(TestSequenceFunctions)  
2. unittest.TextTestRunner(verbosity=2).run(suite)  

Running the revised script from the interpreter or another script produces the following output: 

test_choice (__main__.TestSequenceFunctions) ... ok
test_sample (__main__.TestSequenceFunctions) ... ok
test_shuffle (__main__.TestSequenceFunctions) ... ok

----------------------------------------------------------------------
Ran 3 tests in 0.110s

OK

The above examples show the most commonly used unittest features which are sufficient to meet many everyday testing needs. The remainder of the documentation explores the full feature set from first principles. 

Command-Line Interface 
The unittest module can be used from the command line to run tests from modules, classes or even individual test methods: 

# python -m unittest test_module1 test_module2
# python -m unittest test_module.TestClass
# python -m unittest test_module.TestClass.test_method

You can pass in a list with any combination of module names, and fully qualified class or method names. 

You can run tests with more detail (higher verbosity) by passing in the -v flag: 

# python -m unittest -v test_module

For a list of all the command-line options: 

# python -m unittest -h

Test Discovery 
Unittest supports simple test discovery. In order to be compatible with test discovery, all of the test files must be modules or packages importable from the top-level directory of the project (this means that their filenames must be valid identifiers). 

Test discovery is implemented in TestLoader.discover(), but can also be used from the command line. The basic command-line usage is: 

# cd project_directory
# python -m unittest discover

The discover sub-command has the following options: 
-v, --verbose 

Verbose output

-s, --start-directory directory 

Directory to start discovery (. default)

-p, --pattern pattern 

Pattern to match test files (test*.py default)

-t, --top-level-directory directory 

Top level directory of project (defaults to start directory)

The -s, -p, and -t options can be passed in as positional arguments in that order. The following two command lines are equivalent: 

# python -m unittest discover -s project_directory -p '*_test.py'
# python -m unittest discover project_directory '*_test.py'

As well as being a path it is possible to pass a package name, for example myproject.subpackage.test, as the start directory. The package name you supply will then be imported and its location on the filesystem will be used as the start directory. 
Caution: 

Test discovery loads tests by importing them. Once test discovery has found all the test files from the start directory you specify it turns the paths into package names to import. For example foo/bar/baz.py will be imported as foo.bar.baz.

If you have a package installed globally and attempt test discovery on a different copy of the package then the import could happen from the wrong place. If this happens test discovery will warn you and exit.

If you supply the start directory as a package name rather than a path to a directory then discover assumes that whichever location it imports from is the location you intended, so you will not get the warning.

Test modules and packages can customize test loading and discovery by through the load_tests protocol.

Organizing test code 
The basic building blocks of unit testing are test cases — single scenarios that must be set up and checked for correctness. In unittest, test cases are represented by instances of unittest‘s TestCase class. To make your own test cases you must write subclasses of TestCase, or use FunctionTestCase. 

An instance of a TestCase-derived class is an object that can completely run a single test method, together with optional set-up and tidy-up code. The testing code of a TestCase instance should be entirely self contained, such that it can be run either in isolation or in arbitrary combination with any number of other test cases. The simplest TestCase subclass will simply override the runTest() method in order to perform specific testing code: 

view plaincopy to clipboardprint?

1. import unittest  
2.   
3. class Widget:  
4.     def __init__(self, name):  
5.         self.name = name  
6.         self.width = 50  
7.         self.height = 50  
8.           
9.     def size(self):  
10.         return (self.width, self.height)  
11.   
12. class DefaultWidgetSizeTestCase(unittest.TestCase):  
13.     def runTest(self):  
14.         widget = Widget('The widget')  
15.         self.assertEqual(widget.size(), (50, 50), 'incorrect default size')  
16.           
17.           
18. if __name__ == '__main__':  
19.     unittest.main()  

Note that in order to test something, we use one of the assert*() methods provided by the TestCase base class. If the test fails, an exception will be raised, andunittest will identify the test case as a failure. Any other exceptions will be treated as errors. This helps you identify where the problem is: failures are caused by incorrect results - a 5 where you expected a 6. Errors are caused by incorrect code - e.g., a TypeError caused by an incorrect function call. 

The way to run a test case will be described later. For now, note that to construct an instance of such a test case, we call its constructor without arguments: 

view plaincopy to clipboardprint?

1. testCase = DefaultWidgetSizeTestCase()  

Now, such test cases can be numerous, and their set-up can be repetitive. In the above case, constructing a Widget in each of 100 Widget test case subclasses would mean unsightly duplication. Luckily, we can factor out such set-up code by implementing a method called setUp(), which the testing framework will automatically call for us when we run the test: 

view plaincopy to clipboardprint?

1. import unittest  
2.   
3. class Widget:  
4.     def __init__(self, name):  
5.         self.name = name  
6.         self.width = 50  
7.         self.height = 50  
8.           
9.     def size(self):  
10.         return (self.width, self.height)  
11.       
12.     def resize(self, width, height):  
13.         self.width=width  
14.         self.height=height  
15.   
16. class SimpleWidgetTestCase(unittest.TestCase):  
17.     def setUp(self):  
18.         self.widget = Widget('The widget')  
19.   
20. class DefaultWidgetSizeTestCase(SimpleWidgetTestCase):  
21.     def runTest(self):  
22.         self.assertEqual(self.widget.size(), (50,50),  
23.                          'incorrect default size')  
24.   
25. class WidgetResizeTestCase(SimpleWidgetTestCase):  
26.     def runTest(self):  
27.         self.widget.resize(100,150)  
28.         self.assertEqual(self.widget.size(), (100,150),  
29.                          'wrong size after resize')  
30.           
31.           
32. if __name__ == '__main__':  
33.     unittest.main()  

If the setUp() method raises an exception while the test is running, the framework will consider the test to have suffered an error, and the runTest() method will not be executed. Similarly, we can provide a tearDown() method that tidies up after the runTest() method has been run: 

view plaincopy to clipboardprint?

1. ...  
2. class SimpleWidgetTestCase(unittest.TestCase):  
3.     def setUp(self):  
4.         self.widget = Widget('The widget')  
5.   
6.     def tearDown(self):  
7.         self.widget.dispose()  
8.         self.widget = None  
9. ...  

If setUp() succeeded, the tearDown() method will be run whether runTest() succeeded or not. Such a working environment for the testing code is called a fixture.

Often, many small test cases will use the same fixture. In this case, we would end up subclassing SimpleWidgetTestCase into many small one-method classes such as DefaultWidgetSizeTestCase. This is time-consuming and discouraging, so in the same vein as JUnit, unittest provides a simpler mechanism: 

view plaincopy to clipboardprint?

1. import unittest  
2.   
3. class WidgetTestCase(unittest.TestCase):  
4.     def setUp(self):  
5.         self.widget = Widget('The widget')  
6.   
7.     def tearDown(self):  
8.         self.widget.dispose()  
9.         self.widget = None  
10.   
11.     def test_default_size(self):  
12.         self.assertEqual(self.widget.size(), (50,50),  
13.                          'incorrect default size')  
14.   
15.     def test_resize(self):  
16.         self.widget.resize(100,150)  
17.         self.assertEqual(self.widget.size(), (100,150),  
18.                          'wrong size after resize')  

Here we have not provided a runTest() method, but have instead provided two different test methods. Class instances will now each run one of the test_*()methods, with self.widget created and destroyed separately for each instance. When creating an instance we must specify the test method it is to run. We do this by passing the method name in the constructor: 

view plaincopy to clipboardprint?

1. defaultSizeTestCase = WidgetTestCase('test_default_size')  
2. resizeTestCase = WidgetTestCase('test_resize')  

Test case instances are grouped together according to the features they test. unittest provides a mechanism for this: the test suite, represented byunittest‘s TestSuite class: 

view plaincopy to clipboardprint?

1. widgetTestSuite = unittest.TestSuite()  
2. widgetTestSuite.addTest(WidgetTestCase('test_default_size'))  
3. widgetTestSuite.addTest(WidgetTestCase('test_resize'))  

For the ease of running tests, as we will see later, it is a good idea to provide in each test module a callable object that returns a pre-built test suite: 

view plaincopy to clipboardprint?

1. def suite():  
2.     suite = unittest.TestSuite()  
3.     suite.addTest(WidgetTestCase('test_default_size'))  
4.     suite.addTest(WidgetTestCase('test_resize'))  
5.     return suite  

or even: 

view plaincopy to clipboardprint?

1. def suite():  
2.     tests = ['test_default_size', 'test_resize']  
3.   
4.     return unittest.TestSuite(map(WidgetTestCase, tests))  

Since it is a common pattern to create a TestCase subclass with many similarly named test functions, unittest provides a TestLoader class that can be used to automate the process of creating a test suite and populating it with individual tests. For example, 

view plaincopy to clipboardprint?

1. suite = unittest.TestLoader().loadTestsFromTestCase(WidgetTestCase)  

will create a test suite that will run WidgetTestCase.test_default_size() and WidgetTestCase.test_resize. TestLoader uses the 'test' method name prefix to identify test methods automatically. Note that the order in which the various test cases will be run is determined by sorting the test function names with respect to the built-in ordering for strings. 

Often it is desirable to group suites of test cases together, so as to run tests for the whole system at once. This is easy, since TestSuite instances can be added to a TestSuite just as TestCase instances can be added to a TestSuite: 

view plaincopy to clipboardprint?

1. suite1 = module1.TheTestSuite()  
2. suite2 = module2.TheTestSuite()  
3. alltests = unittest.TestSuite([suite1, suite2])  

You can place the definitions of test cases and test suites in the same modules as the code they are to test (such as widget.py), but there are several advantages to placing the test code in a separate module, such as test_widget.py: 

* The test module can be run standalone from the command line.
* The test code can more easily be separated from shipped code.
* There is less temptation to change test code to fit the code it tests without a good reason.
* Test code should be modified much less frequently than the code it tests.
* Tested code can be refactored more easily.
* Tests for modules written in C must be in separate modules anyway, so why not be consistent?
* If the testing strategy changes, there is no need to change the source code.

Re-using old test code 
Some users will find that they have existing test code that they would like to run from unittest, without converting every old test function to a TestCase subclass. For this reason, unittest provides a FunctionTestCase class. This subclass of TestCase can be used to wrap an existing test function. Set-up and tear-down functions can also be provided. 

Given the following test function: 

view plaincopy to clipboardprint?

1. class Something:  
2.     def __init__(self, name):  
3.         self.name = "test"  
4.   
5. def makeSomething():  
6.     return Something("test")  
7.   
8. def testSomething():  
9.     something = makeSomething()  
10.     assert something.name is not None  

one can create an equivalent test case instance as follows: 

view plaincopy to clipboardprint?

1. testcase = unittest.FunctionTestCase(testSomething)  

If there are additional set-up and tear-down methods that should be called as part of the test case’s operation, they can also be provided like so: 

view plaincopy to clipboardprint?

1. testcase = unittest.FunctionTestCase(testSomething,  
2.                                      setUp=makeSomethingDB,  
3.                                      tearDown=deleteSomethingDB)  

To make migrating existing test suites easier, unittest supports tests raising AssertionError to indicate test failure. However, it is recommended that you use the explicit TestCase.fail*() and TestCase.assert*() methods instead, as future versions of unittest may treat AssertionError differently. 
Note. 

Even though FunctionTestCase can be used to quickly convert an existing test base over to a unittest-based system, this approach is not recommended. Taking the time to set up proper TestCase subclasses will make future test refactorings infinitely easier.

Skipping tests and expected failures 
Unittest supports skipping individual test methods and even whole classes of tests. In addition, it supports marking a test as a “expected failure,” a test that is broken and will fail, but shouldn’t be counted as a failure on a TestResult. 

Skipping a test is simply a matter of using the skip() decorator or one of its conditional variants. 

Basic skipping looks like this: 

view plaincopy to clipboardprint?

1. class MyTestCase(unittest.TestCase):  
2.   
3.     @unittest.skip("demonstrating skipping")  
4.     def test_nothing(self):  
5.         self.fail("shouldn't happen")  
6.   
7.     @unittest.skipIf(mylib.__version__ < (1, 3),  
8.                      "not supported in this library version")  
9.     def test_format(self):  
10.         # Tests that work for only a certain version of the library.  
11.         pass  
12.   
13.     @unittest.skipUnless(sys.platform.startswith("win"), "requires Windows")  
14.     def test_windows_support(self):  
15.         # windows specific testing code  
16.         pass  

This is the output of running the example above in verbose mode: 

test_format (__main__.MyTestCase) ... skipped 'not supported in this library version'
test_nothing (__main__.MyTestCase) ... skipped 'demonstrating skipping'
test_windows_support (__main__.MyTestCase) ... skipped 'requires Windows'

----------------------------------------------------------------------
Ran 3 tests in 0.005s

OK (skipped=3)

Classes can be skipped just like methods: 

view plaincopy to clipboardprint?

1. @unittest.skip("showing class skipping")  
2. class MySkippedTestCase(unittest.TestCase):  
3.     def test_not_run(self):  
4.         pass  

TestCase.setUp() can also skip the test. This is useful when a resource that needs to be set up is not available. 

Expected failures use the expectedFailure() decorator. 

view plaincopy to clipboardprint?

1. class ExpectedFailureTestCase(unittest.TestCase):  
2.     @unittest.expectedFailure  
3.     def test_fail(self):  
4.         self.assertEqual(1, 0, "broken")  

It’s easy to roll your own skipping decorators by making a decorator that calls skip() on the test when it wants it to be skipped. This decorator skips the test unless the passed object has a certain attribute: 

view plaincopy to clipboardprint?

1. def skipUnlessHasattr(obj, attr):  
2.     if hasattr(obj, attr):  
3.         return lambda func: func  
4.     return unittest.skip("{!r} doesn't have {!r}".format(obj, attr))  

The following decorators implement test skipping and expected failures: 
* unittest.skip(reason) 

Unconditionally skip the decorated test. reason should describe why the test is being skipped.

* unittest.skipIf(condition, reason) 

Skip the decorated test if condition is true.

* unittest.skipUnless(condition, reason) 

Skip the decorated test unless condition is true.

* unittest.expectedFailure() 

Mark the test as an expected failure. If the test fails when run, the test is not counted as a failure.

* exception unittest.SkipTest(reason) 

This exception is raised to skip a test.

Usually you can use TestCase.skipTest() or one of the skipping decorators instead of raising this directly.

Skipped tests will not have setUp() or tearDown() run around them. Skipped classes will not have setUpClass() or tearDownClass() run. 

Supplement 
* unittest - Classes and functions