2018年2月8日 星期四

[ Python 常見問題 ] How to properly assert that an exception gets raised in pytest?

Source From Here 
Question 
Check testing code below: 
  1. # coding=utf-8  
  2. import pytest  
  3.   
  4.   
  5. def whatever():  
  6.     return 9/0  
  7.   
  8. def test_whatever():  
  9.     try:  
  10.         whatever()  
  11.     except ZeroDivisionError as exc:  
  12.         pytest.fail(exc, pytrace=True)  
The output will be: 
  1. ================================ test session starts =================================  
  2. platform linux2 -- Python 2.7.3 -- py-1.4.20 -- pytest-2.5.2  
  3. plugins: django, cov  
  4. collected 1 items   
  5.   
  6. pytest_test.py F  
  7.   
  8. ====================================== FAILURES ======================================  
  9. ___________________________________ test_whatever ____________________________________  
  10.   
  11.     def test_whatever():  
  12.         try:  
  13.             whatever()  
  14.         except ZeroDivisionError as exc:  
  15. >           pytest.fail(exc, pytrace=True)  
  16. E           Failed: integer division or modulo by zero  
  17.   
  18. pytest_test.py:12: Failed  
  19. ============================== 1 failed in 1.16 seconds ==============================  
How to make pytest print traceback, so I would see where in the whatever function an exception was raised

How-To 
There are two ways to handle these kind of cases in pytest by catching expected exception (Assertions about expected exceptions): 
* Using pytest.raises function
* Using pytest.mark.xfail decorator

Usage of pytest.raises
  1. def whatever():  
  2.     return 9/0  
  3. def test_whatever():  
  4.     with pytest.raises(ZeroDivisionError):  
  5.         whatever()  
Output of pytest.raises
  1. ============================= test session starts ============================  
  2. platform linux2 -- Python 2.7.10, pytest-3.2.3, py-1.4.34, pluggy-0.4.0 --   
  3. /usr/local/python_2.7_10/bin/python  
  4. cachedir: .cache  
  5. rootdir: /home/ukpdl, inifile:  
  6. collected 1 item  
  7.   
  8. test_fun.py::test_whatever PASSED  
  9.   
  10.   
  11. ======================== 1 passed in 0.01 seconds =============================  

Usage of pytest.mark.xfail
  1. @pytest.mark.xfail(raises=ZeroDivisionError)  
  2. def test_whatever():  
  3.     whatever()  
Output of pytest.xfail marker: 
  1. ============================= test session starts ============================  
  2. platform linux2 -- Python 2.7.10, pytest-3.2.3, py-1.4.34, pluggy-0.4.0 --   
  3. /usr/local/python_2.7_10/bin/python  
  4. cachedir: .cache  
  5. rootdir: /home/ukpdl, inifile:  
  6. collected 1 item  
  7.   
  8. test_fun.py::test_whatever xfail  
  9.   
  10. ======================== 1 xfailed in 0.03 seconds=============================  
As the document says: 
Using pytest.raises is likely to be better for cases where you are testing exceptions your own code is deliberately raising, whereas using @pytest.mark.xfail with a check function is probably better for something like documenting unfixed bugs(where the test describes what “should” happen) or bugs in dependencies.

Regarding to print the trackback from pytestpytest.raises(Exception) as e_info is what you need. Check sample code as below: 
- /tmp/test.py 
  1. import pytest  
  2. import logging  
  3.   
  4. def test_passes():  
  5.     with pytest.raises(Exception) as e_info:  
  6.         x = 1 / 0  
  7.     logging.error('Expected exception: {}'.format(e_info))  
  8.   
  9. def test_passes_without_info():  
  10.     with pytest.raises(Exception):  
  11.         x = 1 / 0  
  12.   
  13. def test_fails():  
  14.     with pytest.raises(Exception) as e_info:  
  15.         x = 1 / 1  
  16.   
  17. def test_fails_without_info():  
  18.     with pytest.raises(Exception):  
  19.         x = 1 / 1  
  20.   
  21. # Don't do this. Assertions are caught as exceptions.  
  22. def test_passes_but_should_not():  
  23.     try:  
  24.         x = 1 / 1  
  25.         assert False  
  26.     except Exception:  
  27.         assert True  
  28.   
  29. # Even if the appropriate exception is caught, it is bad style,  
  30. # because the test result is less informative  
  31. # than it would be with pytest.raises(e)  
  32. # (it just says pass or fail.)  
  33.   
  34. def test_passes_but_bad_style():  
  35.     try:  
  36.         x = 1 / 0  
  37.         assert False  
  38.     except ZeroDivisionError:  
  39.         assert True  
  40.   
  41. def test_fails_but_bad_style():  
  42.     try:  
  43.         x = 1 / 1  
  44.         assert False  
  45.     except ZeroDivisionError:  
  46.         assert True  
Output (By executing pytest -s /tmp/test.py): 
  1. ...  
  2. ../../tmp/test.py ERROR:root:Expected exception: /tmp/test.py:6: ZeroDivisionError: division by zero  
  3. ...  
  4. === test session starts ===  
  5. platform linux2 -- Python 2.7.6 -- py-1.4.26 -- pytest-2.6.4  
  6. collected 7 items   
  7.   
  8. test.py ..FF..F  
  9.   
  10. === FAILURES ===  
  11. ___ test_fails ____  
  12.   
  13.     def test_fails():  
  14.         with pytest.raises(Exception) as e_info:  
  15. >           x = 1 / 1  
  16. E           Failed: DID NOT RAISE  
  17.   
  18. test.py:13: Failed  
  19. ___ test_fails_without_info ____  
  20.   
  21.     def test_fails_without_info():  
  22.         with pytest.raises(Exception):  
  23. >           x = 1 / 1  
  24. E           Failed: DID NOT RAISE  
  25.   
  26. test.py:17: Failed  
  27. ___ test_fails_but_bad_style ___  
  28.   
  29.     def test_fails_but_bad_style():  
  30.         try:  
  31.             x = 1 / 1  
  32. >           assert False  
  33. E           assert False  
  34.   
  35. test.py:43: AssertionError  
  36. === 3 failed, 4 passed in 0.02 seconds ===  
Note that e_info saves the exception object so you can extract details from it. For example, if you want to check the exception call stack or another nested exception inside.

沒有留言:

張貼留言

[Linux 常見問題] What's the best way to send a signal to all members of a process group?

Source From  Here   Question   I want to  kill a whole process tree.  What is the best way to do this using any common scripting languages? ...