[ Python 常見問題 ] Using module 'subprocess' with timeout

 Source From Here

Question
Here's the Python code to run an arbitrary command returning its stdout data, or raise an exception on non-zero exit codes:

view plaincopy to clipboardprint?

1. proc = subprocess.Popen(  
2.     cmd,  
3.     stderr=subprocess.STDOUT,  # Merge stdout and stderr  
4.     stdout=subprocess.PIPE,  
5.     shell=True)  

communicate is used to wait for the process to exit:

view plaincopy to clipboardprint?

1. stdoutdata, stderrdata = proc.communicate()  

The subprocess module does not support timeout--ability to kill a process running for more than X number of seconds--therefore, communicate may take forever to run.

What is the simplest way to implement timeouts in a Python program meant to run on Windows and Linux?

HowTo
In Python 3.3+:

view plaincopy to clipboardprint?

1. from subprocess import STDOUT, check_output  
2.   
3. output = check_output(cmd, stderr=STDOUT, timeout=seconds)  

output is a byte string that contains command's merged stdout, stderr data.

check_output raises CalledProcessError on non-zero exit status as specified in the question's text unlike proc.communicate() method.

I've removed shell=True because it is often used unnecessarily. You can always add it back if cmd indeed requires it. If you add shell=True i.e., if the child process spawns its own descendants; check_output() can return much later than the timeout indicates, see Subprocess timeout failure.

For a quick demonstration:

>>> from subprocess import STDOUT, check_output
>>> check_output('sleep 1 && echo test', stderr=STDOUT, timeout=5, shell=True)
b'test\n'
>>> check_output('sleep 6 && echo test', stderr=STDOUT, timeout=5, shell=True)
...
subprocess.TimeoutExpired: Command 'sleep 6 && echo test' timed out after 5 seconds

Another approach is to use Python threading module to wrap your command execution. e.g.:
- test.py

view plaincopy to clipboardprint?

1. #!/usr/bin/env python3  
2. import subprocess, threading  
3.   
4. class Command(object):  
5.     def __init__(self, cmd):  
6.         self.cmd = cmd  
7.         self.process = None  
8.   
9.     def run(self, timeout):  
10.         def target():  
11.             print('Thread started')  
12.             self.process = subprocess.Popen(self.cmd, shell=True)  
13.             self.process.communicate()  
14.             print('Thread finished')  
15.   
16.         thread = threading.Thread(target=target)  
17.         thread.start()  
18.   
19.         thread.join(timeout)  
20.         if thread.is_alive():  
21.             print('Terminating process')  
22.             self.process.terminate()  
23.             thread.join()  
24.         print(f"command:{self.cmd}\nWith rc={self.process.returncode} (timeout={timeout})")  
25.   
26. command = Command("echo 'Process started'; sleep 2; echo 'Process finished'")  
27. command.run(timeout=3)  
28. command.run(timeout=1)  

Execution result:

# ./test.py
Thread started
Process started
Process finished
Thread finished
command:echo 'Process started'; sleep 2; echo 'Process finished'
With rc=0 (timeout=3)
Thread started
Process started
Terminating process
Thread finished
command:echo 'Process started'; sleep 2; echo 'Process finished'
With rc=-15 (timeout=1)