[ Python 常見問題 ] Is it possible to attach a console into a running process

Source From Here
Question
I just want to see the state of the process, is it possible to attach a console into the process, so I can invoke functions inside the process and see some of the global variables.

How-To
If you have access to the program's source-code, you can add this functionality relatively easily. See Recipe 576515: Debugging a running python process by interrupting and providing an interactive prompt (Python):

This provides code to allow any python program which uses it to be interrupted at the current point, and communicated with via a normal python interactive console. This allows the locals, globals and associated program state to be investigated, as well as calling arbitrary functions and classes. To use, a process should import the module, and call listen() at any point during startup. To interrupt this process, the script can be run directly, giving the process Id of the process to debug as the parameter.

A simple user case can be referred as below. Firstly, our fake main program zombe.py:
- zombe.py

view plaincopy to clipboardprint?

1. #!/usr/bin/env python  
2. import time  
3. import DebugPy  
4. import pdb  
5. import sys  
6.   
7. cnt=0  
8. DebugPy.listen()  # This is key point for the main program being able to be intercepted by Linux signal  
9.   
10.   
11. def test():  
12.     print("This is testing function in zombe")  
13.   
14. while True:  
15.     cnt = cnt + 1  
16.     print("Count %d...(sleep 10 min)" % (cnt))  
17.     time.sleep(1*60)  

Which will use while loop to keep running as zombie. Then is the module which will help us intercept the running Python process for debugging:
- DebugPy.py

view plaincopy to clipboardprint?

1. #!/usr/bin/env python  
2. try: import readline  # For readline input support  
3. except: pass  
4.   
5. import sys, os, traceback, signal, codeop, cStringIO, cPickle, tempfile  
6.   
7. def pipename(pid):  
8.     """Return name of pipe to use"""  
9.     return os.path.join(tempfile.gettempdir(), 'debug-%d' % pid)  
10.   
11. class NamedPipe(object):  
12.     def __init__(self, name, end=0, mode=0666):  
13.         """Open a pair of pipes, name.in and name.out for communication  
14.         with another process.  One process should pass 1 for end, and the  
15.         other 0.  Data is marshalled with pickle."""  
16.         self.in_name, self.out_name = name +'.in',  name +'.out',  
17.         try: os.mkfifo(self.in_name,mode)  
18.         except OSError: pass  
19.         try: os.mkfifo(self.out_name,mode)  
20.         except OSError: pass  
21.   
22.         # NOTE: The order the ends are opened in is important - both ends  
23.         # of pipe 1 must be opened before the second pipe can be opened.  
24.         if end:  
25.             self.inp = open(self.out_name,'r')  
26.             self.out = open(self.in_name,'w')  
27.         else:  
28.             self.out = open(self.out_name,'w')  
29.             self.inp = open(self.in_name,'r')  
30.         self._open = True  
31.   
32.     def is_open(self):  
33.         return not (self.inp.closed or self.out.closed)  
34.   
35.     def put(self,msg):  
36.         if self.is_open():  
37.             data = cPickle.dumps(msg,1)  
38.             self.out.write("%d\n" % len(data))  
39.             self.out.write(data)  
40.             self.out.flush()  
41.         else:  
42.             raise Exception("Pipe closed")  
43.   
44.     def get(self):  
45.         txt=self.inp.readline()  
46.         if not txt:  
47.             self.inp.close()  
48.         else:  
49.             l = int(txt)  
50.             data=self.inp.read(l)  
51.             if len(data) < l: self.inp.close()  
52.             return cPickle.loads(data)  # Convert back to python object.  
53.   
54.     def close(self):  
55.         self.inp.close()  
56.         self.out.close()  
57.         try: os.remove(self.in_name)  
58.         except OSError: pass  
59.         try: os.remove(self.out_name)  
60.         except OSError: pass  
61.   
62.     def __del__(self):  
63.         self.close()  
64.   
65. def remote_debug(sig,frame):  
66.     """Handler to allow process to be remotely debugged."""  
67.     def _raiseEx(ex):  
68.         """Raise specified exception in the remote process"""  
69.         _raiseEx.ex = ex  
70.     _raiseEx.ex = None  
71.   
72.     try:  
73.         # Provide some useful functions.  
74.         locs = {'_raiseEx' : _raiseEx}  
75.         locs.update(frame.f_locals)  # Unless shadowed.  
76.         globs = frame.f_globals  
77.   
78.         pid = os.getpid()  # Use pipe name based on pid  
79.         pipe = NamedPipe(pipename(pid))  
80.   
81.         old_stdout, old_stderr = sys.stdout, sys.stderr  
82.         txt = ''  
83.         pipe.put("Interrupting process at following point:\n" +  
84.                ''.join(traceback.format_stack(frame)) + ">>> ")  
85.   
86.         try:  
87.             while pipe.is_open() and _raiseEx.ex is None:  
88.                 line = pipe.get()  
89.                 if line is None: continue # EOF  
90.                 txt += line  
91.                 try:  
92.                     code = codeop.compile_command(txt)  
93.                     if code:  
94.                         sys.stdout = cStringIO.StringIO()  
95.                         sys.stderr = sys.stdout  
96.                         exec code in globs,locs  
97.                         txt = ''  
98.                         pipe.put(sys.stdout.getvalue() + '>>> ')  
99.                     else:  
100.                         pipe.put('... ')  
101.                 except:  
102.                     txt='' # May be syntax err.  
103.                     sys.stdout = cStringIO.StringIO()  
104.                     sys.stderr = sys.stdout  
105.                     traceback.print_exc()  
106.                     pipe.put(sys.stdout.getvalue() + '>>> ')  
107.         finally:  
108.             sys.stdout = old_stdout # Restore redirected output.  
109.             sys.stderr = old_stderr  
110.             pipe.close()  
111.   
112.     except Exception:  # Don't allow debug exceptions to propogate to real program.  
113.         traceback.print_exc()  
114.   
115.     if _raiseEx.ex is not None: raise _raiseEx.ex  
116.   
117. def debug_process(pid):  
118.     """Interrupt a running process and debug it."""  
119.     os.kill(pid, signal.SIGUSR1)  # Signal process.  
120.     pipe = NamedPipe(pipename(pid), 1)  
121.     try:  
122.         while pipe.is_open():  
123.             txt=raw_input(pipe.get()) + '\n'  
124.             pipe.put(txt)  
125.     except EOFError:  
126.         pass # Exit.  
127.     pipe.close()  
128.   
129. def listen():  
130.     signal.signal(signal.SIGUSR1, remote_debug) # Register for remote debugging.  
131.   
132. if __name__=='__main__':  
133.     if len(sys.argv) != 2:  
134.         print "Error: Must provide process id to debug"  
135.     else:  
136.         pid = int(sys.argv[1])  
137.         debug_process(pid)  

Then you can use it this way:

# ./zombe.py & // Launch zombe in background
[1] 22266
# ./DebugPy.py 22266 // Attach into zombe process with PID
Interrupting process at following point:
File "./zombe.py", line 21, in
time.sleep(1*60)
>>> cnt // Check the variable cnt in zombe process
1
>>> test() // Call the function available in zombe process
This is testing function in zombe

Another implementation of roughly the same concept is provided by rconsole. From the documentation:

rconsole is a remote Python console with auto completion, which can be used to inspect and modify the namespace of a running script. To invoke in a script do:

view plaincopy to clipboardprint?

1. from rfoo.utils import rconsole  
2. rconsole.spawn_server()  

To attach from a shell do:

$ rconsole

Security note: The rconsole listener started with spawn_server() will accept any local connection and may therefore be insecure to use in shared hosting or similar environments!