Timing and profiling code is all sorts of useful, and it’s also just good ol’ fashioned fun (and sometimes surprising!). In this post, I’ll introduce how to do the following through IPython magic functions:
Installation & Setup
Please follow instructions to install ipython:
Most of the functionality we’ll work with is included in the standard library, but if you’re interested in line-by-line or memory profiling, go ahead and run through this setup. First, install the following:
Next, create an IPython profile and extensions directory where we’ll configure a couple of missing magic functions:
Create the following IPython extention files with the contents below to define the magic functions:
Finally, register the extension modules you just created with the default IPython profile we made earlier:
And that’s it! We’re ready to time and profile to our hearts content. Start ipython and test for the following (executing command ipython):
Time profiling does exactly what it sounds like - it tells you how much time it took to execute a script, which may be a simple one-liner or a whole module.
See how long it takes a script to run.
See how long a script takes to run averaged over multiple runs.
See how long it took each function in a script to run:
See how long it took each line in a function to run.
Create and edit a new module named foo.py in the same directory where you started IPython. Paste the following code in the file and jump back to IPython.
See how much memory a script uses line by line. Let’s take a look at the same foo() function that we profiled with %lprun - except this time we’re interested in incremental memory usage and not execution time:
See how much memory a script uses overall. %memit works a lot like %timeit except that the number of iterations is set with -r instead of -n:
* Difference between CPU time and wall time
* Profiling Python Code
* Chapter 3 of the Python Data Analysis book
* Python MotW: Debugging and Profiling