How to profile python 3.5 code line by line in jupyter notebook 5

Question

How to find out execution time taken by each line of python code.

line_profiler works with ipython but doesnt work with jupyter notebook. I tried adding @profile to my function, it gives error saying name 'profile' is not defined. There is one way to do it by time.time() , but i was wondering if there is any inbuilt profiling function which can profile each line of my function and show me the execution time.

def prof_function():
    x=10*20
    y=10+x
    return (y)

Answer 1

You can use line_profiler in jupyter notebook.

1. Install it: pip install line_profiler
2. Within your jupyter notebook, call: %load_ext line_profiler
3. Define your function prof_function as in your example.
4. Finally, profile as follows: %lprun -f prof_function prof_function()

Which will provide the output:

Timer unit: 1e-06 s

Total time: 3e-06 s
File: <ipython-input-22-41854af628da>
Function: prof_function at line 1

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
     1                                           def prof_function():
     2         1          1.0      1.0     33.3      x=10*20
     3         1          1.0      1.0     33.3      y=10+x
     4         1          1.0      1.0     33.3      return (y)

---

Note that you can profile any function in the call hierarchy, not just the top one. If the function you want to profile isn't in the notebook, you must import it. Unfortunately you can only profile functions which are run from source code, not from libraries you've installed (unless they were installed from source).

Example:

 import numpy as np

 def sub_function():
     a = np.random.random((3, 4))
     x = 10 * a
     return x

 def prof_function():
     y = sub_function()
     z = 10 + y
     return z

Output of %lprun -f prof_function prof_function():

 Timer unit: 1e-09 s

 Total time: 3.2e-05 s
 File: /var/folders/hh/f20wzr451rd3m04552cf7n4r0000gp/T/ipykernel_51803/3816028199.py
 Function: prof_function at line 6

 Line #      Hits         Time  Per Hit   % Time  Line Contents
 ==============================================================
      6                                           def prof_function():
      7         1      29000.0  29000.0     90.6      y = sub_function()
      8         1       3000.0   3000.0      9.4      z = 10 + y
      9         1          0.0      0.0      0.0      return z

Output of %lprun -f sub_function prof_function():

 Timer unit: 1e-09 s

 Total time: 2.7e-05 s
 File: /var/folders/hh/f20wzr451rd3m04552cf7n4r0000gp/T/ipykernel_51803/572113936.py
 Function: sub_function at line 1

 Line #      Hits         Time  Per Hit   % Time  Line Contents
 ==============================================================
      1                                           def sub_function():
      2         1      13000.0  13000.0     48.1      a = np.random.random((3, 4))
      3         1      14000.0  14000.0     51.9      x = 10 * a
      4         1          0.0      0.0      0.0      return x

More detail in this nice blog post.