Accessing documentation with ?


In [1]:
help(len)


Help on built-in function len in module builtins:

len(obj, /)
    Return the number of items in a container.


In [2]:
len?

In [3]:
# ? works for many kind of objects
L = [1, 2, 3, 2]

In [4]:
L?

In [5]:
# works for function
L.insert?

In [6]:
# even function user created.
def square(x):
    '''Return square of the value'''
    return x ** 2

In [7]:
square?

In [8]:
#reach the source code
square??

In [9]:
len??

TAB complition


In [10]:
L.count(2)


Out[10]:
2

In [11]:
# press tab after import to see all possible imports
#from itertools import

In [13]:
# press tab after import to see all library in the system
#import

In [14]:
# list all objects ends with Warning
*Warning?

In [15]:
# string method contains the find word in it 
str.*find*?

Magic Commands

1- Line magics denoted by %
2- Cell magics indicated by %%


In [16]:
# run the code that you saved in the text editor in the notebook 
%run myScript.py


1 square is 1
2 square is 4
3 square is 9

In [17]:
%%timeit
L = [n ** 2 for n in range(1000)]


330 µs ± 2.81 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

In [18]:
%%timeit
L = []
for i in range(1000):
    L.append(i ** 2)


386 µs ± 6.97 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

In [19]:
# help on magic functions
%timeit?

In [20]:
#general info about magic functions
%magic

In [21]:
# available magic functions
%lsmagic


Out[21]:
Available line magics:
%alias  %alias_magic  %autocall  %automagic  %autosave  %bookmark  %cat  %cd  %clear  %colors  %config  %connect_info  %cp  %debug  %dhist  %dirs  %doctest_mode  %ed  %edit  %env  %gui  %hist  %history  %killbgscripts  %ldir  %less  %lf  %lk  %ll  %load  %load_ext  %loadpy  %logoff  %logon  %logstart  %logstate  %logstop  %ls  %lsmagic  %lx  %macro  %magic  %man  %matplotlib  %mkdir  %more  %mv  %notebook  %page  %pastebin  %pdb  %pdef  %pdoc  %pfile  %pinfo  %pinfo2  %popd  %pprint  %precision  %profile  %prun  %psearch  %psource  %pushd  %pwd  %pycat  %pylab  %qtconsole  %quickref  %recall  %rehashx  %reload_ext  %rep  %rerun  %reset  %reset_selective  %rm  %rmdir  %run  %save  %sc  %set_env  %store  %sx  %system  %tb  %time  %timeit  %unalias  %unload_ext  %who  %who_ls  %whos  %xdel  %xmode

Available cell magics:
%%!  %%HTML  %%SVG  %%bash  %%capture  %%debug  %%file  %%html  %%javascript  %%js  %%latex  %%perl  %%prun  %%pypy  %%python  %%python2  %%python3  %%ruby  %%script  %%sh  %%svg  %%sx  %%system  %%time  %%timeit  %%writefile

Automagic is ON, % prefix IS NOT needed for line magics.

in and out objects


In [22]:
In


Out[22]:
['',
 'help(len)',
 "get_ipython().magic('pinfo len')",
 '# ? works for many kind of objects\nL = [1, 2, 3, 2]',
 "get_ipython().magic('pinfo L')",
 "# works for function\nget_ipython().magic('pinfo L.insert')",
 "# even function user created.\ndef square(x):\n    '''Return square of the value'''\n    return x ** 2",
 "get_ipython().magic('pinfo square')",
 "#reach the source code\nget_ipython().magic('pinfo2 square')",
 "get_ipython().magic('pinfo2 len')",
 'L.count(2)',
 '# press tab after import to see all possible imports\n#from itertools import ',
 '# press tab after import to see all library in the system\nimport ',
 '# press tab after import to see all library in the system\n#import ',
 "# list all objects ends with Warning\nget_ipython().magic('psearch *Warning')",
 "# string method contains the find word in it \nget_ipython().magic('psearch str.*find*')",
 "# run the code that you saved in the text editor in the notebook \nget_ipython().magic('run myScript.py')",
 "get_ipython().run_cell_magic('timeit', '', 'L = [n ** 2 for n in range(1000)]')",
 "get_ipython().run_cell_magic('timeit', '', 'L = []\\nfor i in range(1000):\\n    L.append(i ** 2)')",
 "# help on magic functions\nget_ipython().magic('pinfo %timeit')",
 "#general info about magic functions\nget_ipython().magic('magic')",
 "# available magic functions\nget_ipython().magic('lsmagic')",
 'In']

In [23]:
Out


Out[23]:
{10: 2,
 21: Available line magics:
%alias  %alias_magic  %autocall  %automagic  %autosave  %bookmark  %cat  %cd  %clear  %colors  %config  %connect_info  %cp  %debug  %dhist  %dirs  %doctest_mode  %ed  %edit  %env  %gui  %hist  %history  %killbgscripts  %ldir  %less  %lf  %lk  %ll  %load  %load_ext  %loadpy  %logoff  %logon  %logstart  %logstate  %logstop  %ls  %lsmagic  %lx  %macro  %magic  %man  %matplotlib  %mkdir  %more  %mv  %notebook  %page  %pastebin  %pdb  %pdef  %pdoc  %pfile  %pinfo  %pinfo2  %popd  %pprint  %precision  %profile  %prun  %psearch  %psource  %pushd  %pwd  %pycat  %pylab  %qtconsole  %quickref  %recall  %rehashx  %reload_ext  %rep  %rerun  %reset  %reset_selective  %rm  %rmdir  %run  %save  %sc  %set_env  %store  %sx  %system  %tb  %time  %timeit  %unalias  %unload_ext  %who  %who_ls  %whos  %xdel  %xmode

Available cell magics:
%%!  %%HTML  %%SVG  %%bash  %%capture  %%debug  %%file  %%html  %%javascript  %%js  %%latex  %%perl  %%prun  %%pypy  %%python  %%python2  %%python3  %%ruby  %%script  %%sh  %%svg  %%sx  %%system  %%time  %%timeit  %%writefile

Automagic is ON, % prefix IS NOT needed for line magics.,
 22: ['',
  'help(len)',
  "get_ipython().magic('pinfo len')",
  '# ? works for many kind of objects\nL = [1, 2, 3, 2]',
  "get_ipython().magic('pinfo L')",
  "# works for function\nget_ipython().magic('pinfo L.insert')",
  "# even function user created.\ndef square(x):\n    '''Return square of the value'''\n    return x ** 2",
  "get_ipython().magic('pinfo square')",
  "#reach the source code\nget_ipython().magic('pinfo2 square')",
  "get_ipython().magic('pinfo2 len')",
  'L.count(2)',
  '# press tab after import to see all possible imports\n#from itertools import ',
  '# press tab after import to see all library in the system\nimport ',
  '# press tab after import to see all library in the system\n#import ',
  "# list all objects ends with Warning\nget_ipython().magic('psearch *Warning')",
  "# string method contains the find word in it \nget_ipython().magic('psearch str.*find*')",
  "# run the code that you saved in the text editor in the notebook \nget_ipython().magic('run myScript.py')",
  "get_ipython().run_cell_magic('timeit', '', 'L = [n ** 2 for n in range(1000)]')",
  "get_ipython().run_cell_magic('timeit', '', 'L = []\\nfor i in range(1000):\\n    L.append(i ** 2)')",
  "# help on magic functions\nget_ipython().magic('pinfo %timeit')",
  "#general info about magic functions\nget_ipython().magic('magic')",
  "# available magic functions\nget_ipython().magic('lsmagic')",
  'In',
  'Out']}

In [26]:
In[10]


Out[26]:
'L.count(2)'

In [28]:
# _ prints the previous output
_


Out[28]:
'L.count(2)'

In [29]:
3 + 4


Out[29]:
7

In [31]:
# suppress the output with ;
3 + 4 ;

In [32]:
%history?

In [36]:
%history -n 30-32


  30: 3 + 4 ;
  31:
# suppress the output with ;
3 + 4 ;
  32: %history?

In [37]:
%rerun


=== Executing: ===
%history -n 30-32
=== Output: ===
  30: 3 + 4 ;
  31:
# suppress the output with ;
3 + 4 ;
  32: %history?

In [38]:
!pwd


/Users/numanyilmaz/Documents/MyProjects/jake/PythonDataScienceHandbookNotes

In [39]:
directory = !pwd
directory


Out[39]:
['/Users/numanyilmaz/Documents/MyProjects/jake/PythonDataScienceHandbookNotes']

In [43]:
# use shell command in the notebook. automagic has to be on.

In [42]:
ls


Chapter1.ipynb    ExampleDirectory/ LICENSE           README.md

Debugging and Errors


In [48]:
def func1(a,b):
    return a / b
def func2(x):
    a = x
    b = x - 1
    return func1(a,b)

In [50]:
func2(1)


---------------------------------------------------------------------------
ZeroDivisionError                         Traceback (most recent call last)
<ipython-input-50-b2e110f6fc8f> in <module>()
----> 1 func2(1)

<ipython-input-48-7eb40a43b4a1> in func2(x)
      4     a = x
      5     b = x - 1
----> 6     return func1(a,b)

<ipython-input-48-7eb40a43b4a1> in func1(a, b)
      1 def func1(a,b):
----> 2     return a / b
      3 def func2(x):
      4     a = x
      5     b = x - 1

ZeroDivisionError: division by zero

In [51]:
%xmode Plain


Exception reporting mode: Plain

In [52]:
func2(1)


Traceback (most recent call last):

  File "<ipython-input-52-b2e110f6fc8f>", line 1, in <module>
    func2(1)

  File "<ipython-input-48-7eb40a43b4a1>", line 6, in func2
    return func1(a,b)

  File "<ipython-input-48-7eb40a43b4a1>", line 2, in func1
    return a / b

ZeroDivisionError: division by zero

In [53]:
%xmode Verbose


Exception reporting mode: Verbose

In [54]:
func2(1)


---------------------------------------------------------------------------
ZeroDivisionError                         Traceback (most recent call last)
<ipython-input-54-b2e110f6fc8f> in <module>()
----> 1 func2(1)
        global func2 = <function func2 at 0x110c97a60>

<ipython-input-48-7eb40a43b4a1> in func2(x=1)
      4     a = x
      5     b = x - 1
----> 6     return func1(a,b)
        global func1 = <function func1 at 0x110c97d08>
        a = 1
        b = 0

<ipython-input-48-7eb40a43b4a1> in func1(a=1, b=0)
      1 def func1(a,b):
----> 2     return a / b
        a = 1
        b = 0
      3 def func2(x):
      4     a = x
      5     b = x - 1

ZeroDivisionError: division by zero

Python Debugger (pdb)


In [55]:
%debug


> <ipython-input-48-7eb40a43b4a1>(2)func1()
      1 def func1(a,b):
----> 2     return a / b
      3 def func2(x):
      4     a = x
      5     b = x - 1

ipdb> a
a = 1
b = 0
ipdb> print(a)
1
ipdb> print(b)
0
ipdb> up
> <ipython-input-48-7eb40a43b4a1>(6)func2()
      2     return a / b
      3 def func2(x):
      4     a = x
      5     b = x - 1
----> 6     return func1(a,b)

ipdb> print(x)
1
ipdb> quit

In [56]:
%pdb on


Automatic pdb calling has been turned ON

Profiler and Timing the code


In [57]:
%timeit sum(range(100))


1.5 µs ± 26.7 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

In [58]:
#profiler
def sum_of_lists(N):
    total = 0
    for i in range(5):
        L = [j ^ (j >> i) for j in range(N)]
        total += sum(L)
    return total

In [60]:
%prun sum_of_lists(1000000)


 

In [63]:
#line by line profiler
!pip install line_profiler


Collecting line_profiler
  Downloading line_profiler-2.0.tar.gz (68kB)
    100% |████████████████████████████████| 71kB 1.5MB/s ta 0:00:01
Requirement already satisfied: IPython>=0.13 in /anaconda/anaconda/lib/python3.6/site-packages (from line_profiler)
Requirement already satisfied: jedi>=0.10 in /anaconda/anaconda/lib/python3.6/site-packages (from IPython>=0.13->line_profiler)
Requirement already satisfied: prompt-toolkit<2.0.0,>=1.0.4 in /anaconda/anaconda/lib/python3.6/site-packages (from IPython>=0.13->line_profiler)
Requirement already satisfied: decorator in /anaconda/anaconda/lib/python3.6/site-packages (from IPython>=0.13->line_profiler)
Requirement already satisfied: appnope; sys_platform == "darwin" in /anaconda/anaconda/lib/python3.6/site-packages (from IPython>=0.13->line_profiler)
Requirement already satisfied: pexpect; sys_platform != "win32" in /anaconda/anaconda/lib/python3.6/site-packages (from IPython>=0.13->line_profiler)
Requirement already satisfied: simplegeneric>0.8 in /anaconda/anaconda/lib/python3.6/site-packages (from IPython>=0.13->line_profiler)
Requirement already satisfied: traitlets>=4.2 in /anaconda/anaconda/lib/python3.6/site-packages (from IPython>=0.13->line_profiler)
Requirement already satisfied: pygments in /anaconda/anaconda/lib/python3.6/site-packages (from IPython>=0.13->line_profiler)
Requirement already satisfied: pickleshare in /anaconda/anaconda/lib/python3.6/site-packages (from IPython>=0.13->line_profiler)
Requirement already satisfied: setuptools>=18.5 in /anaconda/anaconda/lib/python3.6/site-packages (from IPython>=0.13->line_profiler)
Requirement already satisfied: six>=1.9.0 in /anaconda/anaconda/lib/python3.6/site-packages (from prompt-toolkit<2.0.0,>=1.0.4->IPython>=0.13->line_profiler)
Requirement already satisfied: wcwidth in /anaconda/anaconda/lib/python3.6/site-packages (from prompt-toolkit<2.0.0,>=1.0.4->IPython>=0.13->line_profiler)
Requirement already satisfied: ipython-genutils in /anaconda/anaconda/lib/python3.6/site-packages (from traitlets>=4.2->IPython>=0.13->line_profiler)
Requirement already satisfied: packaging>=16.8 in /anaconda/anaconda/lib/python3.6/site-packages (from setuptools>=18.5->IPython>=0.13->line_profiler)
Requirement already satisfied: appdirs>=1.4.0 in /anaconda/anaconda/lib/python3.6/site-packages (from setuptools>=18.5->IPython>=0.13->line_profiler)
Requirement already satisfied: pyparsing in /anaconda/anaconda/lib/python3.6/site-packages (from packaging>=16.8->setuptools>=18.5->IPython>=0.13->line_profiler)
Building wheels for collected packages: line-profiler
  Running setup.py bdist_wheel for line-profiler ... done
  Stored in directory: /Users/numanyilmaz/Library/Caches/pip/wheels/a8/aa/3b/1d4570cc467313caec92c9d51e6543b598b835ab95cfe9c726
Successfully built line-profiler
Installing collected packages: line-profiler
Successfully installed line-profiler-2.0

In [65]:
%load_ext line_profiler

In [66]:
%lprun -f sum_of_lists sum_of_lists(5000)

In [68]:
#memory porfiler
!pip install memory_profiler


Collecting memory_profiler
  Downloading memory_profiler-0.45.tar.gz
Building wheels for collected packages: memory-profiler
  Running setup.py bdist_wheel for memory-profiler ... done
  Stored in directory: /Users/numanyilmaz/Library/Caches/pip/wheels/34/5e/67/68730a6aef67abcabc2e2e5f67d6c8e3221cbc5b16030d312a
Successfully built memory-profiler
Installing collected packages: memory-profiler
Successfully installed memory-profiler-0.45

In [69]:
%load_ext memory_profiler

In [70]:
%memit sum_of_lists(1000000)


peak memory: 109.42 MiB, increment: 63.57 MiB

In [71]:
pwd


Out[71]:
'/Users/numanyilmaz/Documents/MyProjects/jake/PythonDataScienceHandbookNotes'

In [73]:
%%file mprun_demo.py
def sum_of_lists(N):
    total = 0
    for i in range(5):
        L = [j ^ (j >> i) for j in range(N)]
        total += sum(L)
    return total


Writing mprun_demo.py

In [75]:
from mprun_demo import sum_of_lists
%mprun -f sum_of_lists sum_of_lists(10000)




In [76]:
import this


The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

In [ ]: