Python Hidden Treasure

This ebook contains few lesser known Python gems. As usual, I will try to keep them updated and will continue to expand. If you wish to add any new, send them to me at (funmayank @ yahoo . co . in).

Variables

In-place value swapping


In [206]:
a = 10
b = "TEST"
a, b = b, a
print(a, b)


TEST 10

Unicode identifier

Python 3 allows to have unicode identifier's, which allows non-english speaking users to code.


In [216]:
हिन्दी = 10
print(हिन्दी)


10

Integer

Negative round

round is a function to round off the numbers and its normal usage is as follows


In [2]:
num = round(283746.32321, 1)
print(num)


283746.3

The second parameter defines the decimal number to which the number to rounded of. But if we provide a -ve number to it then it starts rounding of the number itself instead of decimal digit as shown in the below example


In [3]:
num = round(283746.32321, -2)
print(num)
num = round(283746.32321, -1)
print(num)
num = round(283746.32321, -4)
print(num)


283700.0
283750.0
280000.0

pow power - pow() can calculate (x ** y) % z


In [4]:
x, y, z = 1019292929191, 1029228322, 222224
pow(x, y, z)


Out[4]:
115681

In [5]:
# Do not run this, please. it will take forever.
##### (x ** y) % z

String

Multi line strings

In python we can have multiple ways to achieve multi line strings.

  • Using triple quotes

In [6]:
txt = """The Supreme Lord said: The indestructible, transcendental living 
entity is called Brahman  and his eternal nature is called the 
self. Action pertaining to the development of these  material
bodies is called karma, or fruitive activities."""

In [7]:
print(txt)


The Supreme Lord said: The indestructible, transcendental living 
entity is called Brahman  and his eternal nature is called the 
self. Action pertaining to the development of these  material
bodies is called karma, or fruitive activities.
  • Using brackets "( )"

In [8]:
txt = ("The Supreme Lord said: The indestructible, transcendental living" 
      "entity is called Brahman  and his eternal nature is called the "
      "self. Action pertaining to the development of these  material"
      "bodies is called karma, or fruitive activities.")

In [9]:
print(txt)


The Supreme Lord said: The indestructible, transcendental livingentity is called Brahman  and his eternal nature is called the self. Action pertaining to the development of these  materialbodies is called karma, or fruitive activities.

In [10]:
txt = "The Supreme Lord said: The indestructible, transcendental living " \
      "entity is called Brahman  and his eternal nature is called the"
print(txt)


The Supreme Lord said: The indestructible, transcendental living entity is called Brahman  and his eternal nature is called the

using string multiply with int results in concatinating string that number of times. Lets print a line on console using -.


In [2]:
print("~^*" * 10)


~^*~^*~^*~^*~^*~^*~^*~^*~^*~^*

Search substring in string


In [12]:
print("ash" in "ashwini")


True

In [3]:
print("ash" is ['a', 's', 'h'])


False

In [4]:
print("ash" is ('a', 's', 'h'))


False

In [14]:
print("ash" is 'ash')


True

In [15]:
### Implicit concatenation without "+" operator

In [5]:
name = "Mayank" " " "Johri"
print(name)


Mayank Johri

In [12]:
try:
    name = "Mayank" " " "Johri" ' .' 
    print(name)
except SyntaxError:
    pass


Mayank Johri .

Join list of strings


In [14]:
list_cities = ["Bhopal", "New Delhi", "Agra", "Mumbai", "Aligarh", "Hyderabad"]

# Lets join the list of string in string using `join`
str_cities = ", ".join(list_cities)
print(str_cities)


Bhopal, New Delhi, Agra, Mumbai, Aligarh, Hyderabad

In [15]:
list_cities = ("Bhopal", "New Delhi", "Agra", "Mumbai", "Aligarh", "Hyderabad")

# Lets join the list of string in string using `join`
str_cities = ", ".join(list_cities)
print(str_cities)


Bhopal, New Delhi, Agra, Mumbai, Aligarh, Hyderabad

Reverse the string

There are few methods to reverse the string, but two are most common

  • using slices

In [17]:
txt = "The Mother Earth"
print(txt[::-1])


htraE rehtoM ehT

In [16]:
txt = "The Mother Earth"
print("".join(list(reversed(txt))))


htraE rehtoM ehT

List / Tuple

tuple / list unpacking


In [18]:
a, b, *remaining = (1, 2, 3, 4, 5, "test")
print(a, b)
print(remaining)


1 2
[3, 4, 5, 'test']

In [19]:
a, b, *remaining = [1, 2, 3, 4, 5, "test"]
print(a, b)
print(remaining)


1 2
[3, 4, 5, 'test']

In [20]:
first,*middle,last = (1, 2, 3, 4, 5, 6, 7, 8)

print(first, last)
print(middle)


1 8
[2, 3, 4, 5, 6, 7]

In [21]:
first,*middle,last = [1, 2, 3, 4, 5, 6, 7, 8]

print(first, last)
print(middle)


1 8
[2, 3, 4, 5, 6, 7]

List/tuple multiplication ;)

similar to String we can literally multiply string and tuples with integer as shown below


In [22]:
lst = [1, 2, 3]
print(lst * 3)


[1, 2, 3, 1, 2, 3, 1, 2, 3]

In [23]:
lst = (1, 2, 3)
print(lst * 3)


(1, 2, 3, 1, 2, 3, 1, 2, 3)

Array Transpose using zip


In [25]:
a = [(1,2), (3,4), (5,6)]
print(list(zip(a)))

print("*" * 33)
print(list(zip(*a)))


[((1, 2),), ((3, 4),), ((5, 6),)]
*********************************
[(1, 3, 5), (2, 4, 6)]

In [31]:
a = [(1, 2, 7), 
     (3, 4, 8), 
     (5, 6, 9)]
print(list(zip(a)))

print("*" * 33)
print(list(zip(*a)))


[((1, 2, 7),), ((3, 4, 8),), ((5, 6, 9),)]
*********************************
[(1, 3, 5), (2, 4, 6), (7, 8, 9)]

enumerate with predefined starting index


In [28]:
lst = ["Ashwini", "Banti", "Bhaiya", "Mayank", "Shashank", "Rahul" ]
list(enumerate(lst))


Out[28]:
[(0, 'Ashwini'),
 (1, 'Banti'),
 (2, 'Bhaiya'),
 (3, 'Mayank'),
 (4, 'Shashank'),
 (5, 'Rahul')]

Now, lets change the starting index to 10


In [29]:
print(list(enumerate(lst, 10)))


[(10, 'Ashwini'), (11, 'Banti'), (12, 'Bhaiya'), (13, 'Mayank'), (14, 'Shashank'), (15, 'Rahul')]

Reverse the list

built-in keyword reversed allows the list to be reversed.


In [34]:
lst = [1, 2, 3, 4, 53]
print(list(reversed(lst)))


[53, 4, 3, 2, 1]

In [35]:
print(lst[::-1])


[53, 4, 3, 2, 1]

Flattening of list


In [45]:
l = [[1,2], [3], [4,5], [6], [7, 8, 9]]
l1 = [[1,2], 3, [4,5], [6], [7, 8, 9]]
l2 = [[1,2], [3], [4,5], [6], [[7, 8], 9], 10]
  • Method 1:

In [39]:
from itertools import chain

flattened_list = list(chain(*l))
print(flattened_list)


[1, 2, 3, 4, 5, 6, 7, 8, 9]

NOTE: this method will fail if any of the element is non list item as shown in the below example


In [41]:
from itertools import chain
try:
    flattened_list = list(chain(*l1))
    print(flattened_list)
except:
    print("Error !!!")


Error !!!

In [44]:
from itertools import chain

flattened_list = list(chain(*l2))
print(flattened_list)


[1, 2, 3, 4, 5, 6, [7, 8], 9]
  • Method 2:

In [163]:
flattened_list = [y for x in l for y in x]
print(flattened_list)


[1, 2, 3, 4, 5, 6, 7, 8, 9]

NOTE: this method will fail if any of the element is non list item as shown in the below example


In [190]:
flattened_list = [y for x in l1 for y in x]
print(flattened_list)


---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-190-7d7abc28b176> in <module>()
----> 1 flattened_list = [y for x in l1 for y in x]
      2 print(flattened_list)

<ipython-input-190-7d7abc28b176> in <listcomp>(.0)
----> 1 flattened_list = [y for x in l1 for y in x]
      2 print(flattened_list)

TypeError: 'int' object is not iterable

Lets update code to handle this situation


In [193]:
flattened_list = [si for i in l1 for si in (i if isinstance(i, list) else [i])]
print(flattened_list)


[1, 2, 3, 4, 5, 6, 7, 8, 9]
  • Method 3:

In [161]:
flattened_list = sum(l, [])
print(flattened_list)


[1, 2, 3, 4, 5, 6, 7, 8, 9]

NOTE: this method will fail if any of the element is non list item as shown in the below example


In [148]:
sum(l1, [])


---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-148-6c8ec61ef5b9> in <module>()
----> 1 sum(l1, [])

TypeError: can only concatenate list (not "int") to list
  • Method 4:

In [160]:
flattened_list = []
for x in l:
    for y in x:
        flattened_list.append(y)
print(flattened_list)


[1, 2, 3, 4, 5, 6, 7, 8, 9]

NOTE: this method will fail if any of the element is non list item as shown in the below example


In [166]:
flattened_list = []
for x in l1:
    for y in x:
        flattened_list.append(y)
print(flattened_list)


---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-166-4967a0a245fb> in <module>()
      1 flattened_list = []
      2 for x in l1:
----> 3     for y in x:
      4         flattened_list.append(y)
      5 print(flattened_list)

TypeError: 'int' object is not iterable
  • Method 5:

In [196]:
from functools import reduce  
flattened_list = reduce(lambda x, y: x + y, l)
print(flattened_list)


[1, 2, 3, 4, 5, 6, 7, 8, 9]

NOTE: this method will fail if any of the element is non list item as shown in the below example


In [197]:
flattened_list = reduce(lambda x, y: x + y, l1)
print(flattened_list)


---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-197-78cb8e6490ba> in <module>()
----> 1 flattened_list = reduce(lambda x, y: x + y, l1)
      2 print(flattened_list)

<ipython-input-197-78cb8e6490ba> in <lambda>(x, y)
----> 1 flattened_list = reduce(lambda x, y: x + y, l1)
      2 print(flattened_list)

TypeError: can only concatenate list (not "int") to list
  • Method 6:

In [186]:
import operator
flattened_list = reduce(operator.add, l)
print(flattened_list)


[1, 2, 3, 4, 5, 6, 7, 8, 9]

NOTE: this method will fail if any of the element is non list item as shown in the below example


In [187]:
import operator
flattened_list = reduce(operator.add, l1)
print(flattened_list)


---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-187-2781850cb615> in <module>()
      1 import operator
----> 2 flattened_list = reduce(operator.add, l1)
      3 print(flattened_list)

TypeError: can only concatenate list (not "int") to list

Infinite Recursion


In [157]:
lst = [1, 2]
lst.append(lst)
print(lst)


[1, 2, [...]]

lets check if really we have infinite recursion, with the following code. We should get RuntimeError: maximum recursion depth exceeded in comparison error later in the execution.

def test(lst):
    for a in lst:
        if isinstance(a, list):
            print("A", a)
            test(a)
        print(a)

test(lst)

Copy a list


In [200]:
ori = [1, 2, 3, 4, 5, 6]
dup = ori
print(id(ori))
print(id(dup))


140397756982280
140397756982280

Both the variables are still pointing to same list, thus change in one will change another also.


In [201]:
dup.insert(0, 29)
print(ori)
print(dup)


[29, 1, 2, 3, 4, 5, 6]
[29, 1, 2, 3, 4, 5, 6]

Deepcopy a list


In [47]:
ori = [1, 2, 3, 4, 5, 6]
dup = ori[:]
print(id(ori))
print(id(dup))


98568776
94952008

In [58]:
dup.insert(0, 29)
print(ori)
print(dup)


[1, 2, 3, [4, 10, 10], 5, 6]
[29, 1, 2, 3, [4, 10, 10], 5, 6]

!!! ouch moment !!!


In [55]:
ori = [1, 2, 3, [4], 5, 6]
dup = ori[:]
print(id(ori))
print(id(dup))


98568968
100768968

In [67]:
ori[3].append(10)
print(ori)
print(dup)
print(id(ori[3]))
print(id(dup[4]))
print(id(ori))
print(id(dup))


[1, 2, 3, [4, 10, 10], 5, 6]
[1, 2, 3, [4], 5, 6]
91489096
1901620832
91689928
91690504

rescue using Deep copy


In [66]:
from copy import deepcopy
ori = [1, 2, 3, [4], 5, 6]
dup = deepcopy(ori)
print(ori)
print(dup)
print(id(ori[3]))
print(id(dup[3]))
print(id(ori))
print(id(dup))
ori[3].append(10)
print(ori)
print(dup)
print(id(ori[3]))
print(id(dup[3]))
print(id(ori))
print(id(dup))


[1, 2, 3, [4], 5, 6]
[1, 2, 3, [4], 5, 6]
91489096
94879944
91689928
91690504
[1, 2, 3, [4, 10], 5, 6]
[1, 2, 3, [4], 5, 6]
91489096
94879944
91689928
91690504

Dictionaries

Reverse the key values in unique dictionary


In [70]:
states_capitals = {'MP': 'Bhopal', 'UP': 'Lucknow', 'Rajasthan': 'Jaipur'}
  • Method 1:

In [117]:
capitals_states = dict(zip(*list(zip(*states_capitals.items()))[::-1]))
print(capitals_states)


{'Jaipur': 'Rajasthan', 'Bhopal': 'MP', 'Lucknow': 'UP'}
  • Method 2:

In [71]:
capitals_states = dict([v, k] for k, v in states_capitals.items())
print(capitals_states)


{'Bhopal': 'MP', 'Lucknow': 'UP', 'Jaipur': 'Rajasthan'}
  • Method 3:

In [123]:
capitals_states = dict(zip(states_capitals.values(), states_capitals.keys()))
print(capitals_states)


{'Jaipur': 'Rajasthan', 'Bhopal': 'MP', 'Lucknow': 'UP'}
  • Method 4:

In [72]:
capitals_states = {states_capitals[k] : k for k in states_capitals}
print(capitals_states)


{'Bhopal': 'MP', 'Lucknow': 'UP', 'Jaipur': 'Rajasthan'}

Creating dictionaries

Multiple methods can be used to create a dictionary. We are going to cover few of the cool ones.

  • Using two lists

In [31]:
states = ["MP", "UP", "Rajasthan"]
capitals = ["Bhopal", "Lucknow", "Jaipur"]

states_capitals = dict(zip(states, capitals))
print(states_capitals)


{'MP': 'Bhopal', 'UP': 'Lucknow', 'Rajasthan': 'Jaipur'}
  • Using arguments

In [32]:
states_capitals = dict(MP='Bhopal', Rajasthan='Jaipur', UP='Lucknow')
print(states_capitals)


{'MP': 'Bhopal', 'UP': 'Lucknow', 'Rajasthan': 'Jaipur'}
  • list of tuples

In [59]:
states_capitals = dict([('MP', 'Bhopal'), ('UP', 'Lucknow'), ('Rajasthan', 'Jaipur')])
print(states_capitals)


{'MP': 'Bhopal', 'UP': 'Lucknow', 'Rajasthan': 'Jaipur'}
  • By adding two dictionary using copy and update

In [229]:
a = {'MP': 'Bhopal', 'UP': 'Lucknow', 'Rajasthan': 'Jaipur'}
b = {'Jaipur': 'Rajasthan', 'Bhopal': 'MP', 'Lucknow': 'UP'}
c = a.copy()
c.update(b)
print(c)


{'MP': 'Bhopal', 'Jaipur': 'Rajasthan', 'Bhopal': 'MP', 'UP': 'Lucknow', 'Lucknow': 'UP', 'Rajasthan': 'Jaipur'}
# for Python >= 3.5: https://www.python.org/dev/peps/pep-0448
c = {**b, **a}
print(c)
  • Using dictionary comprehension

In [ ]:
def  double_bubble(x):
    yield x
    yield x*x
    
d = {k:v for k, v in double_bubble}

In [78]:
{chr(97+i)*2 : i for i in range(5)}


Out[78]:
{'aa': 0, 'bb': 1, 'cc': 2, 'dd': 3, 'ee': 4}

if

Conditional Assignment


In [33]:
y = 10
x = 3 if (y == 1) else 2
print(x)


2

In [34]:
x = 3 if (y == 1) else 2 if (y == -1) else 1
print(x)


1

Functions

default arguments

Dangerous mutable default arguments


In [35]:
def foo(x=[]):
    x.append(1)
    print(x)
    
foo()
foo()
foo()


[1]
[1, 1]
[1, 1, 1]

In [73]:
# instead use:
def fun(x=None):
    if x is None:
        x = []
    x.append(1)
    print(x)

fun()
fun()
fun()


[1]
[1]
[1]

TODO: Add more examples

Function argument unpacking


In [37]:
def draw_point(x, y):
    """You can unpack a list or a dictionary as 
    function arguments using * and **."""
    print(x, y)

point_foo = (3, 4)
point_bar = {'y': 3, 'x': 2}

draw_point(*point_foo)
draw_point(**point_bar)


3 4
2 3

Function arguments


In [76]:
def letsEcho():
    test = "Hello"
    print(test)
    
letsEcho.test = "Welcome"
print(letsEcho.test)
letsEcho()
print(letsEcho.test)


Welcome
Hello
Welcome

Finally returns the ultimate return


In [126]:
def dum_dum():
    try:
        return '`dum dum` returning from try'
    finally:
        return '`dum dum` returning from finally'

print(dum_dum())


`dum dum` returning from finally

OOPS

Attributes

Dynamically added attributes


In [136]:
class Test():
    def __getattribute__(self, name):
        f = lambda: " ".join([name, name[::-1]])
        return f

t = Test()
# New attribute created at runtime
t.rev()


Out[136]:
'rev ver'

operators

Chaining comparison operators


In [77]:
x = 5

In [79]:
1 < x < 100


Out[79]:
True

In [81]:
1 < x > 100


Out[81]:
False

In [83]:
1 > x > 100


Out[83]:
False

In [84]:
1 > x < 100


Out[84]:
False

In [42]:
10 < x < 20


Out[42]:
False

In [88]:
x < 10 < x*10 < 100


Out[88]:
True

In [90]:
x < 10 < x*10 < 50


Out[90]:
False

In [91]:
x < 10 < x*10 <= 50


Out[91]:
True

In [44]:
10 > x <= 9


Out[44]:
True

In [93]:
5 == x > 4


Out[93]:
True

In [94]:
x == 5 > 4


Out[94]:
True

enumerate

Wrap an iterable with enumerate and it will yield the item along with its index.


In [48]:
a = ['a', 'b', 'c', 'd', 'e']
for index, item in enumerate(a): print (index, item)


0 a
1 b
2 c
3 d
4 e

Generators

Sending values into generator functions

https://www.python.org/dev/peps/pep-0342/, also please reaad http://www.dabeaz.com/coroutines/


In [49]:
def mygen():
    """Yield 5 until something else is passed back via send()"""
    a = 5
    while True:
        f = (yield a) #yield a and possibly get f in return
        if f is not None: 
            a = f  #store the new value

g = mygen()
print(next(g))
print(next(g))
g.send(7)
print(next(g))
print(next(g))
g.send(17)
print(next(g))
print(next(g))


5
5
7
7
17
17

Iterators

iter() can take a callable argument


In [50]:
def seek_next_line(f):
    """
    The iter(callable, until_value) function repeatedly calls 
    callable and yields its result until until_value is returned.
    """
    for c in iter(lambda: f.read(1),'\n'):
        pass

I/O

with

open multiple files in a single with.


In [95]:
try:
    with open('a', 'w') as a, open('b', 'w') as b:
        pass
except IOError as e:
    print ('Operation failed: %s' % e.strerror)

In [212]:
#### write file using `print`

In [215]:
with open("outfile.txt" , "w+") as outFile:
    print('Modern Standard Hindi is a standardised and sanskritised register of the Hindustani language.', file=outFile)

Exception

Re-raising exceptions:


In [54]:
# Python 2 syntax
try:
    some_operation()
except SomeError, e:
    if is_fatal(e):
        raise
    handle_nonfatal(e)


  File "<ipython-input-54-bfe95c85d6b3>", line 4
    except SomeError, e:
                    ^
SyntaxError: invalid syntax

In [55]:
def some_operation():
    raise Exception
    
def is_fatal(e):
    return True

# Python 3 syntax
try:
    some_operation()
except Exception as e:
    if is_fatal(e):
        raise
    handle_nonfatal(e)


---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
<ipython-input-55-18cc95aa93e6> in <module>()
      7 # Python 3 syntax
      8 try:
----> 9     some_operation()
     10 except Exception as e:
     11     if is_fatal(e):

<ipython-input-55-18cc95aa93e6> in some_operation()
      1 def some_operation():
----> 2     raise Exception
      3 
      4 def is_fatal(e):
      5     return True

Exception: 

!!! Easter Eggs !!!


In [56]:
from __future__ import braces


  File "<ipython-input-56-6d5c5b2f0daf>", line 1
    from __future__ import braces
SyntaxError: not a chance

In [205]:
import __hello__


Hello world!

Lets encrypt our code using cot13


In [57]:
import codecs
s   = 'The Zen of Python, by Tim Peters'
enc = codecs.getencoder( "rot-13" )
dec = codecs.getdecoder("rot-13")
os  = enc( s )[0]
print(os)
print(dec(os)[0])


Gur Mra bs Clguba, ol Gvz Crgref
The Zen of Python, by Tim Peters

In [58]:
import this


The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!