Python Training - Lesson 5 - Python idioms and Pythonic code

Python guidelines - how to code?

Style and readbility of code - PEP8

PEP8 is a set of common sense practices and rules on how to format the text of code, how to name variables, when to make newline breaks, etc. You should familiarize yourself with this standard, as most of work environments will use this to some extent. https://www.python.org/dev/peps/pep-0008/

Zen of Python code - PEP20

PEP20 is a set of short recommendations on how to write code. https://www.python.org/dev/peps/pep-0020/

It's called Zen, because it's more of a spiritual guide, than a concrete ruleset. I recommend reading it, and a few examples on how this changes the look of code can be found here: https://gist.github.com/evandrix/2030615

Hitch-hiker's guide to Python

The code style article can be found here: http://docs.python-guide.org/en/latest/writing/style/

Glossary

This page has explanations for many words and acronyms used in Python world. https://docs.python.org/2/glossary.html

I would expand here on these examples:

EAFP

Easier to ask for forgiveness than permission.


In [ ]:
# Example of what that means:

# Some dictionary which we obtained, have no control of.
dictionary = {"a":1, "b":2, "c":3}

# List of keys we always check.
some_keys = ["a", "b", "c", "d"]

In [ ]:
# Old-style way - Look before you leap (LBYL)
for k in some_keys:
    if k not in dictionary:
        print("Expected to find key: " + str(k) + " but did not find it.")
        continue
    else:
        print(dictionary[k])

In [ ]:
# Pythonic way - ask for forgiveness, not permission
for k in some_keys:
    try:
        print(dictionary[k])
    except KeyError:
        print("Expected to find key: " + str(k) + " but did not find it.")
        continue
    except Exception as e:
        print("Something terrible happened. Details: " + str(e))
        continue

Observe the differences. In the first case, we always need to now what exactly we should check before we perform our operation (so we "ask for permission"). What if we don't know?

Imagine - you open a file, but first you "ask for permission" - you check if the file exists. It exists, you open it, but then an exception is raised, like "You do not have sufficient rights to read this file". Our program fails.

If you first perform an operation, and "ask for forgiveness", then you have a much greater control, and you communicate something with your code - that it should work most of the time, except some times it does not.

If you always "ask for permission", then you are wasting computation.

Some recommendations for the EAFP rule:

EAFP (Easier to Ask for Forgiveness than Permission)

IO operations (Hard drive and Networking) Actions that will almost always be successful Database operations (when dealing with transactions and can rollback) Fast prototyping in a throw away environment

LBYL (Look Before You Leap):

Irrevocable actions, or anything that may have a side effect Operation that may fail more times than succeed When an exception that needs special attention could be easily caught beforehand

Idiomatic Python

First, I recommend watching this video: https://www.youtube.com/watch?v=OSGv2VnC0go

"Transforming Code into Beautiful, Idiomatic Python"

Most of these examples come from that video.

Pythonic, idiomatic Python

It just means, that the code uses Python idioms, the Python features that make this programming language unique. The code will be more readable, expressive, will be able to do more things than you thought it can. Let's go through some examples.

Change many "if" into a dictionary

To avoid the infamous "if" ladders, it is much much easier to change this into a dictionary.

First examples shows how to change the argument of "print" function with this approach. Try to count how many less "checks" are performed by the system.


In [ ]:
# This is bad:
s = ["a",1,(2,2), 20.00]

for elem in s:
    if isinstance(elem, str):
        print("This is string")
    elif isinstance(elem, int):
        print("This is an integer")
    elif isinstance(elem, tuple):
        print("This is a tuple")
    else:
        print("This is something else. Details:" + str(type(elem)))

In [ ]:
# This is good:
s = ["a", 1, (2,2), 20.00]

helper_dict = {
    str: "This is string",
    int: "This is integer",
    tuple: "This is a tuple"}

for elem in s:
    # Notice "asking for forgiveness" and not "permission"
    try:
        print(helper_dict[type(elem)])
    except Exception as e:
        print("This is something else. Details: " + str(e))

In [ ]:
# Another example, but to store FUNCTIONS instead of VARIABLES
from datetime import datetime
helper_dict = {"amount": float, "counter": int, "date": datetime.strptime}

# Types references are also functions that convert variables between types.

some_dict = {"currency": "USD", "amount": "10000", "source": "Poland", "target": "Poland", "counter": "9298", "date": "20171102"}

for key, value in some_dict.items():
    try:
        converted = helper_dict[key](value)
    except Exception:
        converted = str(value)
    
    print(converted)
    print(type(converted))

Loop over a range of numbers


In [ ]:
# This is not productive
for i in [0,1,2,3,4,5]:
    print(i)

In [ ]:
# This is much better
for i in range(6): 
    print(i)

# The 'range' function does not return a simple list.
# It returns an "iterable" - which gives you elements one at a time,
# so the actual big list is not held there inside the statement.

Loop forwards and backwards through a list


In [ ]:
cars = ['ford', 'volvo', 'chevrolet']

# This is bad
for i in range(len(cars)): print(cars[i])

In [ ]:
# This is better
for car in cars: print(car)

In [ ]:
# Reversed
for car in reversed(cars): print(car)

Loop over a list AND the indexes at the same time


In [ ]:
# I want to know the index of an item inside iteration

# This is bad
for i in range(len(cars)):
    print(str(i) + " " + cars[i])

In [ ]:
# This is better
for i, car in enumerate(cars): print(str(i) + " " + car)

Loop over two lists at the same time


In [ ]:
numbers = [1,2,3,3,4]
letters = ["a", "b", "c", "d", "e"]

# This is bad
for i in range(len(numbers)):
    print(str(numbers[i]) + " " + letters[i])

In [ ]:
# This is better
for number, letter in zip(numbers,letters): print(number,letter)

Calling a function until something happens


In [ ]:
# Lets write a simple file
import os

filename = 'example.txt'

try:
    os.remove(filename)
except OSError:
    pass

with open('example.txt', 'w+') as f:
    [f.write(str(x) + "\n") for x in range(0,20)]

In [ ]:
# Bad way
with open('example.txt', 'r') as f:
    while True:
        line = f.readline()
        if line == '':
            break
        print(line)

In [ ]:
# Better way
with open('example.txt', 'r') as f:
    for line in iter(f.readline, ''):
        print(line)

Looping over dictionary keys and values at the same time


In [ ]:
dictionary = {k:v for k,v in zip(range(0,3), range(0,3))}

# Bad Way
for k in dictionary.keys():
    print(k, dictionary[k])

In [ ]:
# Much better way
for k, v in dictionary.items():
    print(k, v)

Unpacking sequences


In [ ]:
seq = ["a", "b", "c", "d"]

# Bad way
first = seq[0]
second = seq[1]
third = seq[2]
fourth = seq[3]
print(first, second, third, fourth)

In [ ]:
# Better way
first, second, third, fourth = seq
print(first, second, third, fourth)

Unpacking with wildcard "*"


In [ ]:
seq = ["a", "b", "c", "d", "e", "d"]
start, *middle, end = seq
print(start)
print(middle)
print(end)

Updating multiple variables at once


In [ ]:
# Bad fibonacci implementation

def fibonacci(n):
    x = 0
    y = 1
    for i in range(n):
        print(x)
        t = y
        y = x + y
        x = t

fibonacci(8)

In [ ]:
# Simpler implementation

def fibonacci(n):
    x, y = 0, 1
    for i in range(n):
        print(x)
        x, y = y, x + y

fibonacci(8)

In [ ]:
# Multiple updates at the same time
x, y, z, u = range(0,4)
print(x, y, z, u)

x, y, z, u = x + 1, y + z, u - x, z**2
print(x, y, z, u)

Basics of itertools

These tools provide some nice operations you can do on collections.


In [ ]:
import itertools

In [ ]:
# List all the different sequences of a starting list

permutations = itertools.permutations([1,2,3])
print(list(permutations))

In [ ]:
# Cycle constantly through a short sequence
from itertools import cycle

counter = 20
for item in cycle('Adamek'):
    if counter > 0:
        print(item)
        counter -= 1