In [1]:
print("Hi, All!")
Look at Raymond Hettinger's slides.
Brief Bio: Python Core developer and main Python instructor @ Cisco.
In [2]:
import this
TRIVIA: Tim Peters a.k.a Uncle Tim has invented a new sorting algorithm during Python's lists development.
Key concept: in Python everything is an object. Key concept: variables have no type. What does it means in practice? Variables are only "tags", or "names" to refer to an object. Variables are bound to objects with the assignment operator "=". If one tries to access an unassigned variable a NameError exception is raised.
a
NameError: name 'a' is not defined
Key concept: in Python objects have only attributes. What does it means in practice?
When an object's method or attribute is "called" using the "dot notation", but the method/attribute is not present, an "AttributeError" exception is raised. If the method/attribute is present, it gets "called".
In other words, Python will check only attribute presence at run time. Since there's no compiler or static typing is VITAL to TEST the script before use it in production!
a = None
a.strip()
AttributeError: 'NoneType' object has no attribute 'strip'
Key Concept: a module is a "script" which contains objects used by another script who "imports" the module using the import keyword.
Key Concept: Python searches modules into his standard library path or in paths contained into the PYTHONPATH environment variable, which is a "colon" separated list of paths.
Key Concept: a Python package is a directory that contains a special filename: __init__.py, package's modules are all scripts into that directory. Modules may be imported in __init__.py.
It's simple to obtain the list of all Python's builtins:
In [3]:
print(dir(__builtins__))
s = "hello"
s[1]
s[1] = 'i'
TypeError: 'str' object does not support item assignment
In [4]:
i = 10
i
Out[4]:
In [5]:
i = int()
i
Out[5]:
In [6]:
i = int("10")
i
Out[6]:
In [7]:
i = int("F", 16)
i
Out[7]:
In [8]:
f = 10.0
f
Out[8]:
In [9]:
f = float(10)
f
Out[9]:
In [10]:
f = float("10")
f
Out[10]:
No base conversion for float!
Be careful with == !=.
In [11]:
b = True
b
Out[11]:
In [12]:
b = bool()
b
Out[12]:
In [13]:
b = bool(1)
b
Out[13]:
Ok, I think you got it!
GLOSSARY: extension: a python module implemented in C, in form of a shared library.
Although the math module is not a built-in, it is implemented as C extension. For detail:
import math
help(math)
In [14]:
import math
i = 16
f = 16.0
In [15]:
math.sqrt(i)
Out[15]:
In [16]:
math.sqrt(f)
Out[16]:
key concept: The Iterable abstraction is a Python concept that generalizes the idea of "sequence".
We will see that "Iterable" objects are the only objects applicable to the "for" statements: in every iteration an Iterable yields an element of the "sequence" starting from the first element.
Tuples, lists, strings and sets are all Iterables and built-ins types. Tuples and strings are also immutable types.
The built-in function len() can be used to count elements.
NOTE: len() is applicable to all "collection" objects, not only iterables.
In [17]:
t = 1,2,3,4,5
t
Out[17]:
In [18]:
t = (1,2,3,4,5)
t
Out[18]:
In [19]:
t = tuple()
t
Out[19]:
In [20]:
t = tuple("12345")
t
Out[20]:
In [21]:
t[0]
Out[21]:
In [22]:
t[1]
Out[22]:
In [23]:
t[-1]
Out[23]:
t[2] = 'A'
TypeError: 'tuple' object does not support item assignment
In [24]:
t[0:3]
Out[24]:
In [25]:
t[3:]
Out[25]:
In [26]:
t[::2]
Out[26]:
In [27]:
t[1::2]
Out[27]:
In [28]:
t = (1,2,3,4,5)
t += 6,7
t
Out[28]:
In [29]:
t = t * 2
t
Out[29]:
In [30]:
2 in t
Out[30]:
Please note that containment does not apply to "sub-tuples", but only to elements: in this case integers.
In [31]:
(1, 2) in t
Out[31]:
In [32]:
len(t)
Out[32]:
In [33]:
t = 1, 1.0, "hi, there"
t
Out[33]:
In [34]:
i, f, s = t
s
Out[34]:
The built-in function len() can be used to count elements
Lists are Mutable.
Lists can be modified with methods, for example append.
In [35]:
l = [1,2,3,4,5]
l2 = l
id(l)
Out[35]:
In [36]:
id(l2)
Out[36]:
In [37]:
l2 = list(l)
l
Out[37]:
In [38]:
id(l2)
Out[38]:
In [39]:
l3 = l[::]
l3
Out[39]:
In [40]:
id(l3)
Out[40]:
I can modify a list
In [41]:
l3[2] = 9
l3
Out[41]:
In [42]:
id(l3)
Out[42]:
In [43]:
print(dir(l))
In [44]:
l.append(10)
l
Out[44]:
| Operation | Big-O Efficiency |
|---|---|
| index [] | O(1) |
| index assignment | O(1) |
| append | O(1) |
| pop() | O(1) |
| pop(i) | O(n) |
| insert(i,item) | O(n) |
| del operator | O(n) |
| Operation | Big-O Efficiency |
|---|---|
| iteration | O(n) |
| contains (in) | O(n) |
| get slice [x:y] | O(k) |
| del slice | O(n) |
| set slice | O(n+k) |
| reverse | O(n) |
| concatenate | O(k) |
| sort | O(n log n) |
| multiply | O(nk) |
So Python lists are "Array Lists", implemented into the interpreter. Use them as arrays without fear!
A set is a mutable, unordered, unique collection of objects. It is designed to reflect the properties and behavior of a true mathematical set. The following operators apply to a set:
union), & (intersection), - (difference), ^ (simmetric_difference)issubset.
In [45]:
print(dir(set()))
A set can be created with set(). set() takes any iterable as parameters to create a set.
In [46]:
s = set([1,2,2,2,3,4,5,6,6])
s
Out[46]:
In [47]:
l = list(s)
l
Out[47]:
So, sets can be used to remove duplicates!
ATTENTION: Set are "iterables" but the insertion order is not guaranteed.
Sets have an immutable counterpart: frozenset
In [48]:
fs = frozenset("aaabbbcvfzggg")
fs
Out[48]:
String literals can be defined with any of single quotes ('), double quotes (") or triple quotes (''' or """). All give the same result with two important differences.
The following operators apply to strings:
Strings are immutable but have methods for example strip.
In [49]:
s = ''
s
Out[49]:
In [50]:
s = str()
s
Out[50]:
In [51]:
s = "Portuguese Man o' War what a strange animal!"
s
Out[51]:
In [52]:
s = 'yes, is made up of many individuals called "zoids"'
s
Out[52]:
In [53]:
s = """Is incredible isn't it?
watch out from his venom filled tentacles"""
s
Out[53]:
In [54]:
s[0]
Out[54]:
s[0] = 'U'
TypeError: 'str' object does not support item assignment
Immutability!
In [55]:
s = 'Hello!'
s
Out[55]:
In [56]:
s += ' How are you?'
s
Out[56]:
In [57]:
s * 2
Out[57]:
NOTE: due to strings immutability, is advisable to avoid the use of string concatenation to produce long strings of text, this is more true in loops: string concatenation in loops is a performance killer. Is better to accumulate many strings in a list and "join" them, will will see this in next slides.
In [58]:
'Hello' in s
Out[58]:
The "in" operator is an handy way to check for sub-strings. Use it whenever possible.
In [59]:
s = "{} {} is an actor his age is {}".format("Johnny", "Depp", 56)
s
Out[59]:
In [60]:
s = "{1} {0} is an actor his age is {2}".format("Depp", "Johnny", 56)
s
Out[60]:
In [61]:
s = "{firstname} {lastname} is an actor his age is {age}".format(age=56,
firstname="Johnny", lastname="Depp")
s
Out[61]:
In [62]:
print(dir(s))
Split and join are two very useful string methods and are the base of the powerful Python's text processing capabilities.
text.split(separator) "splits" the text string in a list of strings where chunks are interleaved by the separator string. If there's no separator string as argument, the separator is any number of blanks: space or tabs.
separator.join(list) conversely produce a string from a list of strings where string chunks are "joined" by interleaving the separator string. If the separator is "", the empty string, chunks are joined without interleaving.
join can be used to produce a big text string from chunks accumulated into a list of strings in a very efficient way respect to string concatenation. For example the list of strings may contain the lines of a text file and join can be used to produce the final text of the text file.
split is the base for simple parsing tasks of text files or analysis of record oriented text files, like for example /etc/passwd.
In [63]:
s = 'here is a string with many\tspaces'
s
Out[63]:
In [64]:
chunks = s.split()
chunks
Out[64]:
In [65]:
chunks[5] = 'single'
' '.join(chunks)
Out[65]:
In [66]:
s.split(' ')
Out[66]:
In [67]:
s.split('\t')
Out[67]:
In [68]:
''.join(s.split())
Out[68]:
In [69]:
','.join(s.split())
Out[69]:
Files, again, are a Python interpreter built-in type.
File access is very simple, files are always created by the open function that takes two string arguments: the file path to open and the type of operation: read or write
The main operations are "r", "rb", "w", "wb": text read, binary read, text write, binary write respectively. The default operation is "r" if no argument is given.
Python reserves a special treatment for text files: text files are iterable objects, this means that they can be used in a for statement and in every iteration a file object yields a line of text.
In [70]:
f = open('examples/greetings.txt')
print(f.read())
f.close()
Using a "Context Manager" (we will talk better later about this)
In [71]:
with open('examples/greetings.txt') as f:
print(f.read())
In [72]:
print(dir(f))
In [73]:
print(f.closed)
WARNING: Dictionary order is undefined and implementation-specific. It can be different across interpreters, versions, architectures, and more. Even multiple executions in the same environment.
A dictionary is not an iterable. What does it means in practice?
Dictionaries cannot be used in for statements, although is possible to convert them into tuples with the items method.
In [74]:
d = {}
d
Out[74]:
In [75]:
d = dict()
d
Out[75]:
In [76]:
d = {'firstname': 'John', 'lastname': 'Smith'}
d
Out[76]:
The dictionary use a special kind of index operator [key]. Any immutable object can be used as a key, although typical keys are strings or integers, tuples in some cases.
In [77]:
d['lastname']
Out[77]:
In [78]:
d['firstname']
Out[78]:
A Dictionary is mutable! So I can modify it
In [79]:
d['firstname'] = 'Johnny'
d['lastname'] = 'Deep'
d
Out[79]:
I can add keys and values.
In [80]:
import datetime
today = datetime.date.today()
d['age'] = today.year - 1963
d
Out[80]:
d['bankaccountpassword']
KeyError: 'bankaccountpassword'
If a key is not present a KeyError is raised.
If you prefer you can "default" a value.
In [81]:
d.get('bankaccountpassword', 'UNKNOWN')
Out[81]:
| operation | Big-O Efficiency |
|---|---|
| copy | O(n) |
| get item | O(1) |
| set item | O(1) |
| delete item | O(1) |
| contains (in) | O(1) |
So Dictionaries are the equivalent of a Java HashMap built into the interpreter itself.
So one can use them for tasks where insertion ordering or sorting is not important. This is the main differences with lists.
The main task of a Python programmer is to choose among the data structures that Python designers give us not reinventing the wheel when possible.
In [82]:
print(dir(d))
In [83]:
import datetime
today = datetime.date.today()
today
Out[83]:
In [84]:
print(dir(today))
In [85]:
today + datetime.timedelta(days=10)
Out[85]:
In [86]:
now = datetime.datetime.now()
now
Out[86]:
In [87]:
print(dir(now))
In [88]:
now.hour
Out[88]:
In [89]:
ints = list(range(10))
import random
random.shuffle(ints)
ints
Out[89]:
In [90]:
import heapq
heapq.heapify(ints)
ints[0]
Out[90]:
In [91]:
heapq.heappop(ints)
Out[91]:
In [92]:
ints
Out[92]:
In [93]:
import re
import collections
with open('examples/romeoandjuliet.txt') as will:
words = re.findall(r'\w+', will.read().lower())
print(collections.Counter(words).most_common(10))
Is useful for log file analysis isn't it?
In [94]:
import collections
def palindrome(word):
if not word:
return False
if len(word) > 1:
d = collections.deque(word.lower())
while d:
if d.pop() != d.popleft():
return False
return True
print(palindrome('Anna'))
In [95]:
print(palindrome('abracadabra'))
In [96]:
import collections
d = {'CSCO': 26, 'APPL': 108, 'IBM':140, 'INTC': 29}
dv = collections.OrderedDict(sorted(d.items(), key=lambda x: -x[1]))
dv
Out[96]:
Used with the sorted built in function we can have sorted dictionaries.
TRIVIA: sorted is Raymond Hettinger's stuff!
In [97]:
import collections
stocks = [('CSCO', 29), ('APPL', 130), ('APPL', 140),
('CSCO', 27), ('INTC', 28)]
dd = collections.defaultdict(list)
for stock, value in stocks:
dd[stock].append(value)
dd.items()
Out[97]:
In [98]:
from __future__ import print_function
import collections
records = [('CSCO', 27), ('APPL', 140), ('INTC', 28)]
stocks_nt = collections.namedtuple('stocks', ['name', 'value'])
stocks = [stocks_nt._make(record) for record in records]
for stock in stocks:
print(stock.name, stock.value)
This is Raymond Hettinger's stuff too!
Examples?
In [99]:
import csv
with open('examples/stocks.csv') as stocks:
for stock in csv.reader(stocks):
name, value = stock
print(name, value)
import csv
hero_vs_villain = [('hero', 'villain'), ('spiderman', 'goblin'), ('batman', 'jocker')]
with open('examples/comics.csv', 'wb') as comics:
comic_writer = csv.writer(comics)
comic_writer.writerows(hero_vs_villain)
# test some code
def greeting(name):
return 'Hello, ' + name
import unittest
class TestGreeting(unittest.TestCase):
"Test case for 'greeting'."
def test_greeting(self):
"Add many tests as you want as methods."
self.assertEqual('Hello, World', greeting('World'))
unittest.main()
Anyway I would like to mention:
So, my message is: before write something on your own, check in http://www.python.org before!
In Python the code executed when a certain condition is met is called "block". Each line of code in a certain scope must be indented equally and indented more than the surrounding top level scope. The standard (defined in PEP-8) is to use 4 spaces for each level of block indentation. Statements preceding blocks generally end with a colon ":".
Because there are no semi-colons or other end-of-line indicators, breaking lines of code requires either a continuation character (\ as the last char) or for the break to occur inside an unfinished structure (such as open parentheses).
I do really need to indent correctly the code to make it work?
Yeah!
So, there's no block signalling characters like braces and this will be for ever in this way?
from __future__ import braces
SyntaxError: not a chance
Oh, Yeah!
In [100]:
import keyword
print(keyword.kwlist)
In [101]:
from __future__ import print_function
print('Numbers')
In [102]:
print('Numbers', 1, 2, 3)
In [103]:
l = [1, 2, 3, 4]
print(l)
In [104]:
print(l.__str__())
statement can be equal to pass.
pass means "no operation in this if branch".
In [105]:
a = ['a', 'b', 'c']
for i in range(len(a)):
print(a[i])
In [106]:
for c in a:
print(c)
In [107]:
# don't do this
for i in range(len(a)):
print(i + 1, a[i])
In [108]:
for i, c in enumerate(a):
print(i + 1, c)
In [109]:
b = [1, 2, 3]
for c, i in zip(a, b):
print(c, i)
In [110]:
s = 'abracadabra'
j = sorted(s)
print(''.join(j))
In [111]:
j = []
for c in reversed(s):
j.append(c)
print(''.join(j))
In [112]:
d = {'a': 1, 'b': 2, 'c': 3}
for key, value in d.items():
print(key, value)
ATTENTION: the order is not preditable! (But you can use sorted)
Why is more efficient?
Because for loops in Python are very generic, no assumption on iterables or object in iterables is possibile. In list comprehension, instead, the iterable is always a list, so it was possible to add C code into the Interpreter to deal with this idiom.
In [113]:
pows = [x ** 2 for x in range(10)]
pows
Out[113]:
In [114]:
odd_pows = [x ** 2 for x in range(10) if x % 2]
odd_pows
Out[114]:
In [115]:
even_pows = [x ** 2 for x in range(10) if not x % 2]
even_pows
Out[115]:
In [116]:
try:
1/0
except ZeroDivisionError:
print('caught')
Key concept: parameters are "passed by value" if type is immutable, by reference otherwise.
Key concept: if no return value is specified a function will return None.
In [117]:
from __future__ import print_function
def func(i, xs):
"i: int, xs:list."
i = i * 3
print('i', i)
xs[0] = 2
xs.append(42)
print('xs', xs)
k = 1
ys = [1]
func(k, ys)
print('k is immutable', k)
print('ys is mutable', ys)
In [118]:
func(xs=[10,20], i=3)
In [119]:
def func_with_default(xs, i=3):
"xs:list, i:int."
func(i, xs)
func_with_default([11, 22])
Use default values for immutable types. Is possible to use them for mutable types but the default object is created once.
In [120]:
def func_with_mutable_default(i, xs=[1]):
"i:int, xs:list."
func(i, xs)
func_with_mutable_default(4)
func_with_mutable_default(5)
In [121]:
def arg_list_func(*args):
for arg in args:
print(arg)
arg_list_func(1, 'hello', [1, 2, 3])
In [122]:
def kw_arg_func(**kwargs):
for k, v in kwargs.items():
print(k,v)
kw_arg_func(a="hello", b="there", c=1, d=[1,2])
In [123]:
def kw_arg_list_func(*args, **kwargs):
for arg in args:
print(arg)
for k, v in kwargs.items():
print(k,v)
kw_arg_list_func(1,2,"bye",x=1,y=3,s="cartesian",l=[1,2])
In [124]:
def power2(x):
return x ** 2
power2(3)
Out[124]:
In [125]:
def power22(x, y):
return x ** 2, y ** 2
power22(3, 4)
Out[125]:
In [126]:
powerZZ = power22
powerZZ(4, 5)
Out[126]:
In [127]:
def make_adder(x):
def adder(y):
return x + y
return adder
In [128]:
inc = make_adder(1)
inc(2)
Out[128]:
In [129]:
greeter = make_adder('Hello')
greeter(' Closure')
Out[129]:
What happened?
adder inner function can see the x variable and "close" it. The "closure" is over the x variable.Next Step. Suppose we want to write a "something" able to log the argument(s) and return value(s) of a function.
This can be done with a closure.
In [130]:
def logit(func):
def wrapper(*args, **kwargs):
print(args, kwargs)
return_value = func(*args, **kwargs)
print(return_value)
return return_value
return wrapper
In [131]:
def power2(x):
return x ** 2
power2 = logit(power2)
power2(5)
Out[131]:
It's only a way to write something like
power2 = logit(power2)
In a clearer way.
In [132]:
@logit
def power2(x):
return x ** 2
power2(3)
Out[132]:
This is the simplest way to write a Decorator, but I think you got the Idea.
In [133]:
def loop3():
"A toy Generator."
yield 1
yield 2
yield 3
for i in loop3():
print(i)
In Python even the code is iterable!
A Generator is a sort of stoppable function. The context of a Generator is always saved: instruction pointer and local variables.
In [134]:
def looper(x):
a = 0
while a < x:
yield a
a += 1
for i in looper(3):
print(i)
In [135]:
def fib(n):
"Fibonacci generator."
a, b = 0, 1
while a < n:
yield a
prev = a
a = b
b = a + prev
for f in fib(30):
print(f)
Generator have a nice property. They are like an Iterator: they are lazy. They don't create list or tuples that consume memory before the iteration
Python has built in Generators: for example enumerate or os.walk.
They are useful to save memory when is necessary to pile up many transformation and/or filtering over an iterable.
In [136]:
ge = (x ** 2 for x in range(10))
type(ge)
Out[136]:
In [137]:
odd_pows = [x for x in ge if x % 2]
odd_pows
Out[137]:
In [138]:
type(odd_pows)
Out[138]:
In [139]:
class Engine:
def __enter__(self):
print('Ignite!')
def __exit__(*args):
print('Turn key')
def boost(self):
print('Quench my thirst with gasoline')
dominic = Engine()
with dominic:
dominic.boost()
In [140]:
class Pizza:
"All love it."
p = Pizza
type(p)
Out[140]:
In Python, classes are First Class Objects.
In [141]:
po = Pizza()
type(po)
Out[141]:
This is how to create a Pizza's instance.
In [142]:
class BioPizza(Pizza):
"This is healty."
gluten_free = True
BioPizza.gluten_free
Out[142]:
In [143]:
bp = BioPizza()
bp.gluten_free
Out[143]:
This is a class attribute. And see how inherit from base class.
In [144]:
class Pizza:
def __init__(self, toppings=[]):
self.toppings = toppings
def get_toppings(self):
return self.toppings
p = Pizza(['pepperoni'])
p.get_toppings()
Out[144]:
self is the Java this and is explicit._, private methods start with __.
In [145]:
class Pizza:
healthiness = 1
@staticmethod
def default_healthiness():
return Pizza.healthiness
def __init__(self, toppings=[]):
self.toppings = toppings
def get_toppings(self):
return self.toppings
def healty_factor(self):
return self.default_healthiness()
Pizza.default_healthiness()
Out[145]:
In [146]:
p = Pizza()
p.healty_factor()
Out[146]:
In [147]:
class BioPizza(Pizza):
gluten_free = True
@classmethod
def is_gluten_free(cls):
return cls.gluten_free
BioPizza.is_gluten_free()
Out[147]:
In [148]:
bp = BioPizza(['eggplants'])
bp.is_gluten_free()
Out[148]:
https://julien.danjou.info/blog/2013/guide-python-static-class-abstract-methods