Python 101

This introduction is based heavily on the introduction notes found here: https://github.com/scipy-lectures/scipy-lecture-notes

The (climate) scientists needs

  • To get, potentially very large, datasets from simulation, observation, or experiment
  • To manipulate and process it
  • To visualize results, quickly to understand, but also with high quality figures, for reports or publications.

Why Python

I've used pretty much every programming language you can think of... MatLab, IDL, Fortran, C++, C#, Java,... But Python, for me anyway, is the best of the lot. It's flexible, fast, easy to read and portable across all sorts of machines and architectures.

  • Batteries included Rich collection of already existing bricks of classic numerical methods, plotting or data processing tools.
  • Easy to learn Most scientists are not payed as programmers, neither have they been trained so.
  • Easy communication To keep code alive within a lab or a company it should be as readable as a book by collaborators, students, or maybe customers.
  • Efficient code Python numerical modules are computationally efficient.
  • Universal Python is a language used for many different problems.
  • Free As in freedom. Python and it's libraries are all open-source and freely available

Let's compare a bit more specifically against some other languages we might use:

Compiled languages: C, C++, Fortran...

Pros
  • Very fast. For heavy computations, it's difficult to outperform these languages.
Cons
  • Painful usage: no interactivity during development, mandatory compilation steps, verbose syntax, manual memory management. These are **difficult languages** for non programmers.

Matlab

Pros
  • Very rich collection of fast libraries with numerous algorithms, for many different domains.
  • Pleasant development environment: comprehensive and help, integrated editor, etc.
  • Commercial support is available.
Cons
  • Base language is quite poor and can become restrictive for advanced users.
  • Not free.

Julia

Pros
  • Fast code, yet interactive and simple.
  • Easily connects to Python or C.
Cons
  • Ecosystem limited to numerical computing.
  • Still young.

Other scripting languages: Scilab, Octave, R, IDL, etc.

Pros
  • Open-source, free, or at least cheaper than Matlab.
  • Some features can be very advanced (statistics in R, etc.)
Cons
  • Fewer available algorithms than in Matlab, and the language is not more advanced.
  • Some software are dedicated to one domain.

Python

Pros
  • Very rich scientific computing libraries (amonst others - web server, serial port access, etc.).
  • Well thought out language, allowing to write very readable and well structured code: we “code what we think”.
  • Free and open-source software, widely spread, with a vibrant community.
  • A variety of powerful environments to work in, such as IPython, Spyder, Jupyter notebooks, Pycharm
Cons
  • Not all the algorithms that can be found in more specialized software or toolboxes.

The scientific Python ecosystem

Unlike Matlab, or R, Python does not come with a pre-bundled set of modules for scientific computing. Below are the basic building blocks that can be combined to obtain a scientific computing environment:

Core Python

  • The language: flow control, data types (string, int), data collections (lists, dictionaries), etc.
  • Modules of the standard library: string processing, file management, simple network protocols.

Core numeric libraries

  • Numpy - Numerical array types and operations
  • Matplotlib - General plotting routines modelled on MatLab
  • SciPy - Fast linear algebra and optimisation routines

Domain specific libraries

  • Iris - For working with model data
  • CIS - For working with observational data and collocation
  • Pandas - Extensive (time)-series analysis routines
  • sci-kit-learn - Machine learning algorithms
  • ...

Managing libraries: Conda

There are many ways of downloading, installing and managing the various Python installations and libraries, but the easiest and most comprehensive is currently using a tool called Conda.

This is an open source tool included with the Anaconda distribution.

Working in Python

Make sure you now type source activate python_workshop on UNIX based systems and activate python_workshop - this may not be necassary if you have nb_conda_kernels installed.

Python is an interpreted language and there is more than one interpreter we can use:

  • python - The basic built-in interpreter
  • iPython - A more feature rich interpreter with tab-completion, shell access and inline plots
  • Jupyter (iPython) notebooks - A web based iPython interpreter with markdown (and $\LaTeX$!) support

For this course we will be using the iPython notebooks - these are great for working interactively and storing the results, as well the code and any notes we might want to make. For script development you can use a basic text editor or an IDE (or anything in between), we'll be exploring PyCharm later in the course

Basic syntax


In [ ]:
# The obligatory hello world...
print("Hello world!")

In [ ]:
an_integer = 9
a_float = 2.5
a_string = "blah"
a_complex_number = 2.0 + 2.3j

In [ ]:
# Note that variables are case-sensitive
a = 1
A = 2
print(a, A)

In [ ]:
# And can be dynamically reassigned
an_integer = 7
print(an_integer)

Operations


In [ ]:
print(an_integer * 3)
print(a_float + 3)
print(an_integer / 4)
print(a_complex_number ** 2)
print(a_string * 2)

Functions


In [ ]:
def say_hello(name): 
    print("Hello {}!".format(name))

In [ ]:
say_hello("world")

In [ ]:
def say_hello(name, indent=0):
    padding = " "*indent
    print(padding + "Hello {}!".format(name))

In [ ]:
say_hello("Duncan", indent=4)

In [ ]:
say_hello(3, 4.2)

Comments


In [ ]:
# This is a comment
pass  # This is also a comment!

In [ ]:
def nicely_documented_function(oranges):
    """
    This function takes a list of oranges and returns orange juice
    
    :param list oranges: The oranges to be juiced
    """
    pass

In [ ]:
help(nicely_documented_function)

Containers

Lists


In [ ]:
assorted_stuff = ["dog", 3, 2.34, "cat"]

In [ ]:
assorted_stuff[0]

In [ ]:
assorted_stuff[1:]

In [ ]:
assorted_stuff[::-1]

Loops


In [ ]:
for item in assorted_stuff:
    print(item)
    
print("Done.")

Sometimes it's useful to also have a loop counter, the enumerate function can do this


In [ ]:
for n, item in enumerate(assorted_stuff):
    print(item, n)
    
print("Done.")

List comprehensions

We can also easily create lists programatically using loops. These are called 'list comprehensions'


In [ ]:
some_integers = [i for i in range(10)]
some_integers

Dictionaries


In [ ]:
# A collection of key-value pairs
simple_lookup = {"red": 5, "green": 12, "blue": [0.5, 45.8]}

In [ ]:
print(simple_lookup["red"])

In [ ]:
print(simple_lookup["blue"][0])

In [ ]:
print(simple_lookup["yellow"])

In [ ]:
print(simple_lookup.get("yellow", 0))

In [ ]:
simple_lookup["orange"] = 15
print(simple_lookup)

In [ ]:
simple_lookup[4] = "brown"
print(simple_lookup[4])

There are other containers too, but we won't cover them here. Most useful are probably sets and named tuples.

Logic


In [ ]:
# Simple conditional statements
variable = 0.95
if variable > 0.9:
    print("On")

In [ ]:
if variable > 0.9:
    print("On")
elif variable < 0.1:
    print("Off")
elif 0.4 < variable < 0.6:
    print("Intermediate")
else:
    print("Unknown")

In [ ]:
# Python will 'short-cut' logic statements
if "orange" in simple_lookup and simple_lookup["orange"] > 10:
    print(simple_lookup["orange"])

In [ ]:
switch = "On" if abs(variable-1.0) < 0.1 else "Off"
print(switch)