Python general coding guidelines

"The guidelines provided here are intended to improve the readability of code and make it consistent across the wide spectrum of Python code" PEP8

Python PEPs are "Python Enhancement Proposal". The PEP for the conding standard adopted by the Python Software foundation (PSF) is the PEP8.

There are also other coding standards specific to organizations and/or projects, e.g:

Euclid's Python coding standard can be found here
Google's python coding style here.

Python2 code must be compatible with Python3



In [ ]:

    
# this must be included at the top of a python2 src file
# to ensure most python3 features that are backported
# to python2 are available
from __future__ import absolute_import, division, print_function  
from builtins import (bytes, str, open, super, range,  
                      zip, round, input, int, pow, object, map, zip)

In case you missed it, there was a dedicated session on future proofing your code in the second developers workshop last year. http://euclid.roe.ac.uk/attachments/download/6019

Scoping: namespaces in python



In [ ]:

    
# manynames.py   
X = 11                       # Global (module) name/attribute (X, or manynames.X)   

def f():   
    print(X)                 # Access global X (11)   

def g():   
    X = 22                   # Local (function) variable (X, hides module X)   
    print(X)   

class C:   
    X = 33                   # Class attribute (C.X)   
    def m(self):   
        X = 44               # Local variable in method (X)   
        self.X = 55          # Instance attribute (instance.X)



In [ ]:

    
f()



In [ ]:

    
g()



In [ ]:

    
print('C.X = {}'.format(C.X))



In [ ]:

    
my_c = C()
print('my_c.X = {}'.format(my_c.X))
my_c.m()
print('my_c.X = {}'.format(my_c.X))

Avoid as possible global variables for several modules



In [ ]:

    
def scope_test():   
    def do_local():   
        spam = "local spam"  
    def do_nonlocal():   
        nonlocal spam       
        spam = "nonlocal spam"  
    def do_global():   
        global spam              
        spam = "global spam"  

def scope_test():   
    def do_local():   
        spam = "local spam"  
    def do_nonlocal():   
        nonlocal spam       
        spam = "nonlocal spam"  
    def do_global():
        pass

TIP

To examine what is available in the current scope, use the build in functions globals(), locals()

nonlocal is only available for python3 and above. A good example of the usage of nonlocal http://stackoverflow.com/questions/1261875/python-nonlocal-statement?answertab=active#tab-top



In [ ]:

    
x = 0
def outer():
    x = 1
    def inner():
        x = 2
        print("inner:", x)

    inner()
    print("outer:", x)

outer()
print("global:", x)



In [ ]:

    
x = 0
def outer():
    x = 1
    def inner():
        nonlocal x        # binds x to the outer scope (not to the global scope)
        x = 2
        print("inner:", x)

    inner()
    print("outer:", x)

outer()
print("global:", x)

Naming conventions

Python packages and modules should also have short, all-lowercase names

OK



In [ ]:

    
%%file my_module.py
#
#  my module content...
#

NOT OK



In [ ]:

    
%%file My_module.py
#
#  my module content...
#



In [ ]:

    
%%file MyModule.py
#
#  my module content...
#



In [ ]:

    
%%file My_Module.py
#
#  my module content...
#

Almost without exception, class names use mixed case starting with uppercase

OK



In [ ]:

    
def Foo():
    pass

def MyFoo():
    pass

NOT OK



In [ ]:

    
def foo():
    pass

def my_foo():
    pass

Classes for internal use MUST have a leading underscore

OK



In [ ]:

    
%%file my_module.py
class _MyInternalClassThatShouldNotBeAccessedOutsideThisModule():
    pass

def foo():
    my_instance = _MyInternalClassThatShouldNotBeAccessedOutsideThisModule()

NOT OK



In [ ]:

    
%%file my_module.py
class _MyInternalClassThatShouldNotBeAccessedOutsideThisModule():
    pass



In [ ]:

    
%%file my_script.py
import my_module
my_instance = _MyInternalClassThatShouldNotBeAccessedOutsideThisModule()

An exception name MUST include the suffix "Error"

OK



In [ ]:

    
class MyError(Exception):
    def __init__(self, value):
        self.value = value
    def __str__(self):
        return repr(self.value)

NOT OK



In [ ]:

    
class MyWarning(Exception):
    def __init__(self, value):
        self.value = value
    def __str__(self):
        return repr(self.value)
    
class MyException(Exception):
    def __init__(self, value):
        self.value = value
    def __str__(self):
        return repr(self.value)

Developer SHOULD use properties to protect the service from the implementation

OK



In [ ]:

    
class MyClass(object):
    def __init__(self):
        self._excutable = None
    @property
    def executable(self):
        return self._executable
    @executable.setter
    def executable(self, value):
        # check that the executable actually can be found in the OS/system
        # the assign it to the backing variable
        self._executable = value

class MyOtherClass(object):
    def __init__(self):
        self._speed_of_light_si = 3e8 
    @property
    def speed_of_light_si(self):
        return self._speed_of_light_si

NOT OK



In [ ]:

    
class MyClass(object):
    def __init__(self, path_to_exec):
        self.excutable = path_to_exec
        
class MyOtherClass(object):
    def __init__(self):
        self.speed_of_light_si = 3e8



In [ ]:

    
x = MyOtherClass()
x.speed_of_light_si



In [ ]:

    
x.speed_of_light_si = 1

Protected Class Attribute Names MUST be prefixed with a single underscore

OK



In [ ]:

    
class MyClass(object):
    def __init__(self):
        self._safe_combination_pin_code = 541976
        """A private attribute that is not intended to be used outside the class"""

NOT OK



In [ ]:

    
class MyClass(object):
    def __init__(self):
        self._safe_combination_pin_code = 541976
        """A private attribute that is not intended to be used outside the class"""

If your public attribute name collides with a reserved keyword, append a single trailing “_” underscore to your attribute name

OK



In [ ]:

    
class foo(object):
    def __init__(self):
        self._print = None

NOT OK



In [ ]:

    
class foo(object):
    def __init__(self):
        self.print = None

Private Class Attribute Names MUST be prefixed with a double underscore

OK



In [ ]:

    
class Mapping:
    def __init__(self, iterable):
        self.items_list = []
        self.__update(iterable)

    def update(self, iterable):
        for item in iterable:
            self.items_list.append(item)
    
    __update = update   # private copy of original update() method

class MappingSubclass(Mapping):
    def update(self, keys, values):
        # provides new signature for update()
        # but does not break __init__()
        for item in zip(keys, values):
            self.items_list.append(item)

NOT OK



In [ ]:

Function names MUST be lowercase with words separated by underscore

OK



In [ ]:

    
def see_above_examples(x, y):
    pass

NOT OK



In [ ]:

    
def MyBadFunctionName(x, y):
    pass
def my_Bad_function(x, y):
    pass

Always use self for the first argument to instance methods

OK



In [ ]:

    
class MyClass(object):
    def __init__(self):
        pass
    def make_mesh(self):
        pass

NOT OK



In [ ]:

    
class MyClass(object):
    def __init__(this):
        pass
    def make_mesh(this):
        pass

class MyOtherClass(object):
    def __init__(that):
        pass
    def make_mesh(that):
        pass

class MyOtherClass2(object):
    def __init__(asdasdasdasd):
        pass
    def make_mesh(asdasdasdasd):
        pass



In [ ]:

    
MyOtherClass2()

Always use cls for the first argument to class methods

OK



In [ ]:

    
class MyClass(object):
    def __init__(self):
        pass
    def my_method1(self):
        pass
    @classmethod
    def my_class_method_foo(cls):
        pass

NOT OK



In [ ]:

    
class MyClass(object):
    def __init__(self):
        pass
    def my_method1(self):
        pass
    @classmethod
    def my_class_method_foo(self):
        pass

Avoid usage of mutable types (lists, dictionaries) as default values of the arguments of a method

OK



In [ ]:

    
def good_append(new_item, a_list=None):
    if a_list is None:
        a_list = []
    a_list.append(new_item)
    return a_list
my_list = good_append(1)
print(good_append(2, my_list))
print(good_append(3, my_list))

NOT OK (unless it is intentional)



In [ ]:

    
def bad_append(new_item, a_list=[]):
    a_list.append(new_item)
    return a_list
my_list = bad_append(1)
print(bad_append(2))
print(bad_append(3))

Constants are usually defined on a module level and written in all capital letters with underscores separating words

OK



In [ ]:

    
class Foo(object):   
    my_const = "Name"

NOT OK



In [ ]:

    
class Foo(object):   
    my_const = "Name"

Global variables names should be lowercase with words separated by underscores

OK



In [ ]:

    
my_global_variable = 1

NOT OK



In [ ]:

    
MY_GLOBAL_VARIABLE = 1

Files

The parts of a module MUST be sorted

OK



In [ ]:

    
#!/usr/bin/env python     # Shebang line (#!), only for executable scripts
# my module comments
# more comments...
"""
my module docstring
...
...
"""

import os   
import sys
# and other imports...

__all__ = ['MyClass1', 'MyClass2'] # whatever you wish to import with from my_module import *, if any

#
#  Public variables
#

#
#  Public classes, functions...
#

NOT OK



In [ ]:

    
#!/usr/bin/env python     # Shebang line (#!), only for executable scripts
"""
my module docstring
...
...
"""

# my module comments
# more comments...

import os   
import sys
# and other imports...

__all__ = ['MyClass1', 'MyClass2'] # whatever you wish to import with from my_module import *, if any

import numpy

#
#  Public variables
#

#
#  Public classes, functions...
#

Imports SHOULD be grouped, in order: standard lib, 3rd party lib, local lib

OK



In [ ]:

    
import os
import sys
import numpy
import matplotlib

import my_module
import my_module2

NOT OK



In [ ]:

    
import numpy
import matplotlib
import os
import sys
import my_module
import my_module2

Modules designed for use via "from M import *" SHOULD use the all mechanism

OK



In [ ]:

    
%%file my_module.py
__all__ = ['Foo', 'my_func']
manager1 = 1
manager2 = 1
class MyLocalManager():
    pass
class Foo():
    pass
def my_func():
    pass



In [ ]:

    
from my_module import *
print(list(filter(lambda x: 'manager' in x.lower(), globals())))

NOT OK



In [ ]:

    
%%file my_other_module.py
manager1 = 1
manager2 = 1
class MyLocalManager():
    pass
class Foo():
    pass
def my_func():
    pass



In [ ]:

    
from my_other_module import *
print(list(filter(lambda x: 'manager' in x.lower(), globals())))

Statements

If a class inherits from no other base classes, explicitly inherit from object

OK



In [ ]:

    
class Base(object):   
    pass   
 
class Outer(object):    
    class Inner(object):   
        pass   

class Child(Base):   
    """Explicitly inherits from another class already."""
    pass

NOT RECCOMENDED (UNLESS YOU KNOW WHAT YOU ARE DOING)



In [ ]:

    
class Base:   
    pass   
 
class Outer:    
    class Inner:   
        pass

Do not use non-existent pre-increment or pre-decrement operator

OK



In [ ]:

    
x = 1
x += 1
print(x)

NOT OK



In [ ]:

    
x = 1
print(++x)   # +(+x)

Use the "implicit" false if at all possible

OK



In [ ]:

    
#  use meaningful names for boolean variables
data_found = False
if not data_found:
    print('no data found')

NOT OK



In [ ]:

    
if data_found == False:
    print('no data found')

Explicitly close files and sockets when done with them

OK



In [ ]:

    
with open("hello.txt", 'w') as hello_file:   
    for word in ['aaa', 'bbb']:
        hello_file.write(word)
# file is closed automoatically in a context manager

NOT OK



In [ ]:

    
hello_file = open("hello.txt", 'w')
for word in ['aaa', 'bbb']:
    hello_file.write(word)
# easy to forget closing the file

Modules or packages should define their own domain-specific base exception class

OK



In [ ]:

    
%%file my_specific_module.py
class MySpecificError(Exception):   
    """Base class for errors in my package."""    
    def __init__(self, value):   
        self.value = value   
    def __str__(self):   
        return repr(self.value)

try:   
    raise MySpecificError(2*2)
except MySpecificError as e:   
    print('My exception occurred, value:', e.value)



In [ ]:

    
%run my_specific_module

NOT OK



In [ ]:

    
%%file my_specific_module.py
try:   
    raise ValueError("""can not accept bla bla value""")
except ValueError as e:   
    print('Exception occurred')



In [ ]:

    
%run my_specific_module.py

When raising an exception, raise an exception instance and not an exception class

OK



In [ ]:

    
raise ValueError("""this is an instance of the ValueError exception class""")

NOT OK



In [ ]:

    
raise ValueError

When catching exceptions, mention specific exceptions whenever possible

OK



In [ ]:

    
try:   
    import platform_specific_module   
except ImportError:   
    platform_specific_module = None
    print('import error occured')

NOT OK



In [ ]:

    
try:   
    import platform_specific_module   
except:   
    platform_specific_module = None
    print('i caught an exception, but i do not know what it is')

Layout and Comments

Block layout rules

OK



In [ ]:

    
my_list = [   
    1, 2, 3,   
    4, 5, 6,
    ]   

result = some_function_that_takes_arguments(   
    'a', 'b', 'c',   
    'd', 'e', 'f',   
    )   

#or it may be lined up under the first character of the line that starts the multi-line construct, as in   
my_list = [   
    1, 2, 3,   
    4, 5, 6,   
] 

result = some_function_that_takes_arguments(   
    'a', 'b', 'c',   
    'd', 'e', 'f',   
)

NOT OK



In [ ]:

    
# don't line things up under the = sign
my_list = [   
        1, 2, 3,   
        4, 5, 6,
        ]

Compound statements (multiple statements on the same line) are discouraged

OK



In [ ]:

    
do_one()   
do_two()   
do_three()

NOT OK



In [ ]:

    
do_one(); do_two(); do_three()

Function layout rules

OK



In [ ]:

    
# Aligned with opening delimiter   
foo = long_function_name(var_one, var_two,   
                         var_three, var_four)   

# More indentation included to distinguish this from the rest.   
def long_function_name(   
        var_one, var_two, var_three,   
        var_four):   
    print(var_one)

NOT OK



In [ ]:

    
foo = long_function_name(var_one, var_two,
    var_three, var_four)

# Further indentation required as indentation is not distinguishable.
def long_function_name(
    var_one, var_two, var_three,
    var_four):
    print(var_one)

Import layout rules

OK



In [ ]:

    
import os
import sys

NOT OK



In [ ]:

    
import sys, os

brackets and braces SHOULD be used for wrapped lines

OK



In [ ]:

    
from Tkinter import (Tk, Frame, Button, Entry, Canvas, Text,
                LEFT, DISABLED, NORMAL, RIDGE, END)

NOT OK



In [ ]:

    
from Tkinter import Tk, Frame, Button, Entry, Canvas, Text,\
             LEFT, DISABLED, NORMAL, RIDGE, END

Avoid extraneous whitespace in the following situations

OK



In [ ]:

    
my_func(ham[1], {eggs: 2}) 

if x == 4: print(x, y); x, y = y, x

print(x)

dict['key'] = list[index]

x = 1  
y = 2  
long_variable = 3

NOT OK



In [ ]:

    
my_func(ham[ 1 ], { eggs: 2 }) 

if x == 4 : print (x , y) ; x , y = y , x

print (x)

dict ['key'] = list [index] 

x             = 1  
y             = 2  
long_variable = 3

Binary operators SHOULD be surrounded by a single space

OK



In [ ]:

    
i = i + 1
submitted += 1
x = x*2 - 1
hypot2 = x*x + y*y
c = (a+b) * (a-b)

NOT OK



In [ ]:

    
i=i+1
submitted +=1
x = x * 2 - 1
hypot2 = x * x + y * y
c = (a + b) * (a - b)

Blank lines rules

OK



In [ ]:

    
%%file my_module.py
def func1():
    pass


def func2():
    pass


class foo1():
    def method1():
        pass
    
    def method2():
        pass

NOT OK



In [ ]:

    
def func1():
    pass

def func2():
    pass


class foo1():
    def method1():
        pass    
    def method2():
        pass

Block comments rules

OK



In [ ]:

    
%%file my_module.py
# this is intended to be documentation that should not be extracted
# by doxygen or sphinx.

NOT OK



In [ ]:

    
%%file my_module.py
"""
sphinx treats this as a docstring, thus it is not a block comment.
"""

Inline comments rules

OK



In [ ]:

    
if i & (i-1) == 0:  # true if i is a power of 2  
if i & (i-1) == 0:        # true if i is a power of 2

NOT OK



In [ ]:

    
if i & (i-1) == 0:# true if i is a power of 2

Documentation strings ("docstrings") MUST be used for packages, modules, functions, classes, and methods

see sample project directory

Pylint & pep8 & sonarQube

pep8: https://www.python.org/dev/peps/pep-0008/
pylint: https://www.pylint.org/
sonarQube: http://euclid.roe.ac.uk/projects/codeen-users/wiki/User_Cod_Std-Tools

Exercise

We will run pep8 and pylint on actual code from Euclid's IAL project.

  EC/SGS/ST/4-2-03-IAL/drm/trunk/euclid_ial/drm/system/utils.py

We copied this to the exercises directory of this tutorial

   python-euclid2016/exercises/coding-standard



In [ ]:

    
pep8 utils.py



In [ ]:

    
pylint utils.py -f html > utils.html

Currently it scores -1.48/10

Try to increase the score to 5/10

and utimately try to reach as close as possible to 10/10