Descriptors and Properties


Python for Everyone!
Oregon Curriculum Network

Descriptors and Properties in Python

Descriptors

Lets take a look at the descriptor protocol. When and how binding happens, and later lookup, is intimately controlled by __set__ and __get__ methods respectively. When defined for a class of object, getting and setting become mediated operations, without changes to the outward API (user interface).

For example, here is some code directly from the Python docs:


In [1]:
class RevealAccess(object):
    """A data descriptor that sets and returns values
       normally and prints a message logging their access.

       Descriptor Example:       
       https://docs.python.org/3/howto/descriptor.html
    """

    def __init__(self, initval=None, name='var'):
        self.val = initval
        self.name = name

    def __get__(self, obj, objtype):
        print('Retrieving', self.name)
        return self.val

    def __set__(self, obj, val):
        print('Updating', self.name)
        self.val = val
        
class MyClass(object):
    x = RevealAccess(10, 'var "x"')
    y = 5

# Let's test...
m1 = MyClass()
m2 = MyClass()
print("m1.x: ", m1.x)
m1.x = 20
m2.x = 10
print("m1.x: ", m1.x)
print("m2.x: ", m2.x)
print("m1.y: ", m1.y)
print(m1.x is m2.x)


Retrieving var "x"
m1.x:  10
Updating var "x"
Updating var "x"
Retrieving var "x"
m1.x:  10
Retrieving var "x"
m2.x:  10
m1.y:  5
Retrieving var "x"
Retrieving var "x"
True

y's value is an ordinary int, equivalently the value of MyClass.__dict__['y'], whereas the x attribute, a descriptor, will police getting and setting through __get__ and __set__ methods, using the name 'x' as a proxy to x.val behind the scenes (think of x.val as "more secret" as in less directly accessible).

Notice our distinct instances of MyClass nevertheless share both x and y as class level names. Changing the value for one changes it for all. Building on this behavior, a Pythonic way to define setters and getters that store data at the instance level becomes possible.

Properties

The code below is likewise from the Python 3.5 version of the docs at Python.org, and shows how the built-in property() type may be modeled as a pure Python class.


In [2]:
class Property(object):
    "Emulate PyProperty_Type() in Objects/descrobject.c"

    def __init__(self, fget=None, fset=None, fdel=None, doc=None):
        self.fget = fget
        self.fset = fset
        self.fdel = fdel
        if doc is None and fget is not None:
            doc = fget.__doc__
        self.__doc__ = doc

    def __get__(self, obj, objtype=None):
        if obj is None:
            return self
        if self.fget is None:
            raise AttributeError("unreadable attribute")
        return self.fget(obj)

    def __set__(self, obj, value):
        if self.fset is None:
            raise AttributeError("can't set attribute")
        self.fset(obj, value)

    def __delete__(self, obj):
        if self.fdel is None:
            raise AttributeError("can't delete attribute")
        self.fdel(obj)

    def getter(self, fget):
        return type(self)(fget, self.fset, self.fdel, self.__doc__)

    def setter(self, fset):
        return type(self)(self.fget, fset, self.fdel, self.__doc__)

    def deleter(self, fdel):
        return Property(self.fget, self.fset, fdel, self.__doc__)

The getter, setter and deleter methods allow swapping out new versions of fset, fget and fdel by keeping whatever is unchanged and making a new instance with a call to type(self) -- remember that types are callables.

The C class now uses the Property class to fully develop a pet class level attribute named 'x', which in turn fully implements the descriptor protocol, as an instance of the Property descriptor.


In [3]:
class C:
    def getx(self): 
        print("getting...")        
        return self.__x
    def setx(self, value): 
        print("setting...")         
        self.__x = value
    def delx(self): 
        print("deleting...")
        del self.__x
    x = Property(getx, setx, delx, "I'm the 'x' property.")

Think of x as an object prepared to delegate to the three methods. Every time a new C instance is created, that instance is bound to a deeply internal "secret" __x. The public or class level x is a proxy to a set of instance methods living inside C.__dict__. Each instance of C talks to its respective self.__x i.e. don't confuse the shared nature of x with the private individuality of each self.__x

The code below goes a step further in using two instances of C to demonstrate how properties work. In this case, the same Property class is used as a decorator. Notice how .setter() becomes available once the getter is defined, because the decorator has "abucted" the original method and turned it into an instance of something, of which .setter is a now an attribute.


In [4]:
class Generic:
    
    def __init__(self, a=None, b=None):
        self.__dict__['y'] = C()
        self.y = a
        self.__dict__['z'] = C()
        self.z = b

    @Property
    def y(self):
        return self.__dict__['y'].x

    @y.setter
    def y(self, val):
        print("Generic setter for y")
        self.__dict__['y'].x = val

    @Property
    def z(self):
        return self.__dict__['z'].x
        
    @z.setter
    def z(self, val):
        print("Generic setter for z")
        self.__dict__['z'].x = val

The reason for all the __getitem__ syntax i.e. talking to self.__dict__ in "longhand", is to avoid a recursive situation where a setter or getter calls itself. __getitem__ syntax lets us set and get in a way that bypasses __getattribute__ and its internal mechanisms, which are responsible for triggering the descriptor protocol in the first place.

Once we have our instance of C in the guts of a setter or getter, we talk directly to its proxy, an instance of Property, which responds accordingly to setting and getting.


In [5]:
me = Generic(3, "Hello")
print("me.y:", me.y)
print("me.z:", me.z)
little_me = Generic(4, "World")
print("little_me.y:", little_me.y)
print("little_me.z:", little_me.z)
me.y = 5
me.z = "Ciao"
little_me.y = 6
little_me.z = "Mondo"
print("me.y:", me.y)
print("me.z:", me.z)
print("little_me.y:", little_me.y)
print("little_me.z:", little_me.z)


Generic setter for y
setting...
Generic setter for z
setting...
getting...
me.y: 3
getting...
me.z: Hello
Generic setter for y
setting...
Generic setter for z
setting...
getting...
little_me.y: 4
getting...
little_me.z: World
Generic setter for y
setting...
Generic setter for z
setting...
Generic setter for y
setting...
Generic setter for z
setting...
getting...
me.y: 5
getting...
me.z: Ciao
getting...
little_me.y: 6
getting...
little_me.z: Mondo

Fun though that was, there's more indirection going on than necessary.

The methods of Generic are themselves suitable as setters and getters, without needing to delegate to some instance of C with its fancy 'x' property.

What you see below is the more usual Python program, except instead of using the pure Python class above, the equivalent built-in property type (lowercase) is used as a decorator instead.

Reading the pure Python version shows how it works.


In [6]:
class Generic:
    
    def __init__(self, a=None, b=None):
        self.y = a
        self.z = b
    
    @property
    def y(self):
        return self.__y
    
    @y.setter
    def y(self, val):
        print("Generic setter for y")
        self.__y = val

    @property
    def z(self):
        return self.__z
        
    @z.setter
    def z(self, val):
        print("Generic setter for z")
        self.__z = val

This time, we've cut out the middle man, C.

The Property class is where the descriptor protocol gets implemented.

We turn Generic.y and Generic.z into properties by decorating methods of the same names.

Through decorating, two class level Property objects, similar to C.x, get created, with each one providing a set of instance methods happy to work with a specific self.

These four instance methods, defined within Generic itself, consult self.__y and self.__z much as x worked with self.__x behind the scenes.


In [7]:
me = Generic(3, "Hello")
print("me.y:", me.y)
print("me.z:", me.z)
little_me = Generic(4, "World")
print("little_me.y:", little_me.y)
print("little_me.z:", little_me.z)
me.y = 5
me.z = "Ciao"
little_me.y = 6
little_me.z = "Mondo"
print("me.y:", me.y)
print("me.z:", me.z)
print("little_me.y:", little_me.y)
print("little_me.z:", little_me.z)


Generic setter for y
Generic setter for z
me.y: 3
me.z: Hello
Generic setter for y
Generic setter for z
little_me.y: 4
little_me.z: World
Generic setter for y
Generic setter for z
Generic setter for y
Generic setter for z
me.y: 5
me.z: Ciao
little_me.y: 6
little_me.z: Mondo

By the way, notice that method( ) has a single argument 'this', showing that 'self' is by convention and, furthermore, the value of 'this' will depend on whether method( ) is called: on an instance or on a class.

Calling me.method() sets 'this' to the instance object i.e. what 'self' is used for.

However Generic.method(Generic) is likewise legal Python, and in this case you must pass the class explicitly if that's what's needed.

The @classmethod decorator, applied to a method, will pass in the class automatically.


In [8]:
class Generic2(Generic):
    
    def method(this):
        return ("this is: " + 
                ("Instance" if isinstance(this, Generic2) 
                else "Class"))

class Generic3(Generic):
    
    @classmethod
    def method(this):
        return ("this is: " + 
                ("Instance" if isinstance(this, Generic2) 
                else "Class"))
    
me = Generic2(1,2)
print("On an instance: ", me.method())
print("On the class:   ", Generic2.method(Generic2))

me = Generic3(1,2)
print("With @classmethod decorator: ", me.method())


Generic setter for y
Generic setter for z
On an instance:  this is: Instance
On the class:    this is: Class
Generic setter for y
Generic setter for z
With @classmethod decorator:  this is: Class

So that's a lot of fancy theory, but what might be a practical application of the above. Suppose we want a circle to let us modify its radius at will, and to treat area as an ordinary attribute nonetheless...


In [9]:
import math

class Circle:
    
    def __init__(self, r):
        self.radius = r
    
    @property
    def area(self):
        return self.radius ** 2 * math.pi
    
    def __repr__(self):
        return "Circle({})".format(self.radius)
    
the_circle = Circle(1)
print(the_circle) # triggers __repr__ in the absence of __str__
print("Area of the circle: {:f}".format(the_circle.area))
the_circle.radius = 2
print("Area of the circle: {:f}".format(the_circle.area))


Circle(1)
Area of the circle: 3.141593
Area of the circle: 12.566371

In decorating only the area method, we provide the area property with a getter, i.e. fget has been set to this method. No setter proxy (self.fset) has been defined, hence an assignment to the area property, which triggers its __set__ method, raises an AttributeError (see Property.__set__).


In [10]:
try:
    the_circle.area = 90
except AttributeError:
    print("Can't set the area directly")


Can't set the area directly

Might we make both radius and area into properties, such that setting either recalculates the other?

Let's try:


In [11]:
import unittest

class Circle:
    """setting either the radius or area attribute sets the other
       as a dependent value.  Initialized with radius only, unit 
       circle by default.
    """
    
    def __init__(self, radius = 1):
        self.radius = radius
    
    @property
    def area(self):
        return self._area

    @property    
    def radius(self):
        return self._radius
    
    @area.setter
    def area(self, value):
        self._area = value
        self._radius = math.sqrt(self._area / math.pi)
        
    @radius.setter
    def radius(self, value):
        self._radius = value
        self._area = math.pi * (self._radius ** 2)
    
    def __repr__(self):
        return "Circle(radius = {})".format(self.radius)
    
class TestCircle(unittest.TestCase):

    def testArea(self):
        the_circle = Circle(1)
        self.assertEqual(the_circle.area, math.pi, "Uh oh")
        
    def testRadius(self):
        the_circle = Circle(1)
        the_circle.area = math.pi * 4 # power rule
        self.assertEqual(the_circle.radius, 2, "Uh oh")

a = TestCircle()  # the test suite
suite = unittest.TestLoader().loadTestsFromModule(a) 
unittest.TextTestRunner().run(suite)  # run the test suite


..
----------------------------------------------------------------------
Ran 2 tests in 0.002s

OK
Out[11]:
<unittest.runner.TextTestResult run=2 errors=0 failures=0>