Class Warfare in Python 2

Operators

For new style classes, all the Python special "dunder" methods (like __repr__ and the arithmetic and comparison operators such as __add__ and __eq__) must be defined on the class itself. If we want to change the repr, we can't do it in the instance, we have to do so on the class:

>>> new_style_instance.__repr__ = lambda: "Hi, I'm a new instance"
>>> print(repr(new_style_instance))
<NewStyle object at 0x...>
>>> NewStyle.__repr__ = lambda self: "Hi, I'm the new class"
>>> print(repr(new_style_instance))
Hi, I'm the new class

(We monkey-patched a class after its definition here, which does work thanks to some magic well discuss soon, but that's not usually the way its done.)

In contrast, old style classes allowed dunders to be assigned to an instance, even overriding one set on the class, in exactly the same way that regular methods work on both new and old style classes.

>>> OldStyle.__repr__ = lambda self: "Hi, I'm the old class"
>>> print(repr(old_style_instance))
Hi, I'm the old class
>>> old_style_instance.__repr__ = lambda: "Hi, I'm the old instance"
>>> print(repr(old_style_instance))
Hi, I'm the old instance

Old Style Instances are Bigger

On a 64-bit platform, old style instances are one pointer size larger:

>>> import sys
>>> sys.getsizeof(new_style_instance)
64
>>> sys.getsizeof(old_style_instance)
72

Old Style Classes Ignore __slots__

The slots declaration can be used not only to optimize memory usage for small frequently used objects but offer tighter control on what attributes an instance can have, which may be useful in some scenarios. Unfortunately, old style classes accept but ignore this declaration.

>>> class NewWithSlots(object):
...      __slots__ = ('attr',)
>>> new_with_slots = NewWithSlots()
>>> sys.getsizeof(new_with_slots)
56
>>> new_with_slots.not_a_slot = 42
Traceback (most recent call last):
...
AttributeError: 'NewWithSlots' object has no attribute 'not_a_slot'
>>> new_with_slots.attr = 42
>>> sys.getsizeof(new_with_slots)
56

>>> class OldWithSlots:
...      __slots__ = ('attr',)
>>> old_with_slots = OldWithSlots()
>>> sys.getsizeof(old_with_slots)
72
>>> old_with_slots.not_a_slot = 42
>>> sys.getsizeof(old_with_slots)
72

Old Style Classes Ignore __getattribute__

The getattribute dunder method allows an instance to intercept all attribute access. It's what makes it possible to implement transparently persistent objects in pure-Python. Indeed, it's the same basic feature that allowed for descriptors to be implemented in Python 2.2. Old style classes only support __getattr__, which is only used when the attribute cannot be found.

>>> class NewWithCustomAttrs(object):
...   def __getattribute__(self, name):
...       print("Looking up %s" % name)
...       if name == 'computed': return 'computed'
...       return object.__getattribute__(self, name)
...   def __getattr__(self, name):
...       print("Returning default for %s" % name)
...       return 1
>>> custom_new = NewWithCustomAttrs()
>>> custom_new.foo
Looking up foo
Returning default for foo
1
>>> custom_new.foo = 42
>>> custom_new.foo
Looking up foo
42
>>> custom_new.computed = 42
>>> custom_new.computed
Looking up computed
'computed'

>>> class OldWithCustomAttrs:
...   def __getattribute__(self, name):
...       print("Looking up %s" % name)
...       if name == 'computed': return 'computed'
...       return object.__getattribute__(self, name)
...   def __getattr__(self, name):
...       print("Returning default for %s" % name)
...       return 1
>>> custom_old = OldWithCustomAttrs()
>>> custom_old.foo
Returning default for foo
1
>>> custom_old.foo = 42
>>> custom_old.foo
42
>>> custom_old.computed = 42
>>> custom_old.computed
42

Contrary to some information that can be found on the Internet, old style classes can use most descriptors, including @property.

Old Style Classes Have a Simplistic MRO

When multiple inheritance is involved, the MRO of an old style class can be quite surprising. I'll let Guido provide the explanation.

Old Style Classes are Slow

This is especially true on PyPy, but even on CPython 2.7 an old style class is three times slower at basic arithmetic.

>>> import timeit
>>> class NewAdd(object):
...    def __add__(self, other):
...        return 1
>>> timeit.new_add = NewAdd() # Cheating, sneak this into the globals
>>> timeit.timeit('new_add + 1')
0.2...
>>> class OldAdd:
...    def __add__(self, other):
...        return 1
>>> timeit.old_add = OldAdd()
>>> timeit.timeit('old_add + 1')
0.6...

CPython Types

In CPython, all types are defined by a C structure called PyTypeObject. It has a series of members (confusingly also called slots) that are C function pointers that implement the behaviour defined by the type. For example, tp_repr is the slot for the __repr__ function, and tp_getattro is the slot for the __getattribute__ function. When the interpreter needs to get the repr of an object, or get an attribute from it, it always goes through those function pointers. An operation like repr(obj) becomes the C equivalent of type(obj).__repr__(obj).

When we create a new style class, the standard type metaclass makes a C function pointer out of each special dunder method defined in the class's __dict__ and uses that to fill in the slots in the C structure. Special methods that aren't present are inherited. Similarly, when we later set such an attribute, the __setattr__ function of the type metaclass checks to see if it needs to update the C slots.

The C structure that defines the object type uses PyObject_GenericGetAttr for its tp_getattro (__getattribute__) slot. That function implements the "standard" way of accessing attributes of objects. It's the same thing we showed above in old_getattribute when we called object.__getattribute__(self, name). The object type also leaves most other slots (like for __add__) empty, meaning that subclasses must define them in the class __dict__ (or assign to them) for them to be found.

Importantly, though, other types implemented in C don't have to do any of that. They could set their tp_getattro and tp_repr slots, along with any of the other slots, to do whatever they want.

You might now understand how PEP 253, which gave us new style classes, also gave us the ability to subclass C types.

instance types

That's exactly what <type 'instance'> does. That type is defined by PyInstance_Type. The metaclass classobj (recall that's the type of old style classes) sets its tp_call slot (aka __call__) to PyInstance_New, a function that returns a new object of <type 'instance'>, aka PyInstance_Type (explaining while all old instances have the same type). (Incidentally, the metaclass also returns instances of itself, not new sub-types as the type metaclass does, which explains why they all have the same type.)

PyInstance_Type, in turn, overrides almost all of the possible slot definitions to implement the old behaviour of first looking in the instance's __dict__ and only if it's not present then looking up the class hierarchy or invoking __getattr__. For example, tp_repr is instance_repr, which uses instance_getattr to dynamically find an appropriate __repr__ function from the instance or class. This, by the way, is the source of the performance difference between old and new style classes.

Implementing Old Style Classes in Python

Can we re-implement old style classes in pure-Python (for example, on Python 3)? I haven't tried, but we can probably get pretty close. It would be similar to implementing transparent object proxies in pure-Python, which is verbose and tedious.

Footnotes

>>> class NewWithMeta:
...    __metaclass__ = type
>>> isinstance(NewWithMeta, type)
True