The __getattribute__ method and descriptors

Question

according to this guide on python descriptors https://docs.python.org/howto/descriptor.html

method objects in new style classes are implemented using descriptors in order to avoid special casing them in attribute lookup.

the way I understand this is that there is a method object type that implements __get__ and returns a bound method object when called with an instance and an unbound method object when called with no instance and only a class. the article also states that this logic is implemented in the object.__getattribute__ method. like so:

def __getattribute__(self, key):
 "Emulate type_getattro() in Objects/typeobject.c"
 v = object.__getattribute__(self, key)
 if hasattr(v, '__get__'):
 return v.__get__(None, self)
 return v

however object.__getattribute__ is itself a method! so how is it bound to an object (without infinite recursion)? if it is special cased in the attribute lookup does that not defeat the purpose of removing the old style special casing?

• bruno desthuilliers: "he way I understand this is that there is a method object type that implements get" => actually, it's the function type that is the descriptor here (cf wiki.python.org/moin/FromFunctionToMethod).

Answer 1

Actually, in CPython the default __getattribute__ implementation is not a Python method, but is instead implemented in C. It can access object slots (entries in the C structure representing Python objects) directly, without bothering to go through the pesky attribute access routine.

Just because your Python code has to do this, doesn't mean the C code has to. :-)

If you do implement a Python __getattribute__ method, just use object.__getattribute__(self, attrname), or better still, super().__getattribute__(attrname) to access attributes on self. That way you won't hit recursion either.

In the CPython implementation, the attribute access is actually handled by the tp_getattro slot in the C type object, with a fallback to the tp_getattr slot.

To be exhaustive and to fully expose what the C code does, when you use attribute access on an instance, here is the full set of functions called:

• Python translates attribute access to a call to the PyObject_GetAttr() C function. The implementation for that function looks up the tp_getattro or tp_getattr slot for your class.

• The object type has filled the tp_getattro slot with the PyObject_GenericGetAttr function, which delegates the call to _PyObject_GenericGetAttrWithDict (with the *dict pointer set to NULL and the suppress argument set to 0). This function is your object.__getattribute__ method (a special table maps between the name and the slots).

• This _PyObject_GenericGetAttrWithDict function can access the instance __dict__ object through the tp_dict slot, but for descriptors (including methods), the _PyType_Lookup function is used.

• _PyType_Lookup handles caching and delegates to find_name_in_mro on cache misses; the latter looks up attributes on the class (and superclasses). The code uses direct pointers to the tp_dict slot on each class in the MRO to reference class attributes.

• If a descriptor is found by _PyType_Lookup it is returned to _PyObject_GenericGetAttrWithDict and it calls the tp_descr_get function on that object (the __get__ hook).

When you access an attribute on the class itself, instead of _PyObject_GenericGetAttrWithDict, the type->tp_getattro slot is instead serviced by the type_getattro() function, which takes metaclasses into account too. This version calls __get__ too, but leaves the instance parameter set to None.

Nowhere does this code have to recursively call __getattribute__ to access the __dict__ attribute, as it can simply reach into the C structures directly.

• user3848844: I don't think I fully understand. do you mean to say that __getattribute__ is not treated as a method of object and not looked up as a regular method? then who converts it from a function to a method?
• martineau: @user3848844: object.__getattribute__() is a call to the unbound method of the built-in class object, so no, it is not looked up via what you call the regular method.
• Martijn Pieters: @user3848844: I've reworked this to be more accurate. __getattribute__ is not implemented as Python code. It is a C function (no methods in C, C is not object oriented). The CPython interpreter, being coded in C, calls other C functions.
• torek: Side note: while the hg.python.org links still work, I wonder if this should be updated to use the github CPython. (But that's a lot of work...)
• Martijn Pieters: @torek: The fate of hg.python.org is still an open issue. For now, it'll continue to exist. I update answers to point to the github views as needed (usually when I come across an answer that uses the old links and recieved comments or a vote).