Descriptor class with advanced (inspect/metaclass) functionality

Question

Having answered this question over on Programmers.SE, I found myself wondering how much effort it would be to write a descriptor that can automatically figure out what the 'destination' attribute would be, e.g.:

class Demo(object):
 foo = Something()

would redirect access to the foo attribute to _foo by default. To do this, I looked into inspect to determine the name to which the descriptor is being assigned (see BaseDescriptor._get_name).

Given that the BaseDescriptor on its own is pretty pointless, I also created a metaclass to enforce that at least one of the descriptor protocol methods is implemented in any sub-classes of it (which required borrowing part of six to keep it 2.x-and-3.x-compliant).

I've included some basic doctests and a demo sub-class to enforce attribute types; what do you think? Using inspect makes the auto-naming somewhat fragile (it will probably only work with CPython, and assumes that the descriptor is assigned on a single line), but what else haven't I thought of?

from inspect import currentframe, getouterframes
def with_metaclass(meta, *bases):
 """Create a base class with a metaclass.
 Source:
 https://pypi.python.org/pypi/six
 License:
 Copyright (c) 2010-2015 Benjamin Peterson
 Released under http://opensource.org/licenses/MIT
 """
 class metaclass(meta):
 def __new__(cls, name, this_bases, d):
 return meta(name, bases, d)
 return type.__new__(metaclass, 'temporary_class', (), {})
class EnforceDescriptor(type):
 """Ensures that at least one descriptor method is implemented.
 Notes:
 Requires at least one of the three descriptor protocol methods
 (__get__, __set__ and __delete__) to be implemented.
 Attributes:
 REQUIRED (tuple): The names of the required methods.
 Raises:
 TypeError: If none of the three REQUIRED methods are implemented.
 Example:
 >>> class TestClass(with_metaclass(EnforceDescriptor)):
 ... pass
 Traceback (most recent call last):
 ...
 TypeError: 'TestClass' must implement at least one descriptor method
 """
 REQUIRED = ('__get__', '__set__', '__delete__')
 def __new__(cls, name, bases, attrs):
 if all(attrs.get(name) is None for name in cls.REQUIRED):
 msg = '{!r} must implement at least one descriptor method'
 raise TypeError(msg.format(name))
 return super(EnforceDescriptor, cls).__new__(cls, name, bases, attrs)
class BaseDescriptor(with_metaclass(EnforceDescriptor)):
 """Descriptor base class, providing basic set, get and del methods.
 Notes:
 Attempts to determine an appropriate name for the destination
 attribute if one is not explicitly supplied, defaulting to the
 name to which the descriptor is assigned with a leading
 underscore. This uses inspect and is somewhat fragile.
 Arguments:
 name (str, optional): The internal 'destination' attribute to
 redirect access to. Defaults to None.
 Raises:
 ValueError: If the name isn't explicitly supplied and can't be
 determined by inspection.
 Example:
 >>> class TestClass(object):
 ... foo = BaseDescriptor()
 >>> inst = TestClass()
 >>> inst.foo = 'bar'
 >>> inst._foo
 'bar'
 >>> inst.foo
 'bar'
 >>> del inst.foo
 >>> inst.foo
 Traceback (most recent call last):
 ...
 AttributeError: 'TestClass' object has no attribute '_foo'
 """
 def __init__(self, name=None):
 self.name = self._get_name() if name is None else name
 def _get_name(self):
 """Attempt to determine an appropriate name by inspection."""
 try:
 code = next(frame for frame in getouterframes(currentframe())
 if frame[3] not in ('_get_name', '__init__'))[4]
 name = '_{}'.format(code[0].split('=')[0].strip())
 except (IndexError, OSError, TypeError):
 raise ValueError('name could not be determined by inspection '
 'so must be supplied')
 return name
 def __get__(self, obj, typ=None):
 return getattr(obj, self.name)
 def __set__(self, obj, val):
 setattr(obj, self.name, val)
 def __delete__(self, obj):
 delattr(obj, self.name)
class TypedAttribute(BaseDescriptor):
 """Descriptor to limit attribute values to a specified type.
 Arguments:
 type_ (type or tuple, optional): Valid type of the attribute, or
 a tuple of valid types. Defaults to object.
 Example:
 >>> class TestClass(object):
 ... num = TypedAttribute(type_=(int, float))
 >>> inst = TestClass()
 >>> inst.num = 50
 >>> inst.num
 50
 >>> inst.num = 'bar'
 Traceback (most recent call last):
 ...
 TypeError: value must be one of (int, float)
 >>> inst.num
 50
 """
 def __init__(self, name=None, type_=object):
 super(TypedAttribute, self).__init__(name)
 self.type_ = type_
 def __set__(self, obj, val):
 if not isinstance(val, self.type_):
 if isinstance(self.type_, tuple):
 types = ', '.join(typ.__name__ for typ in self.type_)
 msg = 'value must be one of ({})'.format(types)
 else:
 msg = 'value must be {}'.format(self.type_.__name__)
 raise TypeError(msg)
 super(TypedAttribute, self).__set__(obj, val)
if __name__ == '__main__':
 import doctest
 doctest.testmod(verbose=True)

Answer 1

I'll talk about the little stuff first

Name shadowing

Personal preference, I'm not a huge fan of using the same variable name in your generator expression as you did in the parameters to __new__ in your EnforceDescriptor metaclass. I'd prefer something like

if all(attrs.get(func_name) is None for func_name in cls.REQUIRED):

Code comments

Your method of getting the name is a little confusing - I had to read it about three times and check the inspect docs to fully understand how it works (of course, anything involving inspection is usually a little tricky). I, and presumably anyone who wanted to touch this in the future, would appreciate some comments as to why things happen why they do. For example,

code = next(frame for frame in getouterframes(currentframe())
 if frame[3] not in ('_get_name', '__init__'))[4]

could use a comment indicating that you're finding the first frame that wasn't this function or the __init__ function of the descriptor. Then maybe an explanation about why the 4th and 5th items in that frame are important.

Name unused variables with _

I'd rather see

def __get__(self, obj, _):
 return getattr(obj, self.name)

than

def __get__(self, obj, typ=None):
 return getattr(obj, self.name)

My IDE highlights typ=None so I know it's not used, and this is a simple function, but using _ makes it even easier to see that parameter can be ignored.

---

Now here's the more interesting things, imo.

Better way of naming the descriptor's attribute

Like you said, using inspect is pretty fragile. It'll likely fail on non-CPython implementations, and it's pretty hard to read. I think a better way to do this is with a metaclass that enforces naming of all descriptors, like so.

class EnforceNamedDescriptors(type):
 """Ensures that every instance of a BaseDescriptor has a name for
 its attribute.
 """
 def __new__(cls, name, bases, attrs):
 for attr_name, attr_value in attrs.items():
 if (isinstance(attr_value, BaseDescriptor) and
 not hasattr(attr_value, "name")):
 attr_value.name = "_{}".format(attr_name)
 return super(EnforceNamedDescriptors, cls).__new__(cls, name, bases, attrs)

then just make all classes that are going to use this functionality like so

class TestClass(with_metaclass(EnforceNamedDescriptors):
 ...

or even make a mixin class like that so you don't have to do that every time. You could still include the _get_name function as a fallback, in case you don't want this auto-naming to work every time. Then you'd probably change BaseDescriptor to look sort of like

def __init__(self, name=None):
 if name is not None:
 self.name = name
 # Otherwise fall back to _get_name and EnforceNamedDescriptors
def __get__(self, obj, typ=None):
 if not hasattr(self, 'name'):
 self.name = self._get_name()
 return getattr(obj, self.name)

One possible issue with this is if, for some ungodly reason, someone wanted to do

class TestClass(with_metaclass(EnforceNamedDescriptors)):
 __mangled = TypedAttribute(type=(int, float))

you might run into some weird issues... but I figure if someone is doing this much weirdness they probably deserve whatever is coming to them.

Other ways of avoiding using inspection

I think that you might also be able to avoid using inspection AND metaclasses if you don't mind a little overhead on the first time access of the attribute's value. If you pull the code from the meta class into the __get__ of BaseDescriptor you could do something like

def __get__(self, obj, typ=None):
 if not hasattr(self, 'name'):
 for name, value in obj.__dict__.items():
 if (isinstance(value, BaseDescriptor) and not
 hasattr(value, 'name')):
 value.name = "_{}".format(name)
 return getattr(obj, self.name)

which would add names to every descriptor the first time you access one. This might get a little unwieldy if there were a ton of these in a given class, but I suspect that this would be okay in most use cases.