[Python-ideas] make closure writable

Fri Mar 23 07:19:29 CET 2012

On Tue, Mar 20, 2012 at 6:06 AM, Yury Selivanov <yselivanov.ml at gmail.com> wrote:
> I did provide such example earlier in this thread.  I'm copying and
> pasting it to this mail.  Please read the example carefully, as it
> explains why returning new types.FunctionType() is not enough.
>> ----
>> Yes, your approach will work if your decorator is the only one applied.
> But, as I said, if you have many of them (see below), you can't just
> return a new function out of your decorator, you need to change the
> underlying "in-place".  Consider the following:
>> def modifier(func):
>  orig_func = func
>>  while func.__wrapped__:
>   func = func.__wrapped__
>>  # patch func.__code__ and func.__closure__
>  return orig_func # no need to wrap anything
>> def some_decorator(func):
>  def wrapper(*args, **kwargs):
>     # some code
>     return func(*args, **kwargs)
>  functools.wraps(wrapper, func)
>  return wrapper
>> @modifier
> @some_decorator
> def foo():
>  # this code needs to be verified/augmented/etc

Couldn't something like the following work?
def modifier(func):
 """Traverse the decorator "stack" and patch the bottom-most
wrapped function."""
 # relies on __wrapped__ being set at each level of the decorator
stack and on the
 # wrapped function being bound in func.__closure__.
 if not hasattr(func, "__wrapped__"):
 # patch func.__code__ and func.__closure__
 code = ...
 closure = ...
 else:
 code = func.__code__
 closure = list(func.__closure__)
 closure[closure.index(func.__wrapped__)] = modifier(func.__wrapped__)
 return type(func)(code, func.__globals__, func.__name__,
func.__defaults__, tuple(closure))
Also, I'm guessing that your actual use-case looks more like the following:
 from some_third_party_module import foo
 #assert foo.__wrapped__ == foo.__closure__[0]
 foo = modifier(foo) # hacks up foo.__wrapped__
Hacking the innards of an existing function object is touchy stuff,
probably the riskiest kind of monkey-patching. You're basically
taking the chance of breaking (in ugly, unexpected ways) other code
that uses that function you just hacked. Still, there are certainly
valid use-cases (and we're all consenting adults here).
However, I'd hate for projects to start getting blamed for
difficult-to-debug problems that are the result of some other project
that did this sort of hack. It's nice when your expectations for a
function's behavior (or any code for that matter) can remain stable,
regardless of what libraries are installed along-side.
-eric
p.s. I want to reiterate my understanding that nearly everything
involving the internals of functions is pretty delicate (read:
fragile), in part due to being the focal point for optimization.
Hacking it like this is therefore a delicate undertaking and
definitely skirts the edges of creating implementation-specific code.
Don't shy away. Just be extra cautious.
>> So, in the above snippet, if you don't want to discard the
> @some_decorator by returning a new function object, you need to modify
> the 'foo' from the @modifier.
>> In a complex framework, where you can't guarantee that your magic
> decorator will always be called first, rewriting the __closure__
> attribute is the only way.
>> Again, since the __code__ attribute is modifiable, and __closure__
> works in tight conjunction with it, I see no point in protecting it.
>> On 2012年03月20日, at 5:34 AM, Mark Shannon wrote:
>>> Yury Selivanov wrote:
>>> I've created an issue: http://bugs.python.org/issue14369
>>>> I think that creating an issue may be premature, given that you have had
>> no positive feedback on the idea.
>>>> I still think making __closure__ mutable is unnecessary.
>> If you insist that it is it, then please provide an example which would
>> work with your proposed change, but cannot be made to work using
>> types.FunctionType() to create a new closure.
>>>> Cheers,
>> Mark.
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> http://mail.python.org/mailman/listinfo/python-ideas
>> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas

[Python-ideas] make __closure__ writable

[Python-ideas] make closure writable