homepage

This issue tracker has been migrated to GitHub , and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Duck-typing inspect.isfunction()
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.7
process
Status: closed Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: jdemeyer, r.david.murray, scoder, serhiy.storchaka, steven.daprano, terry.reedy
Priority: normal Keywords: patch

Created on 2017年04月14日 11:34 by jdemeyer, last changed 2022年04月11日 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
isfunction.patch jdemeyer, 2017年04月14日 11:40 Patch for Python 3
Messages (13)
msg291647 - (view) Author: Jeroen Demeyer (jdemeyer) * (Python triager) Date: 2017年04月14日 11:34
Python is supposed to encourage duck-typing, but the "inspect" module doesn't follow this advice. A particular problem is that Cython functions are not recognized by the inspect module to be functions: http://cython.readthedocs.io/en/latest/src/userguide/limitations.html#inspect-support 
msg291662 - (view) Author: Steven D'Aprano (steven.daprano) * (Python committer) Date: 2017年04月14日 14:25
Duck typing is not something that "Python" does, it is a style of programming done by Python programmers. You wouldn't expect isinstance() to try to "duck type", and likewise the inspect module should be precise about what it is inspecting. If inspect reports something is a duck, it should be an actual duck, not just something that quacks.
I'm not sure that the CPython inspect module should care about Cython objects. I don't think that Cython functions should count as Python functions, I think they are different kinds of callables.
But even if we decide that Cython function should be recognised by inspect.isfunction(), I don't think your patch is the right way to deal with it. Not every object with a __code__ attribute is a function.
py> from types import SimpleNamespace
py> x = SimpleNamespace(__code__=1, spam=2)
py> '__code__' in dir(x)
True
Your patch would wrongly detect x as a function when it isn't even callable.
msg291665 - (view) Author: Jeroen Demeyer (jdemeyer) * (Python triager) Date: 2017年04月14日 14:42
> If inspect reports something is a duck, it should be an actual duck, not just something that quacks.
The problem is that some Python packages (Sphinx and IPython for example) really need to know whether it quacks. And the only tool they have is inspect.isfunction(), so they use that. It's silly that every single package using inspect.isfunction() should be fixed. Better just fix inspect.isfunction().
>>> from types import SimpleNamespace
>>> x = SimpleNamespace(__code__=1, spam=2)
>>> '__code__' in dir(x)
Of course, you can always break stuff. User code is not supposed to invent new __dunder__ special names.
msg291666 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2017年04月14日 14:50
The python standard library makes extensive use of duck typing. Duck typing is a pretty fundamental part of the design of Python, IMO. Even the ABC module does a bunch of duck typing, rather than requiring strict subclassing or registration.
I think the request is valid, and it is mostly a matter of agreeing on the best way to identify function ducks. (I agree that Steven's example is intentionally trying to quack like a duck and so is not, IMO, a valid counter argument against using __code__). I doubt we would make such a change in anything except a feature release, though. 
Let's see what other devs besides Steven and I think.
msg291667 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017年04月14日 15:01
__code__ is not enough for quacking as a function. Different code can expect other function attributes (for example __name__, __qualname__ and __module__).
See also issue8488. inspect.isroutine() and inspect.ismethoddescriptor() return True for some descriptors, but they don't quack good enough for pydoc.
msg291668 - (view) Author: Jeroen Demeyer (jdemeyer) * (Python triager) Date: 2017年04月14日 15:03
At the very least, the inspect module should use more duck-typing internally. For example, consider this code from "getfile":
 if ismethod(object):
 object = object.__func__
 if isfunction(object):
 object = object.__code__
 if istraceback(object):
 object = object.tb_frame
 if isframe(object):
 object = object.f_code
 if iscode(object):
 return object.co_filename
msg291705 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017年04月15日 06:06
inspect.isfunction(object) is documented to
 Return true if the object is a Python function, which includes functions created by a lambda expression.
This is currently implemented as "isinstance(object, types.FunctionType)".
The docs usually regard a 'Python function' as the result of a def statement or lambda expression. The inspect doc says that a function includes a particular set of attributes. One of them is a code object with its own fairly extensive set of attributes. Some of them are derived from the Python code. But others, in particular co_code, are specific to the current CPython bytecode version. (And co_code is intentionally writable.)
To me, the main purpose of checking that something is a function, as opposed to just being callable, is to know whether one can dependably access the attributes. Given that some are inherently CPython specific, including objects compiled by third-party software seems dubious. (There is also the issue of not being able to test with 3rd party objects.)
The referenced cython doc says
"""While it is quite possible to emulate the interface of functions in Cython’s own function type, and recent Cython releases have seen several improvements here,"""
To me, this implies to me that Cython function (compiled from Cython's extended def statements) do not yet perfectly emulate (fulfill) 'Python functions'. As indicated above, perfect emulation seems impossible for Cython or any other external compiler that does not use the same bytecode.
"""the "inspect" module does not consider a Cython implemented function a "function", because it tests the object type explicitly instead of comparing an abstract interface or an abstract base class. This has a negative impact on code that uses inspect to inspect function objects, but would require a change to Python itself."""
Where the current situation would be annoying is if working code uses isfunction and then Cython is used to speed up the code. But Cython could supply, if it does not now, expanded functions along with the list of cyfunction attributes and an indication of which are compatible with CPython function attributes.
Cython is not the only 3rd party compiler, and not the only one that might ever be linkable to CPython. So any change to CPython should not be limited to Cython.
If it were possible for Cython to makes its CythonFunction class a subclass of FunctionType, the issue would be 'solved', though the incompatibilities would remain.
msg291716 - (view) Author: Jeroen Demeyer (jdemeyer) * (Python triager) Date: 2017年04月15日 15:06
> As indicated above, perfect emulation seems impossible for Cython or any other external compiler that does not use the same bytecode.
True, Cython functions are not implemented using Python bytecode, so perfect emulation is impossible. The use case I care most about is getargspec(), which is fully supported by Cython functions.
> If it were possible for Cython to makes its CythonFunction class a subclass of FunctionType, the issue would be 'solved', though the incompatibilities would remain.
That's an interesting idea. Currently, that is simply impossible because
>>> from types import FunctionType
>>> class X(FunctionType): pass
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
TypeError: type 'function' is not an acceptable base type
Still, one could argue to change the implementation of FunctionType. If you do that, it would be best to define a BaseFunctionType and then have Cython functions and Python functions inherit from that. Personally, I think that's an even better but much more involved solution (I guess it would require a PEP).
msg291724 - (view) Author: Jeroen Demeyer (jdemeyer) * (Python triager) Date: 2017年04月15日 17:24
For the record: the __code__ attribute of a Cython function is a real "code" object (the same type as the __code__ attribute of a Python function). Of course not all fields are relevant, for example co_code is empty.
So I think it's clear that Cython tries really hard to be compatible with Python functions.
msg291728 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017年04月15日 20:04
inspect.getargspec is deprecated in favor of .getfullargspec and .signature and is implemented in with .getfullargspec. This, in turn, calls ._signature_from_callable which ultimately looks for (perhaps after recursive unwrap calls) obj.__signature__. So I expect that the case you 'care most about' already works. True?
It appears that .signature is intended to work for cython functions via the following helper function. Its code is somewhat awkward and tests that the object has needed attributes with needed types.
def _signature_is_functionlike(obj):
 """Private helper to test if `obj` is a duck type of FunctionType.
 A good example of such objects are functions compiled with
 Cython, which have all attributes that a pure Python function
 would have, but have their code statically compiled.
 """
That does leave cases like the inspect.getfile code you quoted. It could be fixed with some fiddly code, but there would still be .getclosurevariables and a couple of other uses of isfunction to review.
I reviewed the function and code attributes listed in
https://docs.python.org/3/library/inspect.html#types-and-members
and I think the necessary differences a function compiled by CPython and anything else are limited to the code object.
Proposal: for a cleaner solution, define a 'mincode' base class that lacks, for instance, co_code, co_consts, co_flags, co_lnotab, and co_stacksize. Make code a subclass of this. Define 'minfunction' as a function whose __code__ is a mincode. Make function a subclass of this. Define 'isminfunction' and replace 'isfunction' where a mincode is sufficient. This might allow, for instance, _signature_is_functionlike to be removed.
Details should perhaps be specified in a relatively short PEP. Discussion could maybe continue on python-ideas.
msg291881 - (view) Author: Jeroen Demeyer (jdemeyer) * (Python triager) Date: 2017年04月19日 11:59
> So I expect that the case you 'care most about' already works.
Yes, it works. That's the most ironic part of this issue: getfullargspec(func) works but packages like Sphinx will not call getfullargspec(func) because they do not detect that "func" is actually a function.
msg314292 - (view) Author: Jeroen Demeyer (jdemeyer) * (Python triager) Date: 2018年03月22日 20:53
See https://mail.python.org/pipermail/python-ideas/2018-March/049398.html 
msg314929 - (view) Author: Jeroen Demeyer (jdemeyer) * (Python triager) Date: 2018年04月04日 13:55
Superseded by https://www.python.org/dev/peps/pep-0575/ 
History
Date User Action Args
2022年04月11日 14:58:45adminsetgithub: 74257
2018年04月04日 13:55:15jdemeyersetstatus: open -> closed

messages: + msg314929
stage: test needed -> resolved
2018年03月22日 20:53:35jdemeyersetmessages: + msg314292
2017年04月19日 11:59:40jdemeyersetmessages: + msg291881
2017年04月15日 20:04:44terry.reedysetmessages: + msg291728
2017年04月15日 17:24:53jdemeyersetmessages: + msg291724
2017年04月15日 15:06:26jdemeyersetmessages: + msg291716
2017年04月15日 06:06:47terry.reedysetnosy: + terry.reedy

messages: + msg291705
stage: test needed
2017年04月14日 15:03:39jdemeyersetmessages: + msg291668
2017年04月14日 15:01:47serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg291667
2017年04月14日 14:50:55r.david.murraysetversions: - Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6
nosy: + r.david.murray

messages: + msg291666

type: enhancement
2017年04月14日 14:42:56jdemeyersetmessages: + msg291665
2017年04月14日 14:25:44steven.dapranosetnosy: + steven.daprano
messages: + msg291662
2017年04月14日 11:40:36jdemeyersetfiles: + isfunction.patch
keywords: + patch
2017年04月14日 11:34:54jdemeyercreate

AltStyle によって変換されたページ (->オリジナル) /