[Python-Dev] PEP 362 6th edition

Thu Jun 21 19:21:00 CEST 2012

Hello,
The new revision of PEP 362 has been posted:
http://www.python.org/dev/peps/pep-0362/
Summary:
1. Signature & Parameter objects are now immutable
2. Signature.replace() and Parameter.replace()
3. Signature has a new default constructor, which
accepts parameters list and a return_annotation.
Parameters list is checked for the correct order
(i.e. keyword-only before var-keyword, not vice-versa)
The second way to instantiate Signatures is to use 
'from_function', which creates a Signature object
for the passed function.
4. Parameter.__str__
5. Positional-only arguments are rendered in '<>'
6. PEP was updated to include new documentation &
small examples.
The implementation is updated and 100% test covered.
Please see the issue: http://bugs.python.org/issue15008
Open questions:
Just one - Should we rename 'replace()' to 'new()'? I like
'new()' a bit better - it suggests that we'll get a new object.
-
Yury
PEP: 362
Title: Function Signature Object
Version: $Revision$
Last-Modified: $Date$
Author: Brett Cannon <brett at python.org>, Jiwon Seo <seojiwon at gmail.com>,
 Yury Selivanov <yselivanov at sprymix.com>, Larry Hastings <larry at hastings.org>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 21-Aug-2006
Python-Version: 3.3
Post-History: 04-Jun-2012
Abstract
========
Python has always supported powerful introspection capabilities,
including introspecting functions and methods (for the rest of
this PEP, "function" refers to both functions and methods). By
examining a function object you can fully reconstruct the function's
signature. Unfortunately this information is stored in an inconvenient
manner, and is spread across a half-dozen deeply nested attributes.
This PEP proposes a new representation for function signatures.
The new representation contains all necessary information about a function
and its parameters, and makes introspection easy and straightforward.
However, this object does not replace the existing function
metadata, which is used by Python itself to execute those
functions. The new metadata object is intended solely to make
function introspection easier for Python programmers.
Signature Object
================
A Signature object represents the call signature of a function and
its return annotation. For each parameter accepted by the function
it stores a `Parameter object`_ in its ``parameters`` collection.
A Signature object has the following public attributes and methods:
* return_annotation : object
 The "return" annotation for the function. If the function
 has no "return" annotation, this attribute is not set.
* parameters : OrderedDict
 An ordered mapping of parameters' names to the corresponding
 Parameter objects.
* bind(\*args, \*\*kwargs) -> BoundArguments
 Creates a mapping from positional and keyword arguments to
 parameters. Raises a ``TypeError`` if the passed arguments do
 not match the signature.
* bind_partial(\*args, \*\*kwargs) -> BoundArguments
 Works the same way as ``bind()``, but allows the omission
 of some required arguments (mimics ``functools.partial``
 behavior.) Raises a ``TypeError`` if the passed arguments do
 not match the signature.
* replace(parameters, \*, return_annotation) -> Signature
 Creates a new Signature instance based on the instance
 ``replace`` was invoked on. It is possible to pass different
 ``parameters`` and/or ``return_annotation`` to override the
 corresponding properties of the base signature. To remove
 ``return_annotation`` from the copied ``Signature``, pass in
 ``Signature.empty``.
Signature objects are immutable. Use ``Signature.replace()`` to
make a modified copy:
::
 >>> def foo() -> None:
 ... pass
 >>> sig = signature(foo)
 >>> new_sig = sig.replace(return_annotation="new return annotation")
 >>> new_sig is not sig
 True
 >>> new_sig.return_annotation != sig.return_annotation
 True
 >>> new_sig.parameters == sig.parameters
 True
 >>> new_sig = new_sig.replace(return_annotation=new_sig.empty)
 >>> hasattr(new_sig, "return_annotation")
 False
There are two ways to instantiate a Signature class:
* Signature(parameters, \*, return_annotation)
 Default Signature constructor. Accepts an optional sequence
 of ``Parameter`` objects, and an optional ``return_annotation``.
 Parameters sequence is validated to check that there are no
 parameters with duplicate names, and that the parameters
 are in the right order, i.e. positional-only first, then
 positional-or-keyword, etc.
* Signature.from_function(function)
 Returns a Signature object reflecting the signature of the
 function passed in.
It's possible to test Signatures for equality. Two signatures are
equal when their parameters are equal, their positional and
positional-only parameters appear in the same order, and they
have equal return annotations.
Changes to the Signature object, or to any of its data members,
do not affect the function itself.
Signature also implements ``__str__``:
::
 >>> str(Signature.from_function((lambda *args: None)))
 '(*args)'
 >>> str(Signature())
 '()'
Parameter Object
================
Python's expressive syntax means functions can accept many different
kinds of parameters with many subtle semantic differences. We
propose a rich Parameter object designed to represent any possible
function parameter.
A Parameter object has the following public attributes and methods:
* name : str
 The name of the parameter as a string. Must be a valid
 python identifier name (with the exception of ``POSITIONAL_ONLY``
 parameters, which can have it set to ``None``.)
* default : object
 The default value for the parameter. If the parameter has no
 default value, this attribute is not set.
* annotation : object
 The annotation for the parameter. If the parameter has no
 annotation, this attribute is not set.
* kind
 Describes how argument values are bound to the parameter.
 Possible values:
 * ``Parameter.POSITIONAL_ONLY`` - value must be supplied
 as a positional argument.
 Python has no explicit syntax for defining positional-only
 parameters, but many built-in and extension module functions
 (especially those that accept only one or two parameters)
 accept them.
 * ``Parameter.POSITIONAL_OR_KEYWORD`` - value may be
 supplied as either a keyword or positional argument
 (this is the standard binding behaviour for functions
 implemented in Python.)
 * ``Parameter.KEYWORD_ONLY`` - value must be supplied
 as a keyword argument. Keyword only parameters are those
 which appear after a "*" or "\*args" entry in a Python
 function definition.
 * ``Parameter.VAR_POSITIONAL`` - a tuple of positional
 arguments that aren't bound to any other parameter.
 This corresponds to a "\*args" parameter in a Python
 function definition.
 * ``Parameter.VAR_KEYWORD`` - a dict of keyword arguments
 that aren't bound to any other parameter. This corresponds
 to a "\*\*kwds" parameter in a Python function definition.
 Always use ``Parameter.*`` constants for setting and checking
 value of the ``kind`` attribute.
* replace(\*, name, kind, default, annotation) -> Parameter
 Creates a new Parameter instance based on the instance
 ``replaced`` was invoked on. To override a Parameter
 attribute, pass the corresponding argument. To remove
 an attribute from a ``Parameter``, pass ``Parameter.empty``.
Parameter constructor:
* Parameter(name, kind, \*, annotation, default)
 Instantiates a Parameter object. ``name`` and ``kind`` are required,
 while ``annotation`` and ``default`` are optional.
Two parameters are equal when they have equal names, kinds, defaults,
and annotations.
Parameter objects are immutable. Instead of modifying a Parameter object,
you can use ``Parameter.replace()`` to create a modified copy like so:
::
 >>> param = Parameter('foo', Parameter.KEYWORD_ONLY, default=42)
 >>> str(param)
 'foo=42'
 >>> str(param.replace())
 'foo=42'
 >>> str(param.replace(default=Parameter.empty, annotation='spam'))
 "foo:'spam'"
BoundArguments Object
=====================
Result of a ``Signature.bind`` call. Holds the mapping of arguments
to the function's parameters.
Has the following public attributes:
* arguments : OrderedDict
 An ordered, mutable mapping of parameters' names to arguments' values.
 Contains only explicitly bound arguments. Arguments for
 which ``bind()`` relied on a default value are skipped.
* args : tuple
 Tuple of positional arguments values. Dynamically computed from
 the 'arguments' attribute.
* kwargs : dict
 Dict of keyword arguments values. Dynamically computed from
 the 'arguments' attribute.
The ``arguments`` attribute should be used in conjunction with
``Signature.parameters`` for any arguments processing purposes.
``args`` and ``kwargs`` properties can be used to invoke functions:
::
 def test(a, *, b):
 ...
 sig = signature(test)
 ba = sig.bind(10, b=20)
 test(*ba.args, **ba.kwargs)
Arguments which could be passed as part of either ``*args`` or ``**kwargs``
will be included only in the ``BoundArguments.args`` attribute. Consider the
following example:
::
 def test(a=1, b=2, c=3):
 pass
 sig = signature(test)
 ba = sig.bind(a=10, c=13)
 >>> ba.args
 (10,)
 >>> ba.kwargs:
 {'c': 13}
Implementation
==============
The implementation adds a new function ``signature()`` to the ``inspect``
module. The function is the preferred way of getting a ``Signature`` for
a callable object.
The function implements the following algorithm:
 - If the object is not callable - raise a TypeError
 - If the object has a ``__signature__`` attribute and if it
 is not ``None`` - return it
 - If it has a ``__wrapped__`` attribute, return
 ``signature(object.__wrapped__)``
 - If the object is a an instance of ``FunctionType`` construct
 and return a new ``Signature`` for it
 - If the object is a method, construct and return a new ``Signature``
 object, with its first parameter (usually ``self`` or ``cls``)
 removed
 - If the object is an instance of ``functools.partial``, construct
 a new ``Signature`` from its ``partial.func`` attribute, and
 account for already bound ``partial.args`` and ``partial.kwargs``
 - If the object is a class or metaclass:
 - If the object's type has a ``__call__`` method defined in
 its MRO, return a Signature for it
 - If the object has a ``__new__`` method defined in its MRO,
 return a Signature object for it
 - If the object has a ``__init__`` method defined in its MRO,
 return a Signature object for it
 - Return ``signature(object.__call__)``
Note that the ``Signature`` object is created in a lazy manner, and
is not automatically cached. However, the user can manually cache a
Signature by storing it in the ``__signature__`` attribute.
An implementation for Python 3.3 can be found at [#impl]_.
The python issue tracking the patch is [#issue]_.
Design Considerations
=====================
No implicit caching of Signature objects
----------------------------------------
The first PEP design had a provision for implicit caching of ``Signature``
objects in the ``inspect.signature()`` function. However, this has the
following downsides:
 * If the ``Signature`` object is cached then any changes to the function
 it describes will not be reflected in it. However, If the caching is
 needed, it can be always done manually and explicitly
 * It is better to reserve the ``__signature__`` attribute for the cases
 when there is a need to explicitly set to a ``Signature`` object that
 is different from the actual one
Some functions may not be introspectable
----------------------------------------
Some functions may not be introspectable in certain implementations of
Python. For example, in CPython, built-in functions defined in C provide
no metadata about their arguments. Adding support for them is out of
scope for this PEP.
Signature and Parameter equivalence
-----------------------------------
We assume that parameter names have semantic significance--two
signatures are equal only when their corresponding parameters are equal
and have the exact same names. Users who want looser equivalence tests,
perhaps ignoring names of VAR_KEYWORD or VAR_POSITIONAL parameters, will
need to implement those themselves.
Examples
========
Visualizing Callable Objects' Signature
---------------------------------------
Let's define some classes and functions:
::
 from inspect import signature
 from functools import partial, wraps
 class FooMeta(type):
 def __new__(mcls, name, bases, dct, *, bar:bool=False):
 return super().__new__(mcls, name, bases, dct)
 def __init__(cls, name, bases, dct, **kwargs):
 return super().__init__(name, bases, dct)
 class Foo(metaclass=FooMeta):
 def __init__(self, spam:int=42):
 self.spam = spam
 def __call__(self, a, b, *, c) -> tuple:
 return a, b, c
 @classmethod
 def spam(cls, a):
 return a
 def shared_vars(*shared_args):
 """Decorator factory that defines shared variables that are
 passed to every invocation of the function"""
 def decorator(f):
 @wraps(f)
 def wrapper(*args, **kwds):
 full_args = shared_args + args
 return f(*full_args, **kwds)
 # Override signature
 sig = signature(f)
 sig = sig.replace(tuple(sig.parameters.values())[1:])
 wrapper.__signature__ = sig
 return wrapper
 return decorator
 @shared_vars({})
 def example(_state, a, b, c):
 return _state, a, b, c
 def format_signature(obj):
 return str(signature(obj))
Now, in the python REPL:
::
 >>> format_signature(FooMeta)
 '(name, bases, dct, *, bar:bool=False)'
 >>> format_signature(Foo)
 '(spam:int=42)'
 >>> format_signature(Foo.__call__)
 '(self, a, b, *, c) -> tuple'
 >>> format_signature(Foo().__call__)
 '(a, b, *, c) -> tuple'
 >>> format_signature(Foo.spam)
 '(a)'
 >>> format_signature(partial(Foo().__call__, 1, c=3))
 '(b, *, c=3) -> tuple'
 >>> format_signature(partial(partial(Foo().__call__, 1, c=3), 2, c=20))
 '(*, c=20) -> tuple'
 >>> format_signature(example)
 '(a, b, c)'
 >>> format_signature(partial(example, 1, 2))
 '(c)'
 >>> format_signature(partial(partial(example, 1, b=2), c=3))
 '(b=2, c=3)'
Annotation Checker
------------------
::
 import inspect
 import functools
 def checktypes(func):
 '''Decorator to verify arguments and return types
 Example:
 >>> @checktypes
 ... def test(a:int, b:str) -> int:
 ... return int(a * b)
 >>> test(10, '1')
 1111111111
 >>> test(10, 1)
 Traceback (most recent call last):
 ...
 ValueError: foo: wrong type of 'b' argument, 'str' expected, got 'int'
 '''
 sig = inspect.signature(func)
 types = {}
 for param in sig.parameters.values():
 # Iterate through function's parameters and build the list of
 # arguments types
 try:
 type_ = param.annotation
 except AttributeError:
 continue
 else:
 if not inspect.isclass(type_):
 # Not a type, skip it
 continue
 types[param.name] = type_
 # If the argument has a type specified, let's check that its
 # default value (if present) conforms with the type.
 try:
 default = param.default
 except AttributeError:
 continue
 else:
 if not isinstance(default, type_):
 raise ValueError("{func}: wrong type of a default value for {arg!r}". \
 format(func=func.__qualname__, arg=param.name))
 def check_type(sig, arg_name, arg_type, arg_value):
 # Internal function that encapsulates arguments type checking
 if not isinstance(arg_value, arg_type):
 raise ValueError("{func}: wrong type of {arg!r} argument, " \
 "{exp!r} expected, got {got!r}". \
 format(func=func.__qualname__, arg=arg_name,
 exp=arg_type.__name__, got=type(arg_value).__name__))
 @functools.wraps(func)
 def wrapper(*args, **kwargs):
 # Let's bind the arguments
 ba = sig.bind(*args, **kwargs)
 for arg_name, arg in ba.arguments.items():
 # And iterate through the bound arguments
 try:
 type_ = types[arg_name]
 except KeyError:
 continue
 else:
 # OK, we have a type for the argument, lets get the corresponding
 # parameter description from the signature object
 param = sig.parameters[arg_name]
 if param.kind == param.VAR_POSITIONAL:
 # If this parameter is a variable-argument parameter,
 # then we need to check each of its values
 for value in arg:
 check_type(sig, arg_name, type_, value)
 elif param.kind == param.VAR_KEYWORD:
 # If this parameter is a variable-keyword-argument parameter:
 for subname, value in arg.items():
 check_type(sig, arg_name + ':' + subname, type_, value)
 else:
 # And, finally, if this parameter a regular one:
 check_type(sig, arg_name, type_, arg)
 result = func(*ba.args, **ba.kwargs)
 # The last bit - let's check that the result is correct
 try:
 return_type = sig.return_annotation
 except AttributeError:
 # Looks like we don't have any restriction on the return type
 pass
 else:
 if isinstance(return_type, type) and not isinstance(result, return_type):
 raise ValueError('{func}: wrong return type, {exp} expected, got {got}'. \
 format(func=func.__qualname__, exp=return_type.__name__,
 got=type(result).__name__))
 return result
 return wrapper
References
==========
.. [#impl] pep362 branch (https://bitbucket.org/1st1/cpython/overview)
.. [#issue] issue 15008 (http://bugs.python.org/issue15008)
Copyright
=========
This document has been placed in the public domain.
..
 Local Variables:
 mode: indented-text
 indent-tabs-mode: nil
 sentence-end-double-space: t
 fill-column: 70
 coding: utf-8
 End: