Function overloading in Python

Question

Motivation

As an exercise, I wanted to try implementing function overloading in Python 3, i.e. to implement a way of defining multiple functions with the same name and then calling the appropriate function based on the given arguments in a function call. I know that this isn't necessary with Python and my understanding is that the Pythonic way to realize this kind of functionality is to define a single function and then inspect the arguments in order to determine the desired course of action, but I wanted to try this anyway and see how it went.

Features

There were a few basic features that I wanted to include:

• Collect functions by name, e.g. def f(x): pass and def f(y): pass should be considered as two versions of the same polymorphic function f (this is basically just the definition of function overloading, right?).

• Handle both stand-alone (unbound) functions and instance methods (bound functions).

• If there is a way to resolve a given function call, then do so (i.e. don't fail unless there is no function that can handle the given arguments).

Implementation

My idea was to use a controller class to store the different function definitions and use a decorator to register the functions with the controller. I wrote a module called overload.py with two classes: an OverloadedFunction class which stores a list of functions corresponding to the different overloaded versions of a given polymorphic function and an Overloader class which associates function names with OverloadedFunction objects.

The OverloadFunction objects have a lookup method which looks through the list of registered functions and returns the first one that matches the given set of arguments. The OverloadFunction objects are also callable and use their __call__ method to call the function obtained via the lookup method.

The Overload class defines an overload decorator which defines two wrapper functions: function_wrapper and method_wrapper. This handles both the cases of stand-alone (unbound) functions and instance methods (bound functions).

Here is the script:

#!/usr/bin/env python3
# coding: ascii
"""overload.py
Allow for multiple functions with different signatures
to have the same name (i.e. function overloading)."""
import inspect
from inspect import (
 getfullargspec,
 getmro,
 isfunction,
 ismethod,
)
from functools import (
 wraps,
)
class OverloadedFunctionError(Exception):
 """Exception class for errors related to the OverloadedFunction class."""
 pass
class OverloadedFunction(object):
 """An overloaded function.
 This is a proxy object which stores a list of functions. When called,
 it calls the first of its functions which matches the given arguments."""
 def __init__(self):
 """Initialize a new overloaded function."""
 self.registry = list()
 def lookup(self, args, kwargs):
 """Return the first registered function
 that matches the given call-parameters."""
 for function in self.registry:
 fullargspec = getfullargspec(function)
 # Make sure that the function can handle
 # the given number of positional arguments
 if(
 len(args) <= len(fullargspec.args) or
 bool(fullargspec.varargs)
 ):
 # Make sure that the function can handle
 # the remaining keyword arguments
 remaining_args = fullargspec.args[len(args):]
 if(
 frozenset(kwargs.keys()).issubset(remaining_args) or
 bool(fullargspec.varkw)
 ):
 return function
 def register(self, function):
 """Add a new function to the registry."""
 self.registry.append(function)
 def __call__(self, *args, **kwargs):
 """Call the first matching registered
 function with the given call parameters."""
 # Get the first function which can handle the given arguments
 function = self.lookup(args=args, kwargs=kwargs)
 # If no function can be found, raise an exception
 if not function:
 raise OverloadedFunctionError(
 "Failed to find matching function for given arguments: "
 "args={}, kwargs={}".format(args, kwargs)
 )
 # Evaluate the function and return the result
 return function(*args, **kwargs)
class Overloader(object):
 """A controller object which organizes OverloadedFunction by name."""
 def __init__(self):
 """Initialize a new OverloadedFunction controller."""
 self.registry = dict()
 def register(self, function):
 """Add a new function to the controller."""
 # Create a new OverloadedFunction for this
 # function name if one does not # already exists
 if function.__qualname__ not in self.registry.keys():
 self.registry[function.__qualname__] = OverloadedFunction()
 self.registry[function.__qualname__].register(function)
 def overload(self, function):
 """Decorator for registering a new function with
 the Overloader overloaded function controller."""
 # Register the new function with the controller
 self.register(function)
 # Handle the case of unbound functions
 if isfunction(function):
 def function_wrapper(*args, **kwargs):
 _function = self.registry[function.__qualname__]
 return _function(*args, **kwargs)
 return function_wrapper
 # Handle the case of bound functions
 if ismethod(function):
 def method_wrapper(_self, *args, **kwargs):
 _function = self.registry[function.__qualname__]
 return _function(self=_self, *args, **kwargs)
 return method_wrapper

Example

I wrote a short example script to test out the module. Here is the script:

#!/usr/bin/env python3
# coding: ascii
"""test_overload.py
Simple tests of the overload.py module.
"""
from overload import Overloader
# Instantiate a controller object
p = Overloader()
# Define several function with the same name "f"
@p.overload
def f():
 return "called: def f()"
@p.overload
def f(x):
 return "called: def f(x) with x={}".format(x)
@p.overload
def f(y):
 return "called: def f(y) with y={}".format(y)
@p.overload
def f(*args, **kwargs):
 return "called: def f(*args, **kwargs) with args={}, kwargs={}".format(args, kwargs)
# Call the overloaded function "f" and print the results
print(f())
print(f(1))
print(f(x=2))
print(f(y=3))
print(f(1, 2, 3, x=4, y=5, z=6))
# Define a class with overloaded methods
class MyClass(object):
 @p.overload
 def g(self, x=1):
 return "called: def MyClass.g(x) with x={}".format(x)
 @p.overload
 def g(self, x=1, y=2):
 return "called: def MyClass.g(x, y) with x={}, y={}".format(x, y)
# Instantiate an object
myobject = MyClass()
# Call the overloaded method
print(myobject.g())
print(myobject.g(1, 2))

When I run the test_overload.py script I get the following output:

called: def f()
called: def f(x) with x=1
called: def f(x) with x=2
called: def f(y) with y=3
called: def f(*args, **kwargs) with args=(1, 2, 3), kwargs={'x': 4, 'y': 5, 'z': 6}
called: def MyClass.g(x) with x=1
called: def MyClass.g(x, y) with x=1, y=2

Comments and Questions

Am I overlooking anything important? Are there any counterintuitive behaviors or unexpected errors that this implementation might lead to? Are there any easily implementable features that I could add? I thought about trying to check for function collision when registering a function, i.e. checking to see if there are multiple functions that could be called for the same arguments. Right now I'm resolving this issue my just relying on the order in which the functions are registered, but maybe there's a better way?

• Graipher: Are you aware that this is already in the standard library since Python 3.4? It is called functools.singledispatch.
• igal: @Graipher Oh geez. No, I was not. That's kind of embarrassing. Thanks for the information.
• igal: @Graipher Actually, if I correctly understand what I'm reading, the functools.singledispatch is slightly different than what I was shooting for here. It only performs dispatching based on the type of a single argument.
• Graipher: You are right, it does not dispatch on different number or name of inputs.

Answer 1

funtools.singledispatch()

Take a look at the code for functools.singledispatch(). It's different in that it dispatches based on the type of the first argument. Look at how it wraps the first function with an object, so a separate controller object isn't needed. It also has code to deal with method resolution order if the overloaded functions are at different places in the class inheritance heirarchy.

inspect.Signature

In Python callable signatures can be pretty complicated. Args can be positional, keyword, or both, or positional only, keyword only, and can have default values. Take a look at the inspect.Signature class. It represents the signature of a callable and has a bind() method that can check if some args match the signature. It can simplify your code and handle the complications.

from inspect import signature
class OverloadedFunction(object):
 """An overloaded function.
 This is a proxy object which stores a list of functions. When called,
 it calls the first of its functions which matches the given arguments."""
 def __init__(self):
 """Initialize a new overloaded function."""
 self.registry = list()
 def register(self, function):
 """Add a new function and it's signature to the registry."""
 self.registry.append(function, signature(function))
 def __call__(self, *args, **kwargs):
 """Call the first matching registered
 function with the given call parameters."""
 # Get the first function which can handle the given arguments
 for function, signature in self.registry:
 try:
 signature.bind(*args, **kwargs)
 return function(*args, **kwargs)
 except TypeError:
 # If no function can be found, raise an exception
 raise OverloadedFunctionError(
 "Failed to find matching function for given arguments: "
 "args={}, kwargs={}".format(args, kwargs)
 )