_functools module

Question

How does this import work, what file does it use?

import _functools

In python 2.5:

import _functools
print _functools.__file__

Gives:

Traceback (most recent call last):
 File "D:\zjm_code\mysite\zjmbooks\a.py", line 5, in <module>
 print _functools.__file__
AttributeError: 'module' object has no attribute '__file__'

How can I get the meaning of partial (from _functools import partial) if I can't read C code?

• Noufal Ibrahim: The implementation of functools is in C for 2.5 and 2.6. If you want to see the source, you'll have to read C. There seems to be a difference in the way it's compiled on my distro but either way, the source is in C. Check out the links to the repo in my answer if you want to see the source.
• Roger Pate: zjm: Don't use non-public (_*) modules directly, and view the documentation for functools.partial at docs.python.org/library/functools.html#functools.partial.
• Noufal Ibrahim: +1 for not using the (semi) private modules prefixed by _.
• zjm1126: thanks@ Noufal Ibrahim,@ Roger Pate
• Break: This is very old but, sometimes I like to use private functions, methods, etc. Particularly in asyncio because there are a lot of API like loop._write_to_self to allow threadsafety in otherwise non-threadsafe methods. Or to "cut corners" and run a function closer to the implementation rather than running through the public version, such as asyncio.tasks_wait over asyncio.wait. It's marginal but I like to optimize as long as I'm aware, and as long as it won't lead to a ridiculous looking codebase or something. But that is subjective, as some are against using private methods at all.

Answer 1

C-coded modules can be built-in (lacking __file__) or live in a .so or .pyd dynamic library (which their __file__ will indicate) -- that's an implementation detail that you should not care about.

If you want to understand how a Python-callable, C-coded function works by studying code, learning to read C is generally best (far less hard than actually productively coding in C;-). However, often you'll find (suggestive, non-authoritative) "sample Python implementations" of C-coded functionality, and you can study those.

A particularly fruitful repository of Python-coded equivalents to Python standard library functionality that's normally coded in C is the pypy project (which does Python implementations coded in Python) -- its sources are browseable here and of course you can download and peruse them on your machine.

In particular, this is pypy's _functools.py implementation:

""" Supplies the internal functions for functools.py in the standard library """
class partial:
 """
 partial(func, *args, **keywords) - new function with partial application
 of the given arguments and keywords.
 """
 __slots__ = ['func', 'args', 'keywords']
 def __init__(self, func, *args, **keywords):
 if not callable(func):
 raise TypeError("the first argument must be callable")
 self.func = func
 self.args = args
 self.keywords = keywords
 def __call__(self, *fargs, **fkeywords):
 newkeywords = self.keywords.copy()
 newkeywords.update(fkeywords)
 return self.func(*(self.args + fargs), **newkeywords)

Pretty trivial to read and understand, I hope!

Answer 2

For Python2.6, The _functools module is a builtin. You can see that if you simply type import _functools ; repr(_functools) and hit enter at your interpreter prompt.

If you want to see the C source of the module, check out http://hg.python.org/cpython/file/d7e85ddb1336/Modules/_functoolsmodule.c

This _functools module doesn't have a __file__ attribute (see next paragraph) since it's compiled into the interpreter.

For Python2.5, The _functools module is a standard library module implemented in C and is available at http://hg.python.org/cpython/file/a78381ead4cf/Modules/_functoolsmodule.c if you want to see it. You can see the location from where the module is loaded up by typing import _functools ; print _functools.__file__ at your interpreter prompt

Answer 3

please be more clear when asking questions next time. I assume you want this

>>> import _functools
>>> _functools.__file__
'/usr/lib/python2.6/lib-dynload/_functools.so'

Answer 4

It is a shared object _functools.so, written in C. I guess, it is imported by the actual functools module. Why are you trying to import from it? Sure, it is possible, however, you can just as well from functools import partial.

Answer 5

The behaviour of the functools.partial() function is described in PEP 309 in which it was defined. The although the actual implementations of built-ins are often in C PEP's including this one usually contain example implementations in Python.

This example code should be sufficient for you to understand the behaviour of functools.partial(). If there is a specific issue about the C implementation that is concerning you are going to have to read the C code. Or perhaps describe it in your question and someone might know the answer.