homepage

The definition of a new nametuple creates a large Python script to create the new type. The code stores the code in a private attribute:
 namespace = dict(__name__='namedtuple_%s' % typename)
 exec(class_definition, namespace)
 result = namespace[typename]
 result._source = class_definition
This attribute wastes memory, I don't understand the purpose of the attribute. It was not discussed in an issue, so I guess that there is no real use case:
changeset: 68879:bffdd7e9265c
user: Raymond Hettinger <python@rcn.com>
date: Wed Mar 23 12:52:23 2011 -0700
files: Doc/library/collections.rst Lib/collections/__init__.py Lib/test/test_collections.py
description:
Expose the namedtuple source with a _source attribute.
Can we just drop this attribute to reduce the Python memory footprint?

I found this issue while using my tracemalloc module to analyze the memory consumption of Python. On the Python test suite, the _source attribute is the 5th line allocating the memory memory:
/usr/lib/python3.4/collections/__init__.py: 676.2 kB

> the 5th line allocating the memory memory
oops, the 5th line allocating the *most* memory

As an alternative, how about turning _source into a property?

In a first version namedtuple had an argument (named echo or verbose) that would cause the source code to be printed out, for use at the interactive prompt. Raymond later changed it to a _source attribute, more easy to work with than printed output.
About the other question you asked on the ML (why isn’t there a base NamedTuple class to inherit): this has been discussed on python-ideas IIRC, and people have written ActiveState recipes for that idea. It should be easy to find the ML archive links from the ActiveState posts.

A while back, because of those python-ideas discussions, Raymond added a link at the bottom of the namedtuple section of the docs at http://docs.python.org/3.4/library/collections.html#namedtuple-factory-function-for-tuples-with-named-fields. The link points to a nice recipe by Jan Kaliszewski.

> As an alternative, how about turning _source into a property?
A class or an instance property? A class property requires a metaclass. I guess that each namedtuple type requires its own metaclass, right?

It does not necessarily require a metaclass. You can accomplish it using a custom descriptor:
class classattr:
 def __init__(self, getter):
 self.getter = getter
 def __get__(self, obj, cls):
 return self.getter(cls)
FWIW, this is a descriptor that may be worth adding somewhere regardless.

namedtuple_source.patch: Replace _source attribute wasting memory with a property generating the source on demand. The patch adds also unit test for the verbose attribute (which is public and documented, even it is said to be "outdated").
The patch removes also repr_fmt and num_fields parameters of the class definition template, compute these values using the list of fields.
I suggested to change Python 3.4.1 and 3.5.
Test script:
---
import email
import http.client
import pickle
import test.regrtest
import test.test_os
import tracemalloc
import xmlrpc.server
snap = tracemalloc.take_snapshot()
with open("dump.pickle", "wb") as fp:
 pickle.dump(snap, fp, 2)
---
With the patch, the memory footprint is reduced by 176 kB.

Also be sure the have Raymond's sign-off before committing anything for this. :)

FWIW, the "verbose" option is mentioned as outdated because the "_source" attribute was added.
Also, there are real use cases, people are using the _source as writing it to a .py file so that the dynamic namedtuple generation step can be skipped on subsequent imports. This is useful when people want to avoid the use of eval or want to run cython on the code.
The attribute can be "dropped". It is part of the API.
Sorry, the memory use bugs you. It is bigger than typical docstrings but is not a significant memory consumer in most applications.
I like the idea of dynamically generating the source upon lookup, but want to think about whether there are any unintended consequences to that space saving hack.

The size of the _source attribute is about 2k per namedtuple class:
>>> from collections import namedtuple
>>> Response = namedtuple('Response', ['code', 'msg', 'compressed', 'written'])
>>> len(Response._source)
2174

Victor, I don't think the added complexity is worth 2k per named tuple class. Every time I've gone down the path of lazy evaluation, I've paid an unexpected price for it down the road. If the savings were huge, it might be worth it, but that isn't the case here. This isn't really different than proposing that all docstring be in a separate module to be lazily loaded only when people look at help.