1414
votes

What are the lesser-known but useful features of the Python programming language?

  • Try to limit answers to Python core.
  • One feature per answer.
  • Give an example and short description of the feature, not just a link to documentation.
  • Label the feature using a title as the first line.

Quick links to answers:

0

191 Answers 191

1 2
3
4 5
...
7
24
votes

I personally love the 3 different quotes

str = "I'm a string 'but still I can use quotes' inside myself!"
str = """ For some messy multi line strings.
Such as
<html>
<head> ... </head>"""

Also cool: not having to escape regular expressions, avoiding horrible backslash salad by using raw strings:

str2 = r"\n" 
print str2
>> \n
2
  • 8
    Four different quotes, if you include ''' Commented Aug 26, 2009 at 17:07
  • I enjoy having ' and " do pretty much the same thing in code. My IDE highlights strings from the two in different colors, and it makes it easy to differentiate short strings (with ') from longer ones (with "). Commented Dec 28, 2010 at 5:41
23
votes

Generators

I think that a lot of beginning Python developers pass over generators without really grasping what they're for or getting any sense of their power. It wasn't until I read David M. Beazley's PyCon presentation on generators (it's available here) that I realized how useful (essential, really) they are. That presentation illuminated what was for me an entirely new way of programming, and I recommend it to anyone who doesn't have a deep understanding of generators.

2
  • 2
    Wow! My brain is fried and that was just the first 6 parts. Starting in 7 I had to start drawing pictures just to see if I really understood what was happening with multi-process / multi-thread / multi-machine processing pipelines. Amazing stuff! Commented Nov 10, 2008 at 19:06
  • 1
    +1 for the link to the presentation Commented Oct 1, 2010 at 10:13
22
votes

Implicit concatenation:

>>> print "Hello " "World"
Hello World

Useful when you want to make a long text fit on several lines in a script:

hello = "Greaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa Hello " \
 "Word"

or

hello = ("Greaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa Hello " 
 "Word")
8
  • 3
    To make a long text fit on several lines, you can also use the triple quotes. Commented Sep 19, 2008 at 13:48
  • Your example is wrong and misleading. After running it, the "Word" part won't be on the end of the hello string. It won't concatenate. To continue on next line like that, you would need implicit line continuation and string concatenation and that only happens if you use some delimiter like () or []. Commented Sep 20, 2008 at 1:26
  • 2
    Anyone who has ever forgotten a comma in a list of strings knows how evil this 'feature' is. Commented Sep 22, 2008 at 5:13
  • 7
    Well, a PEP had been set to get rid of it but Guido decided finally to keep it. I guess it's more useful than hateful. Actually the drawbacks are no so dangerous (no safety issues) and for long strings, it helps a lot. Commented Sep 22, 2008 at 16:30
  • 2
    This is probably my favorite feature of Python. You can forget correct syntax and it's still correct syntax. Commented Dec 30, 2008 at 9:47
22
votes

When using the interactive shell, "_" contains the value of the last printed item:

>>> range(10)
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> _
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>>
5
  • I always forget about this one! It's a great feature. Commented Jul 22, 2010 at 20:17
  • _ automatic variable is the best feature when using Python shell as a calculator. Very powerful calculator, by the way. Commented Aug 4, 2010 at 19:03
  • I still try to use %% in the python shell from too much Mathematica in a previous life... If only %% were a valid variable name, I'd set %% = _... Commented Nov 15, 2010 at 12:42
  • This was already given by someone (I don't know if it was earlier, but it is voted higher). Commented Dec 28, 2010 at 5:39
  • __ for second-last and ___ for third-last Commented Jul 19, 2011 at 11:44
22
votes

Zero-argument and variable-argument lambdas

Lambda functions are usually used for a quick transformation of one value into another, but they can also be used to wrap a value in a function:

>>> f = lambda: 'foo'
>>> f()
'foo'

They can also accept the usual *args and **kwargs syntax:

>>> g = lambda *args, **kwargs: args[0], kwargs['thing']
>>> g(1, 2, 3, thing='stuff')
(1, 'stuff')
community wiki

1
  • The main reason I see to keep lambda around: defaultdict(lambda: 1) Commented Oct 9, 2010 at 17:39
22
votes

The textwrap.dedent utility function in python can come quite in handy testing that a multiline string returned is equal to the expected output without breaking the indentation of your unittests:

import unittest, textwrap
class XMLTests(unittest.TestCase):
 def test_returned_xml_value(self):
 returned_xml = call_to_function_that_returns_xml()
 expected_value = textwrap.dedent("""\
 <?xml version="1.0" encoding="utf-8"?>
 <root_node>
 <my_node>my_content</my_node>
 </root_node>
 """)
 self.assertEqual(expected_value, returned_xml)
21
votes

Using keyword arguments as assignments

Sometimes one wants to build a range of functions depending on one or more parameters. However this might easily lead to closures all referring to the same object and value:

funcs = [] 
for k in range(10):
 funcs.append( lambda: k)
>>> funcs[0]()
9
>>> funcs[7]()
9

This behaviour can be avoided by turning the lambda expression into a function depending only on its arguments. A keyword parameter stores the current value that is bound to it. The function call doesn't have to be altered:

funcs = [] 
for k in range(10):
 funcs.append( lambda k = k: k)
>>> funcs[0]()
0
>>> funcs[7]()
7
3
  • 6
    A less hackish way to do that (imho) is just to use a separate function to manufacture lambdas that don't close on a loop variable. Like this: def make_lambda(k): return lambda: k. Commented Jan 28, 2010 at 21:03
  • "less hackish"?....it's personal preference, I guess, but this is core Python stuff -- not really a hack. You certainly can structure it ( using functions ) so that the reader does not need to understand how Python's default arguments work -- but if you do understand how default arguments work, you will read the "lambda: k=k:k" and understand immediately that it is "saving" the current value of "k" ( as the lambda is created ), and attaching it to the lambda itself. This works the same with normal "def" functions, too. Commented Sep 20, 2011 at 20:34
  • Jason Orendorff's answer is correct, but this is how we used to emulate closures in Python before Guido finally agreed that nested scopes were a good idea. Commented Jan 12, 2012 at 4:23
20
votes

Mod works correctly with negative numbers

-1 % 5 is 4, as it should be, not -1 as it is in other languages like JavaScript. This makes "wraparound windows" cleaner in Python, you just do this:

index = (index + increment) % WINDOW_SIZE
community wiki

2
  • In most languages, number = coefficient x quotient + remainder. In Python (and Ruby), quotient is different than in JavaScript (or C or Java), because integer division in Python rounds towards negative infinity, but in JavaScript it rounds towards zero (truncates). I agree that % in Python makes more sense, but I don't know if / does. See en.wikipedia.org/wiki/Modulo_operation for details on each language. Commented Jan 24, 2011 at 3:42
  • In general, if abs(increment) < WINDOW_SIZE, then you can say index = (index + WINDOW_SIZE + increment) in any language, and have it do the right thing. Commented Jan 29, 2011 at 9:12
19
votes

Not very hidden, but functions have attributes:

def doNothing():
 pass
doNothing.monkeys = 4
print doNothing.monkeys
4
community wiki

4
  • 11
    It's because functions can be though of as objects with __call__() function defined. Commented Jan 14, 2010 at 16:06
  • 2
    It's because functions can be thought of as descriptors with __call__() function defined. Commented May 27, 2010 at 19:15
  • Wait, does __call__() also have a __call__() function? Commented Jun 29, 2011 at 14:20
  • 2
    I'll bet it's __call__() functions all the way down. Commented Aug 5, 2011 at 17:17
19
votes

Ternary operator

>>> 'ham' if True else 'spam'
'ham'
>>> 'ham' if False else 'spam'
'spam'

This was added in 2.5, prior to that you could use:

>>> True and 'ham' or 'spam'
'ham'
>>> False and 'ham' or 'spam'
'spam'

However, if the values you want to work with would be considered false, there is a difference:

>>> [] if True else 'spam'
[]
>>> True and [] or 'spam'
'spam'
2
  • Prior to 2.5, "foo = bar and 'ham' or 'spam'" Commented May 11, 2009 at 4:30
  • @a paid nerd - not quite: 1 == 1 and 0 or 3 => 3. The and short circuits on the 0 (as it equivalent to False - same deal with "" and None). Commented Jan 23, 2011 at 21:45
19
votes

Assigning and deleting slices:

>>> a = range(10)
>>> a
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> a[:5] = [42]
>>> a
[42, 5, 6, 7, 8, 9]
>>> a[:1] = range(5)
>>> a
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> del a[::2]
>>> a
[1, 3, 5, 7, 9]
>>> a[::2] = a[::-2]
>>> a
[9, 3, 5, 7, 1]

Note: when assigning to extended slices (s[start:stop:step]), the assigned iterable must have the same length as the slice.

19
votes

First-class functions

It's not really a hidden feature, but the fact that functions are first class objects is simply great. You can pass them around like any other variable.

>>> def jim(phrase):
... return 'Jim says, "%s".' % phrase
>>> def say_something(person, phrase):
... print person(phrase)
>>> say_something(jim, 'hey guys')
'Jim says, "hey guys".'
4
  • 2
    This also makes callback and hook creation (and, thus, plugin creation for your Python scripts) so trivial that you might not even know you're doing it. Commented Dec 30, 2008 at 9:49
  • 4
    Any langauge that doesn't have first class functions (or at least some good substitute, like C function pointers) it is a misfeature. It is completely unbearable to go without. Commented Nov 17, 2009 at 19:06
  • This might be a stupider question than I intend, but isn't this essentially a function pointer? Or do I have this mixed up? Commented Sep 22, 2010 at 4:13
  • 1
    @inspectorG4dget: It's certainly related to function pointers, in that it can accomplish all of the same purposes, but it's slightly more general, more powerful, and more intuitive. Particularly powerful when you combine it with the fact that functions can have attributes, or the fact that instances of certain classes can be called, but that starts to get arcane. Commented Oct 9, 2010 at 17:48
19
votes

Passing tuple to builtin functions

Much Python functions accept tuples, also it doesn't seem like. For example you want to test if your variable is a number, you could do: 

if isinstance (number, float) or isinstance (number, int): 
 print "yaay"

But if you pass us tuple this looks much cleaner: 

if isinstance (number, (float, int)): 
 print "yaay"
4
  • cool, is this even documented? Commented May 16, 2010 at 7:10
  • Yes, but nearly nobody knows about that. Commented May 16, 2010 at 12:35
  • What other functions support this?? Good tip Commented Jul 22, 2010 at 16:14
  • Not sure about other functions, but this is supposed in except (FooError, BarError) clauses. Commented Jan 22, 2011 at 20:16
19
votes

Nice treatment of infinite recursion in dictionaries:

>>> a = {}
>>> b = {}
>>> a['b'] = b
>>> b['a'] = a
>>> print a
{'b': {'a': {...}}}
community wiki

4
  • 1
    That is just the 'nice treatment' of "print", it doesn't imply a nice treatment across the language. Commented Jun 1, 2010 at 19:59
  • Both str() and repr() return the string you posted above. However, the ipython shell returns something a little different, a little more informative: {'b': {'a': <Recursion on dict with id=17830960>}} Commented Jun 2, 2010 at 2:21
  • 1
    @denilson: ipython uses pprint module, which is available whithin standard python shell. Commented Jun 5, 2010 at 19:49
  • 1
    +1 for the first one that I had absolutely no idea about whatsoever. Commented Dec 28, 2010 at 5:47
18
votes

reversing an iterable using negative step

>>> s = "Hello World"
>>> s[::-1]
'dlroW olleH'
>>> a = (1,2,3,4,5,6)
>>> a[::-1]
(6, 5, 4, 3, 2, 1)
>>> a = [5,4,3,2,1]
>>> a[::-1]
[1, 2, 3, 4, 5]
2
  • 2
    Good to know, but minor point: that only works with sequences not iterables in general. I.e., (n for n in (1,2,3,4,5))[::-1] doesn't work. Commented Jul 15, 2010 at 5:32
  • 3
    That notation will actually create a new (reversed) instance of that sequence, which might be undesirable in some cases. For such cases, reversed() function is better, as it returns a reverse iterator instead of allocating a new sequence. Commented Aug 4, 2010 at 19:08
18
votes

Not "hidden" but quite useful and not commonly used

Creating string joining functions quickly like so

 comma_join = ",".join
 semi_join = ";".join
 print comma_join(["foo","bar","baz"])
 'foo,bar,baz

and

Ability to create lists of strings more elegantly than the quote, comma mess. 

l = ["item1", "item2", "item3"]

replaced by

l = "item1 item2 item3".split()
2
  • I think these both make the thing more long and obfuscated. Commented Feb 16, 2012 at 8:18
  • I don't know. I've found places where judicious use made things easier to read. Commented Feb 16, 2012 at 11:55
18
votes

Arguably, this is not a programming feature per se, but so useful that I'll post it nevertheless.

$ python -m http.server

...followed by $ wget http://<ipnumber>:8000/filename somewhere else.

If you are still running an older (2.x) version of Python:

$ python -m SimpleHTTPServer

You can also specify a port e.g. python -m http.server 80 (so you can omit the port in the url if you have the root on the server side)

17
votes

Multiple references to an iterator

You can create multiple references to the same iterator using list multiplication:

>>> i = (1,2,3,4,5,6,7,8,9,10) # or any iterable object
>>> iterators = [iter(i)] * 2
>>> iterators[0].next()
1
>>> iterators[1].next()
2
>>> iterators[0].next()
3

This can be used to group an iterable into chunks, for example, as in this example from the itertools documentation 

def grouper(n, iterable, fillvalue=None):
 "grouper(3, 'ABCDEFG', 'x') --> ABC DEF Gxx"
 args = [iter(iterable)] * n
 return izip_longest(fillvalue=fillvalue, *args)
community wiki

4
  • 1
    You can do the opposite with itertools.tee -- take one iterator and return n that yield the same but do not share state. Commented Jul 3, 2010 at 4:41
  • 6
    I actually don't see the difference to doing this one: "a = iter(i)" and subsequently "b = a" I also get multiple references to the same iterator -- there is no magic about that to me, no hidden feature it is just the normal reference copying stuff of the language. What is done, is creating the iterator, then (the list multiplication) copying this iterator several times. Thats all, its all in the language. Commented Jul 3, 2010 at 9:46
  • 3
    @Juergen: indeed, a = iter(i); b = a does the same thing and I could just as well have written that into the answer instead of [iter(i)] * n. Either way, there is no "magic" about it. That's no different from any of the other answers to this question - none of them are "magical", they are all in the language. What makes the features "hidden" is that many people don't realize they're possible, or don't realize interesting ways in which they can be used, until they are pointed out explicitly. Commented Jul 3, 2010 at 23:14
  • Well, for one thing, you can do it an arbitrary number of times with [iter(i)]*n. Also, it isn't necessarily well known (to many people's peril) that list*int creates referential, not actual, copies of the elements of the list. It's good to see that that is actually useful somehow. Commented Dec 28, 2010 at 5:38
17
votes

From python 3.1 ( 2.7 ) dictionary and set comprehensions are supported :

{ a:a for a in range(10) }
{ a for a in range(10) }
5
  • there is no such thing as tuples comprehension, and this is not a syntax for dict comprehensions. Commented Jul 18, 2010 at 21:31
  • Edited the typo with dict comprehensions. Commented Jul 18, 2010 at 21:58
  • uh oh, looks like I have to upgrade my version of python so I can play with dict and set comprehensions Commented Jul 19, 2010 at 6:23
  • for dictionaries that way is better but dict( (a,a) for a in range(10) ) works too and your error is probably due to remembering this form Commented Aug 26, 2010 at 15:22
  • I cannot wait to use this feature. Commented Dec 28, 2010 at 5:48
15
votes

Python can understand any kind of unicode digits, not just the ASCII kind:

>>> s = u'10585'
>>> s
u'\uff11\uff10\uff15\uff18\uff15'
>>> print s
10585
>>> int(s)
10585
>>> float(s)
10585.0
community wiki

14
votes

__slots__ is a nice way to save memory, but it's very hard to get a dict of the values of the object. Imagine the following object:

class Point(object):
 __slots__ = ('x', 'y')

Now that object obviously has two attributes. Now we can create an instance of it and build a dict of it this way:

>>> p = Point()
>>> p.x = 3
>>> p.y = 5
>>> dict((k, getattr(p, k)) for k in p.__slots__)
{'y': 5, 'x': 3}

This however won't work if point was subclassed and new slots were added. However Python automatically implements __reduce_ex__ to help the copy module. This can be abused to get a dict of values:

>>> p.__reduce_ex__(2)[2][1]
{'y': 5, 'x': 3}
3
  • Oh wow, I might actually have good use for this! Commented Dec 30, 2008 at 9:46
  • Beware that __reduce_ex__ can be overridden in subclasses, and since it's also used for pickling, it often is. (If you're making data containers, you should think of using it too! or it's younger siblings __getstate__ and __setstate__.) Commented Jun 30, 2010 at 20:53
  • 2
    You can still do object.__reduce_ex__(p, 2)[2][1] then. Commented Jun 30, 2010 at 23:13
14
votes

itertools

This module is often overlooked. The following example uses itertools.chain() to flatten a list:

>>> from itertools import *
>>> l = [[1, 2], [3, 4]]
>>> list(chain(*l))
[1, 2, 3, 4]

See http://docs.python.org/library/itertools.html#recipes for more applications.

14
votes

Manipulating sys.modules

You can manipulate the modules cache directly, making modules available or unavailable as you wish:

>>> import sys
>>> import ham
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
ImportError: No module named ham
# Make the 'ham' module available -- as a non-module object even!
>>> sys.modules['ham'] = 'ham, eggs, saussages and spam.'
>>> import ham
>>> ham
'ham, eggs, saussages and spam.'
# Now remove it again.
>>> sys.modules['ham'] = None
>>> import ham
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
ImportError: No module named ham

This works even for modules that are available, and to some extent for modules that already are imported:

>>> import os
# Stop future imports of 'os'.
>>> sys.modules['os'] = None
>>> import os
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
ImportError: No module named os
# Our old imported module is still available.
>>> os
<module 'os' from '/usr/lib/python2.5/os.pyc'>

As the last line shows, changing sys.modules only affects future import statements, not past ones, so if you want to affect other modules it's important to make these changes before you give them a chance to try and import the modules -- so before you import them, typically. None is a special value in sys.modules, used for negative caching (indicating the module was not found the first time, so there's no point in looking again.) Any other value will be the result of the import operation -- even when it is not a module object. You can use this to replace modules with objects that behave exactly like you want. Deleting the entry from sys.modules entirely causes the next import to do a normal search for the module, even if it was already imported before.

1
  • And you can do sys.modules['my_module'] = MyClass(), to implement read only attributes 'module' if MyClass has the right hooks. Commented Mar 4, 2012 at 19:37
14
votes

You can ask any object which module it came from by looking at its __ module__ property. This is useful, for example, if you're experimenting at the command line and have imported a lot of things.

Along the same lines, you can ask a module where it came from by looking at its __ file__ property. This is useful when debugging path issues.

community wiki

13
votes

Some of the builtin favorites, map(), reduce(), and filter(). All extremely fast and powerful.

community wiki

7
  • 1
    list comprehensions can achieve everything you can do with any of those functions. Commented Jan 1, 2009 at 9:28
  • 2
    It can also obfuscate Python code if you abuse them Commented Aug 11, 2009 at 9:57
  • 4
    @sil: map still exists in Python 3, as does filter, and reduce exists as functools.reduce. Commented Oct 4, 2009 at 1:17
  • 8
    @recursive: I defy you to produce a list comprehension/generator expression that performs the action of reduce() Commented Nov 17, 2009 at 19:08
  • 1
    The correct statement is "reduce() can achieve everything you can do with map(), filter(), or list comprehensions." Commented Jan 12, 2012 at 4:31
13
votes

One word: IPython

Tab introspection, pretty-printing, %debug, history management, pylab, ... well worth the time to learn well.

community wiki

3
  • That's not built in python core is it? Commented Jul 22, 2009 at 6:16
  • You're right, it's not. And probably with good reason. But I recommend it without reservation to any Python programmer. (However, I heartily recommend turning off autocall. When it does something you don't expect, it can be very hard to realize why.) Commented Jul 24, 2009 at 20:13
  • I also love IPython. I've tried BPython, but it was too slow for me (although I agree it has some cool features). Commented Aug 4, 2010 at 19:10
13
votes

Guessing integer base

>>> int('10', 0)
10
>>> int('0x10', 0)
16
>>> int('010', 0) # does not work on Python 3.x
8
>>> int('0o10', 0) # Python >=2.6 and Python 3.x
8
>>> int('0b10', 0) # Python >=2.6 and Python 3.x
2
12
votes

You can build up a dictionary from a set of length-2 sequences. Extremely handy when you have a list of values and a list of arrays.

>>> dict([ ('foo','bar'),('a',1),('b',2) ])
{'a': 1, 'b': 2, 'foo': 'bar'}
>>> names = ['Bob', 'Marie', 'Alice']
>>> ages = [23, 27, 36]
>>> dict(zip(names, ages))
{'Alice': 36, 'Bob': 23, 'Marie': 27}
community wiki

2
  • self.data = {} _i = 0 for keys in self.VDESC.split(): self.data[keys] = _data[_i] _i += 1 I replaced my code with this one-liner :) self.data = dict(zip(self.VDESC.split(), _data)) Thanks for the handy tip. Commented Sep 29, 2009 at 2:32
  • 1
    Also helps in Python2.x where there is no dict comprehension syntax. Sou you can write dict((x, x**2) for x in range(10)). Commented May 29, 2010 at 11:57
12
votes

Extending properties (defined as descriptor) in subclasses

Sometimes it's useful to extent (modify) value "returned" by descriptor in subclass. It can be easily done with super():

class A(object):
 @property
 def prop(self):
 return {'a': 1}
class B(A):
 @property
 def prop(self):
 return dict(super(B, self).prop, b=2)

Store this in test.py and run python -i test.py (another hidden feature: -i option executed the script and allow you to continue in interactive mode):

>>> B().prop
{'a': 1, 'b': 2}
1
  • +1 properties! Cant get enough of them. Commented May 27, 2010 at 19:37
1 2
3
4 5
...
7

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.