Cost of exception handlers in Python

Question

In another question, the accepted answer suggested replacing a (very cheap) if statement in Python code with a try/except block to improve performance.

Coding style issues aside, and assuming that the exception is never triggered, how much difference does it make (performance-wise) to have an exception handler, versus not having one, versus having a compare-to-zero if-statement?

• S.Lott: When you measured it, what did you learn?
• tzot: Related question: stackoverflow.com/questions/1835756
• ns15: Use try/except if chances of control going to except part is less and if/else if chances are more.

Answer 1

Why don't you measure it using the timeit module? That way you can see whether it's relevant to your application.

OK, so I've just tried the following (using Python 3.11.1 on Windows 11):

import timeit
statements=["""\
try:
 b = 10/a
except ZeroDivisionError:
 pass""",
"""\
if a:
 b = 10/a""",
"b = 10/a"]
for a in (1,0):
 for s in statements:
 t = timeit.Timer(stmt=s, setup='a={}'.format(a))
 print("a = {}\n{}".format(a,s))
 print("%.2f usec/pass\n" % (1000000 * t.timeit(number=100000)/100000))

Result:

a = 1
try:
 b = 10/a
except ZeroDivisionError:
 pass
0.06 usec/pass
a = 1
if a:
 b = 10/a
0.05 usec/pass
a = 1
b = 10/a
0.03 usec/pass
a = 0
try:
 b = 10/a
except ZeroDivisionError:
 pass
0.27 usec/pass
a = 0
if a:
 b = 10/a
0.02 usec/pass
a = 0
b = 10/a
Traceback (most recent call last):
 File "<stdin>", line 5, in <module>
 File "C:\Python311\Lib\timeit.py", line 178, in timeit
 timing = self.inner(it, self.timer)
 ^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "<timeit-src>", line 6, in inner
ZeroDivisionError: division by zero

As you can see, there is not much of a difference between using a try/except clause vs. an explicit if statement, unless the exception gets triggered. (And of course, not having any control structure is fastest, though not by much, and it will crash the program if anything goes wrong).

Compare this to the results obtained in 2010:

a = 1
try:
 b = 10/a
except ZeroDivisionError:
 pass
0.25 usec/pass
a = 1
if a:
 b = 10/a
0.29 usec/pass
a = 1
b = 10/a
0.22 usec/pass
a = 0
try:
 b = 10/a
except ZeroDivisionError:
 pass
0.57 usec/pass
a = 0
if a:
 b = 10/a
0.04 usec/pass
a = 0
b = 10/a
ZeroDivisionError: int division or modulo by zero

I appears that the PC I'm using now is about twice as fast as the one I had back then. The cost of handling an Exception appears identical, and the "normal" operations (arithmetic) have been improved even more than the handling of control structures, but the point from all those years ago still stands:

It's all within the same order of magnitude and unlikely to matter either way. Only if the condition is actually met (often), then the if version is significantly faster.

• Thilo: Interesting. So try/except is faster than if a != 0
• Garrett Bluma: Ahh, a fine choice of words: "it's all within the same order of magnitude"... I suspect many people who avoid exceptions do so expecting them to be 10x as slow.
• duleshi: Running your code on my Fedora with python 2.7.5 shows that the "if" version(0.08 usec/pass) is faster than the "try/except" one(0.11 usec/pass) when a=1.
• Basic: @duleshi Interesting. I wonder if it's a x86/x64 thing? Or perhaps different processor extensions?
• lesurp: Hmm they are not, by definition, within the same order of magnitude, since the runtime of the try version is more than 10 times slower than the if version. Whether 50us is too high a cost to pay is another question.

Answer 2

This question is actually answered in the Design and History FAQ:

A try/except block is extremely efficient if no exceptions are raised. Actually catching an exception is expensive.

• Thilo: I was just wondering how efficient "extremely efficient" is. Apparently it is faster then even a very simple "if" statement.
• nitsas: The excerpt you posted is from the Design and History FAQ.
• ebk: Maybe "extremely efficient" means something like what is done in Java?

Answer 3

In Python 3.11,

"Zero-cost" exceptions are implemented. The cost of try statements is almost eliminated when no exception is raised. (Contributed by Mark Shannon in bpo-40222.)

https://docs.python.org/3.11/whatsnew/3.11.html#misc

Answer 4

This question is misleading. If you assume the exception is never triggered, neither one is optimal code.

If you assume the exception is triggered as part of an error condition, you are already outside the realm of wanting optimal code (and you probably aren't handling it at a fine-grained level like that anyway).

If you are using the exception as part of the standard control flow - which is the Pythonic "ask forgiveness, not permission" way - then the exception is going to be triggered, and the cost depends on the kind of exception, the kind of if, and what percentage of time you estimate the exception happens.

Answer 5

Q: Is try/catch costly in python?

Should i be concerned when I use try catch? In what way?

This is just a summary of the answers already given.

A: When there is an exception if is much faster. Otherwise no.

@SuperNova writes that exceptions are at zero cost so it is faster than having an if-statement when no exception. However, handling exceptions is costly so:

Use try for things that can fail. If possible, avoid try for things you know will fail

Example:

1. Good case, use try:

try:
 x = getdata() # an external function 
except:
 print('failed. Retrying')

2. Bad case, here if-version is preferred:

y = f(x) # f never fails but often returns 0
try:
 z = 1 / y # this fails often
except:
 print('failed.')
# if-version
y = f(x)
if y != 0:
 z = 1 / y
else:
 print('failed.')

Answer 6

It looks like this has changed significantly since the original post 15 years ago, and especially with the "'Zero-cost' exceptions" in SuperNova's answer. For my current project, I care more about lookup speed and errors than 1 / 0 errors, so I'm looking into that. I found this blog post doing exactly what I wanted, but in Python 2.7. I updated the test to 3.13, (Windows 10, i9-9900k) with results below.

This compares checking key existence with if key in d to using a try: except: block

'''
The case where the key does not exist:
100 iterations:
with_try (0.016 ms)
with_try_exc (0.016 ms)
without_try (0.003 ms)
without_try_not (0.002 ms)
1,000,000 iterations:
with_try (152.643 ms)
with_try_exc (179.345 ms)
without_try (29.765 ms)
without_try_not (32.795 ms)
The case where the key does exist:
100 iterations:
exists_unsafe (0.005 ms)
exists_with_try (0.003 ms)
exists_with_try_exc (0.003 ms)
exists_without_try (0.005 ms)
exists_without_try_not (0.004 ms)
1,000,000 iterations:
exists_unsafe (29.763 ms)
exists_with_try (30.970 ms)
exists_with_try_exc (30.733 ms)
exists_without_try (46.288 ms)
exists_without_try_not (46.221 ms)
'''

where it looks like the try block has a very small overhead, where if the key exists, an unsafe check and try check are the same. Using in has to hash the key for the check, and again for the access, so it slows by ~30% with the redundant operation for real usage. If the key does not exist, the try costs 5x the in statement, which is the same cost for either case.

So, it does come back to asking if you expect few errors, use try and many use in

And here's the code

import time
def time_me(function):
 def wrap(*arg):
 start = time.time()
 r = function(*arg)
 end = time.time()
 print("%s (%0.3f ms)" % (function.__name__, (end-start)*1000))
 return r
 return wrap
 
# Not Existing
@time_me
def with_try(iterations):
 d = {'somekey': 123}
 for i in range(0, iterations):
 try:
 get = d['notexist']
 except:
 pass
@time_me
def with_try_exc(iterations):
 d = {'somekey': 123}
 for i in range(0, iterations):
 try:
 get = d['notexist']
 except Exception as e:
 pass
 
@time_me
def without_try(iterations):
 d = {'somekey': 123}
 for i in range(0, iterations):
 if 'notexist' in d:
 pass
 else:
 pass
 
@time_me
def without_try_not(iterations):
 d = {'somekey': 123}
 for i in range(0, iterations):
 if not 'notexist' in d:
 pass
 else:
 pass
 
 
 
# Existing
@time_me
def exists_with_try(iterations):
 d = {'somekey': 123}
 for i in range(0, iterations):
 try:
 get = d['somekey']
 except:
 pass
 
@time_me
def exists_unsafe(iterations):
 d = {'somekey': 123}
 for i in range(0, iterations):
 get = d['somekey']
@time_me
def exists_with_try_exc(iterations):
 d = {'somekey': 123}
 for i in range(0, iterations):
 try:
 get = d['somekey']
 except Exception as e:
 pass
 
@time_me
def exists_without_try(iterations):
 d = {'somekey': 123}
 for i in range(0, iterations):
 if 'somekey' in d:
 get = d['somekey']
 else:
 pass
 
@time_me
def exists_without_try_not(iterations):
 d = {'somekey': 123}
 for i in range(0, iterations):
 if not 'somekey' in d:
 pass
 else:
 get = d['somekey']
 
 
print("The case where the key does not exist:")
print("100 iterations:")
with_try(100)
with_try_exc(100)
without_try(100)
without_try_not(100)
print("\n1,000,000 iterations:")
with_try(1000000)
with_try_exc(1000000)
without_try(1000000)
without_try_not(1000000)
print("\n\nThe case where the key does exist:")
print("100 iterations:")
exists_unsafe(100)
exists_with_try(100)
exists_with_try_exc(100)
exists_without_try(100)
exists_without_try_not(100)
print("\n1,000,000 iterations:")
exists_unsafe(1000000)
exists_with_try(1000000)
exists_with_try_exc(1000000)
exists_without_try(1000000)
exists_without_try_not(1000000)