Edit - Stack Overflow

You are not logged in. Your edit will be placed in a queue until it is peer reviewed.

We welcome edits that make the post easier to understand and more valuable for readers. Because community members review edits, please try to make the post substantially better than how you found it, for example, by fixing grammar or adding additional resources and hyperlinks.

Required fields*

Rev

Required fields*

Why is reading lines from stdin much slower in C++ than Python?

I wanted to compare reading lines of string input from stdin using Python and C++ and was shocked to see my C++ code run an order of magnitude slower than the equivalent Python code. Since my C++ is rusty and I'm not yet an expert Pythonista, please tell me if I'm doing something wrong or if I'm misunderstanding something.

(TLDR answer: include the statement: cin.sync_with_stdio(false) or just use fgets instead.

TLDR results: scroll all the way down to the bottom of my question and look at the table.)

C++ code:

#include <iostream>
#include <time.h>
using namespace std;
int main() {
 string input_line;
 long line_count = 0;
 time_t start = time(NULL);
 int sec;
 int lps;
 while (cin) {
 getline(cin, input_line);
 if (!cin.eof())
 line_count++;
 };
 sec = (int) time(NULL) - start;
 cerr << "Read " << line_count << " lines in " << sec << " seconds.";
 if (sec > 0) {
 lps = line_count / sec;
 cerr << " LPS: " << lps << endl;
 } else
 cerr << endl;
 return 0;
}
// Compiled with:
// g++ -O3 -o readline_test_cpp foo.cpp

Python Equivalent:

#!/usr/bin/env python
import time
import sys
count = 0
start = time.time()
for line in sys.stdin:
 count += 1
delta_sec = int(time.time() - start_time)
if delta_sec >= 0:
 lines_per_sec = int(round(count/delta_sec))
 print("Read {0} lines in {1} seconds. LPS: {2}".format(count, delta_sec,
 lines_per_sec))

Here are my results:

$ cat test_lines | ./readline_test_cpp
Read 5570000 lines in 9 seconds. LPS: 618889
$ cat test_lines | ./readline_test.py
Read 5570000 lines in 1 seconds. LPS: 5570000

I should note that I tried this both under Mac OS X v10.6.8 (Snow Leopard) and Linux 2.6.32 (Red Hat Linux 6.2). The former is a MacBook Pro, and the latter is a very beefy server, not that this is too pertinent.

$ for i in {1..5}; do echo "Test run $i at `date`"; echo -n "CPP:"; cat test_lines | ./readline_test_cpp ; echo -n "Python:"; cat test_lines | ./readline_test.py ; done

Test run 1 at Mon Feb 20 21:29:28 EST 2012
CPP: Read 5570001 lines in 9 seconds. LPS: 618889
Python:Read 5570000 lines in 1 seconds. LPS: 5570000
Test run 2 at Mon Feb 20 21:29:39 EST 2012
CPP: Read 5570001 lines in 9 seconds. LPS: 618889
Python:Read 5570000 lines in 1 seconds. LPS: 5570000
Test run 3 at Mon Feb 20 21:29:50 EST 2012
CPP: Read 5570001 lines in 9 seconds. LPS: 618889
Python:Read 5570000 lines in 1 seconds. LPS: 5570000
Test run 4 at Mon Feb 20 21:30:01 EST 2012
CPP: Read 5570001 lines in 9 seconds. LPS: 618889
Python:Read 5570000 lines in 1 seconds. LPS: 5570000
Test run 5 at Mon Feb 20 21:30:11 EST 2012
CPP: Read 5570001 lines in 10 seconds. LPS: 557000
Python:Read 5570000 lines in 1 seconds. LPS: 5570000

Tiny benchmark addendum and recap

For completeness, I thought I'd update the read speed for the same file on the same box with the original (synced) C++ code. Again, this is for a 100M line file on a fast disk. Here's the comparison, with several solutions/approaches:

Implementation	Lines per second
python (default)	3,571,428
cin (default/naive)	819,672
cin (no sync)	12,500,000
fgets	14,285,714
wc (not fair comparison)	54,644,808

Answer*

### tl;dr: Because of different default settings in C++ requiring more system calls.

By default, `cin` is synchronized with stdio, which causes it to avoid any input buffering. If you add this to the top of your main, you should see much better performance:

 std::ios_base::sync_with_stdio(false);

Normally, when an input stream is buffered, instead of reading one character at a time, the stream will be read in larger chunks. This reduces the number of system calls, which are typically relatively expensive. However, since the `FILE*` based `stdio` and `iostreams` often have separate implementations and therefore separate buffers, this could lead to a problem if both were used together. For example:

 int myvalue1;
 cin >> myvalue1;
 int myvalue2;
 scanf("%d",&myvalue2);

If more input was read by `cin` than it actually needed, then the second integer value wouldn't be available for the `scanf` function, which has its own independent buffer. This would lead to unexpected results.

To avoid this, by default, streams are synchronized with `stdio`. One common way to achieve this is to have `cin` read each character one at a time as needed using `stdio` functions. Unfortunately, this introduces a lot of overhead. For small amounts of input, this isn't a big problem, but when you are reading millions of lines, the performance penalty is significant.

Fortunately, the library designers decided that you should also be able to disable this feature to get improved performance if you knew what you were doing, so they provided the [`sync_with_stdio`](http://en.cppreference.com/w/cpp/io/ios_base/sync_with_stdio) method. From this link (emphasis added):

> If the synchronization is turned off, the C++ standard streams are allowed to buffer their I/O independently, **which may be considerably faster in some cases**.

Draft saved

Draft discarded

Edit Summary*

Cancel

182

This should be at the top. It is almost certainly correct. The answer cannot lie in replacing the read with an fscanf call, because that quite simply doesn't do as much work as Python does. Python must allocate memory for the string, possibly multiple times as the existing allocation is deemed inadequate - exactly like the C++ approach with std::string. This task is almost certainly I/O bound and there is way too much FUD going around about the cost of creating std::string objects in C++ or using <iostream> in and of itself.

Karl Knechtel
– Karl Knechtel

2012年02月21日 03:34:32 +00:00
Commented Feb 21, 2012 at 3:34
71

Yes, adding this line immediately above my original while loop sped the code up to surpass even python. I'm about to post the results as the final edit. Thanks again!

JJC
– JJC

2012年02月21日 03:45:25 +00:00
Commented Feb 21, 2012 at 3:45
73

Note that sync_with_stdio() is a static member function, and a call to this function on any stream object (e.g. cin) toggles on or off synchronization for all standard iostream objects.

John Zwinck
– John Zwinck

2015年01月21日 01:16:26 +00:00
Commented Jan 21, 2015 at 1:16
Note that it may be possible to call std::cin.rdbuf().pubsetbuf(_my_large_buffer_, _some_large_number_) to reduce the number of IO calls. But pubsetbuf does nothing in the base class basic_streambuf, and although std::cin necessarily uses a derived streambuf, it is of an opaque type that is completely platform-dependent, and we can't guarantee that the library implementer added the behavior to make it useful. YMMV.

Spencer
– Spencer

2025年06月17日 13:05:26 +00:00
Commented Jun 17, 2025 at 13:05

Add a comment |

How to Edit

Correct minor typos or mistakes
Clarify meaning without changing it
Add related resources or links
Always respect the author’s intent
Don’t use edits to reply to the author

How to Format

create code fences with backticks ` or tildes ~
```
like so
```
add language identifier to highlight code
```python
def function(foo):
print(foo)
```
put returns between paragraphs
for linebreak add 2 spaces at end
_italic_ or **bold**
indent code by 4 spaces
backtick escapes `like _so_`
quote by placing > at start of line
to make links (use https whenever possible)

<https://example.com>

[example](https://example.com)

<a href="https://example.com">example</a>

formatting help »
answering help »

How to Tag

A tag is a keyword or label that categorizes your question with other, similar questions. Choose one or more (up to 5) tags that will help answerers to find and interpret your question.

complete the sentence: my question is about...
use tags that describe things or concepts that are essential, not incidental to your question
favor using existing popular tags
read the descriptions that appear below the tag

If your question is primarily about a topic for which you can't find a tag:

combine multiple words into single-words with hyphens (e.g. python-3.x), up to a maximum of 35 characters
creating new tags is a privilege; if you can't yet create a tag you need, then post this question without it, then ask the community to create it for you

popular tags »

default

CollectivesTM on Stack Overflow

Why is reading lines from stdin much slower in C++ than Python?

Answer*