Java vs. C++ Performance: FAQ

By saying "a few low-level benchmarks" the implication is that there is some body of other benchmarks that is not being discussed.

In fact the paper surveys most or all of the benchmarks available. The cited benchmarks 1 and 2 are to my knowledge the only ones in existence that use real algorithms as opposed to invented (and questionable) microbenchmarks. All of the currently available multi-algorithm benchmarks, whether real algorithms (1,2) or microbenchmarks (3-5) give similar results.

There are no collections of benchmarks of "large object-oriented programs" available in both C++ and Java versions, and almost certainly there never will be, so this statement is speculation.

Some isolated "real" programs have been translated from C++ to Java, e.g. FreeTTS; in this and the other case I'm aware of the performance of the Java version is said to be better than the original.

It's easy to find short programs for which C++ beats Java, and other short programs for which Java beats C++, just as there are programs for which one C++ compiler does much better than another.

To be a meaningful language comparison you should report several real algorithms, if possible, running on several problem sizes, and use several compilers. See the Cache/Memory effects section of the paper for why running on different problem sizes is important. The "almabench" benchmark referenced as [3] in the paper is a good example of a poor benchmark in this regard -- the trig library change from jdk1.3 to 1.4 affected the benchmark by a factor of two... meaning that this benchmark is mostly testing trig functions, and not telling us much about the languages in general.

Actually I've seen people use C++ templates in ways that make me agree that C++ can be elegant and expressive. C++ without templates... I used it for years, but in retrospect it seemed like slowly building Frankenstein's monster, over and over.

Ok, I need to vary my phrases here. The remarks are all real comments that have been sent to me multiple times, but I'm paraphrasing.

Concerning this repeated "blowing away": what are we talking about? A 20% speed difference? That's the speed difference you could get if you switch from one C++ compiler to another. A 50% difference? That's well within the overhead of using something like STL in some cases (see table in the paper). You shouldn't be using C++ if that's important to you.

Email me if you can show more than a factor of two speed difference across several algorithms and problem sizes.