String building performance in Java

Question

I have a performance problem with String Handling. I have a log file (simple text file) that I have to manipulate and performa several changes in text.
So the program adds line by line in one huge String. Basically it do.

while (not_finished) { 
 // create new stringAdd; 
 stringResult=stringResult + stringAdd + "\n"; 
} 
// output to a textArea in window
textArea.setText(stringResult);

Now the performances for this is horrible, so I upgraded to StringBuilder

StringBuilder result= new StringBuilder(); 
while (not_finished) { 
// create new stringAdd; 
result.append( stringAdd +"\n"); 
} 
// output to a textArea in window
textArea.setText(result.toString()); 

This is much faster. String once added to result will not be changed. The problem is not with the performances when there are more than 400,000 lines (one line has from 1 to 70 characters).

How to increase performances of building String? Do you have any idea?

• Simon Verhoeven: you're still creating new Strings, do: result.append(stringAdd).append("\n")
• tudoricc: I think you can decrease the time by makind the string of the file faster,am I right?>
• Thomas Jungblut: You read that from a file? Are you using a BufferedReader? Is it buffer size large enough?
• durron597: Are you just appending to the file or are you modifying it?
• Demosten: Aha I'll try this one result.append(stringAdd).append('\n'); and test again.

Answer 1

Two things can be improved. You are still concatenating strings inside the loop, so you can try:

result.append(stringAdd).append('\n'); 

If you know the size of the string beforehand, you can minimize the number of internal buffer resizing:

// expecting 30k characters:
StringBuilder result= new StringBuilder(30_000); 

• Demosten: Thanks, Regarding new StringBuilder(30_000) is this fix size of the StringBuilder or it is just for orentation? I can quite easily estimate the size of the string +/- 10%
• Thomas Jungblut: @Demosten this is just a hint for the internal buffer size, it will still grow if you put more into it.
• Demosten: Houray! The speed is increased doubled! For 440,000 lines, program now is finished for 9 seconds and before was over 30 seconds! THANKS

Answer 2

To just read a text file, there is absolutely no need to parse the file line by line in the first place. The right tool for the reading part is InputStreamReader, that will provide the translation from file encoding to characters. To Speed up the actual file reading to reasonable performance, put a BufferedInputStream in between the actual FileInputStream and the InputStreamReader.

So to set up the file for reading, in principle you nest three sources like this:

InputStream is = new FileInputStream(...);
InputStream bi = new BufferedInputStream(is);
InputStreamReader reader = new InputStreamReader(bi, (CharSet));

The actual reading is then a simple loop, without any fancy logic:

StringBuilder buffer = new StringBuilder();
// possible optimization would be: new StringBuilder((int) file.length());
int c = reader.read();
while (c >= 0) {
 buffer.append((char) c);
 c = reader.read();
}

(Error handling and resource management omitted, in reality this will need try/catch to deal with the possible exceptions)

Since the text file is already containing line feeds (naturally) there is no reason to break it into lines and then put the lines back together.

At the end just do one simple toString() for the buffer and there's your String:

 String contents = buffer.toString();

This method will not create any intermediate objects per read (StringBuilder adjusting its capacity aside). Its complexity is basically O(N), meaning time needed will increase only linearly with file size.

• Demosten: The base is input from a 2 log files. since there is a lot of changes and searches result file lines are all mixed up compared to input file. I can not jsut read from the file, make chage and append to StringBuffer. That's why I use String[] and Integer[] to parse input files. But the hint for new StringBuilder((int) file.length()) is really good. It would use it and test it.