c++ stack overflow unhandled exception with large array

Question

Right, having looked up this issue, I believe that it is caused because I have in my class definition wchar_t downloadedText[400000]; I have read solutions about how I should deal with this by using the new operator to assign the space, ie: wchar_t *downloadedText; downloadedText = new wchar_t[400000];

However I need to write instances of the class to a file, and assigning the variable like above appears to use pointers to point to data stored in a way that does not get written to my file. It is this same reason why I cannot use std::vector.

I have read that another option I may have is that I can increase the size of the 'stack'. I use VS2010 as my IDE, and I located in my project properties> Linker> System 'Stack Commit Size', 'Stack Reserve Size', 'Heap Commit Size' and 'Heap Reserve Size' fields, but I am unsure if this is how I can deal with my issue, and if it is, what to correctly set the appropriate fields to.

• Qaz: How are you accessing the element to write in each case?
• Qaz: Could you update your question with a code sample of the writing process for static arrays vs. pointer/vector?
• Interminable: fileOutput.write(reinterpret_cast<char *> (&myObjects[i]), sizeof(MyClass)); The process of writing to the file does not change when changing the members of the class (although if I wanted to read object data from the file after adding/removing member variables I would be unable to do so). I obviously cannot use std::vector in the same way that I cannot use std::string because they do not directly contain the data I want to write, instead they are more complex and contain pointers to the actual data I want, etc.
• deebee: how do you write when you have pointer?
• aaronqli: Why not to write a proper write/serialization function, or simply use some serialization library? stackoverflow.com/questions/234724/how-to-serialize-in-c

Answer 1

If you must do it this way... You could just write the array explicitly, after writing out the object. eg.

write((char*)&myObjects[i]), sizeof(MyClass));
write((char*)downloadedText, sizeof(downloadedText[0]) * 400000);

And to read it back in:

read((char*)&myObjects[i]), sizeof(MyClass));
downloadedText = new wchar_t[400000];
read((char*)downloadedText, sizeof(downloadedText[0]) * 400000);

However, this is very fragile and prone to errors. Overwriting objects in memory as done by the read is at best bad form except perhaps if you're using a struct created explicitly for that purpose which would typically contain only PODs. As a minimum, note that you have to set the downloadedText member field after the read writes over it.

• Interminable: My class does only contain PODs. In addition I would prefer to keep downloadedText as a member variable of said class.

Answer 2

You can allocate the whole object with new operator in system heap but not on stack. In this case you can write this object to file in the same manner and have no troubles with stack overflow.

• Interminable: I thought of this too, however it does not like the statement myObjects.push_back(new MyClass(0,0,TEXT("text0円"))); It gives me an error along the lines of error C2664: 'void std::vector<_Ty>::push_back(_Ty &&)' : cannot convert parameter 1 from 'MyClass*' to 'MyClass &&' 1> with 1> [ 1> _Ty=MyClass 1> ] 1> Reason: cannot convert from 'MyClass *' to 'MyClass' 1> No constructor could take the source type, or constructor overload resolution was ambiguous
• Max Lybbert: That error message means that myObjects is a std::vector<MyClass>, which you'll need to change to std::vector<MyClass*>.
• Max Lybbert: Thinking more about this, if you put your objects into a std::vector<MyClass>, then they're already allocated on the heap and off the stack.

Answer 3

Yes, you can increase the stack size in Visual Studio with the linker option /STACK. This linker option is also editable with the project properties Stack Reserve Size and Stack Commit Size. It is enough to set the reserve size, the commit size is optional. Nevertheless you should also be able to use std::vector.

• Interminable: I did initially try this without any success. However I then set it to a more insane value of 1024000000 (1GB?) as I didn't realise it appeared to be requesting a figure in bytes. I cannot write std::vector class members to my file because it is not POD. Unfortunately I can only store around 120 entries before getting some sort of bad alloc error. In addition it starts using a fair amount of memory, which is a little annoying as I'd prefer it to not use the memory for all 400000 characters unless it has to. ie if I'm not storing anything it should hopefully be taking little memory.

Answer 4

It sounds like you're fine with your current serialization strategy (make the object POD, and write it as a POD value). In that case, your question really is "how can a keep these objects from taking up too much space on the stack?" The answer: don't put them on the stack.

Allocate them with new. Or, preferably, wrap them in smart pointers of some kind.

---

You have string-like data. It so happens that C++ offers a string class. In fact, since you're talking about wchar_t characters, you want to look at std::wstring (in the <string> header).

• Interminable: I cannot use std::string as a class member because it is not POD. I will have issues trying to write it to the file in binary mode.
• Max Lybbert: You'll only have issues if you try to write the containing struct or class to the file as if it's POD. If you write each element separately, it's trivial to use std::wstring.
• Interminable: I can write instances of my object just fine provided it's got only POD types like int, char, etc, and arrays declared with their exact length in the class, ie char myArray[200] Trying to write ONLY an std::string or std::wstring does not work because they do not themselves contain the data I want, they contain pointers to it, etc.