I have some generated D code in a single source file ~ 30K LOC. On Windows I can compile it but cannot link it. (On Linux, there's no problem.) Some web search leads me to the following, which seems related to the problem. I wonder if OPTLINK can be fixed now. http://www.digitalmars.com/d/archives/digitalmars/D/bugs/306.html 2004年5月30日 20:54:18 -0700 "Walter" <newshound xx digitalmars.com> writes: This happens when there are more than 16,000 fixups in a single .obj. Unfortunately, this isn't fixable at the moment. http://www.digitalmars.com/d/archives/digitalmars/D/bugs/8283.html http://www.digitalmars.com/d/archives/digitalmars/D/bugs/306.html http://www.digitalmars.com/d/archives/digitalmars/D/41323.html
While important, I don't think it's a blocker. 30,000 lines of code in one module is unusually large, and can be split into smaller source files.
(In reply to comment #1) > While important, I don't think it's a blocker. 30,000 lines of code in one > module is unusually large, and can be split into smaller source files. > It is a blocker at least for one user :-) OK, let me elaborate a little more where I come from, and all the problems I've encountered: When I started to use D on a hobby project, I organize the source code into modules as everyone typically does, but because of bug 386: http://d.puremagic.com/issues/show_bug.cgi?id=386 I was so frustrated that I cannot get my code compile separately, and the compiler error message is useless, each time I spent hours and hours on them and cannot get them fixed. So I write some script to concat all my source files (togather with some generated ones) into a single huge .d file, and comment out all the 'module' and 'import' statements. Then everything works. After bug 386 is fixed. I tried to compile each source file separately again, now I no longer have 'import conflicts' problems, but I still cannot get many of the file compiled, the compiler just hangs: I guess it run into some infinite loop. But unfortunately I cannot isolate the issue to send you a test case. (search "infinite loop" on digitalmars.com will show some user report: e.g. http://www.digitalmars.com/pnews/read.php?server=news.digitalmars.com&group=digitalmars.D&artnum=42668 But I'm not sure, it's the same issue as mine.) Then I went back to my original approach to generate a huge source file (and it works); but maybe as my project grows, at certain point, it stops to link on Windows. (On Linux, it's fine). And this is a OPTLINK problem. I hope you can fix it. Or as a workaround: if the generate obj is too large (> 16,000 fixups, I really don't understand what does it mean), can the compiler split the obj into several obj files? e.g. foo1.obj, foo2.obj, ... If you ask the user the split the generated source files, first it's tedious, and it maybe run into some compiler problem, like the import one again. So please fix this issue: 1) remove the (> 16,000 fixups) constraint. It is a bug anyway. 2) or, the compiler automatically generate multiple .obj files, if (> 16,000 fixups) Thanks.
Another "infinite loop" user report, maybe related: http://www.digitalmars.com/pnews/read.php?server=news.digitalmars.com&group=D.gnu&artnum=2134
Let's not clutter this report up with more than one issue. Please make sure a seprate bug is filed for the infinite loop issue. This report is strictly about th eoptlink fixup limit. Walter, can you please clarify where the limit exists and if it's authored by you or microsoft? I don't do any windows development so I'm unfamiliar with the pieces. Filer, would you please attach to this report an example .d file that fails with the error message? Rather than discussing why or how this programming practice might be ill advised, can we talk about where the problem is and how or if it can be fixed. Each barrier to the use of D, whatever the reason, is a barrier that should at least be addressed in a FAQ and not dismissed as a bad practice. Thanks, Brad
any chance this will be fixed soon? it's a bug anyways.
can this one be fixed before 1.0 (Jan 01 2007)?
No update?
Tried the new linker: http://ftp.digitalmars.com/Digital_Mars_C++/Patch/beta.zip In release mode (-d -O -inline -release), it works, and generate the exe file. But when running the program (using wxd, 0.09 version), the main UI window doesn't show up. In debug mode (-d -g -gc -debug), it errors out: Unexpected OPTLINK Termination at EIP=00412793 GDC (under MingW32) works in both debug & release mode on the same system.
I think if you can post your object file which makes optlink fails could let Walter easier to debug this issue
(In reply to comment #1) > While important, I don't think it's a blocker. 30,000 lines of code in one > module is unusually large, and can be split into smaller source files. I believe I'm running into this too, and my files have a reasonable number of lines. However, the code is structured so that *everything* is templated, and doesn't get instantiated until main.d. This is a very common way to write heavily generic code. The actual library in question in which this is happening OpenMeshD (http://www.dsource.org/projects/openmeshd), which is a port of a C++ template library to D. Why I think I'm seeing this same problem: 1) Optlink is dying and popping up a MsgBox saying "Unexpected OPTLINK Termination at EIP=0044C37B" The EIP number is different that the one previously reported, but I'm guessing that doesn't matter (just a random memory address?). 2) The thing that triggered the crashing was adding a function with a big block of type-checking case statements like this: else if (ti==typeid(VectorT!(byte, 2))) { renderer_ = new PropRenderer!(VectorT!(byte, 2)); } So it's causing a lot of templates to be instantiated, which could be pushing the number of "fixups" in main.d over the 16K limit. 3) If I start commenting out those "else if" lines one by one, eventually compilation will work again. It doesn't seem to matter which lines I comment out. If I get rid of enough of them, then it compiles. 4) Compiling with -O -release instead of -g -debug allows me to uncomment a few more of the lines. 5) "main.obj" is between 1-2MB even in release mode. All that seems to add up to an OPTLINK upper limit bug. If not with # of fixups then with something. I'm using DMD 1.023 on Windows XP. I've checked the code into the OpenMesh/D repository in its buggy state so it can be checked out for debugging. To reproduce, check out revision 111 of OpenMesh from the svn repository: If you don't have dsss installed, install it first (http://www.dsource.org/projects/dsss) Then: $ mkdir scratch $ cd scratch $ svn co -r111 http://svn.dsource.org/projects/openmeshd/trunk . $ cd OpenMeshD\OpenMesh\Apps\GLViewer $ dsss net deps $ dsss build -version=OPTLINKIsMyFriend If there are complaints about Helix things, you may need to go to the Helix dir and do a "dsss build && dsss install"
Upgrading the severity because I just ran into this again adding to an unrelated part of the main .d file. Moving the added code to a different file fixed it for now but that may not always be so easy to do if the code needs to be in the generic, templated part of things. Then it seems it will end up instantiating in the main .obj file no matter where I put it. Also further rearrangement of the code started triggering the "previous definition different blahblah_initZ" bug. So, yeh, this is a bad one, and it affects more than just code with gigantic source files. It appears to affect any template library with a few tens of thousands of lines of code.
Fun fun! Just hit this one again. See thread digitalmars.com digitalmars.D:67093
Changed title to include keyword "fixups" for easy finding.
Probably a duplicate of #2436 or/and #1439
(In reply to comment #14) > Probably a duplicate of #2436 or/and #1439 > Er, I think you mean #2436 and/or #1439 are probably duplicates of this one.
Exactly
Just got bitten by this one too. Voted!
Are you (In reply to comment #17) > Just got bitten by this one too. Voted! > Unfortunately I think the only real solution for this is a new linker. Just out of curiosity, in your case is it the lots of templates getting instantiated in a single file thing? Or did you find some other way to trigger it?
(In reply to comment #18) > Are you (In reply to comment #17) > > Just got bitten by this one too. Voted! > > > > Unfortunately I think the only real solution for this is a new linker. > Just out of curiosity, in your case is it the lots of templates getting > instantiated in a single file thing? Or did you find some other way to trigger > it? > I'm pretty sure that's what's causing the issue I have. It compiles with n template instantiations, and fails with n + 1. Doesn't matter which ones I remove or add. And I have a *lot* of them in one file. And they're the deeply nested variadic code-generating kind. I do horrible things.
> Unfortunately I think the only real solution for this is a new linker. I just thought there maybe a simple solution to implement: add a switch to the DMD compiler, e.g "-mo", means output multiple object files. So for file foo.d the compiler will output foo1.o foo2.o foo3.o .... each only contains a limited number of fixups, so in this way, the user can add all these object files to the linker command line. This should work. What do you think? Walter?
Actually even without such switch, the compiler should split the object files " when there are more than 16,000 fixups in a single .obj" -- after all why the compiler should generate such obj file, if the linker cannot link it, while the compiler have all the necessary info to take the right action. (I would consider this is a minor compiler bug in this sense ;-) And if this happened with no user option specified, the compiler should also print a friendly message, saying: "too many fixups, multiple object file foo1.obj, foo2.obj ... fooN.obj generated, please modify your link command". (In reply to comment #20) > > Unfortunately I think the only real solution for this is a new linker. > > I just thought there maybe a simple solution to implement: add a switch to the > DMD compiler, e.g "-mo", means output multiple object files. > > So for file foo.d the compiler will output > foo1.o > foo2.o > foo3.o > .... > > each only contains a limited number of fixups, so in this way, the user can add > all these object files to the linker command line. This should work. > > What do you think? Walter? >
I disagree about multiple output obj files from one source. This would make linking much harder (e.g. - compilation using makefile). I wouldn't like the build process to depend from some third party tool that would parse compiler output and adjust my linking options.
(In reply to comment #22) > I disagree about multiple output obj files from one source. This would make > linking much harder (e.g. - compilation using makefile). I wouldn't like the > build process to depend from some third party tool that would parse compiler > output and adjust my linking options. > Then have a compiler switch then "-mo+" means on, and "-mo-" turn it off. I just wonder if you have ever bitten by this bug, what else you can do? The user have control of his own build process, but how the user overcome this linker bug?
The compiler already has a switch to generate multiple obj files from one source file: -multiobj. I use it for debugging. But it would be a royal pain to use if there's a lot. A better way is to compile to a library with the -lib switch. That splits the module into a zillion obj files, and stuffs them all into the library. Then, link with the library (the main() program will still have to be separate).
(In reply to comment #24) > The compiler already has a switch to generate multiple obj files from one > source file: -multiobj. I use it for debugging. But it would be a royal pain to > use if there's a lot. Yes, tried that: because there are sequence dependency on the order of the object file, one cannot give "foo*.obj", the linker complains lots of unresolved symbol. I just wonder if the linker can be changed to be ".obj" order insensitive. Yes, it's a royal pain to let the user try to figure out the correct ".obj" sequence, but the linker should have all the information to find the unresolved symbols in those .objs. > > A better way is to compile to a library with the -lib switch. That splits the > module into a zillion obj files, and stuffs them all into the library. Then, > link with the library (the main() program will still have to be separate). > Also tried, when linking with the generated .lib file, it also gives "Unexpected OPTLINK Termination" error, not sure it's the same one. But I guess so.
If someone can attach to this an object file with too many fixups, that would be helpful. Or email it to me.
Open a windows command prompt and run the following line: for /L %i in (1,1,17000) do @echo void func%i(){} >> test.txt This creates test.txt with 17,000 functions. Then create test.d in the same dir: module test; mixin(import("test.txt")); void main() {} Now, compile *with the following commandline*. dmd test.d -g -J. This causes OPTLINK v8.00.1 to crash with EIP=00412793 as sa mentioned. For more testing, delete test.txt and run the shell line above with different values. 16000 does not crash. Interestingly, 16500 neither succeeds nor fails; the linker apparently hangs and does not use any CPU. If you want a real-world test, you're going to have to install some third-party D libraries.
I know you responded with a "thanks" on the newsgroup, Walter, but.. is there any followup? Were you able to reproduce it? Have you investigated the problem? Is there any chance of it being fixed? Is it a fundamental limit that can't be changed? Is this on the back burner? Is this going to magically appear in the next release without warning? Cmooon, communicate! What is the story? There are a lot of people who've run into this bug, a lot who want to see it fixed, and a simple repro. What more do you want or need?
Optlink is written entirely in rather impenetrable assembler code, and is resistant to understanding and modification. Hence, over the last few months I've been very slowly converting it to C, function by function. One might ask, why not convert it to D? The answer is that I don't have a good test suite for optlink, so I have to be very very careful to not make a mistake in the translation. That means do one function at a time, rebuild, and retest, which means the compiled C code has to match the segment, naming and calling conventions used in optlink. I made a custom version of the dmc compiler to do this. Also, C can be made to work without any runtime library support at all, and since optlink does not use the C runtime library, this is useful. Once it is in C and working, it will be trivial to translate it to D and start rewriting it. Anyhow, during this process I stumbled upon what the problem was. Optlink was apparently trying to account for some Borland obscure extension to the OMF. Remove this, and it works, although presumably it will no longer link Borland object files (who cares!). The fix will go out in the next update, if you need it sooner please email me.
(In reply to comment #29) > The fix will go out in the next update, if you need it sooner please email me. Holy-- Joy upon joys! This is REALLY good news. I can't tell you how happy you'll make a lot of people with this.
Fixed dmd 1.051 and 2.036
AltStyle によって変換されたページ (->オリジナル) / アドレス: モード: デフォルト 音声ブラウザ ルビ付き 配色反転 文字拡大 モバイル