Semantic Differences, Microsoft v. Microsoft

While comparing the behavior of various versions of old Microsoft C compilers, I tried building a trivial hello-world type program with CL.EXE from Microsoft C/C++ 7.0 (March 1992) running on top of a 32-bit Windows Server 2003. This seemingly trivial task failed:

Command line error D2018 : cannot create linker response file

A quick search revealed that rather predictably, I wasn’t the first person hitting this problem. And as usual, there were only questions, not answers.

MS C/C++ 7.0 was Microsoft’s first compiler with C++ support included, but that’s not what made it special. It was Microsoft’s first DOS/Windows 3.x compiler which required a 386 host and used a 32-bit DOS extender.

Looking at the C/C++ 7.0 executables a bit closer, it’s apparent that these are in fact 32-bit PE modules, with a DOS extender in their respective DOS stub. The catch is that due to their age (released more than a year before NT 3.1), the PE modules are different enough that no NT release can use them. Hence the compiler must be run through NTVDM as if it were a normal DOS executable.

Now, given that when NT 3.1 was released (1993), Microsoft C/C++ 7.0 was still a current product, one would expect that the compiler would run under NT 3.1... and indeed it did! MS C/C++ 7.0 also works under NT 3.50 (1994), but it no longer functions under NT 3.51 (1995). The compiler fails with the above D2018 error message in NT 3.51 and all subsequent releases. At that point, Visual C++ 1.0/1.5 had taken over and C/C++ 7.0 was more or less obsolete, so perhaps the problem slipped under the radar, but why did the compiler stop working at all? What changed?

The reason why the compiler fails on newer NT versions is its questionable use of the seek function (INT 21h/42h). In DOS, seeking to negative offsets succeeds, although subsequent read/write operations will fail. In NT (Win32 API), seeking to negative offsets fails immediately.

The Win32 API behavior has been consistent since NT 3.1; however, the NTVDM behavior has not. In NT 3.1 and 3.50 NTVDM, seeking to negative offsets succeeds, while in NT 3.51 and later it fails.

Clearly this is a bug which somehow crept into NTVDM, since NTVDM is supposed to emulate DOS semantics. Why Microsoft C/C++ 7.0 wants to seek to negative offsets (on a temporary file) is not clear, but it’s apparent that this is why MS C/C++ 7.0 fails on NT 3.51 and later.

Seeking to negative offsets is a questionable practice and its semantics changed over the years. It is especially problematic when seeking is implemented via the classic lseek() routine:

off_t lseek(int fildes, off_t offset, int whence);

While it makes good sense to for the offset argument to be negative when seeking backwards, there simply are no negative offsets in a regular disk file. Hence the return value indicates a failure when the final file position is negative. If seeking to negative offsets is allowed, it becomes difficult to distinguish between failures and successful seeks to negative offsets.

But it wasn’t always so. The Open Group standard provides some hints, and DOS is of course very old—the seek functionality first appeared in DOS 2.0 and was implemented sometime in 1982 or so, with the semantics being derived from then-current XENIX (likely UNIX Version 7). Moreover, the DOS API indicates errors through the carry flag, which means that there is no trouble distinguishing failed seeks from successful seeks to negative offsets.

There does not appear to be any fix for MS C/C++ 7.0. It’s unclear why the NTVDM semantics changed in NT 3.51, but it is understandable that C/C++ 7.0 was no longer important; extremely few applications would have been affected by the change, though it’s ironic that Microsoft’s own compiler was one of them.

The RBIL notes the difference in behavior between DOS and NT when seeking to negative offsets, although it incorrectly implies that all NT versions behave the same. The OS/2 MVDM, needless to say, behaves correctly and executes Microsoft’s CL.EXE compiler driver without trouble.

MS C/C++ 7.0, a victim of MS C/C++ 7.0?

But wait... there’s more to the story. Microsoft was always aware of the semantic differences between DOS and other operating systems (which included OS/2). The lseek() routine in older Microsoft compilers—at least in C 4.0 (1986), C 5.1 (1988), and C 6.0 (1990)—explicitly guarded against seeking to negative offsets. When a backward seek was attempted, the library first checked whether it would end up at a negative offset, and if so, the call failed.

A new feature of the C/C++ 7.0 run-time library was the LSEEKCHK.OBJ file which, when linked in, forced the lseek() routine to return an error if an attempt was made to seek to a negative file offset on DOS, just like earlier versions of Microsoft C. The exact same error checking in the lseek() routine was present, but the big difference from the previous compiler versions was it was now turned off by default. The Microsoft Visual C++ 1.0 and 1.5 run-time libraries behaved the same as MS C/C++ 7.0.

At this point, it is only a speculation that CL.EXE shipped with Microsoft C/C++ 7.0 used the compiler’s own library with changed lseek() semantics. Detailed analysis was not made, but it seems at least likely.

Why MS C/C++ 7.0 introduced this change at all is unclear, and the documentation does not provide any hints. It seems odd in light of the fact that neither OS/2 nor NT allowed seeking to negative offsets anyway, and older versions of Microsoft’s run-time library forced the more portable semantics. But it is certainly ironic that the CL.EXE tool shipped with C/C++ 7.0 itself ran afoul of these semantic differences and bugs introduced in another Microsoft product.

This entry was posted in DOS, Microsoft, Virtualization. Bookmark the permalink.

14 Responses to Semantic Differences, Microsoft v. Microsoft

  1. Calvin says:

    Funny you mention – I’ve been looking at the old Visual C++ stuff and pre-95 32-bit Windows things in general.

  2. Michal Necasek says:

    The MS C/C++ 7.0 compiler is certainly unique in its use of a DOS extender, and the way 32-bit PE binaries are run on NT as 32-bit extended DOS apps. Talk about jumping through of a lot of extra hoops.

    The version 8.0 compiler (aka Visual C++ 1.0/1.5) is not too different, but it’s new enough that the PE binaries run on NT directly, without the DOS extender and NTVDM sandwiched in between.

  3. Yuhong Bao says:

    I wonder if the problems with seeking to negative offsets was why they restricted file size to 2GB by default when FAT32 support was added to MS-DOS 7.1.

  4. Michal Necasek says:

    That could very well be. I don’t think the old DOS API provides any sensible way to distinguish between a seek to a negative offset vs. a seek beyond 2GB.

  5. dosfan says:

    DOS doesn’t provide a way to distinguish between a seek to a negative offset vs. a seek beyond 2GB, it didn’t have to since with FAT16 the maximum partition size was 2GB (FAT16 allowed up to 65,524 clusters and the maximum cluster size was 32KB hence the 2GB limit) so creating a file with a size >= 2GB wasn’t possible.

  6. Andreas Kohl says:

    Windows Server 2003 shipped without an OS/2 subsystem. So it’s not possible to use the optional OS/2 hosted tools. Or at at least unless you upgraded from Windows NT or 2000? There are fixes from Microsoft (September 1992) for the DOS tools you can find at Hobbes.

  7. Can the March ’92 compiler be run on the contemporary NT builds without having to run it in NTVDM?

  8. Yuhong Bao says:

    I think I forgot to mention the matter of NT-style 64K cluster 4GB FAT16 volumes. While Windows NT 4.0 was not supporting FAT32 and continue to use them, it took until Windows 98 before they put a check in ScanDisk for them. Even Windows Me don’t seem to boot properly on them even after installation with setup /is.

  9. Yuhong Bao says:

    Though DOS did not officially support them and in fact the sector number in the FCB structure even in DOS 5.0 was only 22-bit because they took the top two bits for attributes.

  10. Yuhong Bao says:

    Interestingly BTW even MS’s FAT32 converter never supported 4GB FAT16 volumes.

  11. Yuhong Bao says:

    https://groups.google.com/d/msg/comp.os.ms-windows.nt.pre-release/Mg5ALe5633w/W-pQ4Cv_d7MJ called the lack of FAT32 support in NT 4.0 “strictly a matter of development and test resources and our existing release commitments”. In the meantime, ScanDisk from Windows 95 OSR2 had problems with 64K cluster FAT16 partitions that caused memory corruption with them to the point of a “Cannot load COMMAND.COM, system halted” message.

  12. Yuhong Bao says:

    Interestingly, NT 4 setup is able to display partitions as “Windows 95 (FAT32)” and even recognize the new LBA partition types introduced in Windows 95.

  13. Michal Necasek says:

    I guess that was a lot easier than adding FAT32 support 🙂

  14. Yuhong Bao says:

    BTW, MS-DOS 7.1 CHKDSK would work on 64K cluster FAT16 volumes, but would report false “errors” and happily corrupt the volume if you used CHKDSK /F.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.