Installing IBM OS/2 1.0 in a VirtualBox VM

Some time ago I wrote that IBM OS/2 1.0 and 1.1 cannot be installed in a VM due to the way it switches between real and protected mode. At the time I did not realize that there was another obstacle, namely IBM’s use of the undocumented 80286 LOADALL instruction.

For reasons that are not very clear, Microsoft’s editions of OS/2 used 386 instructions since before OS/2 1.0 was released, but IBM’s versions did not. When Microsoft’s OS/2 kernels detected that they were running on an 80386 (or newer) processor, they used 386-specific code to return from protected mode back to real mode. This was faster than the 286 method which required a CPU reset (OS/2 did not utilize the common method which relied on the PC/AT keyboard controller to reset the CPU and instead caused a processor shutdown through a triple fault, inducing a reset that way; the OS/2 method ought to be faster). Microsoft’s OS/2 kernels also didn’t use the LOADALL instruction when running on a 386.

IBM’s OS/2 1.0/1.1 kernels on the other hand didn’t care one bit about running on a 386, at least not on an AT-compatible machine—perhaps because at the time IBM had no AT-compatible 386 systems on the market or even in the pipeline. IBM’s OS/2 1.0 can still run on a 386 or newer CPU, but it requires a bit of help from the machine’s BIOS.

The calming Big Blue logo

The BIOS must properly support software that sets the CMOS shutdown status byte, resets the CPU (either by triple-faulting, through the keyboard controller, or by any other means), and regains control after the reset while minimally disturbing the machine state. That functionality is common enough because it was also used by DOS extenders and many OS boot loaders.

What’s less common is BIOS support for emulating a 286 LOADALL instruction. Even though the 80386 has its own undocumented LOADALL, a 486 or later CPU has nothing of the sort. However, a 386 or later can emulate enough of 286 LOADALL functionality to satisfy common uses of it, which includes things like old HIMEM.SYS, RAMDRIVE.SYS, or OS/2.

The VirtualBox BIOS has had support for LOADALL emulation for a while now, so it should be able to run IBM’s OS/2 1.0. And it is able to run it… but not install. Or not quite. Fortunately there’s a way around it…

When using the VirtualBox defaults for OS/2 1.x, a 500 MB IDE drive is chosen. OS/2 1.0 can work with that, but the FORMAT utility shipped with IBM OS/2 1.0 can not. The OS/2 installer can create a default partition (limited to FAT-12!) just fine, but trying to format it results in the following error:

FORMAT unsuccessful… but why?

“FORMAT unsuccessful” is not exactly self-explanatory. And good luck getting help from your dealer or the IBM representative that sold you IBM OS/2.

The INSTALL.LOG file is supposed to show the “condition that caused the error”, but really, it does not:

Formatting failed. Isn’t it obvious why?

In other words… the FORMAT utility failed, but the OS does not have the faintest clue what the problem might be.

After I failed to find any problems with disk access, I began suspecting that the old FORMAT might not like “modern” drive geometry. That would not be a first exactly, for example old DOS versions have a known bug that may cause them to fail on a hard disk with more than the then-common 17 sectors per track.

I was able to confirm that this is indeed the problem. The FORMAT utility in OS/2 1.0 works with 17 sectors per track common on MFM hard disks, and it also works with 26 sectors per track common on RLL hard disks. It even works with 40 sectors per track, but not 50 or more.

There are two ways around this in VirtualBox. The complicated one is creating a VHD or VMDK disk image with appropriate physical geometry and letting OS/2 format the disk. The significantly easier workaround is letting OS/2 create a partition and fail during format, then use DOS to format the partition; once the partition is formatted by DOS, the OS/2 installer can be restarted, told to not format the C: drive, and happily continue.

Again, the problem is only in the FORMAT utility shipped with IBM’s OS/2 1.0, the operating system as such has no trouble with IDE disks that use 16 heads and 63 sectors per track.

The steps to install IBM OS/2 1.0 in VirtualBox (version 6.1.x or later ought to work) are as follows:

  1. Create a VM using the defaults for OS/2 1.x
  2. If not already done, patch the installation floppy image to avoid division by zero crashes
  3. Start the installer, let it create a partition, and fail during format
  4. Use more or less any DOS version to format the C: drive
  5. Restart the OS/2 installer, do not format the partition, and install the OS

Mouse Extras

For extra points, it is possible to get working mouse support. IBM OS/2 1.0 does not ship with a driver for PS/2 mice on PC/AT compatible systems. But it is possible to borrow one from a Microsoft OS/2 edition which supports the combination. One possible donor is Nokia OS/2 1.1.

To accomplish this, it’s easiest to install OS/2 1.0 with more or less any mouse driver. It won’t work, but will set up the CONFIG.SYS entries. Then insert Disk 1 from Nokia OS/2 1.0, start an OS/2 command prompt, switch to the C:\OS2 directory, and run the following command:

a:\unpack a:\mousea05.sy@

This will copy and uncompress the MOUSEA05.SYS driver. After that, it is necessary to either modify CONFIG.SYS to use MOUSEA05.SYS instead of the original driver, or copy MOUSEA05.SYS over the driver set up by the OS/2 installer.

Those unable or unwilling to use EDLIN (the only text editor shipped with OS/2 1.0) can boot a DOS floppy and use any DOS editor to fix up OS/2’s CONFIG.SYS.

When everything is done, OS/2 should reboot to the this screen:

See that mouse cursor?

Admittedly this is not quite as straightforward as installing IBM OS/2 1.0 in an emulator that mimics an IBM PC/AT. But it’s doable, and the ancient OS runs blazingly fast on a modern computer.

This entry was posted in OS/2, VirtualBox, Virtualization. Bookmark the permalink.

18 Responses to Installing IBM OS/2 1.0 in a VirtualBox VM

  1. SweetLow says:

    >Then insert Disk 1 from Nokia OS/2 1.0, start an OS/2 command prompt, switch to the C:\OS2 directory, and run the following command:

    >a:\unpack a:\[email protected]

    Something went wrong…

  2. Michal Necasek says:

    Yes, something went wrong… but where? That’s not what I see on the site, even when I’m not signed in. I even saved the page and archive.org isn’t seeing that either. So where are you getting the “email protected” thing from?

  3. arpl says:

    I don’t know but:
    – with scripts enabled for os2museum.com I see:
    a:unpack a:mousea05.sy@
    – with scripts disabled I see:
    a:unpack a:[email protected]
    where [email protected] is a link pointing to
    https://www.os2museum.com/cdn-cgi/l/email-protection

    No need to approve my message(s). Just delete them.

  4. Chris M. says:

    Looks like it may be Cloudflare triggering its e-mail protection due to the @ sign in the filename.

    https://community.cloudflare.com/t/email-protected-is-shown-for-every-email-adress/14078

  5. Michal Necasek says:

    Ah, I see now. It’s triggered by disabling scripts.

  6. Michal Necasek says:

    And the detection is really dumb if it thinks that a string ending with ‘@’ is an e-mail address. Sigh.

  7. SweeetLow says:

    >It’s triggered by disabling scripts.
    Indeed, i prefer reading without scripts…

  8. zeurkous says:

    Scripts? What scripts? (lynxtard here…)

    Either way: cloudflare continues to give me loads of trouble, no matter
    what me sends as “User-Agent”, or whether me accepts cookies or not…

    It doesn’t seem to be a very good swervice.

  9. Joshua Rodd says:

    The PS/2 Model 80 doesn’t emulate 80286 LOADALL, according to various sources. How would (IBM) OS/2 1.0 or 1.1 run on an 8580?

    I’ve personally seen 1.2 EE and 1.3 EE running on an 8580, but never 1.0 or 1.1.

  10. Michal Necasek says:

    IBM announcement letter 287-498 (Nov 3, 1987) clearly states that Model 80 is supported in OS/2 1.0, specifically 8580-311. I’m certain the 386 mode switching code path was there, but only used on PS/2 machines, or more specifically ABIOS machines.

    Why IBM did it that way I’m not sure. I’m pretty sure their position was “there are no IBM AT compatibles with a 386 CPU, therefore that case is not relevant to us”.

  11. Bob Nelson says:

    I am fairly certain I remember running 1.0/1.1 on Model 80 machines (maybe even a Model 75 or two) during development of EE. It didn’t see a lot of use but I do remember it.

  12. Chris M. says:

    Did IBM intend that all 386+ machines use ABIOS at the time? That would likely be another attempt to steer things back their way and behind licensing restrictions in addition to Micro Channel. Of course hardly anyone was using OS/2 and the Microsoft versions supported plain old conventional BIOS machines anyway.

    The protected mode Advanced BIOS system would have been nice to have, yet somehow we were stuck with a Real Mode BIOS until the 2010s widespread adoption of UEFI.

  13. Michal Necasek says:

    I don’t think IBM exactly “intended” that (the cat was long out of the bag with the Deskpro 386), but I’m pretty sure it was true of IBM’s lineup in 1987-1988. And remember, IBM didn’t support OS/2 on non-IBM hardware at the time. If the hardware was compatible enough with IBM’s, great, but if not, you were supposed to get MS OS/2 through an OEM.

    ABIOS was a lot better than EFI in many ways. The run-time services of EFI are nearly nonexistent, and in many ways EFI significantly less capable than the old BIOS (accessing the disk through EFI at OS runtime? forget it, that would be too simple).

  14. Yuhong Bao says:

    I think the problems with accessing the disk as a EFI runtime service involve option ROMs.

  15. Joshua Rodd says:

    ABIOS offers protected mode disk access vis option ROMs just fine, although most ABIOS extensions ended up being loaded dynamically.

    Did anything use ABIOS besides OS/2 and the ABIOSDSK.SYS drivers in NT and 95?

  16. vbdasc says:

    @Michal Necasek:
    “For reasons that are not very clear, Microsoft’s editions of OS/2 used 386 instructions since before OS/2 1.0 was released, but IBM’s versions did not.”

    The curious thing is that the body of this article probably contains all the necessary hints to guess the most likely answer 🙂

    I’m guessing that IBM’s logic of 1987 (don’t confuse with normal logic 🙂 ) was like that. First, we assume that there are no 386-based AT machines, and will never be.

    ASSERT(PC for OS/2 install is either a 286 AT, or low-end 286 PS/2, or ABIOS-based PS/2)

    Then, the algorithm for returning to real mode is obvious.
    1. Check for ABIOS. If there is ABIOS, use its services and goodbye.
    2. There is no ABIOS, but CPU is therefore 286. Use LOADALL.

    Virtualbox fails the ABIOS check, so LOADALL286 is always used, without a check for 386 or higher CPU.

  17. Michal Necasek says:

    That is most likely it… in 1987, in IBM’s world, if you had no ABIOS you by definition couldn’t have a 386. That changed in OS/2 1.2, I’m not sure if IBM changed their mind or if enough code was rewritten and the more generic Microsoft approach won out.

  18. Vikki McDonough says:

    > (version 6.1.x or later ought to work)

    VirtualBox 6.0 and (I think) 5.2 also work.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.