Installing OS/2 1.x in a VirtualBox VM

Installing 16-bit OS/2 in a virtual machine ranges between "tricky" and "impossible", depending on the version of OS/2 and virtualization software used. In VirtualBox 4.0.8, things have moved further away from "impossible" and closer towards merely "tricky". Version 4.0.8 fixed problems with floppy emulation which were making installation difficult, but plenty of hurdles still remain. Most of those have little to do with virtualization and are simply a consequence of OS/2’s age—the PCs of the late 1980s were not quite like today’s PCs.

When creating a VM for OS/2 1.x, the chosen OS type does not matter much. The defaults are generally not suitable. The following settings must or should be modified:

  • The hard disk must be smaller than 512 MB. Sizes of 500 MB or less are usable (and 500 MB is still huge for 16-bit OS/2). Sizes greater than approximately 504 MB require geometry translation, which OS/2 1.x does not support.
  • A floppy drive must be added to the VM—obviously, since the installation media are floppies.
  • The emulated CD-ROM should be removed. There are no ATAPI CD-ROM drivers for 16-bit OS/2, and some versions of OS/2 take noticeably longer to boot when a CD-ROM is present.
  • The memory size assigned to the VM generally does not matter. The minimum (4MB) is enough, and 16-bit OS/2 cannot use more than 16 MB RAM.
  • Any virtualization mode should work—software, VT-x, or AMD-V.

In general, the difficulty of installing OS/2 increases with decreasing version number of OS/2—older versions are more picky. Let’s start from the easier to install versions and work our way back in time.

OS/2 1.30.1 or later, IBM or Microsoft

The following paragraph applies only to OS/2 1.30.1 and OS/2 1.30.2, not the original OS/2 1.3 release. Note that IBM often did not label the releases very clearly, and all 1.30.x releases display "Version 1.30" in the initial splash screen. A quick way to distinguish the original release is by the copyright date—the original 1.3 version shows 1990 copyright, while the updated versions show 1991.

Installation is straightforward as long as the above precautions are observed. For MS OS/2, either installation diskette A or B may be used. Either HPFS or FAT can be chosen for the system partition. Accepting defaults in the installer is recommended for users who do not have experience with OS/2 1.x, except that the mouse type must be manually set to PS/2. After feeding the requested floppies to the installation program, the initial setup is done and OS/2 happily boots from the virtual hard disk. Easy!

IBM OS/2 1.3

Installing the initial release of OS/2 1.3 is much harder because this and most earlier versions crash on machines faster than a 386:

To avoid confusing this issue with other potential problems, verify that it is in fact TRAP 0000 (not some other trap number), CX=0000, and AX=86A0. The contents of most other registers vary depending on host system configuration and the version of OS/2 in use.

The OS/2 keyboard driver contains timer calibration code which uses a simple software loop. On "fast" machines, the maximum number of iterations is reached before the timer ticks even once; note that due to this dependency, the OS does not always crash, and especially on slower host systems using software virtualization, the crash may not occur every time.

The crash is a division by zero (trap 0) on a DIV CX instruction where the CX register contains zero. The way around this problem is patching OS/2 and getting rid of the bad loop. The original code sequence looks like this:

 MOV DX, 3
L1:
 XOR CX, CX
L2:
 LOOP L2 ; loop 65,536 times
 DEC DX ; run the loop 3 times
 JNE L1
 CLI
 ... ; read current tick
 STI
 SUB CX, BX
 MOV AX, 1F4h ; 500
 MOV BX, 0C8h ; 200
 MUL BX ; 500 * 200 = 100,000
 DIV CX ; traps if CX = 0
 SHR CX, 1

This obviously became a problem as far back as 1991, because OS/2 1.30.1 contains slightly different code:

 MOV BX, 0C8h ; 200
 MUL BX
 CMP CX, 0
 JNE L3
 MOV CX, 1Fh ; 31
L3:
 DIV CX

In other words, if CX is zero, it is forced to a non-zero value to avoid the division by zero trap.

As it turns out, the MOV/MOV/MUL/DIV instruction sequence is quite unique and occurs only in the BASEDD0x.SYS drivers. To avoid the crash, it is sufficient to remove the DIV instruction, replacing it by NOPs.

A simple Python script is available to do the hard work. The script can be run either on individual files (such as BASEDD01.SYS) or on floppy images. It is thus suitable for patching physical boot floppies as well as diskette images. The script creates backups of modified files, but should be used carefully.

OS/2 1.2, Microsoft or IBM

OS/2 1.2 requires patching—see above. Installation is not substantially different from OS/2 1.3.

Microsoft OS/2 1.1

MS OS/2 1.1 requires patching. In versions prior to OS/2 1.2, there was no merged BASEDD0x.SYS driver but rather several separate drivers. The driver which needs patching in OS/2 1.1 and 1.0 to avoid division by zero traps is KBD01.SYS.

The installation program was different, and HPFS was not an option. Note that some versions of OS/2 1.1 may still be unstable on fast host CPUs.

Microsoft OS/2 1.0

MS OS/2 1.0 requires patching to avoid the division by zero traps—see above. In addition, OS/2 1.0 crashes or hangs if the VM has more than 16 MB RAM. RAM sizes of 8 MB or less are safe, and more than sufficient.

FDISK must be run from the installer to create a partition for OS/2. Note that this partition cannot be greater than 32 MB, but FDISK can automatically create a primary partition of maximum allowed size. The installer must also format the partition.

MS OS/2 1.0 did not support PS/2 mice. OS/2 can be installed without mouse support (with barely noticeable loss of functionality), or a driver for PS/2 mice can be manually copied from a later OS/2 release.

IBM OS/2 1.0 and 1.1

Versions of IBM OS/2 prior to 1.2 use a 286-style method of switching from protected to real mode, at least on PC/AT-compatible systems. This method involves resetting the CPU and is not supported by VirtualBox (or most other virtualization products). These versions currently cannot be installed in a VM. Microsoft’s releases of OS/2 1.0 and 1.2 used a faster 386-specific method when a 386 or later CPU was detected. IBM’s releases probably did not use that method because at the time, IBM’s 386 systems were all based on the PS/2 architecture.

This entry was posted in OS/2, VirtualBox. Bookmark the permalink.

29 Responses to Installing OS/2 1.x in a VirtualBox VM

  1. Wow that’s amazing!!! I never thought OS/2 1.x installations would happen under emulation any time soon!!

    I’ve also read that OS/2 1.1c included the 386 method of protected mode switching.. Although I’m not sure if that was an IBM or Microsoft thing, I had always (incorrectly) assumed that they were basically the same thing.

    And thanks for the great explanation of the keyboard timing thing!

    I wonder if they moved this code into country.sys for 2.0 LA which causes it to either crash out with

    The country information file “A:\COUNTRY.SYS” is incorrect.

    Or

    TRAP 0003 ERRCD=0000 ERACC=**** ERLIM=********
    EAX=000000eb EBX=000037aa ECX=fe560036 EDX=fe560130
    ESI=00000312 EDI=ffe5108a EBP=00004ef6 FLG=00003213
    etc…

  2. Pingback: Installing OS/2 1.x in VirtualBox | Fun with virtualization

  3. michaln says:

    Microsoft was using the 386 method of mode switching at least since their first OS/2 SDK (May or June 1987). IBM for some reason did not, or at least not on AT compatibles.

    IBM and Microsoft OS/2 indeed were the same thing, much like MS-DOS and PC-DOS. The same OS, but not identical releases.

    The trap 3 with OS/2 2.0 LA probably happens when hardware virtualization isn’t used. OS/2 2.x does not work in VirtualBox without VT-x or AMD-V. The message about missing COUNTRY.SYS is something different, I’m sure what exactly. OS/2 is most likely having trouble reading from the floppy.

  4. michaln says:

    I had a closer look at OS/2 2.0 LA. It turns out that the disk drivers significantly changed between the LA and GA versions. The LA release used a DISK01.SYS driver like OS/2 1.x (instead of IBM1FLPY.ADD etc.), but it’s not quite the same driver that OS/2 1.3 used.

    The LA driver does weird things to the floppy controller. It looks like a subtle bug in the driver code but confuses emulation.

  5. Andreas Kohl says:

    I had no success with MS OS/2 1.0 using this method. The patch works! But fdisk cannot operate the hard disk. After some seconds it quits with: An error occurred when reading data from the fixed disk. So which setting of controller type and Host I/O should be done?
    When adding a second emulated floppy drive to VM I got SYS2067.

    For IBM OS/2 1.0 – is it also true when running with PROTECTONLY=YES? CONFIG.SYS of MS OS/2 is also using this setting.

  6. michaln says:

    There are two versions of MS OS/2 1.0 (from Microsoft) with different disk drivers. One works, the other does not. Also, removing any CD-ROMs may help. Using IDE is right.

    IBM OS/2 1.0 will not work at this point, there’s no point in trying.

  7. Vadym says:

    Hello Michal

    I have question about “country.sys” error… again :). As I know this problem can be occured when somthing wrong with floppy or CMOS. But I don’t know what does system check exactly. I’m trying to resolve this problem for QEMU emulator. And always I got this error. Maybe you know somehting interesting and can help me.

  8. michaln says:

    Sorry, if you don’t want to use VirtualBox, you’re on your own 🙂

  9. Dan says:

    So how does one bypass the floppy problem in VirtualBox? it keeps saying my floppy is unbootable….

  10. michaln says:

    One doesn’t. If you’re getting a message that a floppy is unbootable, it’s rather likely that the floppy is unbootable.

  11. Dan says:

    The floppy is quite bootable on my decades old PS/2. I imaged it to try and boot virtualbox with it and I get the “disk read error occurred” error. then I just gave up bought a USB floppy drive thinking it was an image problem, and it doesn’t work. so looks like VirtualBox still can’t boot OS/2 1.x boot disks.

  12. michaln says:

    If you can provide the image, I’ll tell you why it’s not booting… but let me assure you that all the screenshots of OS/2 1.x on this site were made with VirtualBox. So I know it works.

  13. Andreas Kohl says:

    I got similar problems by adding a second virtual floppy drive to VM.

  14. David says:

    This is such a great website you have. Very useful.

    Just for fun, I’m trying various OS/2 versions under VirtualBox 4.2.16 r86992

    I have MS OS2 1.1 (the one that says NOKIA when you load the installer disk). It installs and runs under VirutalBox, but as soon as I click in the window to give it focus, VirtualBox will completely freeze if I move the mouse. If I’m careful not to let the mouse move, I can navigate with keyboard controls. Here’s a quick video I made demonstrating the problem. http://www.youtube.com/watch?v=Yxz598ZG4QE

    Is there a way to make the mouse work? If not, is there a way to disable it so that it doesn’t cause VirtualBox to lock up?

  15. Michal Necasek says:

    By “VirtualBox will completely freeze” you seem to mean “the guest OS stops responding to input”, right? The actual VM still works fine and can be closed etc. I’ll check if it works for me currently.

    Which mouse driver do you have installed in the guest? There should be MOUSEAxx.SYS, but which one? MOUSEA05.SYS should be the right one for PS/2 mice.

  16. David says:

    Correct. The VirutalBox software itself continues to run. I can press RIGHT CTRL and then shutdown the VM, while “inisde” of the VM, I have no control and pressing RIGHT CTRL + DEL will not reboot the VM.

    I looked at my CONFIG.SYS and it says DEVICE=C:\OS2\MOUSEA05.SYS

    I’ve also been trying to get 1.0 working (again, purely for the sake of technical curiosity), but I’m not even able to get it to boot. I successfully applied the python patch to the OS2 1.0 boot/install disk, but when booting, I don’t get a logo or any see any indication that the vm is even booting. The screen remains black and just says “Press F12 to select boot device.” But pressing F12 does nothing and I’m unable to reboot with RIGHT CTRL + DEL. I have to give control back to the VirtualBox window and it down that way.

  17. Michal Necasek says:

    I just tried the Nokia OEM release of MS OS/2 on VirtualBox 4.2.16 on my 2012 MacBook Pro. I installed OS/2 more or less with defaults and selected PS/2 mouse. I didn’t see any problem, the mouse works fine. Tried with or without hardware virtualization enabled, worked either way. The VM used the defaults for “Other OS/2” reduced to 8MB RAM and 500MB disk. I don’t know why it doesn’t work for you because it works for me…

    About OS/2 1.0: IBM OS/2 won’t work in VirtualBox. MS OS/2 1.0 should, but there are various OEM releases so YMMV. I just tried a Tandy OEM release of MS OS/2 1.0 and it does boot up without trouble (well, after patching). OS/2 1.0 will crash if the system (VM) has more than 8MB or so RAM.

    If you get no output on the screen whatsoever, I’d strongly suspect a busted floppy (image).

  18. David says:

    I tried deleting my OS/2 1.1 install and starting over. I ran into the same problem though. Here’s a look at what I did. Maybe you can see something I’m doing wrong?

    http://www.youtube.com/watch?v=aezCv0d83Zc

    About OS/2 1.0. I’m not 100% certain if mine is MS OS/2 1.0 or IBM. The download was labeled as MS OS/2 1.0, so I assume it is. (The others were correctly labeled so there’s no reason to think this one wasn’t.)

    Here’s a link to the image I have. Can you determine if the image is ok?

    This is the .dsk format that I originally got it in.

    http://db.tt/V0kFX2Qa

    And this is after I ran LOADDSKF to convert it to an image file, and then I patched the image file (and the python script did not complain).

    http://db.tt/ejsxRtlD

  19. Michal Necasek says:

    The OS/2 1.0 release you have there is definitely the IBM one. Only IBM used IBMBIO.COM and IBMDOS.COM, for example. The image looks fine but is not expected to work.

    I don’t think I’m going to see much in a video. But if you provide your OS/2 1.1 VM exported in OVA format, I’d be happy to take a look at it and see if I can spot anything.

  20. David says:

    Regarding OS/2 1.0, I’ll have to see if I can hunt down the MS variant and try again.

    Regarding OS/2 1.1, here’s the export: http://db.tt/YScltDK0

    Very interested in your results.

  21. Michal Necasek says:

    Your OVA works fine here (OS X 10.8.4, VirtualBox 4.2.16). I have no idea why mouse doesn’t work for you.

    You could try making sure that Num Lock and Caps Lock is off on the host before the VM is started, but I doubt it’ll make any difference.

  22. David says:

    Wow that is bizarre. I guess I’ll try running it from my WinXP machine just out of curiosity. Shouldn’t make a difference, but who knows.

  23. Michal Necasek says:

    Yes, that is bizarre. Trying the OVA on different machines would be interesting. And I guess you don’t have trouble with input in other VMs? Though they’d have to use the PS/2 mouse emulation to be apples to apples comparison (i.e. not Windows 2000 or later, for example).

  24. David says:

    The mouse is working fine with my Windows XP machine. Hmm… I’m wondering what this popup means (one that on my Windows 7 machine, I read once and then told it not to show the message again.)

    On the XP machine, it says, “The Virtual Machine reports that the guest OS does not support mouse pointer integration in the current video mode. You need to capture the mouse (by clicking over the VM display or pressing the host key) in order to use the mouse inside the guest OS.”

    I’ll have to google that one and see if it has something to do with my problem on my other machine.

    At any rate, do you have a different 1.0 set of disk image I can try. I can’t get my 1.0 INSTALL image to boot under VMware or VirtualBox. And in this case, using Windows XP hasn’t helped.

  25. Michal Necasek says:

    The popup message means exactly what it says. It is simply a usage hint.

  26. David says:

    Regarding MS OS/2 1.1, I was messing around with it on my other Windows 7 computer and was having the same problem with the mouse. I turned off VT-x/AMD-V and that took care of the problem. Weird. (Could have sworn I’d already tried that, but obviously not.)

    When I load system, I get a VirutalBox error telling me that VT-x/AMD-V hardware acceleration has been enabled, but is not operational.

    It goes on to say that certain guests like OS/2 require the feature.

    Interesting contradiction in my particular case. The required feature breaks the system, and turning off the required feature fixes it.

    Whatever. Problem solved.

    I’m thinking I’ll be able to get IBM OS/2 1.2 working now as well. The problem I was having there was A.) No PS/2 mouse support out of the box. And B.) When I copied the mouse drivers over from 1.30.1, I started experiencing the same problem as with 1.1 … the mouse causing the whole guest os to freeze.

  27. Michal Necasek says:

    The message about requiring hardware virtualization applies to OS/2 2.x (and it is accurate). OS/2 1.x is not really a supported guest 🙂 You could set the guest type to something else (DOS? “Other”?) to get rid of the warning.

    If turning off VT-x fixed the problem, there is probably some timing sensitive component. VT-x vs. software virtualization has very different performance characteristics and sometimes timing-sensitive code only works one or the other way.

    What’s odd is that for me it works either with or without VT-x, on two different computers. What kind of CPU do you have?

    Hmm, IBM OS/2 1.2 does come with a PS/2 mouse driver (MSPS201.SYS)… but for some reason it doesn’t work. The 1.3 version driver does. I wonder why.

    The fun thing about OS/2 1.2 is how much it likes like Windows 3.0… which it predates.

  28. David says:

    Both of my Windows 7 computers are Intel i7 processors. My desktop computer is the new 4th gen Haswell, and my laptop is the 3rd gen Ivy Bridge.

    Good idea about the guest type. Setting the guest type to Other -> Unknown takes care of the warning.

  29. Michal Necasek says:

    That should be pretty similar to my hardware (Sandy Bridge, Ivy Bridge Core i7). Magic touch I guess 😛

Comments are closed.