IBM XENIX: One Step Forward, One Step Back

A while ago I wrote about why IBM PC XENIX 1.0 can’t work on any CPU other than a 286. But not content to leave well enough alone, I attempted to patch this version of XENIX so that it would run on a post-286 processor.

While doing that, I had to change my original assessment of the problem. IBM PC XENIX does not put garbage into the reserved word of a descriptor table entry, it actually stores data there! That makes the behavior much harder to excuse, and also much harder to fix.

After some quality time spent with a disassembler, greatly aided by the symbol table included in the xenix and xenix.fd kernel files, I established that XENIX stores the segment size in the reserved descriptor word (this is the size in the file which may differ from the segment limit). Unfortunately, all my attempts to patch the XENIX kernel failed and I could never get the installation floppy to boot.

Or so I thought. A few days ago, I more or less by accident booted the patched XENIX floppy, and lo and behold:

IBM XENIX MaintenanceIt actually worked! It turned out that in my previous attempts, I assigned 8MB RAM to the XENIX VM. That was a mistake, because IBM PC XENIX 1.0 crashes with more than 4MB RAM! Duh... such a common problem which in various incarnations plagues many, many old operating systems (for example OS/2 1.0 crashes with more than 8MB RAM).

EGA Patch

An unrelated problem is that IBM PC XENIX 1.0 is incompatible with EGA (and by extension, VGA) adapters—it does contain code for EGA support, but that is broken.

This is something which was noticed three decades ago, and is easy to patch. XENIX contains an egainit() routine (called when an EGA is detected) which in turn calls initCRTC(); the latter destroys the EGA state. Simply preventing initCRTC() from running is enough to make IBM PC XENIX 1.0 display text properly on EGA and VGA compatible hardware.

This problem probably stems from the fact that IBM XENIX effectively predates EGA and the XENIX + EGA combination was likely never tested.

Drive Types

Another IBM XENIX problem discovered decades ago is its very poor hard disk support. IBM designed the PC/AT BIOS to provide disk geometry information (based on the drive type stored in CMOS).

For reasons that are not entirely clear, IBM XENIX queries the CMOS for the disk type but carries its own disk tables... replicated in several modules and different from the PC/AT BIOS tables. The upshot is that support for disks other than the standard 20MB and perhaps 30MB drives offered with the PC/AT was very problematic.

I decided to use the shotgun approach—force the CMOS drive type to 2 (PC/AT 20MB disk) and use a virtual disk big enough to cover the 615/4/17 CHS geometry that XENIX works well with. Since XENIX uses strictly CHS addressing, this actually works, at the easily affordable expense of wasting some megabytes of virtual hard disk space.

What Works, What Doesn’t

It’s possible to install IBM PC XENIX 1.0 in a VM and boot it from hard disk:

IBM XENIX BootThat’s quite a success for an OS that doesn’t run on anything other than a true blue PC/AT in original configuration.

The system generally works, and it is for example possible to log in remotely over a serial port (once logins are enabled in /etc/ttytype). The great majority of commands works well, and in general IBM PC XENIX 1.0 is a decent UNIX System III implementation:

IBM XENIX CommandsBut there are problems. Running vi reliably panics the OS, although ex (identical executable) sometimes works. The tset utility installed on hard disk crashes, although the one on the boot floppy does not. The vsh “visual shell” sometimes crashes, sometimes works:

IBM XENIX VSH

The problematic executables have something in common—they either have symbols or are “middle model” executables. That strongly hints that the XENIX kernel patch has unwanted side effects and prevents these programs from running.

What Next?

And why is IBM PC XENIX 1.0 important, anyway? It was the first protected-mode operating system available for the PC/AT, and one of the first 286 operating systems. It was a multi-tasking, multi-user operating system which, unlike PC/IX on the IBM PC/XT, was actually usable and could replace a modest UNIX workstation.

Unfortunately I do not have access to either the development system or the text formatting package for IBM XENIX, or any applications. That puts a damper on further research.

Update: It appears that IBM PC XENIX 1.0 can be hybridized with Intel Xenix 286 from November 1984. Intel’s release targeted Intel development systems rather than PCs, but it was likewise supplied by Microsoft and it is very very similar to the IBM PC XENIX. More about this later.

This entry was posted in 286, 386, IBM, VirtualBox, Xenix. Bookmark the permalink.

5 Responses to IBM XENIX: One Step Forward, One Step Back

  1. jack says:

    IBM Xenix virtual shell have interface like Word
    for dos, Multiplan for dos, Chart for dos and
    Microsoft Interface Manger

  2. Joshua Rodd says:

    What’s “middle model” again?

    I assume it’s this: you have three code segments, perhaps 30k each, total 120k. You then organise it like this:

    30k – procedures needed only in segment A
    30k – procedures useful to both segment A and B, we’ll call this “M”
    30k – procedures needed only in segment B

    Now people living in either segment A or B can call procedures in M using near calls. All the code in the overlapping part must be position-independent (i.e. all near calls), since it has no idea what its CS might be.

    I recall this is easily possible to do with MASM and the DOS linker, but I’ve never seen a DOS program actually written this way.

  3. Michal Necasek says:

    “Middle model” is what’s called “medium” model in DOS. Multiple code segments, single data segment.

  4. dosfan says:

    tiny model: COM file (org 100h), single code segment, single data segment
    small model: single code segment, single data segment
    medium model: multiple code segments, single data segment
    compact model: single code segment, multiple data segments
    large model: multiple code segments, multiple data segments
    huge model: multiple code segments, multiple data segments, arrays > 64KB

  5. Pingback: IBM XENIX: Two Steps Forward | OS/2 Museum

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.