Booting Is Hard

So I had this brilliant idea of using SCSI drives with old 286/386/486 boards which have old BIOSes that can’t handle IDE drives bigger than 500-ish megabytes. The SCSI HBA is the first one I happened to grab, an Adaptec 1542C (good because it has a floppy controller onboard); I got the HBA some time ago for free but sans microcode and BIOS chips, and eventually plugged in EPROMs that I burned myself.

To make it more fun, at the other end of the SCSI cable is an Acard IDE-to-SCSI adapter which has a CF-to-IDE adapter attached to it, and plugged into that is a CF card (or a microdrive). Yes, that actually works… up to a point.

CF to IDE to SCSI Frankendrive

Smaller CF cards are a bit problematic because the Adaptec HBA uses a different geometry than IDE. Remember that SCSI drives have no “physical” geometry and everything is strictly LBA, but the BIOS has to present a logical geometry. For drives smaller than 1 GB, the Adaptec HBA uses 64 heads and 32 sectors per track, which has the nice property that 1 cylinder = 1 MB. But IDE drives essentially never use such logical geometry. For larger drives, the Adaptec HBA uses 255 heads and 63 sectors per track, which happens to match typical IDE large-drive logical geometry. What that means is that larger CF cards should have the same logical geometry when they’re plugged straight into IDE or go via SCSI, and should be accessible and bootable without trouble. Well, I was half right about that.

I first tried this in a 386 board which booted off IDE (CF-to-IDE) and the SCSI (CF-to-IDE-to-SCSI) drive was the second BIOS fixed disk (81h). No problem there, DOS partitions were accessible without problem.

But when I tried booting from any of my bootable CF cards (with PC DOS 2000) attached to SCSI, all I got was

Non-System disk or disk error
Replace and press any key when ready

Now, this message comes from the DOS boot sector (or possibly IBMBIO.COM), and not from the MBR. If the boot gets past the MBR, that usually means there is no geometry issue, and I was able to access the disk from booted DOS which corroborates that.

Okay, so I tried smaller (less than 1 GB) and bigger (32 GB) CF cards. No go. Tried FDISK, FORMAT, SYS. None of that made a whit difference, the machine still wouldn’t boot.

In desperation, I completely zapped a random CF card, attached a floppy drive, booted the PC DOS installation disks, and ran the installer. I fully expected that the system still wouldn’t boot from the SCSI-attached drive, but voilà, it booted to DOS just fine! That was a surprise.

And the machine kept happily booting from the SCSI drive until… I removed the floppy drive! At that point, it was back to “Non-System disk or disk error”. What the heck?!

After some experimentation, I found out that the system does not need a floppy drive to be physically attached or even configured in the BIOS in order to boot. But as soon as the floppy controller is completely disabled on the Adaptec 1542C, poof, no more booting.

This admittedly does not make much sense. DOS does not require a floppy drive to be present in order to boot from hard disk. But either the system BIOS or the Adaptec BIOS does not cooperate with the PC DOS 2000 boot sector when there’s no floppy drive. I don’t know why, but I would like to find out.

Pros and Cons

What are the advantages and disadvantages of the CF-to-IDE-to-SCSI solution? Other than the booting difficulty (which is easy to fix with a DIP switch once you know about it), it’s mostly good.

The advantages are:

  • Ability to use big drives (up to 8 GB no problem with DOS, much bigger drives with better operating systems such as OS/2)
  • Ability to move the CF card to a modern system and easily move data
  • SCSI transfer speeds are noticeably faster
  • It’s really cool

The transfer speed is worth expanding on. Old systems have no support for IDE DMA or even faster PIO speeds, and ISA IDE adapters don’t handle such fancy features anyway. So the IDE transfer speed will probably not get much past 1 MB/s—which is a lot better than an ancient IDE disk but much worse than what CF cards are capable of.

In my test system (classic 40 MHz Am386) , the CF-to-IDE-to-SCSI solution transfers 2.5 MB/s without any tuning (i.e. using the default 5 MB/s maximum transfer rate of the HBA). The same CF card attached via IDE only transfers 1 MB/s. A 150% speed improvement is nothing to scoff at!

The disadvantages are:

  • System takes longer to boot (approx. 7 seconds)
  • HBA BIOS takes up potential UMB space
  • Default 330h I/O base conflicts with MPU-401

On a 286, UMBs are not much of an issue. On a 386 with a memory manager it is an issue, but fortunately such old systems generally have small ROMs so the extra 16 KB of adapter ROM isn’t a big deal, and it’s well worth the ease of getting past 500 MB drive size. The boot speed can be tuned a bit by not scanning the entire SCSI bus for devices, but if you really want to reboot a lot, IDE is better. The I/O conflict is easy to fix by flipping a DIP switch.

All in all, it actually works pretty well.

This entry was posted in 386, BIOS, CompactFlash, DOS, SCSI. Bookmark the permalink.

21 Responses to Booting Is Hard

  1. This is likely due to hardcoded stupidity in the boot sector or a BIOS bug. When working in real mode, int 13h provides the interface for accessing both floppy controllers and hard drive controllers, and can work in CHS addressing mode, or LBA if the later is supported by the BIOS.

    Normally, int 13h maps things like so
    DL == 0h A:
    DL == 1h B:
    DL == 80h C: (first hard drive)
    DL == 81h D: (second hard drive)

    On some systems however, when there’s absolutely no floppy controller, the BIOS is idiotic, and the system gets confused so the hard drive becomes DL == 0H (A:). At least Microsoft DOS 5.0 didn’t have any issues with this, and would be completely content with a huge A:\; likely as a side effect of supporting SD (2.8) floppies, I believed it would always use LBA addressing if possible. Other systems are likely less tolerant.

    It’s been a *long* time since I checked how Option BIOS + external media interact, but what I’m guessing is happening is you’re dealing with a bug in the Option ROM interacting with a bug in the BIOS. Option ROMs override the initial IVH vector setup by the BIOS as a hook, so when the SCSI Option ROM kicks off, it overrides int 13h. It’s likely trying to enumerate all devices known by the BIOS controller when int 11 (hardware detection) kicks so it checks if 0, 1, then 80, etc. exist. It’s doing this via int 13h, AL=15h, which allows it to query if the drive is present.

    On some systems with no floppy controller, int 13h, 15 will still respond present, but return failure on initialization, on others, int 13h will respond (properly) as non-present. 2ドル dollars says what’s happening is the main BIOS is responding int 13h, 15 says A:\ is MIA, and then the SCSI logic controller is mapping C:\ to A:\ in a misguided attempt to be backwards compatible since a few of these adapters allow you to fake a floppy for booting. Your boot sector is hardcoded to look for 80 or 81 to boot, and ejects its brain.

  2. John Heritage says:

    There are apparently ISA cards that have a BIOS on them that take over the disk portion of the system BIOS to go beyond the 528 MB limit. Supposedly these are cheap. Might be something to try for the fun of it.

    I’m a little surprised by the slow IDE speeds. 16bit ISA theoretical max is 16 megabytes a second..

  3. @John: Technically speaking, if you have a EEPROM reader/writer, you could take any board with an option ROM, replace it, and the option ROM will get mapped in somewhere in the Cx00 address space. As long as it has the correct header, the BIOS will automatically execute it and said Option ROM could patch the BIOS dynamically to enable LBA addressing or CHS modes that allow you to bust the 528 limit without having to give up UMB space for it.

  4. Bill says:

    A few years ago I spent wayyy too much money on some 8-bit ISA Future Domain SCSI cards with ROM for IBM PC and XT machines because the MFM / RLL drives are so hard to find in working condition.

    I used to have access to an eprom writer and UV eraser at work and as I recall I had to make ROMs of a specific version to boot my the older systems.

  5. Chris M. says:

    …and there is an open source option ROM that does exactly that. The AT Version of the XTIDE BIOS supports large drives on any AT-class machine.

    http://www.xtideuniversalbios.org/

    SCSI has another advantage over IDE, most cards support bus mastered DMA transfers, so less CPU usage. Just one reason why SCSI was faster back in the day.

  6. @Chris: That’s absolutely glorious. I didn’t know it existed before hand, or anyone had actually written an open source Option ROM for it. I was just working from my previous experience with real mode on how you could actually do it 🙂

    IDE is mostly a hack overall to be cheap to implement and easy to manage in software. SCSI requires a relatively expensive control card, and requires active clocking and mangement. IDE is much much simpler, and easier to work with from the perspective of the CPU since you essentially do can do I/O read/write to get to it, and then setup DMA if desired for speed.

  7. MiaM says:

    This reminds me of my experiments with an old (NEC branded) Trantor T-128 SCSI controller (or was it maybe some other similar Trantor?) in an old Panasonic Senior Partner portable 8088 PC (with 9″ CRT and thermo printer buildt in). I found out that the BIOS on the Trantor card uses instructions that doesn’t exist in an 8088 but became available in NEC V20 and 80188 and upwards. However the Panasonic motherboard BIOS doesn’t work with an V20, and that BIOS is soldered to the motherboard.

    The SCSI controller works when booting from floppy and loading drivers from the floppy, so the hardware is fine. However the computer only has 128K RAM so I’m not that keen on loading more stuff from disk.

    My future plan is to burn an XTIDE bios and put it on a network cards boot socket (I want network anyway) and install a 16-bit multi-io card and just live with loosing 50% of an IDE drive’s capacity. Maybe I’ll do some ugly hack on the multi-io card and solder in a 512k sram chip with adress decoding e.t.c. to go up to full 640k capacity (or even solder in two 512k ram chips and arrange so they are enabled on everey free part of memory space, giving some 768k continous free ram and some UMB aswell.

    Or probably I won’t find the time to do anything anytime soon 🙂

  8. Michal Necasek says:

    Cx00 segment is potential UMB space. Any ROM in that range takes away from the available UMBs. Also it depends on the card where the ROM is mapped. For example the Adaptec default is the DC00 segment IIRC. But yes, in general you’re right, it would be possible to install a “BIOS upgrade” this way.

  9. Off-topic but I don’t have a better way to contact you, I’ve been looking at the XENIX 2.2.3 stuff, and I can confirm at least the N disks are not in fact corrupt. I’ve also had some luck in virtualization though I can’t get it successfully look at the IDE controller. I’m going to probably end up patching VBox’s source to get it to work I was able to extract every single file from them:

    http://pastebin.com/Z6iJPzCU

    As far as virtualization goes, QEMU crapped itself out really early in boot. I got the boot prompt, and then it died. VBox worked better, but suicided when I attached an IDE controller, likely due to the timing bug you brought up. I’d like to collobrate with you seeing if we can actually got past the IDE hurdle. I’ve had no luck getting past that as of yet. Any hints?

  10. Michal Necasek says:

    Yes, there are such cards, but I don’t have one. But I have several SCSI HBAs instead 🙂

    16 MB/s for ISA? What??? I see you’re not the only person making that claim, but it’s utter nonsense. 16 MB/s is a purely theoretical and completely worthless value obtained by multiplying ~8 MHz by 16 bits, but an actual ISA bus data transfer takes a minimum of two bus cycles. So the theoretical maximum is around 8 MB/s with zero wait states, not 16 MB/s! Once wait states are introduced it only goes down from there.

    But for PIO IDE transfers (the only kind these systems know) it’s much worse. Everything has to be shoveled through the CPU and you can forget 8 MB/s. I don’t know what the theoretical maximum would be but it will take at least one bus cycle to transfer from device to CPU and another cycle to go from CPU to memory (when reading from disk). The INSW/OUTSW instructions aren’t particularly fast (15 cycles on a 386 for OUTSW, 14 cycles for INSW) and before you know it, you’re in the 1 MB/s range for actual real-world transfer speeds.

  11. Michal Necasek says:

    The system boots fine from IDE when there is no floppy drive. So it’s not the system BIOS being completely broken.

    Part of the problem is that given the BIOS data area design, it’s actually quite difficult to indicate there are no floppy drives. There is a good deal of software (including IBM’s QCONFIG shipped with PC DOS) which thinks there’s a 360K drive when there’s in fact nothing.

    I think you’re right that the Adaptec HBA BIOS is getting confused somehow but it’s not that simple. Remember that the MBR has no trouble. And for example NTLDR works too and can boot NT (well, it bombs on the 386 but it has no trouble loading a few hundred kilobytes off the CF card). There is some interaction with the PC DOS boot sector, I just don’t know what yet. The Adaptec BIOS actually prints the BIOS drive number it’s using and it’s 80h when there’s no IDE drive, 81h when there is one. Exactly what you’d expect.

    I’m almost certain that something is trying to access the floppy drive even though none is configured, and when there’s no FDC, things get a bit more upset than when there is a FDC with no drive attached.

  12. Michal Necasek says:

    Exactly. As I explained on another comment, with PIO (the only kind available) IDE, everything needs to go first to the CPU, then to memory. The Adaptec HBA uses bus-mastering (it’s a pretty clever thing they came up with there, not something IBM had in mind when they designed the PC) and the transfers go straight to memory without truly using the DMA controller. Lower CPU usage isn’t an issue in DOS (it would be in OS/2 or NT or Unix), but the fact that data doesn’t have to go through the CPU actually increases the transfer speeds quite a bit. Cutting out the middleman so to speak.

    I will have to try playing with the system’s ISA bus speed a bit and also see if I can bump up the transfer rate on the Adaptec HBA a little. Even so, I think 2.5 MB/s real-world disk transfer speed on an ISA system is fantastic. Back when I had an ISA 386 I couldn’t get anywhere near 1 MB/s with the drives available at the time.

  13. @Michal: MBRs are really tiny, shouldn’t be hard to disassemble it and see what its actually doing, and then pin down what series of int accesses it’s doing. From some basic Googling, at least Windows 2000 used extended addressing if at all possible; I can’t find any info on WinNT’s MBR. I am rather curious at the underlying bug if you can find it, or the cause.

    Not sure if you saw my earlier comment: re Xenix. I’m currently trying to build a debug VBox to keep investigating.

  14. Pingback: Resetting Disks Is Hard | OS/2 Museum

  15. zeurkous says:

    Thanks for testing the SCSI->ATA->CF bridge config. I had been investigating it myself — it will probably solve a couple of problems on this end.

  16. Yuhong Bao says:

    NTLDR in NT4 did not support extended Int 13h. They never fixed this in even NT4 service packs, but you can use NT5’s NTLDR with NT4.

  17. techfury90 says:

    I always wondered why they cased HDs the way they did for INT 13 drive numbers.

    Oddly, the NEC PC-98’s equivalent, INT 1B, uses drive numbers that correspond to physical hardware. HDs are 80/81H for SASI/IDE (PC-98 IDE emulates SASI at a BIOS level), A0-A6 for SCSI. The lower nibble of that one corresponds directly to the SCSI ID. There’s also C0-C6 for SCSI direct command pass-through mode. The existence of that last one was why ASPI was fairly rare on 98s.

  18. techfury90 says:

    INT 1B’s argument format is also good for up to CHS 65536/256/256, at least. It leads to some wacky translations though: many mid 90s BIOSes use a fixed 8/17 heads/sectors for IDE, eventually topping out at 4.5 GB. They then moved it to the SCSI translation formulas, which were typically 8/32 for 8 GB and less and 8/128 for up to 32 GB. The SCSI HBA in mine lets you specify higher heads/sector values in its setup though. Good for up to 2 TB that way.

    Amazingly, NEC MS-DOS 6.20 can actually use drives that large fine with no kludges needed. Of course, you run into the 16 partition limit and FAT16’s limits first.

  19. Michal Necasek says:

    Interesting. Yeah, direct SCSI command block is basically most of what ASPI does, so I can see why it wouldn’t be too popular.

  20. techfury90 says:

    I think IO.SYS in NEC MS-DOS 3.3C and later actually uses it, too. At least, that’s how I strongly suspect IO.SYS’s built in support for SCSI MO drives works behind the covers.

  21. Yuhong Bao says:

    I wonder why NEC didn’t just expose the values from the IDE interface directly without translation.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.