Chapter One: The Operating System of Your Dreams

You may wonder, if you've never written an operating system before, how you could possibly be qualified to write one. Well, as a computer professional, or even a computer user, you have spent many hours running one application or another...perhaps, in addition to the occasional game. When you run these applications, you are subject to the user interface, file system, and peripheral support for monitors and printers and perhaps modems, that the operating system provides. So, who would be more qualified to design an operating system for users, than a user?

Now, your natural response may be to say, "Well...how about those dedicated scholars in the halls of academe, who devote their very souls to the management of the digital hardware I purchase? They have degrees upon degrees, and therefore must know more about these things than one such as I." If so, I can only reply that you've got to get out more. Haven't you been paying attention? It's those very scholars who have given us the operating systems we already have! And, while I won't argue those systems have made someone a lot of money—a lot of money—that money has come from you. And not just in the form of cash either. Let us consider for a moment:

The Operating System of Your Nightmares

Suppose you wanted to design a truly bad operating system. What might you do? Here's a suggested list:

• Base it on a truly old operating system, such as Unix or CP/M.
• Include "backward compatibility" support for every obsolete program anyone has ever written. This includes support for FCBs (CP/M file control blocks), "threads", monolithic non-protected memory blocks, real-mode programs, and text-mode programs.
• Base it on a file system that has not changed in its essentials in over 25 years.
• Tack on to it an object-oriented, graphical user interface that is not based on an underlying object-oriented structure.
• Add networking support so flimsy that it requires a network specialist even to plug a couple of PCs together.
• Then superimpose an underlying object model that is so complex the author of the book describing it admits he doesn't understand it.

Oops...seems I've just described the current versions of Microsoft DOS, Microsoft Windows, and OS/2!

I Can Dream, Can't I?

Now, let's look at a few "requirements" for an operating system that might be fun to write, and more fun to use:

• It should run optimally on the kind of computer practically everyone has, i.e., the IBM-compatible PC, with a 80386 or better CPU, and VGA or better monitor.
• It should incorporate the good things we've learned about user interfaces—in other words, the fact that users respond as well to object-oriented concepts as programmers do.
• It should carry that object-oriented ideal down to every level. In other words, it should be a truly object-oriented operating system, one in which there is no component that is not an "object" and can therefore be treated as such.
• Instead of a "file system", it should make use of hard and floppy disk space to store objects. Objects should be stored in a way that provides optimal accessibility and minimal load time. Networking should be such a basic part of this scheme that plugging in another computer simply adds on to the available object storage space.
• It should run in protected mode, take advantage of segmented memory (instead of being hobbled by it), and make use of the built-in 80386 virtual memory system to turn hard, floppy, and remote disk space into mere virtual memory. (And this ties in with the previous idea!)
• It should not rely on traditional programming techniques to create new applications. No one knows better than a user what his or her programming needs are, so why shouldn't that person be able to "assemble" a new application from standard parts that come with the operating system?
• Finally—and here's the most significant point—it should be designed simply enough that anyone who wants to make a modest effort, should be able to understand the entire thing.

There are those, of course, who will maintain frantically that this last point makes the entire project impossible. And well they should; their jobs rely on the industry believing the opposite! They are the ones who like to stick useless letters in front of otherwise meaningful variable names. (This is a variation of the useless letters they stick after their own personal names.) Their attitude seems to be, "If you can understand it, I haven't done my job right. Let me enhance it some more until you can't."

The problem is, when your software (at any level) is so complex, it is prone to bugs. And sure enough, note the following:

• Various components (DLLs) in Windows 3.1 have been replaced several times since its release.
• Microsoft Word 6.0 has, as of this writing, in less than a year had four bug-fix releases, bringing it to version 6.0d.
• Borland C++ 4.0 was so filled with bugs that its disastrous release precipitated the resignation of Philippe Kahn, Borland's founder, as the company's CEO.

Why are things this way? Basically because the primary product of software companies is no longer software; it's profits. That's why I'm "selling" this operating system as a book. I am interested in the art of programming computers, not the money. (And believe me, book authors don't make a lot of money unless they write about Hollywood wives in trouble.)

A Plan Of Attack

So how do we do this? How do we just sit down and write an operating system?

First, we'll need certain tools: An existing operating system plus compiler(s), for example, to carry us along until we have equivalent tools on the new operating system. I intend to use Windows and Microsoft Visual C++ and MASM in this book, but there are a fair number of alternatives you can safely use in place of these if you prefer.

We'll need to pick languages. Operating systems are traditionally written in assembler, and certainly there are components that must be written in assembler. My preference is to write a "first draft" in C or C++, capture the assembler translation from the compiler, and tweak that.

We'll need to identify and map out a memory management scheme...and then implement it.

And we'll need a technique for booting from and installing the new operating system.

But, most importantly, we'll need a name. Since this is intended to be an extremely focused operating system (as opposed to one whose components were developed willy-nilly by a thousand programmers who never met one another, much less talked, over a period of years and including pieces thrown in for a hundred reasons no one remembers anymore), I propose naming it after the most focused object in the known Universe. I call this new operating system, "Singularity".

Each of the following chapters will cover an aspect of the operating system, in the order in which these aspects must be developed. So, we'll be covering:

• A floppy disk bootstrap sector program, and how to enter protected mode
• How to access real mode BIOS routines from protected mode, and the protected mode interrupt system
• The Task Manager: a protected mode switch
• Protected mode tasks, virtual memory, and the task list
• A protected mode keyboard driver
• A protected mode VGA video driver

Chapter 2: Floppy Disk Bootstrapping

For the first few iterations of Singularity we'll place the operating system on a single floppy disk. There are several advantages to this:

• It delays our having to work with a hard disk partition table
• It simplifies test boots of Singularity: either boot from the floppy (Singularity) or from the hard disk (Windows)
• It simplifies our management of virtual memory, since we are automatically limited to the space available on a floppy disk

Floppy Disks 101

It will be necessary at this point to understand how floppy disks work.

A floppy disk, of course, is a flat, soft platter that is made of a material similar to that used for audio or video cassette tapes, except that it is coated on both sides with particles capable of holding a magnetic field. Most modern floppy disks are contained within a stiff plastic housing and can hold over a million bytes of storage; just a few years ago a soft, less protective envelope was more common and storage was limited to 360 kilobytes! Although it is common to refer to the disk plus housing as a "floppy disk", in reality only the platter itself is the "disk".

If you look at a floppy disk under a microscope, you'll see a homogeneous field of particles embedded in a thin plastic medium. All the organization, all the information in the disk exists as magnetic lines of force which of course do not show up under a microscope.

If they did, however, you would see that the disk is organized into groups of data packets called sectors. On a DOS disk sectors are 512 bytes long, but that is not a requirement...with one exception: The very first sector on the first side of a floppy must be 512 bytes long, because that's what the PC's BIOS expects it to be.

(Just for completeness, perhaps I should mention that, on a PC, a byte is a group of eight bits, and the smallest directly addressable piece of memory. Hopefully, you knew that.)

A sector is the smallest block of memory that can be written to or read from a floppy. There is nothing stopping you from creating one-byte sectors, if you wish (except that the BIOS limits you to a minimum of 128 bytes. But you could always write your own floppy disk driver!). However, each sector has a certain amount of overhead associated with it. It has, for example, an ID number. It also has a checksum, used to guarantee data integrity. So the fewer, larger sectors on your disk, the more usable space you'll find. (That's why floppies are marked "2 MB unformatted; 1.44 MB formatted".)

Sectors are organized in concentric circles called cylinders. The word was inherited from hard disks, which often consist of parallel platters; a "cylinder" is the series of circles extending through each usable side of all parallel platters. The outermost circle is cylinder 0, the circle within is cylinder 1, and so on up to cylinder 79. This implies that your floppy is also divided into sides, which is true. However, since access to the floppy is provided by the hardware, and a side is read or written by a read/write head, the side you want is actually specified by requesting a head—either head 0 or 1.

The sector ID is also commonly called the "sector number" but this is misleading. True, sectors are usually numbered from one to eighteen, but they don't have to be. There's nothing stopping you from numbering the sectors on a given cylinder 3, 4, 56, 57, 62, 66, 71, 72, 73, 74, 80, 81, 82, 84, 100, 110, 120, for example. You don't have to number them in order, either. But, except for copy protection—a perversion of which I do not approve (possibly the only one!)—I've never been able to figure out an advantage to such a numbering scheme.

On older PCs, there was an advantage to staggering the numbers—for example, number 1, 4, 2, 5, 3, 6... and so on. This was because when two sectors were requested in a row, it was quite possible that the PC's floppy controller would not be able to transfer the first sector before the adjacent one had already passed beneath the drive's read/write heads. By staggering the IDs, the next requested sector would actually be the second or third physical sector following, and read times would be shortened. However, this scheme is not helpful for '386-based computers (or better); if anything, it can slow data transfers.

Floppies, then, are organized into sectors which are located by head, cylinder and ID. Sectors can be sized as a multiple of 128 bytes, except for sector 0, cylinder 0, head 0, which must be 512 bytes.

The Boot Sector

That first, magical sector is more restricted than the others because this is the one the PC examines to learn the physical characteristics of the rest of the floppy. Every floppy drive (and hard drive, too, for that matter) can position itself to address 0-0-0, and if the PC BIOS knows to request a sector of 128 bytes, that request cannot fail on a properly formatted floppy.

The PC BIOS also expects certain things of this sector. The very first thing is a JMP around the boot sector's data. This way, exactly where in the sector the code is placed doesn't have to be known by the BIOS, and can change from disk to disk. The first data item is the BIOS Parameter Block, which requires a special note. This table supplies the BIOS (and, later, the operating system) with information it needs in order to work correctly with the disk. The table is not identical for low and high density disks. It is not the same for 5.25" and 3.5" disks, either. So when you put this boot program onto a disk, you must be sure the values in the BIOS Parameter Block are correct for that disk.

The disk parameter table is always 32 bytes long. This information is copied by the BIOS into a certain location in memory, but the BIOS itself doesn't make use of the information. This is one of those places—fortunately, there are few—where the line between the BIOS and MS-DOS blurs. The BIOS stores the BPB, but it doesn't define most of its fields. Yet, the BIOS builds "sample" BPBs for each of the supported floppy drive types and densities, and stores them; these BPBs are the ones MS-DOS could use.

If you want, you can place additional information following the DPT, though of course the BIOS won't copy it into memory for you. But that's why the executable code's start location isn't hard-coded into the BIOS.

Following the BPB, you must supply code that will be executed if the floppy is used to boot. Here's how it works. When a PC is first powered on, the BIOS runs a "power-on self-test" (POST). It then examines the various disks attached to the PC. Some newer BIOSes offer other options, but generally floppy drive a: is checked for the presence of a bootable disk; if one is not found, hard drive c: is checked. A bootable disk is defined as one that possesses a drive parameter table; so by definition if the disk has been properly formatted, it is bootable. The BIOS then loads the entire sector into memory (at location 2000:0000), makes a JMP to the JMP instruction that began your boot sector, and therefore JMPs to your code. So the question is, what do you want that code to do?

As it turns out, bootable DOS disks have the same boot sector program as non-bootable disks! The program loads the disk directory (a DOS artifact) from the floppy, and checks to see if an operating system is present on disk. If it is, the files are loaded into memory and execution is turned over to them. If not, a message is displayed, saying "This disk is not bootable," which from the user's point of view is true. However, with all the code for locating and loading an operating system in there, there isn't much room for a user-friendly message like, "Excuse me, but I think you've accidentally left this data disk in your floppy drive. Why don't you remove it?" followed by a polling of the drive to automatically boot from the hard disk as soon as the floppy is removed. (My first published article explained how to write such a replacement boot sector program.)

A Singularity Bootable Disk

With Singularity, a system disk is inherently different from a data disk, so we won't have to cram everything into a single, one-size-fits-all boot sector program. The boot sector program for a system disk will know that the operating system is present on the disk; so it can concentrate on the job at hand, which is loading that system into memory.

Where MS-DOS may reside anywhere on a floppy, and so the system directory must be consulted to find out where it is, Singularity will always occupy (or at least start at) the same location. This simplifies our job a lot.

Help From The BIOS

I've mentioned the BIOS several times without really explaining what it is. Surely you've heard the term before! It stands for Basic Input Output System and is perhaps better described as the PC's autonomic nervous system. It provides pre-written functions that access all the basic peripherals on the PC, in a basic way: the keyboard, the video, the floppy disk controller. The BIOS functions are triggered via software interrupts. For example, to send a string to the monitor, the following assembler instructions are all that is needed:

Likewise, there are BIOS instructions for reading (and writing) single and multiple sectors from the floppy disk.

The BIOS function that writes several sectors at once is limited to reading the sectors from a single cylinder, and you have to provide a little table of sectors desired. There's no harm in reading in the remaining sectors of the first cylinder every time—even though we aren't using them now, we probably will eventually. We'll just have to remember this when arranging Singularity's virtual memory scheme; that should make Singularity load even faster!

The Boot Sector

Let's go through the boot sector assembler code, line-by-line...or, at least, section by section.

To begin with, we have to include the usual assembler directives that enable the assembler to optimize the instructions for the segment registers in use. We also have to name the segment, and issue an address ORG directive, before placing that all-important initial JMP:

The ORG directive tells the assembler that addresses in this segment should start at 0000h. It would do that by default; but since most .COM programs start at 0100h, I like to make clear my intentions. (A boot sector resembles a .COM file in that it occupies one segment and all inter-address fixups are already made, prior to loading. We'll have to assemble and link as if this were a .COM program.)

That first label, ProgramEntry, is of course the entry point. You know you'll see it used again, once, in the END directive. Likewise, ProgramInit will appear as the label where the code actually starts.

First, though, we must declare and initialize the BIOS Parameter Block:

Incidentally, there is an interesting anomaly regarding the OEM field of the BIOS Parameter Block. These eight characters identify the operating system for which the floppy is intended. IBM received some well-deserved bad press a few years ago when they modified PC-DOS (licensed from Microsoft) so that the identifier must begin with the letters "IBM", or PC-DOS would refuse to read the disk. Singularity disks will not be readable by any version of DOS, so we won't have to accommodate that, and can use the field for the purpose intended for it.

All we need to know is:

• What will we use for the version identifier?

  How about "SING 010"? That will allow for up to 99 major and 10 minor releases, which ought to be adequate. It also has the benefit of being readable in ASCII.

• How big are the sectors?

  This question requires a bit of thought. As mentioned, the smaller the sector size, the greater overhead on disk. On the other hand, the less space wasted "padding" unfilled sectors, the better. In the end, I propose sticking with the 512-byte sector size, as a good compromise between too big and too small.

• In what sector does Singularity start? And how many sectors does the basic kernel occupy?

  Singularity will always start in sector one. The number of sectors to be loaded will be the same per version release; we can code it as a constant in the assembler code.

  For now this value will be one, primarily because we don't have an operating system to load, yet. We'll have to use a stand-in: a simple assembler program that merely displays "I am loaded!" on the monitor and then quits. This will easily fit in a 512-byte sector!

• Where should Singularity be loaded into memory?

  Again, this value can be a modifiable constant. But a good location is following the BIOS data area. This area's size varies according to the BIOS, but there is a BIOS function we can use to query the size, and thus calculate Singularity's starting address.

• What should the operating system do while the load is taking place?

  That is, how should users be informed that the load is proceeding? Current versions of MS-DOS simply display the text, "MS-DOS Loading..." and a variation of this seems like a reasonable approach.

  If the load were going to take a long time, you could justify a "progress bar" that would let the user decide whether there was enough time remaining to run for a cup of coffee. But Singularity should take a lot less time to load than DOS and Windows, and there's no point in masking that fact with elaborate displays.

We've already talked about the OEM and BytesPerSector fields. Most of the remaining fields have to do with the "FAT" style of disk management, which the BIOS assumes we are going to use. But are we?

FAT stands for File Allocation Table, and it is a truly mind-boggling way of dealing with a small number of disk sectors on the kind of disks used by CP/M, while using a minimum amount of memory. Sectors are not referenced on a sector-by-sector basis. Rather, they are referenced as "clusters", a collection of contiguous sectors. The primary purpose of the FAT is to link discontiguous data; its secondary purpose is to mark "bad" sectors so the operating system can work around them. Thanks to the cluster system, when one sector is bad, all the sectors in the cluster are marked bad. And, when a sector is allocated for use, so is the rest of the cluster! It's not a very efficient way of managing disk space.