Beginner Mistakes: Difference between revisions

The idea to "write your own OS" brought you here. This Wiki is about giving you help, pointers, and references in your undertaking.

However, it is quite common that newcomers make certain mistakes or have common misconceptions about what is involved in the topic. That is not bad--many others made these beginner mistakes before, and many will do so in the future. This page is about making sure you know what you're about before digging into the provided information.

What this is NOT

This may look like a set of tutorials to copy & paste, plus a forum to ask your questions whenever you get stuck. This is not so. We fully expect you to carry your own weight, and be an experienced programmer of user-space applications before you set out to write your own OS. We also expect you to have read about OS design, and that you have studied relevant documentation of your platform of choice. Do not expect this Wiki, or the forum, to be some kind of complete guide to my own OS, let alone a guide to programming skills in general.

What you find here is documentation left behind by those who came before you, who found out about these things by reading technical documentation, available source code, and forum postings as well as by trial & error programming. Some of it was written so those people could look it up again later, some of it so that we could point to a Wiki article instead of explaining a subject all over again.

What you find here are little hints and roadsigns that might help you on your path. It's not a complete map to Oz. Nor should it be.

If you don't already have a good idea what a stack is, or how to use a debugger, we won't go out of our way to explain it to you. Visit those two pages; you will see that they mostly deal with OS specifics, not a general introduction to the topic. That is not a flaw, that's by design. If you are looking for general programming enlightenment, you should rather visit general programming sites like StackOverflow and become a developer first before you aspire to become an OS developer.

This Wiki will not be expanded into a beginner's handbook, because that is not its purpose. It is for answering the advanced questions that arise when people feel they're ready for the plunge into kernel space programming.

A Hard Truth

No one who isn't already a seasoned developer with years of experience in several languages and environments should even be considering OS Dev yet. A decade of programming, including a few years of low-level coding in assembly language and/or a systems language such as C, is pretty much the minimum necessary to even understand the topic well enough to work in it.

What's more, this is growing ever more true by the year, as the number of different 'standards', computer and mobile device models, peripherals, and design concepts continues to expand.

It was one thing for someone like Dennis Ritchie, Richard Greenblatt, Gary Kildall, or Steve Wozniak to write a simple OS for hardware which was relatively straight-forward in comparison to current machines, which they already knew intimately, and which had no standards specifications to adhere to or existing cruft to maintain backwards compatibility with. This is no longer true on current-generation stock hardware. Furthermore, each of them was already a seasoned engineer who had already done years of systems programming. Even then, they only provided the foundations for the systems; the bulk of the work was done by small armies of subordinate developers after the nucleus of the system--the kernel, in modern jargon--was in place.

There are exceptions to this rule, but not many; don't expect yourself to be that one in a thousand. If you think you are, read up on the Dunning-Kruger Effect and consider the matter again.

Oh, and for the record, Linus Torvalds wasn't quite one of them--he was a graduate student when he wrote the Linux kernel and had been coding in C for years. While he was well short of that ten year mark, as a grad student who had turned his hobby into his master's thesis, he had more time on his hands to work on the project than most people would. In any case, the 'Linux 0.0.1' release he famously posted to USENET in 1991 was little more than a round-robin scheduler, nowhere close to a full system. Getting to that point took him a year. Get the picture?

While it is true that most of the contributors to this wiki started much sooner, for most of us, that was a mistake born out of a lack of experience. Most of the pioneers of this group had no idea of the sheer scale and complexity of even a small OS project, no inkling of what they were getting themselves into. This was a difficult pill to swallow, especially back before resources like this Wiki were widely available. We cannot force you to learn from our mistakes, but at least we can pass on this warning.

Now, you shouldn't get too discouraged by this; the point is not that you can't do this, but that--if you are like most of us were when we started out--you probably can't do it yet. Patience can be a virtue when starting out on a project this large.

Is there a tutorial on...?

Because this place cannot and does not cater to beginner developers, the question of which other place provides a tutorial, good explanations, or easy-to-understand reading is often asked. However, they do not exist. Difficult subjects cannot be described with light prose, just like how there are enough things that are too complicated for a monkey to properly learn. If you have trouble reading official documentation, this would be a good time to practice.

In fact, the vast majority of tutorials currently out there are known to be broken in at least one way, so you are generally better off not trusting them in the first place.

Scope

Deadlines

Whether for university, hobby, or commercial uses, operating system development takes time. The Linux kernel took over one year of very dedicated work to get into a semblance of usefulness, and all Linus Torvalds did was mimic existing and well-documented behaviour to get an already-existing userspace to run on it. Moreover, for every project as successful as Linux, there are literally hundreds of projects that consumed a man-year or more of work without ever getting as far as hosting a functional shell.

Therefore, plan a reasonable road map of what you want to get done. Do not assume that in 3 months your OS will have a GUI and voice recognition, because operating system development does not contain any RAD tools in it at all. In fact, it is completely void of them. (void. It's a joke. Get it?)

End Goal Definition

When starting out a project, you should estimate your end goals, your end users, the purpose for which the project is developed, etc. Operating system development is no different to this. Having a rough idea about your goals will give you motivation and the direction where you need to head. However, don't get stuck at your initial end goals when something better comes to your mind.

Unfortunately, many OS developers don't estimate what their final OS will look like; therefore, they don't know in which direction to head and resort to asking, "what next?"

It should be mentioned, however, that to estimate your end goals you should be aware of the whole (technical) concept of how existing operating systems work.

Commercial OSDev

Don't get your mind set that building such a great OS will make you rich. If anything, history has shown us that the best operating systems never receive any commercial success, while the ones that have a near total lack of design and inspiration do, because of clever business moves and being in the right place, at the right time, with the right cover-up.

Despite that, and despite the relative emptiness of the "Jobs" section of the forum, a few sole-developer operating systems have had some commercial success. One example is 01000101 with "Drop-in Network Security", an OS designed to act as a specialized deep-packet analyzer. Note that it's specialized to a particular application. To market your OS in this way, you need to understand the application just as well as you do OS dev, perhaps more, and you need to be quite good at coding both. Your customers will expect you to be professional.

Another possibility is marketing your OS to companies which presently use MS-DOS for process control. This may seem easier in that you're not responsible for the final application, but you will need to be professional and responsive to your clients problems and queries. They may require long-term support. They may not like a feature you really want to implement.

Assuming It Will Go Nowhere

In contrast to the above, some people assume their OS will go nowhere. For this reason, their projects have ugly code, don't take important aspects into account, and in general rely on ugly hacks. Worst of all, they make decisions that don't result in usability and extensibility. This way, their assumption becomes a self-fulfilling prophecy.

In reality, while the chances of getting your OS running outside of your test machines are low, there are enough advanced OS projects that started from this very community.

Avoid Ignorance

Beginners often ask "What is the easiest way to do X?" rather than "What is the best, proper, correct way to do X?" This is dangerous as the newcomer doesn't invest time into understanding the superior way to implement something but instead picks a conceptually simpler method copied from a tutorial. Indeed, the simpler route is often too simple and ends up causing more problems in the long run, because the beginner is ignorant of the superior alternative and doesn't know when it is better to switch. What's so bad about taking the hard route instead?

Common examples include being too lazy to use a Cross-Compiler, not understanding Real Mode, Unreal Mode, Protected Mode, and Long Mode and jumping far too fast from one to the other without first understanding how to gather all of the vital configuration and use all of its capabilities fully (especially detecting basic capabilities with the BIOS during boot and initialization), relying on BIOS calls that are not standard, not learning to write hardware drivers in your own OS and under Windows and Linux for greatest convenience, exposing test capabilities globally, using flat binaries instead of ELF, not learning about executable, archive, graphics, document and other file formats and compression algorithms, and so on. Experienced developers use the superior alternatives for a reason, which you might not understand yet. Experienced developers choose to use the inferior method in some cases, but that's because they can carefully analyze whether it is appropriate, and they know when not to use it. As a beginner or intermediate developer you will likely not know these methods and technologies well enough to reason whether the inferior solution is good enough for you. Remember that if you oppose a method, you should know it well enough to know everything that is wrong (and right) with it. Either way, laziness and ignorance only leads to trouble down the road. When in doubt, pick what appears to be the conservative choice rather than the simpler.

Design

GUI Design

It will likely take you several years, starting from scratch, to get to the point where you actually do anything GUI-related. The looks of a GUI are secondary at best, as they can easily be changed retroactively; what really decides the usability of a GUI is the functionality, and that isn't expressed in mock-up graphics. If your aim is creating a better look instead of a better OS, consider implementing an X Window Manager or a Desktop Environment instead of a whole OS.

Popularity

My OS will be more popular than Windows, Mac OS, and Linux!

This is extremely unlikely. To achieve this requires quite a lot of time, money, and knowledge. Not everyone will download your operating system because:

• they may not know what an operating system is or how to install one
• your operating system has security risks
• your operating system supports fewer applications
• your operating system is not fully functional (minimal command line interface or bad GUI)

You will be lucky to get a few people to use yours. The only reason why the popular operating systems of today are popular is because they were available and addressed needs decades ago when there were fewer alternatives.

Memory usage

I want to use less than a couple of kilobytes for my OS!

Sorry, that's probably impossible. An OS using such little memory will be almost unusable. Forget about filesystem drivers, disk drivers, usb drivers, etc. For developing something small, just make a simple bootloader, not an OS.

Another option would be to try a native Forth; it's like a little OS with the kernel, command interpreter, and assembler all in one tiny binary! Parts of the kernel can even be scripted. It doesn't even need a filesystem; many Forths directly edit and load disk sectors. There's a big catch: Forth is not light on skill. Writing good Forth is a whole other planet which programmers would need to understand to get beyond the bare basics. Surprisingly, some Forth fans are good at writing Forth interpreters but are poor Forth programmers. An OS developer will need to be a good(-ish) Forth programmer to fit OS design concepts to Forth, or to figure out which concepts are not needed. Otherwise, the OS would be much larger than it needs to be. ;)

It can take a lot of time and thought to write compact Forth code. Charles Moore, inventor of Forth, wrote a CAD program which was structured around just 5 lines of code; it took him about 2 years.

OS Emulation

My OS will be able to run programs from Windows, Mac OS, Linux, and even PDP-11 programs!!!

I'm sorry to burst your bubble, but it probably won't. Emulating even just a single system takes years of work, especially when it's proprietary, such as Windows or Mac OS (Linux is probably the easiest out of the four.). Wine, despite being in development since 1993 and being written in userspace, has still got problems with many programs.

So instead of focusing on emulating other systems, focus on your own. Design it, develop it, and be friends with it.

Programming Languages

Some languages are well suited to write a kernel, others are less so. Read the page about using some language other than C.

Kernel Image

Booting Problems

Especially in early stages and with a self-built bootloader, the reason for booting problems is frequently that too few sectors are fetched from disk. Either adjust the amount of sectors you fetch from disk, or have the boot loader/second stage loader parse the file system.

Troubleshooting / Asking for Help

Before asking for help on the forums or IRC, you should be taking all the possible steps to diagnose the nature of the problem yourself. In the case of problems like triple faults or "random" exceptions, it's a common mistake to make assumptions about the cause of a problem. Make use of a debugger or print statements to locate the exact point when an exception occurs. Using and emulator and a debugger (such as GDB and Bochs/QEMU) will help you to locate problems which are difficult to trace. If you provide some theory about the problem and actions you have already taken to solve it, people will be able to help you much easier (even if your theory is not correct, it at least gives people an idea of your views on the problem and the strategies you might have already tried, as well as what you might have missed).

In general, put as much work as possible into solving a problem yourself before asking others for help. Before you post, ask yourself, "Have I done everything I can to diagnose and solve this problem?" Often you will learn a lot in the process, and you'll likely be able to solve the problem (and similar problems in the future) yourself, without the help of others, which is a good thing. When you ask for help, provide the code relevant to your problem. However, the problem may be located somewhere else, so give others a way to look at the other parts of your code as well (if you're using something like GitHub or Bitbucket, this is slightly easier, but there are certainly many other ways).

In regards to the forum, read the forum rules. They are required reading and will improve the quality of your posts and make people more likely to help you. On the IRC channel, if you ask a question, don't expect an answer within 10 seconds, or even 5 minutes. Other people have lives too. If you need to leave to do something and want to check if you missed anything, there are logs available for you to look through. See the links in the IRC topic for those.

For a much more in-depth guide on how to be a good community member when asking for help, consult How to ask questions. It's a great read, and ideally, you should read the whole thing before asking for help anywhere (it's not that long, and well worth the read).

Promotion

Naming

Naming is usually the last problem to be solved, even while we all wish for a cool name for our cool concept. Since the "coolness" of a name is a matter of taste, we cannot help you in finding it. Moreover, if you tie your project name to a certain feature, you might discover somewhere down the road that no concept is perfect and that you want to change what you initially thought was a key feature. Nothing would be more stupid than to not evolve just because you "want to stick to a name"...

A lot of good information on naming can be found in this thread. Simply put, naming your operating system <name>OS (JackOS, FredOS, etc.) may seem like a good idea, until you get a second project member. A good idea (courtesy of Solar) is to choose a codename (like Longhorn, Chicago, etc.) and then make up a better name closer to release time.

Project Website

Many newcomers to osdev create project websites before they have anything worth showing on the website. There's no value in creating a website making dramatic claims about the future plans for your project before you've got any idea where your project is heading or any code, screenshots, or downloads to showcase what you've already produced. Such a website looks dead and creates a bad reputation. Announcing your website (such as in the Announcements forum on the OSDev.org forums) or linking to it in your signature when there's nothing more than a "welcome to <insert fancy project name here>" message on it just makes people lose interest in your project before you've even started, and if/when you do eventually produce something worth showing off, you'll be faced with an already-terrible reputation that will be hard to overcome.

Teamwork

The number one beginner mistake is seen in the Announcements forum. It usually comes in one of two forms, although the forms have quite a bit of overlap:

Community Projects

Don't overestimate your chances of getting people interested in your project. Even the more successful projects usually consist of one, perhaps two people actually working on the code. And that is not due to a lack of need.

Brooks' Law states that the more people there are on a project, the longer the project takes. The only way around this is to split the project into parts which you get people working on, and only on. Good luck.

Recruiting

There are some things you need for standing a chance (and avoid being painfully told that you are a failure):

1. If you have no established codebase, people will not join because they can see you lack experience and expect the project to fail.
2. If you lack a (worked out) design, people will not join you because they can't see how your OS is more interesting than their own design.
3. If your reputation doesn't precede you, especially the more experienced people will be very wary of you and lack the trust to join.
4. If you don't have project management skills, the few rare people that do join will quit shortly because they are discussing stuff and do not get to code.

People that join nevertheless are usually worse programmers than the people for which this list was composed.

• OS Development