An overhead of running program on Linux vs embedded bare-metal

Question

Let's say I have some multi-core ARM Cortex-A.

Now I have there a Linux, with RT_PREEMPT patch (so called real-time Linux). I run a processes with affinity set to one core and high priority (above 50).

Thanks to real-time patch, even the kernel scheduler cannot preempt my process, and thanks to affinity it can run all the time on a single core (and OS on others).

In such conditions, is there any difference (i.e. overhead) of running such code on Linux vs bare-metal?
Will the performance be the same? If not, why?

• Rev: Well, wouldn't the overhead still be the whole operating system with all of its high level drivers and stuff?
• Marcus Müller: @Rev1.0 no, unless you can show that these drivers a) are used, and b) are significant less efficient than a driver that actually exists and is used on a bare-bone implementation.

Answer 1

Overhead does not relate to preemption. Preemption stops your process and runs another process. If you disable that for e.g. one CPU core, you have the CPU core alone for your process.

Still, there's an overhead if you do I/O through the Linux I/O functions instead e.g. controlling the I/O lines directly through your process. But, for any reasonably complicated I/O, you had to implement a function with similar overhead yourself and you can assume the Linux folks are better than you in that – more experience and more test cases.

So, the only way you could win the overhead race is with very basal I/O functions (e.g. bitbanging an exotic, fast protocol) you implement yourself in "your kernel" instead of running it through the various abstraction layers of the Linux GPIO subsystem and do the same in "your userspace process".

• Marcus Müller: Thank you very much. I get this far too often: "but kernel abstraction makes things slooow!" and the honest truth is that you'll probably want to do pretty much the same thing as the kernel if your usage of a device is sufficiently complex. And in that case, writing your own kernel-space code is usually the easier way to achieve success. So your answer is very good for my tortured soul.
• Janka: People sometimes feel insecure about claim using the GPIO subsystem will take about 20 times longers than a single out instruction. That's true but then, who does that for time-crictical things? If you have to shove out things that fast, think about DMA and do it much much faster.
• Marcus Müller: Exactly. On practically all ARM processors, all IO is memory-mapped, so there's literally zero kernel overhead if you use just map that memory in your process space.
• zupazt3: Yep, I forgot to mention that I mean running exactly the same C code bare-metal vs Linux. So even if using Linux I don't mean using anything that comes from Linux (drivers, systems calls etc). Just pure C. So I understand that then there is no any overhead of using Linux then? I was wondering about, for example, the fact that Linux uses virtual memory (while bare-metal uses direct addresses). If I declare some variables they will have virtual memory address on Linux. There is no overhead of translating virtual address to physical one? Some paging etc?
• Janka: The MMU is a piece of hardware, and it's the speed of the RAM. You always have virtual addresses in your program, not using the MMU simply means having only a fixed mapping table loaded into it on system start. If you want to disable paging for a page or process, you can advise the kernel to mark the mappings for your process as non-pageable with mlock() and mlockall().