Linux's thread local storage implementation

Question

__thread Foo foo;

How is foo actually resolved? Does the compiler silently replace every instance of foo with a function call? Is foo stored somewhere relative to the bottom of the stack, and the compiler stores this as "hey, for each thread, have this space near the bottom of the stack, and foo is stored as 'offset x from bottom of stack'"?

Answer 1

It's a little complicated (this document explains it in great detail), but it's basically neither. Instead the compiler puts a special .tdata section in the executable, which contains all the thread-local variables. At runtime, a new data section for each thread is created with a copy of the data in the (read-only) .tdata section, and when threads are switched at runtime, the section is also switched automatically.

The end result is that __thread variables are just as fast as regular variables, and they don't take up extra stack space, either.

• Roger Pate: @anon: Linux threads are actually separate processes with most of their resources shared (e.g. most of their memory space), but all resources don't have to be shared; man 2 clone may be helpful. You can write a test program for the situation you describe.
• anon: How do I print out the address of .tdata that belongs to a particular thread?
• Dean Harding: I don't believe you can, but the address of individual variables will be different so you can just print out the address of a single __thread variable from different threads and see that they're different.
• jleahy: For people who are too lazy to read the attached document, the missing piece is that the address of the thread local storage region is stored in the GS segment register (FS in x86-64), so to access TLS you just do mov eax, GS:ptr.
• jrpear: Also, the fs/gs register is set by the threading library using clone with CLONE_SETTLS