System: ASUS A8V, Athlon 64 X2 4400+, NetBSD simonb-timecounters, TSC
timecounter, MP mode
Timecounters provide a scalable MP
capable
high precision time keeping infrastructure. They where developed by
Poul-Henning Kamp for the FreeBSD project. For more information have a
look at his
paper:
Timecounters: Efficient and precise timekeeping in SMP kernels.
Poul-Henning Kamp
The FreeBSD Project
ABSTRACT
The FreeBSD timecounters are an architecture-independent
implementation of a binary timescale using whatever hardware support
is at hand for tracking time. The binary timescale converts using
simple multiplication to canonical timescales based on micro- or
nano-seconds and can interface seamlessly to the NTP PLL/FLL
facilities for clock synchronisation. Timecounters are implemented
using lock-less stable-storage based primitives which scale
efficiently in SMP systems. The math and implementation behind
timecounters will be detailed as well as the mechanisms used for
synchronisation.
Possible Future Developments:
- Cleanup Timescale usage: Some services need wall clock time
others need monotonic time. There are many areas in the kernel where
not the right scale is uses (e. g. timeout are calculated using wall
clock time though the should actuall use monotonic time)
- timeout functions for select/poll/kevent/nanosleep have been
converted
- overhaul all kernel internal timekeeping/-storage to use struct
bintime and just convert at the API boundaries.
- implement higher precision timeouts when single shot hw timers
are available (could get also rid of periodic interrupt and maybe even
reduce power consumption
Graphics before and after:
The following graphs show time offsets
and clock frequency corrections applied by ntp, before and after
timecounters. Pay particular attention to the difference in
vertical scale (up to a factor of 21000 for the SOEKRIS case)
between the before and after graphs.
No Timecounters: Multiprocessor Stratum
1 operation before on the A8V
Timekeeping on multiprocessor before time counters
Timecounters: Stratum 1 (
Meinberg
GPS167) - Sample of a busy A8V + Athlon 64 X2 doing 2*setiathome +
build.sh -j 3 release:
Loopfilter graph
At this state the synchronisation
results are usually strongly influenced by temperature changes and
interrupt latencies.
No Timecounters: SOEKRIS4801 with NetBSD-current:
Timekeeping on SOEKRIS without Timecounters
Timecounters: Sample of an idle SOEKRIS4801 Stratum 1 Server (
Meinberg
GPS167)
Loopfilter graph
Timecounters: Now if we could get rid of the up to 8us spikes I am
seeing it would look more like this:
Loopfilter graph
Converting a port to Timecounters
Ideally all ports
should be converted to timecounters so that the old microtime
implementation can be completely retired and we can get rid of the dual
code (__HAVE_TIMECOUNTER). The requirements for timecounter support are
minimal.
Requirements for timecounter implementations:
- fixed frequency incrementing counter wrapping at a power of two.
- counter must not wrap within 2/hz - if the counter is wrapping
too fast consider increasing hz (e.g. ELAN SC520 boards need a HZ of
150 to utilize the counter in that architecture)
- if no hardware counters exist for microtime interpolation then a
port may quickly be converted by fixing the setting of the clock and
defining __HAVE_TIMECOUNTER as a default timecounter "clockinterrupt"
based on clock ticks is always registered.
- count down counters must be converted to up counting ones (result = MAX_COUNT - current_count).
- counters not wrapping at a power of two need to be converted. see
also arch/i386/isa/clock.c
for examples.
- you need mutex(IPL_SPLHIGH) guards
when
you need to manipulate/access state that is updated by
clock interrupts or by counter reading. This includes chip access
(address, data phases) and local state manipulation for emulation of
power of two timers.
- for MP operation you need to insure that the returned counter
values are consistent across cpus. either a single and atomically
readable hw counter is available..