HighPerTimer – High precision time keeping on Linux systems
MOTIVATION
In contemporary high-speed communication systems, by modelling autonomous mobile robots or in industrial control environments, acquisition of the system time can occur hundred thousand times a second and even more frequently. So the time of this system time acquisition as well as the CPU costs for this must be kept as low as possible. However, during the implementation of any kind of time critical applications, a number of pitfalls on the level of OS can appear soon. Though the Linux OS has already proved itself suitable for embedding into many sizes of projects and offers the ideal development platform, in some applications Linux still shows itself not enough effective. This project aims to solve those problems and improve accuracy of system calls for timing measurements along with reduced CPU resources. While the time acquisition via Unix system interfaces costs up to 2 microseconds, the suggested solution provides time acquisition means with CPU costs of about 100 nsec and less on a standard PC platform and about 200 ns on ARM Cortex A series. The approach for precise timing is available on user-space beyond the standard C++ library. Additionally, the library possesses microsecond-precise process suspension and resuming functions. This feature extends the horizon of time-critical applications in the industrial communications and automation, which can be implemented in high-level programming languages on Linux OS.
LIBRARY DESCRIPTION
The HPTimer (High Performance Timer) and the corresponding library HighPerTimer [7] present a simple interface for gathering time values from Linux user space. During the initialization stage HPTimer is assigned to the fastest and the most reliable time source available on the given platform. The pitfall here is different architectures use different timer hardware and the given library takes upon itself the identification of the platform and the timer source. Consequently, the further interaction by HPTimer occurs directly with time registers avoiding standard Linux system calls (e.g. clock_gettime or gettimeofday). On ARM Cortex A8/A9/A5 processors, for accessing time hardware, the library needs an additional device driver to be installed and loaded as a module beforehand, as the appropriate time registers are mapped into the protected kernel space. For PC platforms no special drivers are required.
The library provides the following features:
- automatic detection of the reliable time hardware and ability to change the default time source inside the main routine;
- manipulation with time accuracy up to nanoseconds;
- precise sleep functions with reduced CPU usage and appropriate interrupt handlers of the sleeping thread;
- a set of mathematical and comparison operators of an HPTimer C++ object;
- support of Intel, AMD, VIA, ARM Cortex A8/A9/A5 processors.
PERFORMANCE INFORMATION
Below the performance information of some processors is presented. In the table mean values of the cost of setting a timer are shown. Tests have been performed in a loop of 100 million values and possible peaks due to hardware interrupts have not been filtered.
| Processor (CPU) | Software clock Frequency | Cost of system call, µsec | Cost of setting HPTimer, µsec |
|---|---|---|---|
| Intel® Core™ i7-2600 | 1600 MHz | 0.031 | 0.017 |
| Intel® Core™ i5-3337U | 1800 MHz | 0.044 | 0.019 |
| AMD Opteron™ 4238 | 3300 MHz | 0.061 | 0.031 |
| AMD Athlon™ X2 BE-2350 | 1000 MHz | 1.117 | 1.063 |
| VIA Nano X2 U4025 | 1067 MHz | 0.102 | 0.038 |
| ARM Cortex A8 | 24 MHz | 1.262 | 0.391 |
| ARM Cortex A9 | 32.768 KHz1 | –2 | 0.508 |
| ARM Cortex A5 | 12 MHz | 2.069 | 0.636 |
Table 1: Examples of costs of setting timer on different processors
———————————-
1HPTimer uses a 19 MHz clock instead
2In the case of the ARM Cortex A9 processor, the system timer has 32.768 K?z frequency with 30.518 usec resolution respectively. In fact, it means that with the standard system call only intervals of more than 30 µsec can be tracked, even with the monotonic time source and CLOCK_MONOTONIC flag
The important advantage of HighPerTimer is a significantly improved precision of sleep functions. In the context of this paper, the term „miss“ of sleep times refers to a delayed wake up from the synchronous sleep call, whereby the quantity of this miss is the time difference between the real return time of sleep against the target wake-up time. Though in the version 2.6 of the Linux kernel, a redesign of timers took place, even in the 3.x kernel versions, this miss is still in the range of 50 µsec (for PC platform) and 100-400 µsec (for ARM processors). So often CPU-costly busy-waits are inserted into communication and control programs to assure precise protocol timings. At this point, the HighPerTimer can provide precise sleep functions, where the CPU overhead during the sleep keeps at 1-1.89 % in contrary to 100 % of busy waits. At the same time the benefits of very accurate process resuming times on long waits are maintained. Below the table of the comparison miss values is shown. It should be considered that for all methods the miss value depends on the sleep interval, so the Table 2 consists of two part, where sleep time is more then 1 msec and where is less. The measurements were performed on the Intel® Core™ i7-2600 processor, for the calculating the mean value it was used a loop of 10000 steps and total performance time of measurements is equal about 830 minutes.
| Sleep time >= 1 msec | Sleep time < 1 msec | CPU usage | |
|---|---|---|---|
| Mean miss, µsec | Mean miss, µsec | ||
| System sleep | 61.985 | 50.879 | 0.016 % |
| Busy-waiting loop | 0.160 | 0.070 | 99.681 % |
| HPTimer sleep | 0.258 | 0.095 | 1.892 % |
Table 2: The comparison of miss values of different sleep methods
CURRENT RESEARCH STAGE
At the current stage of the project, the library is being tested under different real-time environment. Applying available Linux real-time scheduler policies and addition RT patches, the guaranties of being precise with HighPerTimer library is being investigated. As known, being able to run RT systems under the standard Linux OS ensures that all state machine deadlines are met. However, the typical RT system doesn’t ensure enough precision and fine-grain time evaluation. That’s way, a special timing access is needed beyond the default programming API to the system time function. Therefore, this approach additionally targets to guarantee response within strict time constraints on single- and multiprocessor platforms with the perspective to satisfy requirements of the existing real time systems.
The HighPerTimer library is an open-source project and the current version is ready to use. Please contact us in case of questions.
Workshops, Talks:
1. High-precision sleep operations from Linux user space, Open Source Automation Development Lab (OSADL) Networking Day, May 2016
2. High Precision Timing from Linux User Space on SoC, Open Source Automation Development Lab (OSADL) Networking Day, May 2015
Publications:
1. I. Fedotova, B. Krause and E. Siemens, Upper Bounds Prediction of the Execution Time of Programs Running on ARM Cortex-A Systems, IFIP AICT (Advances in Information and Communication Technology book series), Springer, Cham, Volume 499// DoCEIS 2017, May 2017, pp. 220-229
2. K. Karpov, I. Fedotova, E. Siemens and D. Kachan, Impact of Virtualization on Timing Precision under Stressful Network Conditions”. Proc. of: the 17th IEEE International Conference on Smart Technologies (EUROCON), Ohrid, Macedonia, 2017, pp.157-163.
3. K. Karpov, I. Fedotova and E. Siemens, Impact of Machine Virtualization on Timing Precision for Performance-critical Tasks Journal of Physics:Conference Series (JPCS), ICMIE 2017. – Vol. 870. Issue 1. – P. 52-60.
4. I. Fedotova, B. Krause and E. Siemens, Applicability of Extreme Value Theory to the Execution Time Prediction of Programs on SoCs. Proc. of: The 5th International Conference on Applied Innovations in IT (ICAIT), Köthen, Germany, March 2017, pp. 71-80.
5. I. Fedotova, E. Siemens, System Time Issues for the ARM Cortex A8 Processor. Proc. of: the 2nd International Conference on Applied Innovations in IT (ICAIIT), Köthen, Germany, March 2014, pp. 7-9.
6. I. Fedotova, E. Siemens, Self-configurable time source initialization for obtaining high-precision user-space timing. Vestnik SibSUTIS, 4.2012, Nov. 2012, Novosibirsk, Russia, pp. 22-30.
7. I. Fedotova, E. Siemens, H. Hu, A High-precision Time Handling Library. JCC Journal, USA, David publishing Company, Number 7 (Volume 10) November 2013, pp. 1076 – 1086.
8. I. Fedotova, E. Siemens, Usage of high-precision timers in the wind turbines control systems. Supercomputers Journal, Publishing House SCR-Media LTD, Moscow, Russia, Number 16, winter 2013, pp. 38-42.
9. I. Fedotova, The implementation of new methods of high-precision sleeps for the Linux OS. Proc. of: the 15th International Supercomputer Conference Scientific services in the Internet: All Facets of Parallelism, Novorossiysk, Russia, Sept 2013, pp. 567-572.