2 Time and Time Correction in Erlang
2.1 New Extended Time Functionality
As of OTP 18 (ERTS version 7.0) the time functionality of Erlang has been extended. This both includes a new API for time, as well as time warp modes which alters the behavior of the system when system time changes.
The default time warp mode has the same behavior as before, and the old API will still work, so you are not required to change anything unless you want to. However, you are strongly encouraged to use the new API instead of the old API based on erlang:now/0. erlang:now/0 has been deprecated since it is and forever will be a scalability bottleneck. By using the new API you will automatically get scalability and performance improvements. This will also enable you to use the multi time warp mode which improves accuracy, and precision of time measurements.
2.2 Some Terminology
In order to make it easier to understand this document we first define some terminology. This is a mixture of our own terminology (Erlang/OS system time, Erlang/OS monotonic time, time warp) and globally accepted terminology.
Monotonically Increasing
In a monotonically increasing sequence of values, all values that have a predecessor are either larger than, or equal to its predecessor.
Strictly Monotonically Increasing
In a strictly monotonically increasing sequence of values, all values that have a predecessor are larger than its predecessor.
UT1
Universal Time. Based on the rotation of the earth. Conceptually mean solar time at 0° longitude.
UTC
Coordinated Universal Time. UTC almost align with UT1, however, UTC uses the SI definition of a second which is not exactly of the same length as the second used by UT1. This means that UTC slowly drifts from UT1. In order to keep UTC relatively in sync with UT1, leap seconds are inserted, and potentially also deleted. That is, an UTC day may be 86400, 86401, or 86399 seconds long.
POSIX Time
Time since Epoch. Epoch is defined to be 00:00:00 UTC, January 1, 1970. A day in POSIX time is defined to be exactly 86400 seconds long. Strangely enough Epoch is defined to be a time in UTC, and UTC have another definition of how long a day is. Quoting the Open Group "POSIX time is therefore not necessarily UTC, despite its appearance". The effect of this is that when an UTC leap second is inserted, POSIX time either stops for a second, or repeats the last second. If an UTC leap second would be deleted (has never happened yet), POSIX time would make a one second leap forward.
Time Resolution
The shortest time interval that can be distinguished when reading time values.
Time Precision
The shortest time interval that can be be distinguished repeatedly and reliably when reading time values. Precision is limited by the resolution, but resolution and precision might differ significantly.
Time Accuracy
The correctness of time values.
Time Warp
A time warp is a leap forwards or backwards in time. That is, the difference of time values taken before and after the time warp will not correspond to the actual elapsed time.
OS System Time
The operating systems view of POSIX time. It can be retrieved by calling os:system_time(). This may or may not be an accurate view of POSIX time. This time may typically be adjusted both backwards and forwards without limitation. That is, time warps may be observed. You can get information about the Erlang runtime system's source of OS system time by calling erlang:system_info(os_system_time_source).
OS Monotonic Time
A monotonically increasing time provided by the operating system. This time does not leap and have a relatively steady frequency although not completely correct. However, it is not uncommon that the OS monotonic time stops if the system is suspended. This time typically increase since some unspecified point in time that is not connected to OS system time. Note that this type of time is not necessarily provided by all operating systems. You can get information about the Erlang runtime system's source of OS monotonic time by calling erlang:system_info(os_monotonic_time_source).
Erlang System Time
The Erlang runtime systems view of POSIX time. It can be retrieved by calling erlang:system_time(). This time may or may not be an accurate view of POSIX time, and may or may not align with OS system time. The runtime system works towards aligning the two system times. Depending on time warp mode used, this may be achieved by letting the Erlang system time perform a time warp.
Erlang Monotonic Time
A monotonically increasing time provided by the Erlang runtime system. The Erlang monotonic time increase since some unspecified point in time. It can be retrieved by calling erlang:monotonic_time(). The accuracy, and precision of Erlang monotonic time heavily depends on the accuracy and precision of OS monotonic time, the accuracy and precision of OS system time as well as on the time warp mode used. On a system that is lacking OS monotonic time, the Erlang monotonic time can only guarantee monotonicity and can more or less not give any other guarantees. The frequency adjustments made to the Erlang monotonic time depends on the time warp mode used.
Internally in the runtime system the Erlang monotonic time is the "time engine" that is used for more or less everything that has anything to do with time. All timers regardless of it is a receive ... after timer, BIF timer, or a timer in the timer module are triggered relative Erlang monotonic time. Even Erlang system time is based on Erlang monotonic time. By adding current Erlang monotonic time with current time offset you get current Erlang system time. Current time offset can be retrieved by calling erlang:time_offset/0.
2.3 Introduction
Time is vital to an Erlang program and, more importantly, correct time is vital to an Erlang program. As Erlang is a language with soft real time properties and we have the possibility to express time in our programs, the Virtual Machine and the language has to be very careful about what is considered a correct point in time and in how time functions behave.
In the beginning, Erlang was constructed assuming that the wall clock time in the system showed a monotonic time moving forward at exactly the same pace as the definition of time. That more or less meant that an atomic clock (or better) was expected to be attached to your hardware and that the hardware was then expected to be locked away from any human (or unearthly) tinkering for all eternity. While this might be a compelling thought, it's simply never the case.
A "normal" modern computer can not keep time. Not on itself and not unless you actually have a chip level atomic clock wired to it. Time, as perceived by your computer, will normally need to be corrected. Hence the NTP protocol that together with the ntpd process will do it's best to keep your computers time in sync with the "real" time in the universe. Between NTP corrections, usually a less potent time-keeper than an atomic clock is used.
But NTP is not fail safe. The NTP server can be unavailable, the ntp.conf can be wrongly configured or your computer may from time to time be disconnected from the internet. Furthermore you can have a user (or even system administrator) on your system that thinks the right way to handle daylight saving time is to adjust the clock one hour two times a year (a tip, that is not the right way to do it...). To further complicate things, this user fetched your software from the internet and has never ever thought about what's the correct time as perceived by a computer. The user simply does not care about keeping the wall clock in sync with the rest of the universe. The user expects your program to have omnipotent knowledge about the time.
Most programmers also expect time to be reliable, at least until they realize that the wall clock time on their workstation is of by a minute. Then they simply set it to the correct time, maybe or maybe not in a smooth way. Most probably not in a smooth way.
The amount of problems that arise when you expect the wall clock time on the system to always be correct may be immense. Therefore Erlang introduced the "corrected estimate of time", or the "time correction" many years ago. The time correction relies on the fact that most operating systems have some kind of monotonic clock, either a real time extension or some built in "tick counter" that is independent of the wall clock settings. This counter may have microsecond resolution or much less, but generally it has a drift that is not to be ignored.
2.4 Time Correction
If time correction is enabled, the Erlang runtime system will make use of both OS system time and OS monotonic time, in order to make adjustments of the frequency of the Erlang monotonic clock. Time correction will ensure that Erlang monotonic time will not warp, and that the frequency is relatively accurate. The type of adjustments made to the frequency depends on the time warp mode used. This will be discussed in more details in the time warp modes section below.
By default time correction will be enabled if support for it on the specific platform exist. Support for it includes both an OS monotonic time provided by the OS, and an implementation in the Erlang runtime system utilizing the OS monotonic time. You can check if your system has support for OS monotonic time by calling erlang:system_info(os_monotonic_time_source), and you can check if time correction is enabled on your system by calling erlang:system_info(time_correction).
Time correction is enabled or disabled by passing the +c [true|false] command line argument to erl.
If time correction is disabled, Erlang monotonic time may warp forwards, it may stop and even freeze for extended periods of time, and there are no guarantees that the frequency of the Erlang monotonic clock is accurate or stable.
You typically never want to disable time correction. Previously there was a performance penalty associated with time correction, but nowadays it is most often the other way around. By disabling time correction you are likely to get bad scalability, bad performance, and bad time measurements.
2.5 Time Warp Safe Code
Time warp safe code is code that is able to handle a time warp of Erlang system time.
erlang:now/0 behaves very bad when Erlang system time warps. When Erlang system time do a time warp backwards, the values returned from erlang:now/0 will freeze (if you disregard the micro second increments made due to the actual call) until OS system time reach the point of the last value returned by erlang:now/0. This freeze might continue for very long periods of time. It might take years, decades, and even longer than this until the freeze stops.
All uses of erlang:now/0 are not necessarily time warp unsafe. If you do not use it to get time, it will be time warp safe. However all uses of erlang:now/0 are suboptimal from a performance and scalability perspective. So you really want to replace the usage of it with other functionality. For examples of how to replace the usage of erlang:now/0, see the Dos and Donts section.
2.6 Time Warp Modes
Current Erlang system time is determined by adding current Erlang monotonic time with current time offset. The time offset is managed differently depending on which time warp mode you use. The time warp mode is set by passing the +C [no_time_warp|single_time_warp|multi_time_warp] command line argument to erl.
No Time Warp Mode
The time offset is determined at runtime system start and will after this not change. This is the default behavior. Not because it is the best mode (which it isn't). It is default only because this is how the runtime system always has behaved up until ERTS version 7.0, and you have to ensure that your Erlang code that may execute during a time warp is time warp safe before you can enable other modes.
Since the time offset is not allowed to change, time correction needs to adjust the frequency of the Erlang monotonic clock in order to smoothly align Erlang system time with OS system time. A big downside of this approach is that we on purpose will use a faulty frequency on the Erlang monotonic clock if adjustments are needed. This error may be as big as 1%. This error will show up in all time measurements in the runtime system.
If time correction is not enabled, the Erlang monotonic time will freeze when the OS system time leap backwards. The freeze of the monotonic time will continue until OS system time catch up. The freeze may continue for a very long time. When OS system time leaps forwards, Erlang monotonic time will also leap forward.
Single Time Warp Mode
This mode is more or less a backwards compatibility mode as of its introduction.
On an embedded system it is not uncommon that the system has no power supply at all, not even a battery, when it is shut off. The system clock on such a system will typically be way off when the system boots. If the no time warp mode is used, and the Erlang runtime system is started before the OS system time has been corrected, the Erlang system time may be wrong for a very long time, even centuries or more.
If you for some reason need to use Erlang code that is not time warp safe, and you need to start the Erlang runtime system before the OS system time has been corrected, you may want to use the single time warp mode. Note that there are limitations to when you can execute time warp unsafe code using this mode. If it is possible to only utilize time warp safe code, it is much better to use the multi time warp mode instead.
Using the single time warp mode, the time offset is handled in two phases:
- Preliminary Phase
-
The preliminary phase starts when the runtime system starts. A preliminary time offset based on current OS system time is determined. This offset will from now on be fixed during the whole preliminary phase.
If time correction is enabled, adjustments to the Erlang monotonic clock will be made to keep its frequency as correct as possible, but no adjustments will be made trying to align Erlang system time and OS system time. That is, during the preliminary Erlang system time and OS system time might diverge from each other, and no attempt to prevent this will be made.
If time correction is disabled, changes in OS system time will effect the monotonic clock the same way as when the no time warp mode is used.
- Final Phase
-
The final phase begin when the user finalize the time offset by calling erlang:system_flag(time_offset, finalize). The finalization can only be performed once.
During finalization, the time offset is adjusted and fixated so that current Erlang system time align with current OS system time. Since the time offset may change during the finalization, the Erlang system time may do a time warp at this point. The time offset will from now on be fixed until the runtime system terminates. If time correction has been enabled, the time correction will from now on also make adjustments in order to align Erlang system time with OS system time. When the system is in the final phase it behaves exactly as in the no time warp mode.
In order for this to work properly there are two requirements that the user needs to ensure are satisfied:
- Forward Time Warp
The time warp made when finalizing the time offset can only be done forwards without encountering problems. This implies that the user has to ensure that the OS system time is set to a time earlier or equal to actual POSIX time before starting the Erlang runtime system. If you are not completely sure the OS system time is correct, set it to a time that is guaranteed to be earlier than actual POSIX time before starting the Erlang runtime system just to be safe.
- Finalize Correct OS System Time
The OS system time needs to be correct when the the user finalizes the time offset.
If these requirements are not fulfilled, the system may behave very bad.
Assuming that the requirements above are fulfilled, time correction is enabled, and that the OS system time is adjusted using some time adjustment protocol like NTP or similar, only small adjustments of the Erlang monotonic time should be needed in order to keep system times aligned after finilization. As long as the system is not suspended, the largest adjustments needed should be for inserted (or deleted) leap seconds.
In order to be able to use this mode you have to ensure that all Erlang code that will execute in both phases are time warp safe.
Code that only execute in the final phase does not have to be able to cope with the time warp.
Multi Time Warp Mode
Multi time warp mode in combination with time correction is the preferred configuration. This since, on almost all platforms, the Erlang runtime system will have better performance, will scale better, will behave better, and since the accuracy, and precision of time measurements will be better. Only Erlang runtime systems executing on ancient platforms will benefit from another configuration.
The time offset may change at any time without limitations. That is, Erlang system time may perform time warps both forwards and backwards at any time. Since we align the Erlang system time with the OS system time by changing the time offset, we can enable a time correction that tries to adjust the frequency of the Erlang monotonic clock to be as correct as possible. This will make time measurements using the Erlang monotonic time more accurate and precise.
If time correction is disabled, Erlang monotonic time will leap forward if OS system time leaps forward. If the OS system time leaps backwards, Erlang monotonic time will stop briefly but it does not freeze for extended periods of time. This since the time offset is changed in order to align Erlang system time with OS system time.
In order to be able to use this mode you have to ensure that all Erlang code that will execute on the runtime system is time warp safe.
2.7 The New Time API
The old time API is based on erlang:now/0. The major issue with erlang:now/0 is that it was intended to be used for so many unrelated things. This tied these unrelated operations together and unnecessarily caused performance, scalability as well as accuracy, and precision issues for operations that do not need to have such issues. The new API spreads different functionality over multiple functions in order to improve on this.
In order to be backwards compatible erlang:now/0 will remain as is, but you are strongly discouraged from using it. A lot of uses of erlang:now/0 will also prevent you from using the new multi time warp mode which is an important part of this new time functionality improvement.
Some of the new BIFs on some systems, perhaps surprisingly, return negative integer values on a newly started run time system. This is not a bug, but a memory usage optimization.
The new API consists of a number of new BIFs:
and a number of extensions of existing BIFs:
The New Erlang Monotonic Time
The Erlang monotonic time as such is new as of ERTS version 7.0. It has been introduced in order to be able to detach time measurements such as elapsed time from calender time. It is very common that one is interested in measuring elapsed time or specifying a time relative to another point in time without having any need to know what the involved times are in UTC or any other globally defined time scale. By introducing a time scale that has a local definition of where it starts, it is possible to manage time that do not concern calender time on that time scale. Erlang monotonic time use such a time scale with a locally defined start.
The introduction of Erlang monotonic time gives us the possibility to adjust the two Erlang times (Erlang monotonic time and Erlang system time) separately. By doing this, accuracy of elapsed time does not have to suffer just because the system time happened to be wrong at some point in time. Separate adjustments of the two times are only performed in the time warp modes, and only fully separated in the multi time warp mode. All other modes than the multi time warp mode are there for backwards compatibility reasons, and when using these the accuracy of Erlang monotonic time suffer since the adjustments of Erlang monotonic time in these modes are more or less tied to the Erlang system time.
The adjustment of system time could have been made smother than using a time warp approach, but we think that would be a bad choice. Since we are able to express and measure time that aren't connected to calender time by the use of Erlang monotonic time, it is better to expose the change in Erlang system time immediately. This since it makes it possible for the Erlang applications executing on the system to react on the change in system time as soon as possible. This is also more or less exactly how most OSes handle this (OS monotonic time and OS system time). By adjusting system time smoothly we would just hide the fact that system time changed and make it harder for the Erlang applications to react to the change in a sensible way.
In order to be able to react to a change in Erlang system time you have to be able to detect that it happened. The change in Erlang system time occurs when current time offset is changed. We have therefore introduced the possibility to monitor the time offset using erlang:monitor(time_offset, clock_service). A process monitoring the time offset will be sent a message on the following format when the time offset is changed:
{'CHANGE', MonitorReference, time_offset, clock_service, NewTimeOffset}Unique Values
Besides reporting time erlang:now/0 also produce unique and strictly monotonically increasing values. In order to detach this functionality from time measurements we have introduced erlang:unique_integer().
Dos and Don'ts
Previously erlang:now/0 was the only option for doing quite a lot of things. We will look at a few different things erlang:now/0 could be used for, and how you want to do this using the new API:
Retrieve Erlang System Time
use erlang:now/0 in order to retrieve current Erlang system time.
use erlang:system_time/1 in order to retrieve current Erlang system time on the time unit of your choice.
If you want the same format as returned by erlang:now/0, use erlang:timestamp/0.
Measure Elapsed Time
take timestamps with erlang:now/0 and calculate the difference in time with timer:now_diff/2.
take timestamps with erlang:monotonic_time/0 and calculate the time difference using ordinary subtraction. The result will be in native time unit. If you want to convert the result to another time unit you can do this using erlang:convert_time_unit/3.
Another easier way of doing this is to use erlang:monotonic_time/1 with desired time unit. However, you may lose accuracy, and precision this way.
Determine Order of Events
determine the order of events by saving a timestamp with erlang:now/0 when the event happens.
determine the order of events by saving the integer returned by erlang:unique_integer([monotonic]) when the event happens. These integers will be strictly monotonically ordered on current runtime system instance corresponding to creation time.
Determine Order of Events With Time of the Event
determine the order of events by saving a timestamp with erlang:now/0 when the event happens.
determine the order of events by saving a tuple containing monotonic time and a strictly monotonically increasing integer like this:
Time = erlang:monotonic_time(),
UMI = erlang:unique_integer([monotonic]),
EventTag = {Time, UMI}These tuples will be strictly monotonically ordered on the current runtime system instance according to creation time. Note that it is important that the monotonic time is in the first element (the most significant element when comparing 2-tuples). Using the monotonic time in the tuples, you can calculate time between events.
If you are interested in the Erlang system time at the time when the event occurred you can also save the time offset before or after saving the events using erlang:time_offset/0. Erlang monotonic time added with the time offset corresponds to Erlang system time.
If you are executing in a mode where time offset may change and you want to be able to get the actual Erlang system time when the event occurred you can save the time offset as a third element in the tuple (the least significant element when comparing 3-tuples).
Create a Unique Name
use the values returned from erlang:now/0 in order to create a name unique on the current runtime system instance.
use the value returned from erlang:unique_integer/0 in order to create a name unique on the current runtime system instance. If you only want positive integers, you can use erlang:unique_integer([positive]).
Seed Random Number Generation With a Unique Value
seed random number generation using erlang:now().
seed random number generation using a combination of erlang:monotonic_time(), erlang:time_offset(), erlang:unique_integer(), and other functionality.
To sum this section up: Don't use erlang:now/0!
2.8 Supporting Both New and Old OTP Releases
Your code may be required to be able to run on a variety of OTP installations of different OTP releases. If so, you can not just use the new API out of the box, since it will not be available on old pre OTP 18 releases. The solution is not to avoid using the new API, since your code then won't be able to benefit from the scalability and accuracy improvements made. Instead you want to use the new API when available, and fall back on erlang:now/0 when it is not available. Fortunately almost all of the new API can easily be implemented using existing primitives (except for erlang:system_info(start_time), erlang:system_info(end_time), erlang:system_info(os_monotonic_time_source), and erlang:system_info(os_system_time_source)). By wrapping the API with functions that fall back on erlang:now/0 when the new API is not available, and using these wrappers instead of using the API directly the problem is solved. These wrappers can for example be implemented as in $ERL_TOP/erts/example/time_compat.erl.