Time and Time Keeping

May 2, 2017 Andrew Thomson

Welcome to our Time & Time-keeping blog!

In this series we’ll be looking at time and time-keeping, and the challenges and opportunities when applied to packet capture and replay. In this space, accuracy is key - especially when correlating between recordings. But achieving consistency across multiple pieces of equipment deployed to several sites can be difficult.

We'll attempt to address some of the common problems that occur and look at tested solutions, as well as covering some of the background to explain what is going on.

So let’s start by looking at something very topical, and proving to be quite contentious.

Leap Seconds

Leap seconds, like leap years, are required to keep clocks in sync with the movement of the earth. Leap years are used to cope with the fact that the earth actually takes a little over 365 days to go round the sun (365 days, 5 hours, 48 minutes, and 45 seconds in fact). Leap seconds operate at the other end of the scale, and are used to compensate for tiny variations in the rotation of the earth, typically caused by tides and other effects. They are usually added at either the end of June or the end of December, as determined by the IERS (International Earth Rotation and Reference Systems Service), and insert or remove a second as required. They make sure that midday remains mid-day.

At the time of writing, there have been a total of 27 leap seconds since they were introduced in 1972. Every one has been positive, indicating that an extra second should be inserted. They occur on average every 18 months, but there is no regularity - sometimes there are two in a year, sometimes none for several years. The last leap second was on 31st December 2016 at 23:59:60 UTC, delaying New Year 2017 by one second; the next one has not yet been announced. However, depending on where you are in the world and your timezone, leap seconds may well not fall at midnight local time.

Leap seconds cause problems for many time-dependent systems since they are frequently mis-handled or ignored, causing widespread synchronisation issues. Some systems recognise 23:59:60 as a valid time, others repeat 23:59:59, and still others 'smear' the leap second over a much longer period.

None of these solutions suit everyone, and so there is an ongoing debate about what to do. With no consensus, various organisations are adopting their own approaches. Google, for example, has decided to stretch every second a tiny amount for 10 hours before and after the leap point; Amazon stretches seconds for 12 hours before and after; and Bloomberg stretches the first 2000 seconds after. This mixed approach can only lead to more complications.

There are a number of problems caused by leap seconds when making packet captures which it's easy to get caught out by:

The difference between two timestamps, one either side of the leap second, will be wrong by 1s, as calculations assume that every day has the same 86,400s.
Packet timestamps may be duplicated if the OS simply repeats 23:59:59 to handle inserting a leap second, affecting all sorts of correlation and rate calculations

Do you know what your packet capture system would do?

Background

The Packet Company make network and telecoms traffic recorders. These are used in test labs and cyber forensics for capturing and replaying a wide variety of signal formats and speeds. Precision time-stamping is increasingly becoming an important requirement for customers, both to allow precise analysis of packet ordering and spacing, and also to enable highly accurate replay. As network speeds increase, resolution and accuracy must keep pace; and as cyber attacks become more sophisticated, so the need for good quality recording increases.

Welcome to our Time & Time-keeping blog!

Leap Seconds

Further reading:

Background