Time Synchronization: The Devil Is In The Details

One of the coolest birthday gifts I received this year was a Kindle, which is letting me finally tap into the collection of unread e-books sitting on my laptop. One of them is the first in a series called "Stealing the Network: How to Own the Box." The chapter I'm reading reminded me of a pet peeve of mine that drives me nuts during incident response: time synchronization.The "Stealing the Network" series is a collaborative effort by well-known "hackers" and security industry greats, including Johnny Long (Hacker for Charities -- go donate now!), Dan Kaminsky, FX, Joe Grand, and Tim Mullen, jst to name a few. The chapter I'm in involves a forensics expert who must investigate a compromise against a client's database server that is believed to have been attacked through the Web server.

The first issue the investigator had to deal with was determining the logs' time zones and skew given a reference system. Apparently, the server time zones were not documented or configured to sync with a time server regularly. Isn't time synchronization a basic tenet of good system administration? How many of you out there have ever had to proceed with an investigation in which the systems were not synchronized?

In a recent risk assessment, I was working with one sysadmin who looked puzzled when asked if his systems had their times synched. After digging a little further, it was clear he had no idea, but it turned out that his department was part of a larger Microsoft Active Directory and was automatically configured. The domain admins had configured their domain controllers to sync with public time servers, which, in turn, provided time synchronization to servers and workstations throughout the company.

Not all admins are created equal. That's probably an understatement, but I've found a correlation between lack of knowledge on time synchronization and lack of centralized logging. Groups that do centralized logging have configured all of their systems to synchronize their times with a central time server. Groups failing at one or the other are highly likely to be failing at both.

I try not to let little things like that bug me, but it's those little things that can lead to nightmares when you have numerous hacked machines -- and none have the same time. Ever experienced that? Not fun.

John H. Sawyer is a senior security engineer on the IT Security Team at the University of Florida. The views and opinions expressed in this blog are his own and do not represent the views and opinions of the UF IT Security Team or the University of Florida. When John's not fighting flaming, malware-infested machines or performing autopsies on blitzed boxes, he can usually be found hanging with his family, bouncing a baby on one knee and balancing a laptop on the other. Special to Dark Reading.