SYSTEMS OPERATIONAL
VPS

VPS Clock Drift: Fix NTP/Chrony Before It Breaks Everything

Getwebup 5 min read

Cron jobs firing at the wrong hour. SSL handshakes failing for no visible reason. MySQL replication silently falling behind. If you've chased any of these on a VPS and found nothing wrong in the app itself, check the one thing almost nobody thinks to check first: what time your server actually thinks it is.

Symptom: things break in ways that don't point at the clock

Clock drift rarely announces itself. It shows up as a grab-bag of unrelated-looking failures, which is exactly why it gets missed:

  • Cron jobs run early, late, or twice in the same hour after a DST change
  • API calls fail with 401 Unauthorized or "token not yet valid" even though the credentials are correct
  • TLS handshakes fail with certificate validation errors, even for certs that are nowhere near expiry
  • MySQL or PostgreSQL replication lag grows for no query-level reason
  • Log timestamps across your app server, database, and load balancer don't line up, making incident timelines useless
  • Docker containers show a different time than the host

None of these say "clock" on the tin. That's the trap.

Cause: the hypervisor clock and NTP daemon disagree, or NTP was never running

A few things cause a VPS clock to drift, and they're not all the same fix:

1. NTP sync was never enabled

Some minimal cloud images ship with time sync disabled or the daemon installed but not running. The clock then just free-runs on the VM's virtual hardware clock, which drifts a few seconds to a few minutes per day depending on host load - worse on busy shared hypervisors, where the guest's sense of elapsed time gets distorted under CPU contention.

2. Two time services fighting each other

Ubuntu and Debian ship systemd-timesyncd by default. If you install chrony or ntpd on top without disabling it, both try to own the clock and you get inconsistent corrections instead of a clean sync.

3. Wrong timezone, not wrong time

This one isn't drift at all, but it looks identical from the app's point of view: the system clock is accurate but set to the wrong timezone, so cron and logs are "off" by a fixed number of hours. Always rule this out first - it's the more common issue and the easier fix.

4. Large, sudden jumps instead of gradual correction

If NTP is misconfigured or was down for a long stretch, the next sync can slew the clock by minutes in one jump. A sudden backward jump is worse than gradual drift - it can make timestamps appear to go back in time, which breaks anything relying on monotonically increasing timestamps (session tokens, some database indexes, log aggregators).

Fix it: get a real NTP client running

Step 1 - Check current status

timedatectl status

Look at three lines specifically:

FieldWhat it should say
Local time / Universal timeCorrect current time (compare against your phone)
Time zoneWhat you actually intend - usually UTC on a server
System clock synchronizedyes
NTP serviceactive

If System clock synchronized says no, that's your confirmation - stop guessing and go straight to Step 2.

Step 2 - Set the timezone correctly (usually UTC)

For servers, UTC avoids DST-jump bugs in cron and logs entirely:

sudo timedatectl set-timezone UTC

Only use a local timezone if something on the box genuinely depends on it (a legacy cron schedule written for a specific region, for example) - and document why, because the next person will assume UTC.

Step 3 - Install and enable chrony

Chrony syncs faster and handles intermittent connectivity better than the older ntpd, and it's the default on most modern distros. If it's not already there:

# Debian/Ubuntu
sudo apt update && sudo apt install chrony -y

# AlmaLinux/Rocky/CentOS
sudo dnf install chrony -y

If systemd-timesyncd is running alongside it, disable that one so only chrony is in charge:

sudo systemctl disable --now systemd-timesyncd
sudo systemctl enable --now chronyd

Step 4 - Force an immediate sync and verify

sudo chronyc makestep
chronyc tracking

Check the System time line in the output - it should show an offset under a second. Anything in the tens of seconds or more means chrony just corrected a real problem; re-run it a minute later to confirm it's holding.

Step 5 - Confirm the sources are reachable

chronyc sources -v

You want at least one source marked with a * (currently selected, in sync). If every source shows an X or a blank, your outbound UDP port 123 is probably blocked by a firewall or security group - check that before assuming chrony is broken.

Prevention: make drift impossible to ignore next time

  • Keep servers on UTC. It sidesteps an entire category of DST-related cron and log bugs.
  • Don't run two time daemons. Pick chrony (recommended) or timesyncd, not both.
  • Add a drift check to your monitoring. Netdata, Zabbix, and most agent-based monitors can alert on NTP offset - a few seconds of drift is normal, tens of seconds is not.
  • Watch it after resizing or migrating a VPS. Moving a virtual machine between hosts (a snapshot restore, a live migration, a resize) is one of the most common triggers for a sudden clock jump - the guest's virtual clock and the new host's real clock rarely start in perfect agreement.
  • Check containers separately. Docker containers usually share the host clock, but if you're running VMs inside VMs (nested virtualization) or an older container runtime, verify the guest clock independently.

Clock drift is one of those problems that costs you an hour of debugging application code before someone thinks to run timedatectl status. Run it first next time - it takes five seconds and rules out an entire class of "nothing about this makes sense" bugs.

Frequently asked questions

How much clock drift is actually a problem?

A second or two of offset is normal and harmless. Once you're seeing 10+ seconds of drift, expect token-based auth (JWT, OAuth, some API signatures) to start failing intermittently. Drift in the minutes will break TLS validation, cron scheduling, and database replication outright.

My server shows the correct time but cron still runs at the wrong hour. What's going on?

That's almost always a timezone mismatch, not drift. Run timedatectl status and check the Time zone line - if it's set to a region other than what you assumed, cron is running exactly on schedule for that zone, just not the one you expected.

Do I need chrony if I'm on Ubuntu with systemd-timesyncd already running?

No, timesyncd is a perfectly valid NTP client for most single-purpose VPS setups. Switch to chrony if you need faster resync after downtime, tighter accuracy for time-sensitive workloads (database clusters, distributed systems), or if you're troubleshooting drift and want more detailed diagnostics via chronyc.

Why did my VPS clock jump after a snapshot restore?

Restoring a snapshot pauses the guest's sense of elapsed time at the moment the snapshot was taken. When it boots back up, the virtual clock is behind reality by however long the snapshot sat unused. NTP will correct it automatically within a few minutes if it's running - if it isn't, the drift can persist indefinitely.

Can clock drift cause a valid SSL certificate to fail?

Yes. TLS certificate validation checks the current system time against the certificate's notBefore and notAfter dates. If your server's clock has drifted enough to fall outside that window - or jumped past a certificate's expiry that hasn't actually happened yet - the handshake fails even though the certificate itself is fine.

#vps #ntp #chrony #clock-drift #server-time #linux

Keep reading

Chat with Support