Skip to content

doc: state the host clock source requirement for clock_realtime and KVM PTP - #6244

Open
yuchen-plori wants to merge 1 commit into
firecracker-microvm:mainfrom
plori-ai:doc/clock-realtime-ptp-tsc-host
Open

yuchen-plori wants to merge 1 commit into
firecracker-microvm:mainfrom
plori-ai:doc/clock-realtime-ptp-tsc-host

Conversation

@yuchen-plori

Copy link
Copy Markdown

Changes

Documentation only:

  • docs/snapshotting/snapshot-support.md: the host that creates the snapshot must
    use a TSC-based clock source for clock_realtime: true, and the error you get
    when it does not.
  • FAQ.md (wall-clock answer): /dev/ptp0 appears only when the host kernel uses
    a TSC-based clock source, and how to check the host.

Reason

On a host that is itself a KVM guest without an invariant TSC, both
clock_realtime and the KVM PTP time source fail. The docs do not name this
precondition, and the PTP failure is silent. We observed it with Firecracker
v1.17.0 on such a host:

  • host kernel 6.8, current_clocksource = kvm-clock (available:
    kvm-clock acpi_pm);
  • dmesg: tsc: Marking TSC unstable due to TSCs unsynchronized and
    kvm: SMP vm created on host with unstable TSC; guest TSC will not be reliable;
  • no constant_tsc / nonstop_tsc in /proc/cpuinfo.

Results:

  • LoadSnapshot with clock_realtime: true fails with
    clock_realtime requested but not present in the snapshot state (snapshot
    created and loaded on the same host and Firecracker build).
  • The guest (kernel 6.18, CONFIG_PTP_1588_CLOCK_KVM=y) has no /dev/ptp0, and
    the kernel logs nothing about it. The same guest kernel on a host with a
    TSC clock source gets /dev/ptp0, and chrony uses it as a PHC refclock.

Both come from one condition in KVM (Linux v6.8):

  • __get_kvmclock() sets KVM_CLOCK_REALTIME only when the VM uses the master
    clock and kvm_get_walltime_and_clockread() succeeds
    (x86.c#L3056).
    The master clock needs a TSC-based host clock
    (gtod_is_based_on_tsc()).
    Firecracker then returns ClockRealtimeNotInState
    (vm.rs).
  • kvm_pv_clock_pairing() returns -KVM_EOPNOTSUPP under the same check
    (x86.c#L9875),
    so the guest's ptp_kvm driver does not register /dev/ptp0.

A nested host is the common case, not the only one: a nested host that gets an
invariant TSC can use tsc, so the docs describe the clock source condition and
give a way to check it. NTP in the guest, which the FAQ already names as the
canonical solution, works on these hosts; we serve NTP from the host to the
guests.

Related: the TSC-stability question in the review of #5809
(#5809 (comment))
and the test fixes for non-metal hosts in #5946.

License Acceptance

By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache 2.0 license. For more information on following Developer
Certificate of Origin and signing off your commits, please check
CONTRIBUTING.md.

PR Checklist

  • I have read and understand CONTRIBUTING.md.
  • I have run tools/devtool checkbuild --all to verify that the PR passes
    build checks on all supported architectures. (Documentation only; no code
    changes.)
  • I have run tools/devtool checkstyle --no-clippy to verify that the PR
    passes the automated style checks. (Ran the same mdformat 0.7.22 with the
    gfm, frontmatter and footnote plugins on both files; mdformat --check
    passes.)
  • I have described what is done in these changes, why they are needed, and
    how they are solving the problem in a clear and encompassing way.
  • I have updated any relevant documentation (both in code and in the docs)
    in the PR.
  • I have mentioned all user-facing changes in CHANGELOG.md. (Not needed:
    documentation only.)
  • If a specific issue led to this PR, this PR closes the issue. (No issue.)
  • When making API changes, I have followed the
    Runbook for Firecracker API changes. (No API change.)
  • I have tested all new and changed functionalities in unit tests and/or
    integration tests. (Documentation only.)
  • I have linked an issue to every new TODO. (No TODOs.)

  • This functionality cannot be added in rust-vmm. (Not applicable.)

— Plori AI team (plori.ai)

@zulinx86 zulinx86 self-assigned this Sep 30, 2026
@zulinx86
zulinx86 self-requested a review September 30, 2026 14:08
@zulinx86 zulinx86 removed their assignment Sep 30, 2026
@zulinx86 zulinx86 added the Status: Awaiting assignee Indicates that an issue or pull request is awaiting action from its assignee. label Sep 30, 2026
@zulinx86 zulinx86 self-assigned this Sep 30, 2026

@zulinx86 zulinx86 left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution, and for the thorough investigation behind it!

You are right that both clock_realtime and KVM PTP depend on the host using a TSC-based clock source, and documenting this precondition, especially the silent PTP failure, would save users a lot of debugging. I agree with the overall direction.

I left a few minor comments, mostly about wording precision.

Thanks!

Comment thread docs/snapshotting/snapshot-support.md Outdated
Comment thread FAQ.md Outdated
Comment thread FAQ.md Outdated
@yuchen-plori
yuchen-plori force-pushed the doc/clock-realtime-ptp-tsc-host branch from 69f5621 to e88d560 Compare October 2, 2026 22:46
@zulinx86 zulinx86 added Status: Awaiting author Indicates that an issue or pull request requires author action and removed Status: Awaiting assignee Indicates that an issue or pull request is awaiting action from its assignee. labels Oct 2, 2026
zulinx86
zulinx86 previously approved these changes Oct 2, 2026

@zulinx86 zulinx86 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the quick update!

I left one more small nit, which is not blocking. Feel free to take it or leave
it as is.

LGTM, thanks again for documenting this.

Comment thread FAQ.md Outdated
@zulinx86

zulinx86 commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

Thanks for applying the suggestion! Could you squash the "drop it to keep the
paragraph shorter" commit into the first one? It is missing a Signed-off-by
line, so the DCO check fails, and it would also need a commit body to pass our
commit message checks. Keeping the original commit message of the first commit
is fine, since it still describes the change well.

@yuchen-plori
yuchen-plori force-pushed the doc/clock-realtime-ptp-tsc-host branch from 2d582d8 to 409f058 Compare October 2, 2026 23:36
@yuchen-plori

Copy link
Copy Markdown
Author

Squashed into the first commit and force-pushed (409f058). The branch now has one commit with the original message and a Signed-off-by line; the content is unchanged from the previous head.

— Plori AI team

@zulinx86 zulinx86 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks for the contribution again!

@zulinx86

zulinx86 commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

Agh... CI now fails on gitlint for the commit message
(https://buildkite.com/firecracker/firecracker-pr/builds/19257):

  • the title is 75 characters, above the 72-character limit
  • line 8 of the body is 73 characters, above the same limit

Could you shorten the title, for example to "doc: document host clock source
requirement for clock_realtime and PTP" (70 characters), and rewrap the body to
72 columns?

Both `clock_realtime: true` on snapshot load and the KVM PTP device
(/dev/ptp0) need the host kernel to use a TSC-based clock source. KVM
sets KVM_CLOCK_REALTIME in KVM_GET_CLOCK, and answers the
KVM_HC_CLOCK_PAIRING hypercall, only when the host clock is TSC-based.
On a host that uses kvm-clock, for example a KVM guest without an
invariant TSC, the snapshot load fails with "clock_realtime requested
but not present in the snapshot state" and the guest has no /dev/ptp0,
with no message from the driver.

Document the requirement in the snapshot guide and in the wall-clock FAQ
answer, with a way to check the host.

Signed-off-by: Plori AI Team <agent@plori.ai>
@yuchen-plori
yuchen-plori force-pushed the doc/clock-realtime-ptp-tsc-host branch from 1eabfab to 5716930 Compare October 2, 2026 23:56
@yuchen-plori

Copy link
Copy Markdown
Author

Reworded and force-pushed (5716930): the title is now "doc: document host clock source requirement for clock_realtime and PTP" (70 characters) and the body is wrapped at 72 columns. gitlint with the repo config and framework/gitlint_rules.py passes on it locally. I rebased the commit onto current main instead of keeping the merge commit, so the branch is one commit on top of f23a213 with the same content as before.

— Plori AI team

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Status: Awaiting author Indicates that an issue or pull request requires author action

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants