vmm: x86_64: synchronize TSC offsets after restoring vCPUs - #6200
JackThomson2 wants to merge 2 commits into
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #6200 +/- ##
==========================================
+ Coverage 83.04% 83.09% +0.05%
==========================================
Files 277 277
Lines 31437 31478 +41
==========================================
+ Hits 26106 26157 +51
+ Misses 5331 5321 -10
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
zulinx86
left a comment
zulinx86
left a comment
There was a problem hiding this comment.
Don't we need to follow the instruction described in https://docs.kernel.org/virt/kvm/devices/vcpu.html#group-kvm-vcpu-tsc-ctrl ?
Yeah I think we could follow this in the future, but I read it as a possible algorithm for maintaining a consistent TSC during live migration. Here the goal is just keeping the vCPUs in sync. |
zulinx86
left a comment
zulinx86
left a comment
There was a problem hiding this comment.
Don't we need to follow the instruction described in docs.kernel.org/virt/kvm/devices/vcpu.html#group-kvm-vcpu-tsc-ctrl ?
Yeah I think we could follow this in the future, but I read it as a possible algorithm for maintaining a consistent TSC during live migration. Here the goal is just keeping the vCPUs in sync.
OK. I'm fine to keep this PR focused on the offset synchronization.
I'll dive deeper into whether it's fine to not account for elapsed time especially when clock_realtime=true.
When I introduced |
Restoring TSC MSRs separately can leave vCPUs with different offsets, allowing guest clocks backed by raw TSC to move backwards when a task migrates. Add device-attribute accessors and copy the restored vCPU0 offset to all vCPUs before starting their threads. Preserve the reference timeline and existing VM clock restore policy. Probe for attribute support and keep synchronization best-effort on hosts where it is unavailable or fails. No snapshot format changes or runtime test hooks are introduced. Signed-off-by: Jack Thomson <jackabt@amazon.com>
Exercise signed TSC offset access and synchronization using real KVM vCPUs restored into a fresh VM. Cover unequal offsets, an already synchronized pair and a single vCPU, then verify that VM clock restoration preserves the reference offset. Extend the raw-TSC snapshot test with bounded CPU migration sampling across both vCPUs. Require non-realtime clock restoration to resume near snapshot time rather than merely differ from elapsed host time. Keep skew injection in Rust test code instead of adding production environment hooks. Signed-off-by: Jack Thomson <jackabt@amazon.com>
a292420 to
ac1a211
Compare
JackThomson2
added
the
Status: Awaiting review
zulinx86
left a comment
zulinx86
left a comment
There was a problem hiding this comment.
LGTM (with a minor comment)
| if not clock_realtime: | ||
| assert 0 <= guest_delta < 5.0, f"Unexpected clock delta: {guest_delta:.6f}s" |
There was a problem hiding this comment.
What do we want to ensure with this? How is this related to the TSC offset synchronization between vCPUs?
There was a problem hiding this comment.
I left it in while asserting the behaviour before and after this change. I can remove if we want
There was a problem hiding this comment.
I think the assertion itself is useful, but could we move it below the diagnostic prints, next to assert jumped == clock_realtime? That would keep the measurements and jumped calculation together, followed by the assertions.
|
Do you think it's worth adding this fix to the changelog? |
3 participants
Changes
Restoring TSC MSRs separately can leave vCPUs with different offsets,
allowing guest clocks backed by raw TSC to move backwards when a task
migrates. Add device-attribute accessors and copy the restored vCPU0
offset to all vCPUs before starting their threads. Preserve the
reference timeline and existing VM clock restore policy. Probe for
attribute support and keep synchronization best-effort on hosts where it
is unavailable or fails. No snapshot format changes or runtime test
hooks are introduced.
Related:
Reason
...
License Acceptance
By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache 2.0 license. For more information on following Developer
Certificate of Origin and signing off your commits, please check
CONTRIBUTING.md.PR Checklist
tools/devtool checkbuild --allto verify that the PR passesbuild checks on all supported architectures.
tools/devtool checkstyle --no-clippyto verify that the PRpasses the automated style checks.
how they are solving the problem in a clear and encompassing way.
in the PR.
CHANGELOG.md.Runbook for Firecracker API changes.
integration tests.
TODO.rust-vmm.