Skip to content

[pciehp 1/4] Support 32 PCIe buses - #6226

Open
ilstam wants to merge 5 commits into
mainfrom
ilstam-hotplug-1
Open

ilstam wants to merge 5 commits into
mainfrom
ilstam-hotplug-1

Conversation

@ilstam

@ilstam ilstam commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

Firecracker currently supports a single PCI bus. This PR is the groundwork for extending that to 32 buses.

Today Firecracker presents a single PCI bus to the guest, with no Type 1
devices other than the host bridge itself. Virtio devices appear as Root
Complex Integrated Endpoints attached directly to the host bridge, on
bus 0.

Native PCIe hot-plug needs root ports, and each root port starts a
secondary bus, so configuration cycles have to be able to reach buses
other than 0. Introduce PciBuses holding the buses a segment has; the
root bus plus one secondary bus per root port.

No visible change for the guest at this point, nothing creates root
ports and secondary buses yet.

Signed-off-by: Ilias Stamatis <ilstam@amazon.com>
Until now PCI_MMIO_CONFIG_SIZE_PER_SEGMENT was meant to cover every
device and function in a single bus because it was assumed that there's
a single PCI bus in the system. Since we are going to expand the
topology to support up to 32 buses this needs to be updated.

Rename PCI_MMIO_CONFIG_SIZE_PER_SEGMENT to PCI_MMIO_CONFIG_SIZE_PER_BUS
which is a more accurate name and move the constant to pci/bus.rs
instead of repeating its definition per arch. And add a new
PCI_MMIO_CONFIG_SIZE_PER_SEGMENT that represents the ECAM space for 32
buses.

The ECAM regions grows from 1 MiB to 32 MiB which is 's still well
inside the 256 MiB PCI_MMCONFIG_SIZE window which could be used to cover
every possible BDF (256 buses).

The test_mmio_bus() test used to access the entire 1MiB ECAM region, but
that's not possible after growing the ECAM region to 32 MiB since the
stack is not large enough to accommodate such a large local variable.
Adjust the test to instead check that the beginning and end of the ECAM
region can be accessed, whereas accessing just before or just after it
fails.

Signed-off-by: Ilias Stamatis <ilstam@amazon.com>
Two bus ranges are advertised to the guest via ACPI tables and both
describe bus 0 only. Subsequent patches will extend the bus topology so
secondary buses need to be advertised too.

Make the MCFG ECAM entry cover every bus a PCI segment can possibly
have, and add the full bus range of present buses in the host bridge's
_CRS.

Signed-off-by: Ilias Stamatis <ilstam@amazon.com>
The FDT advertises a single PCI bus to the guest. Change this to
advertise the full range of buses present in the system.

Signed-off-by: Ilias Stamatis <ilstam@amazon.com>
Add an _OSC method to the host bridge. This can be used by the OS to
request it's granted control of certain capabilities (PCIe hotplug,
power management events, etc). Otherwise the platform keeps control of
them.

Grant whatever the guest asks for by returning the capabilities buffer
unchanged. Granting control of features Firecracker does not implement
is harmless. The guest only drives a feature if it also finds the
corresponding PCI capability.

Signed-off-by: Ilias Stamatis <ilstam@amazon.com>
@ilstam ilstam changed the title [pciehp 1/4] Support multi-bus PCI configuration and grant the OS PCIe hot-plug control [pciehp 1/4] Support 32 PCIe buses Sep 17, 2026
@ilstam
ilstam marked this pull request as ready for review September 17, 2026 17:15
@codecov

codecov Bot commented Sep 17, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.38710% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 83.07%. Comparing base (23b09b9) to head (93557a0).

Files with missing lines Patch % Lines
src/vmm/src/devices/pci/pci_segment.rs 95.45% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6226      +/-   ##
==========================================
+ Coverage   83.05%   83.07%   +0.02%     
==========================================
  Files         277      277              
  Lines       31481    31508      +27     
==========================================
+ Hits        26147    26176      +29     
+ Misses       5334     5332       -2     
Flag Coverage Δ
5.10-m5n.metal 83.31% <98.33%> (+0.01%) ⬆️
5.10-m6a.metal 82.69% <98.33%> (+0.02%) ⬆️
5.10-m6g.metal 80.15% <95.55%> (+0.01%) ⬆️
5.10-m6i.metal 83.30% <98.33%> (+0.01%) ⬆️
5.10-m7a.metal-48xl 82.68% <98.33%> (+0.02%) ⬆️
5.10-m7g.metal 80.15% <95.55%> (+0.01%) ⬆️
5.10-m7i.metal-24xl 83.28% <98.33%> (+0.01%) ⬆️
5.10-m7i.metal-48xl 83.28% <98.33%> (+0.01%) ⬆️
5.10-m8g.metal-24xl 80.15% <95.55%> (+0.01%) ⬆️
5.10-m8g.metal-48xl 80.15% <95.55%> (+0.01%) ⬆️
5.10-m8i.metal-48xl 83.28% <98.33%> (+0.01%) ⬆️
5.10-m8i.metal-96xl 83.28% <98.33%> (+0.01%) ⬆️
5.10-m9g.metal-48xl 80.15% <95.55%> (+0.01%) ⬆️
6.1-m5n.metal 83.33% <98.33%> (+0.01%) ⬆️
6.1-m6a.metal 82.71% <98.33%> (+0.03%) ⬆️
6.1-m6g.metal 80.14% <95.55%> (+<0.01%) ⬆️
6.1-m6i.metal 83.33% <98.33%> (+0.02%) ⬆️
6.1-m7a.metal-48xl 82.70% <98.33%> (+0.02%) ⬆️
6.1-m7g.metal 80.15% <95.55%> (+0.01%) ⬆️
6.1-m7i.metal-24xl 83.34% <98.33%> (+0.02%) ⬆️
6.1-m7i.metal-48xl 83.34% <98.33%> (+0.02%) ⬆️
6.1-m8g.metal-24xl 80.15% <95.55%> (+0.01%) ⬆️
6.1-m8g.metal-48xl 80.14% <95.55%> (+<0.01%) ⬆️
6.1-m8i.metal-48xl 83.34% <98.33%> (+0.01%) ⬆️
6.1-m8i.metal-96xl 83.35% <98.33%> (+0.02%) ⬆️
6.1-m9g.metal-48xl 80.15% <95.55%> (+0.01%) ⬆️
6.18-m5n.metal 83.33% <98.33%> (+0.02%) ⬆️
6.18-m6a.metal 82.70% <98.33%> (+0.01%) ⬆️
6.18-m6g.metal 80.25% <95.55%> (+<0.01%) ⬆️
6.18-m6i.metal 83.33% <98.33%> (+0.02%) ⬆️
6.18-m7a.metal-48xl 82.70% <98.33%> (+0.02%) ⬆️
6.18-m7g.metal 80.26% <95.55%> (+0.01%) ⬆️
6.18-m7i.metal-24xl 83.34% <98.33%> (+0.02%) ⬆️
6.18-m7i.metal-48xl 83.35% <98.33%> (+0.02%) ⬆️
6.18-m8g.metal-24xl 80.26% <95.55%> (+0.01%) ⬆️
6.18-m8g.metal-48xl 80.26% <95.55%> (+0.01%) ⬆️
6.18-m8i.metal-48xl 83.34% <98.33%> (+0.02%) ⬆️
6.18-m8i.metal-96xl 83.34% <98.33%> (+0.02%) ⬆️
6.18-m9g.metal-48xl 80.26% <95.55%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ilstam
ilstam added this pull request to stack #6230 September 17, 2026 17:16
@ilstam ilstam added the Status: Awaiting review Indicates that a pull request is ready to be reviewed label Sep 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Status: Awaiting review Indicates that a pull request is ready to be reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant