Bump runc to 1.3.3 and containerd to 2.1.5 in main

[krnowak]

• Changelog entries added in the respective changelog/ directory (user-facing change, bug fix, security fix, update)
• Inspected CI output for image differences: /boot and /usr size, packages, list files for any missing binaries, kernel modules, config files, kernel modules, etc.

[krnowak]

Tested by building them locally.

[github-actions]

Build action triggered: https://github.com/flatcar/scripts/actions/runs/19171756331

[hakman]

@krnowak @chewi do you have any hint when this change will reach stable?

[tormath1]

@krnowak @chewi do you have any hint when this change will reach stable?

Hello @hakman, are you looking for containerd 2.1.5 in particular? Because runc-1.3.3 update is available on current stable (4459.2.0 and 4459.2.1)

[hakman]

@tormath1 I am looking for the contained update. I need to use the image volumes feature that is only available in 2.1.

[tormath1]

@tormath1 I am looking for the contained update. I need to use the image volumes feature that is only available in 2.1.

Understood. New major stable has been relased in Nov. so we're planning to have it around for a while (5 to 6 months) to let folks use containerd v2.0.x and provide any additional feedback on this container runtime upgrade (cc @sayanchowdhury to confirm).

In the meantime, you can:

• use the next Beta (release in December) to run your workloads, we trust enough Beta to recommend folks to use it in production - they run through the exact same series of test
• override the built-in containerd with a sysext image of containerd 2.1.x: https://flatcar.github.io/sysext-bakery/containerd/ & https://www.flatcar.org/docs/latest/provisioning/sysext/#remove-docker-and--or-containerd-from-flatcar

[hakman]

Thanks for the advice, I was just asking to know what to communicate to our users that are hitting this, see kubernetes/kops#17780.

Not sure I understand what you mean. containerd 2.1 is stable and it was released 6 months ago. With the release of contained 2.2 and has not become the previous stable. Generally speaking, Kubernetes works best with newer containerd versions, as this is what it's tested against.

[tormath1]

@hakman thanks for providing more context here with the kops issue!

containerd 2.1 is stable and it was released 6 months ago.

We usually rely on Gentoo folks to upgrade containerd packages and/or security concerns (like this current PR). This is a trade-off between security and stability: we don't want to introduce new packages (especially container runtimes) to Stable without going through the stabilization process (alpha -> beta -> stable) to ensure users workloads stability.

Generally speaking, Kubernetes works best with newer containerd versions, as this is what it's tested against.

FWIW Flatcar is running automated tests against current Kubernetes major versions (at this time 1.32, 1.33 and 1.34). We could investigated on extending our test suite to run kops tests but it seems you're already testing Flatcar in kops CI?

[hakman]

Thanks for the detailed explanation @tormath1, much appreciated.
kOps is running CI tests against Flatcar, but only informing. We could probably pull the official binaries for runc and containerd, but until now there were no features that we missed.
Btw, are you running the full CI, or just the conformance?

[tormath1]

@hakman ok, I understand. We are discussing about enabling back our automation to quickly catch-up on newer containerd releases.
For the CI, we have our own test suite that spawns kubernetes clusters with multiple version with multiple CNIs with multiple scenarios, we don't rely on the Kubernetes CI. :)

So now, to get back on the initial issue, I only see the two options mentioned above:

• use the next Beta (release in December) to run your workloads, we trust enough Beta to recommend folks to use it in production they run through the exact same series of test
• override the built-in containerd with a sysext image of containerd 2.1.x: https://flatcar.github.io/sysext-bakery/containerd/ & https://www.flatcar.org/docs/latest/provisioning/sysext/#remove-docker-and--or-containerd-from-flatcar

I never tried kOps, so I don't really know how the second option could apply.

[hakman]

Thanks @tormath1, I appreciate the effort and the insights.
Beta release should be good enough for now, we are already using Alpha in our tests and works fine, so I am confident Beta will be good as well.
I will try to provide a workaround on our end, so Flatcar users should not be blocked when upgrading.