Swap encryption pr2 swap capability#6785
Open
DevVegeta wants to merge 29 commits into
Open
Conversation
…il.IssueCommand Replace all raw ['gcloud', ...] list + vm_util.IssueCommand calls in swap_encryption_benchmark.py with PKB's existing GcloudCommand infrastructure: - _create_benchmark_node_pool: cluster._GcloudCommand() + cmd.flags + cmd.Issue - _delete_default_node_pool: cluster._GcloudCommand() + cmd.Issue - _attach_swap_disk: gcp_util.GcloudCommand(_GcpZonalResource) for create+attach - _delete_disk_by_name: gcp_util.GcloudCommand for describe/detach/delete Add _GcpZonalResource shim: pins zone for gcloud compute operations. GcloudCommand auto-injects --project and --zone/--region, handles auth token refresh -- matching PKB standards.
…fix imports
Replace manual temp-file + kubectl apply in _deploy_daemonset() with
PKB's kubernetes_commands.ApplyManifest():
- Remove _daemonset_yaml() helper
- _deploy_daemonset() delegates to kubernetes_commands.ApplyManifest(
'cluster/swap_encryption_daemonset.yaml.j2', **kwargs)
- Add kubernetes_commands import; remove vm_util import (now unused)
- Fix import order: providers.gcp before resources.container_service
… remove cgroup hack Address Ajay review comments on PR GoogleCloudPlatform#6776: Comment #r3457877984 (linuxConfig.swapConfig): Extend --system-config-from-file YAML with linuxConfig blocks: linuxConfig.swapConfig.enabled: true -- GKE sets up node-level swap dedicatedLocalSsdProfile.diskCount: N -- LSSD: use local NVMe for swap linuxConfig.sysctl: vm.swappiness=100, vm.min_free_kbytes=200, vm.watermark_scale_factor=500 Ref: https://cloud.google.com/kubernetes-engine/docs/how-to/node-memory-swap Comment #r3457928855 (cgroup hack): Remove memory.swap.max=max loop from swap_encryption_daemonset.yaml.j2. With kubeletConfig.memorySwapBehavior=LimitedSwap the kubelet manages per-container swap allocation; the cgroup hack is unnecessary.
…5); manifest moved to data/cluster and rendered via vm_util
Per Ajay's review comment on PR GoogleCloudPlatform#6758: - Add _GKE_KUBELET_MEMORY_SWAP flag (default LimitedSwap) so the benchmark nodepool is created with kubeletConfig.memorySwapBehavior set via --system-config-from-file, enabling pod-level swap usage. - Wrap gcloud IssueCommand in try/finally to clean up the temp YAML. - Update nodepool creation log to include kubelet_swap value.
DevVegeta
force-pushed
the
swap-encryption-pr2-swap-capability_new
branch
from
f83f078 to
6afbfe1
Compare
- Add SwapDaemonSet(resource.BaseResource) in resources/container_service/swap_daemonset.py - _Create(): apply Jinja2 manifest + wait for Running + /tmp/pkb_ready - _Delete(): in-pod swapoff/dmsetup/losetup/pkill teardown; kubectl delete - PodExec(): transient-reset retry, rc=137 OOM detection, pod recovery - Add SwapNodePool(resource.BaseResource) in resources/container_service/swap_nodepool.py - _Create(): gcloud node-pools create with linuxConfig.swapConfig + optional swap disk - _Delete(): detach+delete disk; delete nodepool - DeleteDefaultPool(): remove dummy e2-medium pool after DaemonSet pod Running - Rewrite benchmark to thin pattern: Prepare() uses resource.Create() + spec.resources - Cleanup() is empty - PKB framework auto-deletes spec.resources - Run() uses daemonset.PodExec() throughout - Addresses Zac review: resources pattern, no infra code in benchmark file - Fix COS_CONTAINERD -> UBUNTU_CONTAINERD (r3472549985) - swapConfig auto-enables memorySwapBehavior=LimitedSwap (r3472513706)
… NodepoolSpec field BREAKING: replaces SwapNodePool (standalone nodepool lifecycle) with the correct PKB pattern: swap configuration declared in BENCHMARK_CONFIG and applied by the existing GKE cluster creation flow. New files: - resources/container_service/swap_config.py - GkeSwapConfig(BaseResource): WriteLinuxConfigYaml(), ValidHyperdiskThroughput() - EksSwapConfig(BaseResource): stub for nodeadm config (deferred to PR GoogleCloudPlatform#6780) Core framework changes: - configs/container_spec.py: add SwapConfigSpec(BaseSpec) + _SwapConfigDecoder + swap_config field on NodepoolSpec - resources/container_service/container.py: add swap_config attr to BaseNodePoolConfig - resources/container_service/container_cluster.py: propagate swap_config in _InitializeNodePool() (mirrors sandbox_config pattern) - providers/gcp/google_kubernetes_engine.py: _AddNodeParamsToCmd() reads nodepool_config.swap_config - applies --system-config-from-file, UBUNTU_CONTAINERD, --no-enable-autorepair, boot-disk-provisioned-iops/throughput Thin benchmark: - BENCHMARK_CONFIG declares benchmark nodepool with swap_config (no separate nodepool create needed - GKE cluster creation handles it) - Prepare(): deploy SwapDaemonSet + delete default-pool - Run(): verify swap_active + swap_encrypted; report samples - Cleanup(): empty (PKB auto-deletes spec.resources) Addresses Ajay reviews: - r3457826290: swap as base resource plugged into GKE cluster creation flow - r3457877984: linuxConfig.swapConfig via --system-config-from-file (GkeSwapConfig) - r3457928855: removed memory.swap.max hack - r3457964593: UBUNTU_CONTAINERD set per-nodepool in _AddNodeParamsToCmd - r3472513706: swapConfig auto-enables memorySwapBehavior=LimitedSwap - r3472549985: UBUNTU_CONTAINERD required for dm-crypt
… NodepoolSpec field BREAKING: replaces SwapNodePool (standalone nodepool lifecycle) with the correct PKB pattern: swap configuration declared in BENCHMARK_CONFIG and applied by the existing GKE cluster creation flow. New files: - resources/container_service/swap_config.py - GkeSwapConfig(BaseResource): WriteLinuxConfigYaml(), ValidHyperdiskThroughput() - EksSwapConfig(BaseResource): stub for nodeadm config (deferred to PR GoogleCloudPlatform#6780) Core framework changes: - configs/container_spec.py: add SwapConfigSpec(BaseSpec) + _SwapConfigDecoder + swap_config field on NodepoolSpec - resources/container_service/container.py: add swap_config attr to BaseNodePoolConfig - resources/container_service/container_cluster.py: propagate swap_config in _InitializeNodePool() (mirrors sandbox_config pattern) - providers/gcp/google_kubernetes_engine.py: _AddNodeParamsToCmd() reads nodepool_config.swap_config - applies --system-config-from-file, UBUNTU_CONTAINERD, --no-enable-autorepair, boot-disk-provisioned-iops/throughput Thin benchmark: - BENCHMARK_CONFIG declares benchmark nodepool with swap_config (no separate nodepool create needed - GKE cluster creation handles it) - Prepare(): deploy SwapDaemonSet + delete default-pool - Run(): verify swap_active + swap_encrypted; report samples - Cleanup(): empty (PKB auto-deletes spec.resources) Addresses Ajay reviews: - r3457826290: swap as base resource plugged into GKE cluster creation flow - r3457877984: linuxConfig.swapConfig via --system-config-from-file (GkeSwapConfig) - r3457928855: removed memory.swap.max hack - r3457964593: UBUNTU_CONTAINERD set per-nodepool in _AddNodeParamsToCmd - r3472513706: swapConfig auto-enables memorySwapBehavior=LimitedSwap - r3472549985: UBUNTU_CONTAINERD required for dm-crypt
GkeSwapConfig and EksSwapConfig now both inherit from BaseSwapConfig(BaseResource). Common sysctl attrs (swappiness, min_free_kbytes, watermark_scale_factor) live in the base class. Cloud-specific attrs remain in each subclass. Addresses Zac review: GkeSwapConfig & EksSwapConfig should inherit from BaseSwapConfig.
…EksSwapConfig and GKE wiring
…etries arg, suppress invalid-name on from_spec base class definition
DevVegeta
force-pushed
the
swap-encryption-pr2-swap-capability_new
branch
from
e884b5c to
ebab7f2
Compare
- Prepare() uses SwapNodePool + SwapDaemonSet from spec.resources - Cleanup() is empty - PKB framework auto-deletes spec.resources - All _pod_exec(pod, ...) calls replaced with daemonset.PodExec(...) - DaemonSet: lean (fio/cryptsetup/mdadm/sysstat/nvme-cli only) - Remove cgroup v2 memory.swap.max hack (r3457928855) - Add _configure_eks_kubelet_swap() stub for nodeadm (PR GoogleCloudPlatform#6780) - Fix COS_CONTAINERD -> UBUNTU_CONTAINERD (r3472549985) - swapConfig auto-enables memorySwapBehavior=LimitedSwap (r3472513706)
DevVegeta
force-pushed
the
swap-encryption-pr2-swap-capability_new
branch
2 times, most recently
from
79aca1c to
6b37fa6
Compare
DevVegeta
force-pushed
the
swap-encryption-pr2-swap-capability_new
branch
from
6b37fa6 to
c7a1578
Compare
DevVegeta
force-pushed
the
swap-encryption-pr2-swap-capability_new
branch
from
c7a1578 to
9bf362f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
feat(swap_encryption/pr2): swap device setup for all 8 test-matrix cells
Stacked PR 2/5. Inherits PR1 framework (swap_config as NodepoolSpec field).
Benchmark nodepool provisioned automatically by GKE cluster creation — no manual nodepool lifecycle in Prepare().
Adds dm-crypt swap device setup paths for all 8 test-matrix cells:
Changes vs PR1:
setup_gke_hyperdisk(),setup_gke_lssd(),setup_gke_boot_disk()— dm-crypt config per disk typesetup_eks_instance_store(),setup_eks_io2()— EKS dm-crypt setup_configure_eks_kubelet_swap()stub for nodeadm (deferred to PR Add swap_config to node-pool spec #6780)memory.swap.maxhack (r3457928855)swapConfigauto-enablesmemorySwapBehavior=LimitedSwap(r3472513706)