From 992bb78174104bb27335a4bd221f9d23a99bb997 Mon Sep 17 00:00:00 2001 From: Bjordis Collaku Date: Wed, 10 Jun 2026 16:57:33 -0700 Subject: [PATCH 1/4] rootfs: isolate build workspace from source tree Problem - ROOTFS_DIR was bound to $WORKDIR/rootfs while the script itself resides under rootfs/scripts/ in the same repository. - The preprocessing stage performs `rm -rf "$ROOTFS_DIR"`; when invoked from qcom-build-utils/, that path overlaps the source tree and can remove the script directory used by subsequent invocations. Root cause - The working rootfs output directory and the repository source directory shared the same basename (rootfs) and filesystem root. Implementation - Rename the runtime extraction/build directory from $WORKDIR/rootfs to $WORKDIR/rootfs_work. Operational impact - Prevents destructive collision between generated rootfs state and checked-in script sources. - Enables repeated invocations in a single CI job (e.g. multi-kernel loops) without losing rootfs/scripts/build-rootfs.sh mid-run. Scope - Path-safety fix only; no package-selection or install-order behavior changes. Signed-off-by: Bjordis Collaku --- rootfs/scripts/build-rootfs.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rootfs/scripts/build-rootfs.sh b/rootfs/scripts/build-rootfs.sh index 71b1c1a4..da9e2325 100755 --- a/rootfs/scripts/build-rootfs.sh +++ b/rootfs/scripts/build-rootfs.sh @@ -139,7 +139,7 @@ fi WORKDIR=$(pwd) MNT_DIR="$WORKDIR/mnt" -ROOTFS_DIR="$WORKDIR/rootfs" +ROOTFS_DIR="$WORKDIR/rootfs_work" ROOTFS_IMG="rootfs.img" mkdir -p "$MNT_DIR" "$ROOTFS_DIR" From 5d1f84043139a6d5d3c536e3937a6e9c95bbee99 Mon Sep 17 00:00:00 2001 From: Bjordis Collaku Date: Wed, 10 Jun 2026 16:57:33 -0700 Subject: [PATCH 2/4] rootfs: add local-debs flow; kernel deb optional Context - The script previously required --kernel-package, which blocked valid flows where kernels are provided by apt sources or installed later via overlay manifests. - Local .deb injection relied on direct package install assumptions and did not provide a first-class dependency-resolving ingestion path. Implementation - Relax the required CLI contract: - keep --product-conf and --seed mandatory, - make --kernel-package optional with conditional validation, copy, and install behavior. - Introduce a repeatable --local-debs input: - accepts file or directory paths, - validates each path preflight, - stages all discovered debs into /opt/local-debs in the rootfs. - Inside the chroot, create a temporary local apt repository from the staged debs: - dpkg-scanpackages index generation, - file:///opt/local-debs source registration, - package-name extraction from deb metadata, - apt-driven installation to resolve inter-package dependencies. Why an apt-backed local repo - `dpkg -i` alone is non-closure-preserving for dependency graphs. - A local apt repository allows the resolver to satisfy dependency chains across the provided package set in deterministic order. Operational behavior - If no --local-debs are provided, local-repo stages are true no-ops. - If directories are provided but empty, execution remains stable (no hard failure). Scope - Packaging ingress expansion and input-contract modernization; the bootloader sequencing change lands in a subsequent commit. Signed-off-by: Bjordis Collaku --- rootfs/scripts/build-rootfs.sh | 98 ++++++++++++++++++++++++++++++---- 1 file changed, 88 insertions(+), 10 deletions(-) diff --git a/rootfs/scripts/build-rootfs.sh b/rootfs/scripts/build-rootfs.sh index da9e2325..7e9da703 100755 --- a/rootfs/scripts/build-rootfs.sh +++ b/rootfs/scripts/build-rootfs.sh @@ -17,7 +17,9 @@ # - Supports JSON package manifest for additional package installation # (via apt or local .deb) inside the rootfs. # - Supports injecting custom apt sources from the package manifest. -# - Injects custom kernel and firmware .deb packages. +# - Optionally injects custom kernel and firmware .deb packages. +# - Supports installing local .deb packages (--local-debs) with full dependency resolution +# via a temporary local APT repository built inside the rootfs. # - Installs user-specified packages from seed and/or overlay manifest. # - Dynamically deduces and generates base and custom package manifests. # - Configures GRUB bootloader, hostname, DNS, and other system settings. @@ -27,18 +29,28 @@ # ./build-rootfs.sh \ # --product-conf qcom-product.conf \ # --seed seed_file \ -# --kernel-package kernel.deb \ +# [--kernel-package kernel.deb] \ # [--firmware firmware.deb] \ # [--overlay package-manifest.json] \ -# [--variant desktop] +# [--variant desktop] \ +# [--local-debs pkg-a.deb] \ +# [--local-debs debs/] # # ARGUMENTS: # --product-conf Required. Product configuration file. # --seed Required. Seed file: one package per line (# comments allowed). -# --kernel-package Required. Custom kernel package. +# --kernel-package Optional. Custom kernel package. # --firmware Optional. Custom firmware package. # --overlay Optional. JSON manifest specifying extra packages/apt sources. # --variant Optional. System variant (default: desktop). +# --local-debs Optional. A .deb file OR a directory of .deb files to install +# via a local APT repo (repeatable). May be specified multiple +# times. Inter-package dependencies are resolved automatically +# via a temporary local APT repository built inside the rootfs. +# Installed after manifest packages (--overlay). +# Examples: +# --local-debs mypkg.deb +# --local-debs debs/ # # OUTPUT: # rootfs.img Flashable ext4 rootfs image. @@ -69,21 +81,26 @@ MANIFEST="" # internal name retained (overlay JSON) KERNEL_DEB="" FIRMWARE_DEB="" VARIANT_INPUT="" # New variable to hold the variant argument +LOCAL_DEBS=() # Array of local .deb file paths (--local-debs, repeatable) USE_CONF=0 USE_MANIFEST=0 TARGET="" print_usage() { echo "Usage:" - echo " $0 --product-conf --seed --kernel-package [--firmware ] [--overlay ] [--variant ]" + echo " $0 --product-conf --seed [--kernel-package ] [--firmware ] [--overlay ] [--variant ] [--local-debs ] ..." echo echo "Arguments:" echo " --product-conf Required. qcom-product.conf" echo " --seed Required. Seed file (one package per line; supports # comments)" - echo " --kernel-package Required. Kernel .deb" + echo " --kernel-package Optional. Kernel .deb" echo " --firmware Optional. Firmware .deb" echo " --overlay Optional. package-manifest.json (same schema as current manifest)" echo " --variant Optional. System variant (default: desktop)" + echo " --local-debs Optional. A .deb file OR a directory of .deb files to install via a" + echo " local APT repo (repeatable). Specify once per path. Dependencies" + echo " between packages are resolved automatically. Installed after manifest." + echo " Examples: --local-debs mypkg.deb --local-debs debs/" } # Parse named options @@ -102,6 +119,8 @@ while [[ $# -gt 0 ]]; do MANIFEST="${2-}"; shift 2 ;; --variant) VARIANT_INPUT="${2-}"; shift 2 ;; + --local-debs) + if [[ -n "${2-}" ]]; then LOCAL_DEBS+=("${2-}"); fi; shift 2 ;; -h|--help) print_usage exit 0 @@ -115,7 +134,7 @@ while [[ $# -gt 0 ]]; do done # Validate required args -if [[ -z "${CONF}" || -z "${SEED}" || -z "${KERNEL_DEB}" ]]; then +if [[ -z "${CONF}" || -z "${SEED}" ]]; then echo "[ERROR] Missing required argument(s)." print_usage exit 1 @@ -129,13 +148,25 @@ fi [[ -f "$CONF" ]] || { echo "[ERROR] Config file not found: $CONF"; exit 1; } [[ -f "$SEED" ]] || { echo "[ERROR] Seed file not found: $SEED"; exit 1; } -[[ -f "$KERNEL_DEB" ]] || { echo "[ERROR] Kernel package not found: $KERNEL_DEB"; exit 1; } +if [[ -n "$KERNEL_DEB" ]]; then + [[ -f "$KERNEL_DEB" ]] || { echo "[ERROR] Kernel package not found: $KERNEL_DEB"; exit 1; } +fi if [[ -n "$FIRMWARE_DEB" ]]; then [[ -f "$FIRMWARE_DEB" ]] || { echo "[ERROR] Firmware package not found: $FIRMWARE_DEB"; exit 1; } fi if [[ "$USE_MANIFEST" -eq 1 && -n "$MANIFEST" ]]; then [[ -f "$MANIFEST" ]] || { echo "[ERROR] Manifest/overlay file not found: $MANIFEST"; exit 1; } fi +for _deb in "${LOCAL_DEBS[@]}"; do + if [[ -d "$_deb" ]]; then + : # directory — may contain zero or more .deb files; handled gracefully at copy time + elif [[ -f "$_deb" ]]; then + : # valid .deb file path + else + echo "[ERROR] --local-debs path not found (not a file or directory): $_deb" + exit 1 + fi +done WORKDIR=$(pwd) MNT_DIR="$WORKDIR/mnt" @@ -391,7 +422,9 @@ fi # Step 4: Inject Kernel, Firmware, and Working resolv.conf # ============================================================================== echo "[INFO] Copying kernel and firmware packages into rootfs..." -cp "$KERNEL_DEB" "$ROOTFS_DIR/" +if [[ -n "$KERNEL_DEB" ]]; then + cp "$KERNEL_DEB" "$ROOTFS_DIR/" +fi if [[ -n "$FIRMWARE_DEB" ]]; then cp "$FIRMWARE_DEB" "$ROOTFS_DIR/" fi @@ -463,6 +496,27 @@ apt install -y ${APT_INSTALL_LIST[@]} EOF chmod +x "$ROOTFS_DIR/install_manifest_pkgs.sh" +# ============================================================================== +# Step 6.5: Copy local .deb packages into rootfs (if provided) +# ============================================================================== +if [[ ${#LOCAL_DEBS[@]} -gt 0 ]]; then + echo "[INFO] Copying local .deb packages into rootfs for local APT repository..." + mkdir -p "$ROOTFS_DIR/opt/local-debs" + for _deb in "${LOCAL_DEBS[@]}"; do + if [[ -d "$_deb" ]]; then + echo "[INFO] -> directory: $_deb" + for _f in "$_deb"/*.deb; do + [[ -f "$_f" ]] || continue # skip if glob matched nothing (empty dir) + echo "[INFO] $(basename "$_f")" + cp "$_f" "$ROOTFS_DIR/opt/local-debs/" + done + else + echo "[INFO] -> $(basename "$_deb")" + cp "$_deb" "$ROOTFS_DIR/opt/local-debs/" + fi + done +fi + # ============================================================================== # Step 7: Bind Mount System Directories for chroot # ============================================================================== @@ -516,6 +570,13 @@ else CMD_FW_INSTALL="echo '[CHROOT] Skipping firmware installation.'" fi +CMD_KERNEL_INSTALL="" +if [[ -n "$KERNEL_DEB" ]]; then + CMD_KERNEL_INSTALL="yes \"\" | dpkg -i /$(basename "$KERNEL_DEB")" +else + CMD_KERNEL_INSTALL="echo '[CHROOT] Skipping kernel installation (no kernel package provided).'" +fi + echo "[INFO] Entering chroot to install packages and configure GRUB..." env DISTRO="$DISTRO" CODENAME="$CODENAME" VARIANT="$VARIANT" \ chroot "$ROOTFS_DIR" /bin/bash -c " @@ -562,7 +623,7 @@ dpkg-query -W -f='\${Package} \${Version}\n' > /tmp/\${CODENAME}_base.manifest echo '[CHROOT] Installing custom firmware and kernel...' $CMD_FW_INSTALL -yes \"\" | dpkg -i /$(basename "$KERNEL_DEB") +$CMD_KERNEL_INSTALL # Run update-grub explicitly: the zz-update-grub hook skips it in a chroot # because systemd is not running (/run/systemd/system absent). @@ -575,6 +636,23 @@ usermod -aG sudo qcom echo '[CHROOT] Installing manifest packages (if any)...' /install_manifest_pkgs.sh || true +echo '[CHROOT] Installing local .deb packages via local APT repository (if any)...' +if ls /opt/local-debs/*.deb >/dev/null 2>&1; then + echo '[CHROOT] Setting up local .deb APT repository...' + apt-get install -y --no-install-recommends dpkg-dev + cd /opt/local-debs + dpkg-scanpackages . /dev/null > Packages + cd / + echo 'deb [trusted=yes] file:///opt/local-debs ./' > /etc/apt/sources.list.d/local-debs.list + apt-get update + LOCAL_PKG_NAMES=\$(for deb in /opt/local-debs/*.deb; do dpkg-deb --field \"\$deb\" Package; done | tr '\n' ' ') + echo \"[CHROOT] Installing local packages: \$LOCAL_PKG_NAMES\" + apt-get install -y \$LOCAL_PKG_NAMES + echo '[CHROOT] Local .deb packages installed successfully.' +else + echo '[CHROOT] No local .deb packages found; skipping.' +fi + echo '[CHROOT] Capturing post-install package list...' dpkg-query -W -f='\${Package} \${Version}\n' > /tmp/\${CODENAME}_post.manifest From 602443e9abab7cd726db165969ed78c3cdf6682c Mon Sep 17 00:00:00 2001 From: Bjordis Collaku Date: Wed, 10 Jun 2026 16:57:33 -0700 Subject: [PATCH 3/4] rootfs: run update-grub after all package installs Problem - update-grub was executed immediately after the direct kernel `dpkg -i`, before overlay apt packages and local-debs repo packages were installed. - When the effective kernel came from overlay manifest apt sources, GRUB generation could observe stale kernel state. Implementation - Move post-bootloader actions to after all install sources complete: 1. firmware `dpkg -i` (optional) 2. kernel `dpkg -i` (optional) 3. overlay manifest apt packages 4. local-debs apt repository packages 5. update-grub 6. grub cleanup/normalization 7. package manifest capture/delta generation - Keep the explicit update-grub invocation in the chroot (do not rely on hook execution gated by systemd runtime presence). Result - Bootloader state reflects final package-set convergence rather than an intermediate state. - The kernel delivery path (direct deb vs apt overlay vs local deb repo) no longer changes GRUB correctness. Scope - Install-order and bootloader-timing correction only; the DTB policy cleanup remains separate. Signed-off-by: Bjordis Collaku --- rootfs/scripts/build-rootfs.sh | 57 ++++++++++++++++++++-------------- 1 file changed, 33 insertions(+), 24 deletions(-) diff --git a/rootfs/scripts/build-rootfs.sh b/rootfs/scripts/build-rootfs.sh index 7e9da703..58368486 100755 --- a/rootfs/scripts/build-rootfs.sh +++ b/rootfs/scripts/build-rootfs.sh @@ -18,6 +18,8 @@ # (via apt or local .deb) inside the rootfs. # - Supports injecting custom apt sources from the package manifest. # - Optionally injects custom kernel and firmware .deb packages. +# - Supports installing kernel packages via APT through the --overlay manifest +# (source: apt); update-grub runs after all installs regardless of kernel path. # - Supports installing local .deb packages (--local-debs) with full dependency resolution # via a temporary local APT repository built inside the rootfs. # - Installs user-specified packages from seed and/or overlay manifest. @@ -625,10 +627,6 @@ echo '[CHROOT] Installing custom firmware and kernel...' $CMD_FW_INSTALL $CMD_KERNEL_INSTALL -# Run update-grub explicitly: the zz-update-grub hook skips it in a chroot -# because systemd is not running (/run/systemd/system absent). -update-grub - adduser --disabled-password --gecos '' qcom echo 'qcom:qcom' | chpasswd usermod -aG sudo qcom @@ -653,20 +651,15 @@ else echo '[CHROOT] No local .deb packages found; skipping.' fi -echo '[CHROOT] Capturing post-install package list...' -dpkg-query -W -f='\${Package} \${Version}\n' > /tmp/\${CODENAME}_post.manifest - -echo '[CHROOT] Sorting and computing package delta...' -sort /tmp/\${CODENAME}_base.manifest > /tmp/sorted_base.manifest -sort /tmp/\${CODENAME}_post.manifest > /tmp/sorted_post.manifest -DATE=\$(date +%Y-%m-%d) -comm -13 /tmp/sorted_base.manifest /tmp/sorted_post.manifest > /tmp/packages_\${DATE}.manifest - -echo '[CHROOT] Cleaning up intermediate files...' -rm -f /tmp/\${CODENAME}_post.manifest /tmp/sorted_base.manifest /tmp/sorted_post.manifest - -echo '[CHROOT] Base package list preserved as /tmp/\${CODENAME}_base.manifest' -echo '[CHROOT] Custom installed packages saved to /tmp/packages_\${DATE}.manifest' +# ============================================================================== +# Run update-grub after ALL installs (firmware, kernel via dpkg or apt, manifest, +# local-debs). This ensures GRUB sees whichever kernel was installed last, +# regardless of the delivery path. +# The zz-update-grub hook skips update-grub in a chroot because systemd is not +# running (/run/systemd/system absent), so we call it explicitly here. +# ============================================================================== +echo '[CHROOT] Running update-grub after all package installs...' +update-grub # ============================================================================== # GRUB Configuration Cleanup & Standardization @@ -686,27 +679,43 @@ sed -i 's/root=\/dev\/[^ ]* //g' /boot/grub/grub.cfg # Device Tree Configuration for Debian platforms # ============================================================================== -if [ \"\${distro_lc}\" = \"debian\" ]; then +if [ "\${distro_lc}" = "debian" ]; then echo '[INFO][CHROOT] Debian target detected. Configuring platform Device Tree...' # Locate the platform Device Tree Blob (DTB) in standard library or firmware paths - DTB_PATH=\$(find /usr/lib /lib/firmware -name \"glymur-crd.dtb\" -print -quit) + DTB_PATH=\$(find /usr/lib /lib/firmware -name "glymur-crd.dtb" -print -quit) - if [ -n \"\$DTB_PATH\" ]; then - echo \"[INFO][CHROOT] Platform DTB resolved: \$DTB_PATH\" + if [ -n "\$DTB_PATH" ]; then + echo "[INFO][CHROOT] Platform DTB resolved: \$DTB_PATH" # Ensure DTB is accessible in the bootloader's filesystem scope - ln -sf \"\$DTB_PATH\" /boot/dtb + ln -sf "\$DTB_PATH" /boot/dtb # Inject the devicetree directive into the generated GRUB configuration. # This appends the command immediately following the 'initrd' load. - sed -i \"/^[[:space:]]*initrd/a \ devicetree /boot/dtb\" /boot/grub/grub.cfg + sed -i "/^[[:space:]]*initrd/a \ devicetree /boot/dtb" /boot/grub/grub.cfg echo '[SUCCESS][CHROOT] Device Tree directive injected into /boot/grub/grub.cfg' else echo '[WARN][CHROOT] Target DTB (glymur-crd.dtb) not found. Skipping injection.' fi fi + +echo '[CHROOT] Capturing post-install package list...' +dpkg-query -W -f='\${Package} \${Version}\n' > /tmp/\${CODENAME}_post.manifest + +echo '[CHROOT] Sorting and computing package delta...' +sort /tmp/\${CODENAME}_base.manifest > /tmp/sorted_base.manifest +sort /tmp/\${CODENAME}_post.manifest > /tmp/sorted_post.manifest +DATE=\$(date +%Y-%m-%d) +comm -13 /tmp/sorted_base.manifest /tmp/sorted_post.manifest > /tmp/packages_\${DATE}.manifest + +echo '[CHROOT] Cleaning up intermediate files...' +rm -f /tmp/\${CODENAME}_post.manifest /tmp/sorted_base.manifest /tmp/sorted_post.manifest + +echo '[CHROOT] Base package list preserved as /tmp/\${CODENAME}_base.manifest' +echo '[CHROOT] Custom installed packages saved to /tmp/packages_\${DATE}.manifest' + " # ============================================================================== From 218f13111c45dc3afe0d93bdc1a8193e8b9247a9 Mon Sep 17 00:00:00 2001 From: Bjordis Collaku Date: Wed, 10 Jun 2026 16:57:33 -0700 Subject: [PATCH 4/4] rootfs: drop Debian-specific DTB injection Context - The script carried a platform-specific Debian path that: - searched for glymur-crd.dtb, - created a /boot/dtb symlink, - patched grub.cfg to inject devicetree directives. - This behavior is board-specific policy, not generic rootfs assembly responsibility. Implementation - Remove the Debian-only DTB injection block entirely, including the conditional guard, filesystem probing, symlink synthesis, and GRUB mutation. Architectural rationale - Keep build-rootfs.sh focused on distribution-agnostic image construction. - Avoid embedding platform policy and hardcoded DTB assumptions in a shared reusable tool. - Defer DTB/boot policy to dedicated metadata/build layers that own board-specific behavior. Operational impact - Reduces implicit side effects in the generated GRUB config. - Improves portability and maintainability of rootfs construction across product lines. Signed-off-by: Bjordis Collaku --- rootfs/scripts/build-rootfs.sh | 26 -------------------------- 1 file changed, 26 deletions(-) diff --git a/rootfs/scripts/build-rootfs.sh b/rootfs/scripts/build-rootfs.sh index 58368486..04444ffa 100755 --- a/rootfs/scripts/build-rootfs.sh +++ b/rootfs/scripts/build-rootfs.sh @@ -675,32 +675,6 @@ sed -i 's/search --no-floppy --fs-uuid --set=root .*/search --no-floppy --label # conflicts with our 'root=LABEL=system' argument. sed -i 's/root=\/dev\/[^ ]* //g' /boot/grub/grub.cfg -# ============================================================================== -# Device Tree Configuration for Debian platforms -# ============================================================================== - -if [ "\${distro_lc}" = "debian" ]; then - echo '[INFO][CHROOT] Debian target detected. Configuring platform Device Tree...' - - # Locate the platform Device Tree Blob (DTB) in standard library or firmware paths - DTB_PATH=\$(find /usr/lib /lib/firmware -name "glymur-crd.dtb" -print -quit) - - if [ -n "\$DTB_PATH" ]; then - echo "[INFO][CHROOT] Platform DTB resolved: \$DTB_PATH" - - # Ensure DTB is accessible in the bootloader's filesystem scope - ln -sf "\$DTB_PATH" /boot/dtb - - # Inject the devicetree directive into the generated GRUB configuration. - # This appends the command immediately following the 'initrd' load. - sed -i "/^[[:space:]]*initrd/a \ devicetree /boot/dtb" /boot/grub/grub.cfg - - echo '[SUCCESS][CHROOT] Device Tree directive injected into /boot/grub/grub.cfg' - else - echo '[WARN][CHROOT] Target DTB (glymur-crd.dtb) not found. Skipping injection.' - fi -fi - echo '[CHROOT] Capturing post-install package list...' dpkg-query -W -f='\${Package} \${Version}\n' > /tmp/\${CODENAME}_post.manifest