Error: storage error: Failed to read ownership xattr #1578

Activity

1. DorianZheng

   assigned

   ltstriker

   on Sep 21, 2026
2. DorianZheng commented on Sep 22, 2026

   

   DorianZheng

   on Sep 22, 2026

   Member

   More actions

   @boxliteai investigate the issue

   👀React with 👀1Reacted by BoxLite
3. boxliteai commented on Sep 22, 2026

   

   boxliteai

   on Sep 22, 2026

   Contributor

   More actions

   @DorianZheng I traced this to permission ordering in fix_rootfs_permissions; it is not a corrupt Fedora xattr.

   fedora:latest arm64 contains:

   ---x--x--x 0/0 136976 ... usr/bin/sudoreplay
   

   The extractor writes user.containers.override_stat before applying the final 0111 mode. Later, fix_rootfs_permissions does the reverse of what macOS requires:

   1. Reads the ownership xattr while the file is still 0111.
   2. macOS authorizes getxattr with KAUTH_VNODE_READ_EXTATTRIBUTES (XNU source).
   3. The read returns EACCES.
   4. Only afterward would BoxLite temporarily add write permission.

   The copy-mount symlink error is separate and already triggers the intended extraction fallback.

   Recommended fix: temporarily grant owner read/write before both xattr operations, then restore the original mode on every exit path:

    let mode = metadata.permissions().mode() & 0o7777;
   +let needs_access = mode & 0o600 != 0o600;
   +if needs_access {
   +    set_mode(path, mode | 0o600)?;
   +}
   
   -let recorded = OverrideStat::read_xattr(path)?;
   +let operation_result = (|| {
   +    let recorded = OverrideStat::read_xattr(path)?;
   +    // Preserve recorded uid/gid and update the xattr as today.
   +    ...
   +})();
   +
   +if needs_access {
   +    set_mode(path, mode)?;
   +}
   +operation_result?;

   This should retain the current protection against malformed ownership records; treating EACCES as “xattr absent” would risk replacing the only recorded UID/GID with 0:0.

   A macOS regression test should create an xattr-bearing file, chmod it to 0111, call fix_rootfs_permissions, and verify both the ownership record and 0111 mode survive.

   Commands supporting the diagnosis:

   curl ... registry.fedoraproject.org/v2/fedora/... | tar -tzvvf - | rg usr/bin/sudoreplay
   git blame -L 145,225 src/boxlite/src/rootfs/operations.rs
   git clone --depth 1 https://github.com/apple-oss-distributions/xnu.git

   I could not run the Rust tests because this investigation VM does not have make installed. No repository files were changed.

   _{📦 Ran in an isolated BoxLite microVM · reply with a mention to follow up}
4. DorianZheng commented on Sep 22, 2026

   

   DorianZheng

   on Sep 22, 2026

   Member

   More actions

   @boxliteai in short

   👀React with 👀1Reacted by BoxLite
5. boxliteai commented on Sep 22, 2026

   

   boxliteai

   on Sep 22, 2026

   Contributor

   More actions

   @DorianZheng Fedora’s sudoreplay is execute-only (0111). BoxLite reads its ownership xattr before adding temporary read permission, matching the macOS Permission denied failure.

   Proposed fix: grant owner read/write before accessing xattrs, preserve ownership, and restore the original permissions afterward. The diagnosis still needs a macOS reproduction.

   _{📦 Ran in an isolated BoxLite microVM · reply with a mention to follow up}
6. DorianZheng commented on Sep 22, 2026

   

   DorianZheng

   on Sep 22, 2026

   Member

   More actions

   @boxliteai we already have similar fix

   👀React with 👀1Reacted by BoxLite
7. boxliteai commented on Sep 22, 2026

   

   boxliteai

   on Sep 22, 2026

   Contributor

   More actions

   @DorianZheng Yes—#1177 already handles this in the ext4 source scan: grant owner read permission before reading xattrs, then restore the original mode.

   The gap is the earlier fix_rootfs_permissions traversal, which still reads xattrs before adjusting permissions. This failure occurs before the existing fix can help. I missed that precedent in my first reply.

   _{📦 Ran in an isolated BoxLite microVM · reply with a mention to follow up}
8. DorianZheng commented on Sep 22, 2026

   

   DorianZheng

   on Sep 22, 2026

   Member

   More actions

   @boxliteai illustrate the workflow and denote these 2 site

   👀React with 👀1Reacted by BoxLite
9. boxliteai commented on Sep 22, 2026

   

   boxliteai

   on Sep 22, 2026

   Contributor

   More actions

   @DorianZheng For a new image disk, running as a regular user:

   flowchart TD
       A["RootfsBuilder.prepare"] --> B["Copy cached layers"]
       B -->|success| E
       B -->|"failure: reported symlink error"| C["Fallback: extract layers and finalize modes"]
       C --> D["① fix_rootfs_permissions<br/>Read ownership xattr BEFORE chmod"]
       D -->|"reported EACCES on sudoreplay"| X["Abort initialization"]
       D -->|success| E["create_ext4_from_dir"]
       E --> F["② scan_source_tree — existing fix<br/>Grant owner read BEFORE reading xattrs"]
       F --> G["mke2fs populates ext4"]
       G --> H["Restore intended image ownership/modes<br/>and original host modes"]
   

   Loading

   • ① Remaining gap: operations.rs:188 reads the xattr first. Its later permission adjustment adds write only, so it does not protect this read.
   • ② Existing fix: ext4.rs:271–281 grants owner read before accessing ownership xattrs.

   The reported fallback fails at ①, so ② is never reached.

   _{📦 Ran in an isolated BoxLite microVM · reply with a mention to follow up}
10. DorianZheng commented on Sep 22, 2026

    

    DorianZheng

    on Sep 22, 2026

    Member

    More actions

    @boxliteai can we extract the fix from 2? grant before 1 and restore in 2. make a simple code design here.

    👀React with 👀1Reacted by BoxLite
11. boxliteai commented on Sep 22, 2026

    

    boxliteai

    on Sep 22, 2026

    Contributor

    More actions

    @DorianZheng Yes. Extract SourceModeGuard and its permission-widening logic into a shared helper. Let PreparedRootfs own the guard across both stages.

    struct PreparedRootfs {
        path: PathBuf,
        modes: SourceModeGuard, // Records originals; Drop restores host modes.
    }
    
    // RootfsBuilder: after extraction and finalize, before ①
    let prepared = PreparedRootfs::grant_access(dest)?;
    fix_rootfs_permissions(&prepared)?; // ① Uses saved ORIGINAL modes.
    return Ok(prepared);
    
    // ImageDiskManager moves the whole prepared value into spawn_blocking.
    // Ext4 builder consumes it:
    fn create_ext4_from_rootfs(source: PreparedRootfs, ...) -> Result<Disk> {
        let scan = scan_source_tree(&source.path)?; // ② No longer grants access.
        let disk = populate_with_mke2fs(&source.path, ...)?;
        normalize_inodes_with_debugfs(&disk, &scan, &source.modes)?;
        Ok(disk)
        // source drops here: restores host modes.
    }

    Three details matter:

    • Grant access top-down: add owner read/write to files and read/write/search to directories; skip symlinks. Record each original mode before changing it.
    • ① must write the saved original mode into the xattr, otherwise temporary permissions become guest permissions.
    • ② restores image modes through debugfs; the guard restores host modes bottom-up. The guard also restores on failure before reaching ②.

    Copy-success and direct-directory ext4 callers should construct the same guarded input. This keeps one permission implementation and one owner of its lifetime.

    _{📦 Ran in an isolated BoxLite microVM · reply with a mention to follow up}