Skip to content
Open
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions pkgs/code_assets/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,14 @@
## 1.3.0-wip

- Validate the container family expected for the target operating system and
the target architecture family of bundled dynamic libraries by reading
their ELF, Mach-O, or PE headers. Reject multi-architecture Mach-O output
because hooks run once per target architecture. Unrecognized files produce
a warning when a logger is supplied instead of failing.
- Add `NativeLibraryValidator` for invoker-supplied validation without
exposing the built-in header model. The extension point is compatible with
future custom operating system and architecture support.

## 1.2.1

- Avoid throwing a null-assertion error/exception when `input.config.code` is
Expand Down
5 changes: 5 additions & 0 deletions pkgs/code_assets/lib/code_assets.dart
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,11 @@ export 'src/code_assets/link_mode.dart'
LookupInProcess,
StaticLinking;
export 'src/code_assets/link_mode_preference.dart' show LinkModePreference;
export 'src/code_assets/native_library_validation.dart'
show
NativeLibraryValidation,
NativeLibraryValidationContext,
NativeLibraryValidator;
export 'src/code_assets/os.dart' show OS;
export 'src/code_assets/sanitizer.dart' show Sanitizer;
export 'src/code_assets/testing.dart' show testCodeBuildHook;
32 changes: 29 additions & 3 deletions pkgs/code_assets/lib/src/code_assets/extension.dart
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,15 @@
// BSD-style license that can be found in the LICENSE file.

import 'package:hooks/hooks.dart';
import 'package:logging/logging.dart';

import 'architecture.dart';
import 'c_compiler_config.dart';
import 'code_asset.dart';
import 'config.dart';
import 'link_mode.dart';
import 'link_mode_preference.dart';
import 'native_library_validation.dart';
import 'os.dart';
import 'sanitizer.dart';
import 'validation.dart';
Expand Down Expand Up @@ -55,6 +57,16 @@ final class CodeAssetExtension extends ProtocolExtension {
/// runtime libraries and memory allocators across the FFI boundary.
final Sanitizer? sanitizer;

/// Receives warnings from validations which are skipped rather than failed,
/// such as an unrecognized native library header.
final Logger? logger;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rather have the logger from hooks_runner be used instead of a logger here in the constructor, but adding a logger argument to the validateXxx methods would be a breaking change.

See https://github.com/flutter/flutter/blob/e69e703dce29fda392f8024109c1961306cb365a/packages/flutter_tools/lib/src/isolated/native_assets/targets.dart#L237. We don't want to have to construct the same logger there as we did in https://github.com/flutter/flutter/blob/e69e703dce29fda392f8024109c1961306cb365a/packages/flutter_tools/lib/src/isolated/native_assets/native_assets.dart#L561

Maybe we can add the logger to the ValidationContext?

WDYT?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm wait the code assets extension creates the validation context, so it also doesn't have access to the logger.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed it's breaking even as an optional named param, since an external ProtocolExtension subclass that doesn't also declare it would fail invalid_override, so I won't touch those signatures. The additive way to your ValidationContext idea: add a mutable Logger? logger to ProtocolExtension, have NativeAssetsBuildRunner set e.logger before the override call in its existing for (final e in extensions) loop, and CodeAssetExtension forwards it into the context it builds. That covers your follow-up: the extension still creates the context, but now has the runner's logger to put in it, so Flutter never builds a second one. I'll leave the existing constructor logger as a fallback so the change stays additive.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correction on the mechanism I proposed. Keeping CodeAssetExtension's existing final Logger? logger alongside a new logger on ProtocolExtension does not work: the subclass field shadows the base getter, so an e.logger assignment from the runner lands on the base field while every read dispatches to the subclass getter, a silent no-op (confirmed with a quick repro). The workable shape is a single storage: put Logger? logger; on ProtocolExtension, drop the field from CodeAssetExtension, and set this.logger in its constructor body so existing callers still pass it. If you would rather the runner not mutate a caller-owned field, a void setupValidationLogger(Logger logger) {} default hook does the same job.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding a method to ProtocolExtension like setupValidationLogger is then not breaking because we made it a base class.

Then we could also consider something along the lines of:

base class ProtocolExtension {
  // ..
  late Logger _logger;

  Logger get logger => _logger;

  /// Sets up context by the hooks_runner that uses the protocol extensions.
  ///
  /// Guaranteed to be run before any other methods are called on this object.
  void setupLogger(Logger logger) {
    this._logger = logger;
  }
}

We might get passed the same protocol extension object in multiple build/link methods in one run. So we do end up setting the logger more than once potentially. So the field can't be final.

I think technically this is still a breaking change, because package:hooks_runner might resolve to an older version while package:code_assets might resolve to a newer version, causing the setupLogger never to be called. However, I don't think this will happen in Dart or Flutter because all packages are rolled at the same time.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added setupLogger on ProtocolExtension, a non-final late field with a logger getter, since the same extension object can be reused across a run. The hooks_runner calls it before any other method in both build and link. CodeAssetExtension's own logger field is removed so nothing shadows the base getter.


/// Additional validators for bundled dynamic libraries.
///
/// These validators are supplied by the hook invoker, not by packages that
/// produce hook output. The iterable is copied during construction.
final List<NativeLibraryValidator> additionalLibraryValidators;

/// Constructs a [CodeAssetExtension].
CodeAssetExtension({
required this.targetArchitecture,
Expand All @@ -65,7 +77,11 @@ final class CodeAssetExtension extends ProtocolExtension {
this.iOS,
this.macOS,
this.sanitizer,
});
this.logger,
Iterable<NativeLibraryValidator> additionalLibraryValidators = const [],
}) : additionalLibraryValidators = List.unmodifiable(
additionalLibraryValidators,
);

@override
void setupBuildInput(BuildInputBuilder input) {
Expand Down Expand Up @@ -103,13 +119,23 @@ final class CodeAssetExtension extends ProtocolExtension {
Future<ValidationErrors> validateBuildOutput(
BuildInput input,
BuildOutput output,
) => validateCodeAssetBuildOutput(input, output);
) => validateCodeAssetBuildOutput(
input,
output,
logger: logger,
additionalLibraryValidators: additionalLibraryValidators,
);

@override
Future<ValidationErrors> validateLinkOutput(
LinkInput input,
LinkOutput output,
) => validateCodeAssetLinkOutput(input, output);
) => validateCodeAssetLinkOutput(
input,
output,
logger: logger,
additionalLibraryValidators: additionalLibraryValidators,
);

@override
Future<ValidationErrors> validateApplicationAssets(
Expand Down
296 changes: 296 additions & 0 deletions pkgs/code_assets/lib/src/code_assets/native_library_headers.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,296 @@
// Copyright (c) 2026, the Dart project authors. Please see the AUTHORS file
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.

import 'dart:io';
import 'dart:typed_data';

import 'architecture.dart';

/// ELF identification and machine values from the
/// [System V ABI machine table](https://gabi.xinuos.com/elf/a-emachine.html)
/// and the
/// [RISC-V ELF psABI](https://github.com/riscv-non-isa/riscv-elf-psabi-doc/blob/master/riscv-elf.adoc).
abstract final class _Elf {
static const magic = [0x7f, 0x45, 0x4c, 0x46];
static const class32 = 1;
static const class64 = 2;
static const littleEndian = 1;
static const bigEndian = 2;
static const machineOffset = 18;
static const machineX86 = 3;
static const machineArm = 40;
static const machineX64 = 62;
static const machineArm64 = 183;
static const machineRiscV = 243;
}

/// Mach-O magic and CPU values from Apple's
/// [`loader.h`](https://github.com/apple-oss-distributions/xnu/blob/main/EXTERNAL_HEADERS/mach-o/loader.h),
/// [`fat.h`](https://github.com/apple-oss-distributions/cctools/blob/main/include/mach-o/fat.h),
/// and
/// [`machine.h`](https://github.com/apple-oss-distributions/xnu/blob/main/osfmk/mach/machine.h).
abstract final class _MachO {
static const magic32 = 0xfeedface;
static const magic64 = 0xfeedfacf;
static const swappedMagic32 = 0xcefaedfe;
static const swappedMagic64 = 0xcffaedfe;
static const fatMagic32 = 0xcafebabe;
static const fatMagic64 = 0xcafebabf;
static const cpuTypeX86 = 0x00000007;
static const cpuTypeX64 = 0x01000007;
static const cpuTypeArm = 0x0000000c;
static const cpuTypeArm64 = 0x0100000c;
}

/// PE offsets, signatures, and machine values from Microsoft's
/// [PE format specification](https://learn.microsoft.com/en-us/windows/win32/debug/pe-format).
abstract final class _Pe {
static const dosMagic = [0x4d, 0x5a];
static const peHeaderOffset = 0x3c;
static const signature = [0x50, 0x45, 0x00, 0x00];
static const machineX86 = 0x014c;
static const machineArm = 0x01c4;
static const machineRiscV64 = 0x5064;
static const machineX64 = 0x8664;
static const machineArm64 = 0xaa64;
}

/// The binary format of a native library file.
enum BinaryFormat {
/// The Executable and Linkable Format used on Linux, Android, and Fuchsia.
elf('ELF'),

/// The Mach object format used on macOS and iOS.
machO('Mach-O'),

/// The Portable Executable format used on Windows.
pe('PE');

const BinaryFormat(this.displayName);

/// The name of this format in messages.
final String displayName;
}

/// The result of reading a native library's header.
sealed class NativeLibraryHeader {
const NativeLibraryHeader();
}

/// A header whose binary format was recognized.
final class RecognizedHeader extends NativeLibraryHeader {
/// Constructs a [RecognizedHeader].
const RecognizedHeader(
this.format,
this.architectures,
this.machineValues, {
int? architectureCount,
}) : architectureCount = architectureCount ?? architectures.length;

/// The binary format of the file.
final BinaryFormat format;

/// The architectures the file was built for, aligned with [machineValues].
///
/// A `null` entry means the format was recognized but its machine value is
/// not one of the [Architecture]s. Fat Mach-O files contain one entry per
/// slice; the other formats always contain exactly one.
final List<Architecture?> architectures;

/// The raw machine values from the header, for diagnostics.
final List<int> machineValues;

/// The number of architectures declared by the header.
///
/// This can exceed [architectures].length when a fat header contains too
/// many entries to read safely.
final int architectureCount;
}

/// A file whose binary format was not recognized.
final class UnrecognizedHeader extends NativeLibraryHeader {
/// Constructs an [UnrecognizedHeader].
const UnrecognizedHeader();
}

/// Reads just enough of [file] to identify its binary format and the
/// architectures it was built for.
///
/// Never throws for file contents: anything that cannot be identified,
/// including short and empty files, is an [UnrecognizedHeader].
NativeLibraryHeader readNativeLibraryHeader(File file) {
final RandomAccessFile raf;
try {
raf = file.openSync();
} on FileSystemException {
return const UnrecognizedHeader();
}
try {
final length = raf.lengthSync();
final head = raf.readSync(64);
if (head.length >= 20 && _hasElfMagic(head)) return _readElf(head);
if (head.length >= 8) {
final bigEndianMagic = _u32(head, 0, littleEndian: false);
if (bigEndianMagic == _MachO.fatMagic32 ||
bigEndianMagic == _MachO.fatMagic64) {
return _readFatMachO(
raf,
head,
is64: bigEndianMagic == _MachO.fatMagic64,
);
}
final thin = _readThinMachO(head);
if (thin != null) return thin;
}
if (head.length >= _Pe.peHeaderOffset + 4 &&
head[0] == _Pe.dosMagic[0] &&
head[1] == _Pe.dosMagic[1]) {
return _readPe(raf, head, length);
}
return const UnrecognizedHeader();
} on FileSystemException {
return const UnrecognizedHeader();
} finally {
raf.closeSync();
}
}

bool _hasElfMagic(Uint8List head) =>

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for all of the magic numbers everywhere in this code, if we have sources lets add links to the sources in doc comments.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added source links for these constants in 8a67fb2 and updated the System V, fat-header, and machine-header links in 7560bd4.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do. Each format's magic numbers become private static consts in its own file with a doc-comment link to the format spec (the ELF e_machine table, Apple's loader.h/fat.h/machine.h, and the Microsoft PE spec).

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the source links for the magic numbers in doc comments; they moved into MachOValidator with the split.

head[0] == _Elf.magic[0] &&
head[1] == _Elf.magic[1] &&
head[2] == _Elf.magic[2] &&
head[3] == _Elf.magic[3];

NativeLibraryHeader _readElf(Uint8List head) {
final elfClass = head[4];
final elfData = head[5];
if ((elfClass != _Elf.class32 && elfClass != _Elf.class64) ||
(elfData != _Elf.littleEndian && elfData != _Elf.bigEndian)) {
return const UnrecognizedHeader();
}
final machine = _u16(
head,
_Elf.machineOffset,
littleEndian: elfData == _Elf.littleEndian,
);
final architecture = switch (machine) {
_Elf.machineX86 => Architecture.ia32,
_Elf.machineArm => Architecture.arm,
_Elf.machineX64 => Architecture.x64,
_Elf.machineArm64 => Architecture.arm64,
_Elf.machineRiscV =>
elfClass == _Elf.class32 ? Architecture.riscv32 : Architecture.riscv64,
_ => null,
};
return RecognizedHeader(BinaryFormat.elf, [architecture], [machine]);
}

Architecture? _machOArchitecture(int cpuType) => switch (cpuType) {
_MachO.cpuTypeX86 => Architecture.ia32,
_MachO.cpuTypeX64 => Architecture.x64,
_MachO.cpuTypeArm => Architecture.arm,
// arm64e uses CPU_TYPE_ARM64 and identifies the ABI in cpusubtype. Until
// TODO(https://github.com/dart-lang/native/issues/3379) reads that subtype,
// treat arm64e as arm64 rather than claiming to validate its ABI.
_MachO.cpuTypeArm64 => Architecture.arm64,
_ => null,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please leave a comment what you'd expect for arm64e with a TODO referring to:

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the arm64e note with a TODO pointing to #3379 in 8a67fb2.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do. I'll add a TODO in the Mach-O parser referencing #3379: arm64e uses the same CPU_TYPE_ARM64 and identifies its ABI in the cpusubtype, so until we read that subtype we treat it as arm64 rather than claiming to validate its ABI.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the arm64e comment with a TODO referencing #3379; it moved into the per-format Mach-O file with the split.

};

NativeLibraryHeader? _readThinMachO(Uint8List head) {
final littleEndianMagic = _u32(head, 0, littleEndian: true);
final bool littleEndian;
if (littleEndianMagic == _MachO.magic32 ||
littleEndianMagic == _MachO.magic64) {
littleEndian = true;
} else if (littleEndianMagic == _MachO.swappedMagic32 ||
littleEndianMagic == _MachO.swappedMagic64) {
littleEndian = false;
} else {
return null;
}
final cpuType = _u32(head, 4, littleEndian: littleEndian);
return RecognizedHeader(
BinaryFormat.machO,
[_machOArchitecture(cpuType)],
[cpuType],
);
}

NativeLibraryHeader _readFatMachO(
RandomAccessFile raf,
Uint8List head, {
required bool is64,
}) {
// The fat header and arch entries are always big-endian. The number of
// slices is tiny in practice. Java class files share the 0xcafebabe magic,
// but their version number generally makes the implied architecture table
// larger than the file.
final sliceCount = _u32(head, 4, littleEndian: false);
if (sliceCount == 0) return const UnrecognizedHeader();
final entrySize = is64 ? 32 : 20;
final entriesLength = sliceCount * entrySize;
if (raf.lengthSync() < 8 + entriesLength) {
return const UnrecognizedHeader();
}
if (sliceCount > 32) {
return RecognizedHeader(
BinaryFormat.machO,
const [],
const [],
architectureCount: sliceCount,
);
}
raf.setPositionSync(8);
final entries = raf.readSync(entriesLength);
if (entries.length < entriesLength) {
return const UnrecognizedHeader();
}
final machineValues = [
for (var slice = 0; slice < sliceCount; slice++)
_u32(entries, slice * entrySize, littleEndian: false),
];
return RecognizedHeader(BinaryFormat.machO, [
for (final cpuType in machineValues) _machOArchitecture(cpuType),
], machineValues);
}

NativeLibraryHeader _readPe(RandomAccessFile raf, Uint8List head, int length) {
final peOffset = _u32(head, _Pe.peHeaderOffset, littleEndian: true);
if (peOffset + 6 > length) return const UnrecognizedHeader();
raf.setPositionSync(peOffset);
final signature = raf.readSync(6);
if (signature.length < 6 ||
signature[0] != _Pe.signature[0] ||
signature[1] != _Pe.signature[1] ||
signature[2] != _Pe.signature[2] ||
signature[3] != _Pe.signature[3]) {
return const UnrecognizedHeader();
}
final machine = _u16(signature, 4, littleEndian: true);
final architecture = switch (machine) {
_Pe.machineX86 => Architecture.ia32,
_Pe.machineArm => Architecture.arm,
_Pe.machineRiscV64 => Architecture.riscv64,
_Pe.machineX64 => Architecture.x64,
_Pe.machineArm64 => Architecture.arm64,
_ => null,
};
return RecognizedHeader(BinaryFormat.pe, [architecture], [machine]);
}

int _u16(Uint8List bytes, int offset, {required bool littleEndian}) =>
littleEndian
? bytes[offset] | (bytes[offset + 1] << 8)
: (bytes[offset] << 8) | bytes[offset + 1];

int _u32(Uint8List bytes, int offset, {required bool littleEndian}) =>
littleEndian
? bytes[offset] |
(bytes[offset + 1] << 8) |
(bytes[offset + 2] << 16) |
(bytes[offset + 3] << 24)
: (bytes[offset] << 24) |
(bytes[offset + 1] << 16) |
(bytes[offset + 2] << 8) |
bytes[offset + 3];
Loading
Loading