-
Notifications
You must be signed in to change notification settings - Fork 129
[code_assets] Validate dynamic library architecture from file headers #3484
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from 8 commits
695d62f
8a67fb2
23f70e0
24218e4
7560bd4
933b5ef
32f5f07
136e3f2
ccf2f0a
97a40d6
550ff1b
18dad6d
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,296 @@ | ||
| // Copyright (c) 2026, the Dart project authors. Please see the AUTHORS file | ||
| // for details. All rights reserved. Use of this source code is governed by a | ||
| // BSD-style license that can be found in the LICENSE file. | ||
|
|
||
| import 'dart:io'; | ||
| import 'dart:typed_data'; | ||
|
|
||
| import 'architecture.dart'; | ||
|
|
||
| /// ELF identification and machine values from the | ||
| /// [System V ABI machine table](https://gabi.xinuos.com/elf/a-emachine.html) | ||
| /// and the | ||
| /// [RISC-V ELF psABI](https://github.com/riscv-non-isa/riscv-elf-psabi-doc/blob/master/riscv-elf.adoc). | ||
| abstract final class _Elf { | ||
| static const magic = [0x7f, 0x45, 0x4c, 0x46]; | ||
| static const class32 = 1; | ||
| static const class64 = 2; | ||
| static const littleEndian = 1; | ||
| static const bigEndian = 2; | ||
| static const machineOffset = 18; | ||
| static const machineX86 = 3; | ||
| static const machineArm = 40; | ||
| static const machineX64 = 62; | ||
| static const machineArm64 = 183; | ||
| static const machineRiscV = 243; | ||
| } | ||
|
|
||
| /// Mach-O magic and CPU values from Apple's | ||
| /// [`loader.h`](https://github.com/apple-oss-distributions/xnu/blob/main/EXTERNAL_HEADERS/mach-o/loader.h), | ||
| /// [`fat.h`](https://github.com/apple-oss-distributions/cctools/blob/main/include/mach-o/fat.h), | ||
| /// and | ||
| /// [`machine.h`](https://github.com/apple-oss-distributions/xnu/blob/main/osfmk/mach/machine.h). | ||
| abstract final class _MachO { | ||
| static const magic32 = 0xfeedface; | ||
| static const magic64 = 0xfeedfacf; | ||
| static const swappedMagic32 = 0xcefaedfe; | ||
| static const swappedMagic64 = 0xcffaedfe; | ||
| static const fatMagic32 = 0xcafebabe; | ||
| static const fatMagic64 = 0xcafebabf; | ||
| static const cpuTypeX86 = 0x00000007; | ||
| static const cpuTypeX64 = 0x01000007; | ||
| static const cpuTypeArm = 0x0000000c; | ||
| static const cpuTypeArm64 = 0x0100000c; | ||
| } | ||
|
|
||
| /// PE offsets, signatures, and machine values from Microsoft's | ||
| /// [PE format specification](https://learn.microsoft.com/en-us/windows/win32/debug/pe-format). | ||
| abstract final class _Pe { | ||
| static const dosMagic = [0x4d, 0x5a]; | ||
| static const peHeaderOffset = 0x3c; | ||
| static const signature = [0x50, 0x45, 0x00, 0x00]; | ||
| static const machineX86 = 0x014c; | ||
| static const machineArm = 0x01c4; | ||
| static const machineRiscV64 = 0x5064; | ||
| static const machineX64 = 0x8664; | ||
| static const machineArm64 = 0xaa64; | ||
| } | ||
|
|
||
| /// The binary format of a native library file. | ||
| enum BinaryFormat { | ||
| /// The Executable and Linkable Format used on Linux, Android, and Fuchsia. | ||
| elf('ELF'), | ||
|
|
||
| /// The Mach object format used on macOS and iOS. | ||
| machO('Mach-O'), | ||
|
|
||
| /// The Portable Executable format used on Windows. | ||
| pe('PE'); | ||
|
|
||
| const BinaryFormat(this.displayName); | ||
|
|
||
| /// The name of this format in messages. | ||
| final String displayName; | ||
| } | ||
|
|
||
| /// The result of reading a native library's header. | ||
| sealed class NativeLibraryHeader { | ||
| const NativeLibraryHeader(); | ||
| } | ||
|
|
||
| /// A header whose binary format was recognized. | ||
| final class RecognizedHeader extends NativeLibraryHeader { | ||
| /// Constructs a [RecognizedHeader]. | ||
| const RecognizedHeader( | ||
| this.format, | ||
| this.architectures, | ||
| this.machineValues, { | ||
| int? architectureCount, | ||
| }) : architectureCount = architectureCount ?? architectures.length; | ||
|
|
||
| /// The binary format of the file. | ||
| final BinaryFormat format; | ||
|
|
||
| /// The architectures the file was built for, aligned with [machineValues]. | ||
| /// | ||
| /// A `null` entry means the format was recognized but its machine value is | ||
| /// not one of the [Architecture]s. Fat Mach-O files contain one entry per | ||
| /// slice; the other formats always contain exactly one. | ||
| final List<Architecture?> architectures; | ||
|
|
||
| /// The raw machine values from the header, for diagnostics. | ||
| final List<int> machineValues; | ||
|
|
||
| /// The number of architectures declared by the header. | ||
| /// | ||
| /// This can exceed [architectures].length when a fat header contains too | ||
| /// many entries to read safely. | ||
| final int architectureCount; | ||
| } | ||
|
|
||
| /// A file whose binary format was not recognized. | ||
| final class UnrecognizedHeader extends NativeLibraryHeader { | ||
| /// Constructs an [UnrecognizedHeader]. | ||
| const UnrecognizedHeader(); | ||
| } | ||
|
|
||
| /// Reads just enough of [file] to identify its binary format and the | ||
| /// architectures it was built for. | ||
| /// | ||
| /// Never throws for file contents: anything that cannot be identified, | ||
| /// including short and empty files, is an [UnrecognizedHeader]. | ||
| NativeLibraryHeader readNativeLibraryHeader(File file) { | ||
| final RandomAccessFile raf; | ||
| try { | ||
| raf = file.openSync(); | ||
| } on FileSystemException { | ||
| return const UnrecognizedHeader(); | ||
| } | ||
| try { | ||
| final length = raf.lengthSync(); | ||
| final head = raf.readSync(64); | ||
| if (head.length >= 20 && _hasElfMagic(head)) return _readElf(head); | ||
| if (head.length >= 8) { | ||
| final bigEndianMagic = _u32(head, 0, littleEndian: false); | ||
| if (bigEndianMagic == _MachO.fatMagic32 || | ||
| bigEndianMagic == _MachO.fatMagic64) { | ||
| return _readFatMachO( | ||
| raf, | ||
| head, | ||
| is64: bigEndianMagic == _MachO.fatMagic64, | ||
| ); | ||
| } | ||
| final thin = _readThinMachO(head); | ||
| if (thin != null) return thin; | ||
| } | ||
| if (head.length >= _Pe.peHeaderOffset + 4 && | ||
| head[0] == _Pe.dosMagic[0] && | ||
| head[1] == _Pe.dosMagic[1]) { | ||
| return _readPe(raf, head, length); | ||
| } | ||
| return const UnrecognizedHeader(); | ||
| } on FileSystemException { | ||
| return const UnrecognizedHeader(); | ||
| } finally { | ||
| raf.closeSync(); | ||
| } | ||
| } | ||
|
|
||
| bool _hasElfMagic(Uint8List head) => | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. for all of the magic numbers everywhere in this code, if we have sources lets add links to the sources in doc comments.
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Will do. Each format's magic numbers become private static consts in its own file with a doc-comment link to the format spec (the ELF
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Added the source links for the magic numbers in doc comments; they moved into |
||
| head[0] == _Elf.magic[0] && | ||
| head[1] == _Elf.magic[1] && | ||
| head[2] == _Elf.magic[2] && | ||
| head[3] == _Elf.magic[3]; | ||
|
|
||
| NativeLibraryHeader _readElf(Uint8List head) { | ||
| final elfClass = head[4]; | ||
| final elfData = head[5]; | ||
| if ((elfClass != _Elf.class32 && elfClass != _Elf.class64) || | ||
| (elfData != _Elf.littleEndian && elfData != _Elf.bigEndian)) { | ||
| return const UnrecognizedHeader(); | ||
| } | ||
| final machine = _u16( | ||
| head, | ||
| _Elf.machineOffset, | ||
| littleEndian: elfData == _Elf.littleEndian, | ||
| ); | ||
| final architecture = switch (machine) { | ||
| _Elf.machineX86 => Architecture.ia32, | ||
| _Elf.machineArm => Architecture.arm, | ||
| _Elf.machineX64 => Architecture.x64, | ||
| _Elf.machineArm64 => Architecture.arm64, | ||
| _Elf.machineRiscV => | ||
| elfClass == _Elf.class32 ? Architecture.riscv32 : Architecture.riscv64, | ||
| _ => null, | ||
| }; | ||
| return RecognizedHeader(BinaryFormat.elf, [architecture], [machine]); | ||
| } | ||
|
|
||
| Architecture? _machOArchitecture(int cpuType) => switch (cpuType) { | ||
| _MachO.cpuTypeX86 => Architecture.ia32, | ||
| _MachO.cpuTypeX64 => Architecture.x64, | ||
| _MachO.cpuTypeArm => Architecture.arm, | ||
| // arm64e uses CPU_TYPE_ARM64 and identifies the ABI in cpusubtype. Until | ||
| // TODO(https://github.com/dart-lang/native/issues/3379) reads that subtype, | ||
| // treat arm64e as arm64 rather than claiming to validate its ABI. | ||
| _MachO.cpuTypeArm64 => Architecture.arm64, | ||
| _ => null, | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. please leave a comment what you'd expect for arm64e with a TODO referring to:
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Will do. I'll add a TODO in the Mach-O parser referencing #3379: arm64e uses the same
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Added the arm64e comment with a |
||
| }; | ||
|
|
||
| NativeLibraryHeader? _readThinMachO(Uint8List head) { | ||
| final littleEndianMagic = _u32(head, 0, littleEndian: true); | ||
| final bool littleEndian; | ||
| if (littleEndianMagic == _MachO.magic32 || | ||
| littleEndianMagic == _MachO.magic64) { | ||
| littleEndian = true; | ||
| } else if (littleEndianMagic == _MachO.swappedMagic32 || | ||
| littleEndianMagic == _MachO.swappedMagic64) { | ||
| littleEndian = false; | ||
| } else { | ||
| return null; | ||
| } | ||
| final cpuType = _u32(head, 4, littleEndian: littleEndian); | ||
| return RecognizedHeader( | ||
| BinaryFormat.machO, | ||
| [_machOArchitecture(cpuType)], | ||
| [cpuType], | ||
| ); | ||
| } | ||
|
|
||
| NativeLibraryHeader _readFatMachO( | ||
| RandomAccessFile raf, | ||
| Uint8List head, { | ||
| required bool is64, | ||
| }) { | ||
| // The fat header and arch entries are always big-endian. The number of | ||
| // slices is tiny in practice. Java class files share the 0xcafebabe magic, | ||
| // but their version number generally makes the implied architecture table | ||
| // larger than the file. | ||
| final sliceCount = _u32(head, 4, littleEndian: false); | ||
| if (sliceCount == 0) return const UnrecognizedHeader(); | ||
| final entrySize = is64 ? 32 : 20; | ||
| final entriesLength = sliceCount * entrySize; | ||
| if (raf.lengthSync() < 8 + entriesLength) { | ||
| return const UnrecognizedHeader(); | ||
| } | ||
| if (sliceCount > 32) { | ||
| return RecognizedHeader( | ||
| BinaryFormat.machO, | ||
| const [], | ||
| const [], | ||
| architectureCount: sliceCount, | ||
| ); | ||
| } | ||
| raf.setPositionSync(8); | ||
| final entries = raf.readSync(entriesLength); | ||
| if (entries.length < entriesLength) { | ||
| return const UnrecognizedHeader(); | ||
| } | ||
| final machineValues = [ | ||
| for (var slice = 0; slice < sliceCount; slice++) | ||
| _u32(entries, slice * entrySize, littleEndian: false), | ||
| ]; | ||
| return RecognizedHeader(BinaryFormat.machO, [ | ||
| for (final cpuType in machineValues) _machOArchitecture(cpuType), | ||
| ], machineValues); | ||
| } | ||
|
|
||
| NativeLibraryHeader _readPe(RandomAccessFile raf, Uint8List head, int length) { | ||
| final peOffset = _u32(head, _Pe.peHeaderOffset, littleEndian: true); | ||
| if (peOffset + 6 > length) return const UnrecognizedHeader(); | ||
| raf.setPositionSync(peOffset); | ||
| final signature = raf.readSync(6); | ||
| if (signature.length < 6 || | ||
| signature[0] != _Pe.signature[0] || | ||
| signature[1] != _Pe.signature[1] || | ||
| signature[2] != _Pe.signature[2] || | ||
| signature[3] != _Pe.signature[3]) { | ||
| return const UnrecognizedHeader(); | ||
| } | ||
| final machine = _u16(signature, 4, littleEndian: true); | ||
| final architecture = switch (machine) { | ||
| _Pe.machineX86 => Architecture.ia32, | ||
| _Pe.machineArm => Architecture.arm, | ||
| _Pe.machineRiscV64 => Architecture.riscv64, | ||
| _Pe.machineX64 => Architecture.x64, | ||
| _Pe.machineArm64 => Architecture.arm64, | ||
| _ => null, | ||
| }; | ||
| return RecognizedHeader(BinaryFormat.pe, [architecture], [machine]); | ||
| } | ||
|
|
||
| int _u16(Uint8List bytes, int offset, {required bool littleEndian}) => | ||
| littleEndian | ||
| ? bytes[offset] | (bytes[offset + 1] << 8) | ||
| : (bytes[offset] << 8) | bytes[offset + 1]; | ||
|
|
||
| int _u32(Uint8List bytes, int offset, {required bool littleEndian}) => | ||
| littleEndian | ||
| ? bytes[offset] | | ||
| (bytes[offset + 1] << 8) | | ||
| (bytes[offset + 2] << 16) | | ||
| (bytes[offset + 3] << 24) | ||
| : (bytes[offset] << 24) | | ||
| (bytes[offset + 1] << 16) | | ||
| (bytes[offset + 2] << 8) | | ||
| bytes[offset + 3]; | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd rather have the logger from hooks_runner be used instead of a logger here in the constructor, but adding a logger argument to the
validateXxxmethods would be a breaking change.See https://github.com/flutter/flutter/blob/e69e703dce29fda392f8024109c1961306cb365a/packages/flutter_tools/lib/src/isolated/native_assets/targets.dart#L237. We don't want to have to construct the same logger there as we did in https://github.com/flutter/flutter/blob/e69e703dce29fda392f8024109c1961306cb365a/packages/flutter_tools/lib/src/isolated/native_assets/native_assets.dart#L561
Maybe we can add the logger to the
ValidationContext?WDYT?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hm wait the code assets extension creates the validation context, so it also doesn't have access to the logger.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed it's breaking even as an optional named param, since an external
ProtocolExtensionsubclass that doesn't also declare it would failinvalid_override, so I won't touch those signatures. The additive way to yourValidationContextidea: add a mutableLogger? loggertoProtocolExtension, haveNativeAssetsBuildRunnersete.loggerbefore the override call in its existingfor (final e in extensions)loop, andCodeAssetExtensionforwards it into the context it builds. That covers your follow-up: the extension still creates the context, but now has the runner's logger to put in it, so Flutter never builds a second one. I'll leave the existing constructor logger as a fallback so the change stays additive.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Correction on the mechanism I proposed. Keeping
CodeAssetExtension's existingfinal Logger? loggeralongside a newloggeronProtocolExtensiondoes not work: the subclass field shadows the base getter, so ane.loggerassignment from the runner lands on the base field while every read dispatches to the subclass getter, a silent no-op (confirmed with a quick repro). The workable shape is a single storage: putLogger? logger;onProtocolExtension, drop the field fromCodeAssetExtension, and setthis.loggerin its constructor body so existing callers still pass it. If you would rather the runner not mutate a caller-owned field, avoid setupValidationLogger(Logger logger) {}default hook does the same job.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Adding a method to
ProtocolExtensionlikesetupValidationLoggeris then not breaking because we made it a base class.Then we could also consider something along the lines of:
We might get passed the same protocol extension object in multiple
build/linkmethods in one run. So we do end up setting the logger more than once potentially. So the field can't be final.I think technically this is still a breaking change, because
package:hooks_runnermight resolve to an older version whilepackage:code_assetsmight resolve to a newer version, causing thesetupLoggernever to be called. However, I don't think this will happen in Dart or Flutter because all packages are rolled at the same time.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added
setupLoggeronProtocolExtension, a non-finallatefield with aloggergetter, since the same extension object can be reused across a run. The hooks_runner calls it before any other method in bothbuildandlink.CodeAssetExtension's ownloggerfield is removed so nothing shadows the base getter.