Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
75 commits
Select commit Hold shift + click to select a range
760fe11
Add Foundation Models 27 prompt attachments
qoli Oct 9, 2026
f2de319
Add Foundation Models 27 request options
qoli Oct 9, 2026
379ee3c
Fix attachment prompt replay
qoli Oct 11, 2026
2d1a32b
Preserve attachments in tool outputs
qoli Oct 11, 2026
cd1d6fc
Add session properties to dynamic instructions
qoli Oct 11, 2026
9481380
Add session properties to dynamic instructions
qoli Oct 11, 2026
a03fd6b
Add Foundation Models 27 dynamic profiles
qoli Oct 11, 2026
8cc3a74
Preserve dynamic profiles across tool continuations
qoli Oct 11, 2026
dc79bbf
Document attachments in tool outputs
qoli Oct 11, 2026
b9738ab
Document dynamic profiles
qoli Oct 11, 2026
f96d6de
Generate fresh prompt segment identifiers
qoli Oct 12, 2026
c7aaa57
Preserve Gemini tool output segments
qoli Oct 12, 2026
d70bf12
Generate fresh prompt segment identifiers
qoli Oct 12, 2026
ddfa8f1
Preserve Gemini tool output segments
qoli Oct 12, 2026
da63029
Keep session property requests observable
qoli Oct 12, 2026
8b3a234
Keep session property requests observable
qoli Oct 12, 2026
9525809
Keep session property requests observable
qoli Oct 12, 2026
5fdf90e
Keep session property requests observable
qoli Oct 12, 2026
d9de68a
Avoid relowering recorded prompt attachments
qoli Oct 13, 2026
9ab55ea
Clarify supported attachment platforms
qoli Oct 13, 2026
a03fe05
Avoid relowering recorded prompt attachments
qoli Oct 13, 2026
51fa0a9
Clarify supported attachment platforms
qoli Oct 13, 2026
d0cae54
Isolate dynamic profile request state
qoli Oct 13, 2026
1ac8207
Isolate dynamic profile request state
qoli Oct 13, 2026
c350013
Fix dynamic profile continuation boundaries
qoli Oct 13, 2026
4879808
Isolate dynamic profile request state
qoli Oct 13, 2026
51c5bcc
Fix dynamic profile continuation boundaries
qoli Oct 13, 2026
f57be8c
Merge upstream main into tool output attachments
qoli Oct 14, 2026
810861f
Merge updated attachment support into documentation
qoli Oct 14, 2026
d1c8413
Merge remote-tracking branch 'upstream/main' into codex/upstream-dyna…
qoli Oct 14, 2026
49f681f
Merge updated dynamic profiles into tool continuations
qoli Oct 14, 2026
375ebd7
Merge branch 'codex/upstream-profile-tool-continuations' into codex/u…
qoli Oct 14, 2026
1892a39
Inline Gemini tool output file attachments
qoli Oct 15, 2026
8c9e8d9
Merge branch 'codex/upstream-tool-output-attachments' into codex/upst…
qoli Oct 15, 2026
93e4d4d
Fix session property observation and typed defaults
qoli Oct 15, 2026
e99341e
Merge branch 'codex/upstream-session-properties' into codex/upstream-…
qoli Oct 15, 2026
cdeed51
Fix dynamic profile callback concurrency
qoli Oct 15, 2026
8e218df
Merge reviewed profile fixes into tool continuations
qoli Oct 15, 2026
9a27552
Merge branch 'codex/upstream-profile-tool-continuations' into codex/u…
qoli Oct 15, 2026
2f435bd
Format Gemini attachment MIME mapping
qoli Oct 16, 2026
446f111
Merge branch 'codex/upstream-tool-output-attachments' into codex/upst…
qoli Oct 16, 2026
d211109
Allow sampling mode without tool calling policy
qoli Oct 16, 2026
3ad28fb
Keep session history bindings isolated and instruction free
qoli Oct 16, 2026
1fde50e
Merge branch 'codex/upstream-request-options' into codex/upstream-dyn…
qoli Oct 16, 2026
05f892a
Merge branch 'codex/upstream-session-properties' into codex/upstream-…
qoli Oct 16, 2026
9b76d74
Preserve profile history and request callback identity
qoli Oct 16, 2026
ea464ed
Preserve pending profile history across model handoffs
qoli Oct 16, 2026
af1fcfe
Merge branch 'codex/upstream-profile-tool-continuations' into codex/u…
qoli Oct 16, 2026
4d9cd91
Inline local Anthropic image attachments during transport
qoli Oct 17, 2026
6354a86
Merge branch 'codex/upstream-tool-output-attachments' into codex/upst…
qoli Oct 17, 2026
2c5c488
Invalidate stale history commits and evaluate property defaults outsi…
qoli Oct 17, 2026
905194e
Merge branch 'codex/upstream-session-properties' into codex/upstream-…
qoli Oct 17, 2026
d9e5f31
Preserve profile lifecycle identity and evolving callback history
qoli Oct 17, 2026
ea7de61
Merge branch 'codex/upstream-dynamic-profiles' into codex/upstream-pr…
qoli Oct 17, 2026
ef80f14
Carry callback history edits and original schema options through tool…
qoli Oct 17, 2026
20d4a04
Merge branch 'codex/upstream-profile-tool-continuations' into codex/u…
qoli Oct 17, 2026
2eb4abd
Merge remote-tracking branch 'upstream/main' into codex/upstream-tool…
qoli Oct 17, 2026
86d6952
Merge branch 'codex/upstream-attachment-docs' into codex/upstream-too…
qoli Oct 17, 2026
5a17ad3
Snapshot local tool image data before persisting output
qoli Oct 17, 2026
790ce15
Merge remote-tracking branch 'upstream/main' into codex/upstream-sess…
qoli Oct 17, 2026
0a3580b
Match HistoryView collection shape and bind session properties in con…
qoli Oct 17, 2026
4cf0f06
Reshape dynamic profiles as composition-only API on current session p…
qoli Oct 17, 2026
c40cf05
Separate shared profile execution SPI from provider Tool loops
qoli Oct 17, 2026
fb42cd2
Integrate OpenAI profile Tool continuations
qoli Oct 17, 2026
455a415
Integrate OpenResponses profile Tool continuations
qoli Oct 17, 2026
572fe1e
Integrate Anthropic profile Tool continuations without changing ordin…
qoli Oct 17, 2026
020cab2
Integrate Gemini profile Tool continuations
qoli Oct 17, 2026
8528755
Integrate Ollama profile Tool continuations
qoli Oct 17, 2026
40c3d22
Isolate native profile history projection
qoli Oct 17, 2026
c048ce9
Merge branch 'codex/upstream-profile-tool-continuations' into codex/u…
qoli Oct 17, 2026
cfbaf5a
Merge branches 'codex/upstream-profile-openai', 'codex/upstream-profi…
qoli Oct 17, 2026
cc2263b
Integrate attachment snapshots with the split profile stack
qoli Oct 17, 2026
ace3039
Synchronize Anthropic profile integration with attachment snapshots
qoli Oct 17, 2026
fe28fb8
Synchronize Gemini profile integration with attachment snapshots
qoli Oct 17, 2026
bc88e15
Merge branches 'codex/upstream-profile-anthropic' and 'codex/upstream…
qoli Oct 17, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
123 changes: 113 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -384,19 +384,22 @@ pass the model's multimodal projector with `mmprojPath:`.

### Prompt attachments

Image attachments use the Foundation Models 27 prompt construction syntax:
Image attachments use the Foundation Models 27 prompt construction syntax,
including ordinary Tool outputs:

```swift
let prompt = Prompt {
"Describe this image"
Attachment(imageURL: fileURL)
func call(arguments: Arguments) async throws -> Prompt {
Prompt {
"Image read from the selected file"
Attachment(imageURL: fileURL)
}
}
```

On Apple platforms, `Attachment(CGImage, orientation:)` retains typed image
content and orientation. Prompt builders preserve the order of text and images,
and the OS 26 compatibility transcript continues to store images using its
existing `Transcript.Segment.image` Codable representation.
content and orientation. Prompts, builders, and Tool outputs preserve the order
of text and images, and the OS 26 compatibility transcript continues to store
images using its existing `Transcript.Segment.image` Codable representation.

`SystemLanguageModel` and `FoundationLanguageModel` bridge prompt attachments
to Foundation Models only in apps built with Swift 6.4 or later
Expand All @@ -406,9 +409,14 @@ they throw an error when a prompt has an attachment.
The system bridge requires local file URLs; other provider adapters retain URL
images in their existing transcript representation.

This first compatibility surface covers prompt `CGImage` and `imageURL`
attachments. Tool outputs, instruction attachments, labels, `CIImage`, pixel
buffers, and newer data attachments remain outside this change.
Ordinary Tool outputs read local image files once when the output is created,
and retain the image bytes and MIME type in the compatibility transcript.
Continuations and saved transcript replay therefore do not depend on the original
file remaining at its URL. Remote image URLs are retained without downloading.

This compatibility surface covers `CGImage` and `imageURL` attachments in
prompts and ordinary Tool outputs. Instruction attachments, labels, `CIImage`,
pixel buffers, and newer data attachments remain outside this change.

### Tool Calling

Expand Down Expand Up @@ -515,6 +523,86 @@ but never become part of the session's transcript.
> Otherwise, it throws `SystemLanguageModel.Error.dynamicInstructionsUnavailable`
> for a session with dynamic instructions.

### Foundation Models 27 request options

`ContextOptions` mirrors Foundation Models 27's schema-in-prompt and reasoning-level
settings for compatibility-layer request construction. `GenerationOptions` also
provides the OS 27 `samplingMode` spelling and `ToolCallingMode` values:

```swift
let context = ContextOptions(
includeSchemaInPrompt: true,
reasoningLevel: .moderate
)
let options = GenerationOptions(
samplingMode: .greedy,
toolCallingMode: .required
)
```

The older `sampling` property and initializer remain source-compatible.
Tool-calling mode is preserved in Codable transcripts and forwarded by the
Foundation Models bridge on OS 27. Other model providers can read the same
option from their request's `GenerationOptions`.

### Dynamic Profiles

`LanguageModelSession.DynamicProfile` mirrors the Foundation Models 27 profile
API on supported AnyLanguageModel deployment targets. A profile chooses one
active `Profile` for each actual model request and can change its model,
instructions, tools, generation options, context options, and transcript
projection without rebuilding the session:

```swift
extension SessionPropertyValues {
@SessionPropertyEntry var isReviewing = false
}

struct EditingProfile: LanguageModelSession.DynamicProfile {
let model: any LanguageModel
@SessionProperty(\.isReviewing) private var isReviewing

var body: some LanguageModelSession.DynamicProfile {
if isReviewing {
Profile {
Instructions("Review the current draft.")
ReviewTool()
}
.model(model)
.temperature(0.1)
} else {
Profile {
Instructions("Help edit the current draft.")
EditTool()
}
.model(model)
.temperature(0.7)
}
}
}

let session = LanguageModelSession(
profile: EditingProfile(model: model),
history: savedHistory
)
session.properties.isReviewing = true
```

Value modifiers use call-site, inner-profile, then outer-profile precedence.
Lifecycle callbacks accumulate. `historyTransform` changes only the transcript
projection for that request; `session.transcript` remains the canonical history.
`@SessionProperty` exposes isolated session state to profiles, dynamic
instructions, and tools. The built-in `history` property is writable from a
profile or lifecycle callback and read-only while dynamic instructions or a Tool
is active.

External compatibility models that own Tool continuation use
`resolvedRequestContext(including:options:)` before every provider request. The
`including:` entries are the canonical Tool calls and outputs completed in the
current response but not yet committed to the session transcript. Calls execute
against the tools in the producing request context; the next resolved context is
used only for the continuation request.

### Reasoning in the transcript

Reasoning is transcript content, separate from the answer in `response.content`.
Expand Down Expand Up @@ -641,6 +729,10 @@ say which API they follow.
[image inputs](#image-inputs).
Foundation Models 27-style [prompt attachments](#prompt-attachments) are also
available for ordered text and image prompt construction.
- `Tool.makeOutputSegments(from:)`, exposed only through the `Compatibility` SPI:
an integration extension for back-deployment provider adapters to lower normal
Tool outputs into the compatibility transcript. It is not an Apple Tool API;
OS 27 canonical executors use Foundation Models' own Tool execution instead.
- `ToolExecutionDelegate`, `ToolExecutionDecision`, and `toolExecutionDelegate`:
[observing and controlling tool calls](#tool-calling).
- `transcriptErrorHandlingPolicy` and `waitForResponseCompletion()`:
Expand All @@ -654,6 +746,17 @@ say which API they follow.
- `DynamicInstructions`, its builder, and `LanguageModelSession.init(model:dynamicInstructions:history:)`:
[dynamic instructions](#dynamic-instructions),
which follow the Foundation Models 27 API.
- `ContextOptions`, `GenerationOptions.samplingMode`, and
`GenerationOptions.ToolCallingMode`:
[request options](#foundation-models-27-request-options), which follow the
Foundation Models 27 API while remaining available on the package's existing
deployment targets.
- `LanguageModelSession.DynamicProfile`, profile modifiers, lifecycle callbacks,
session properties, and `LanguageModelSession.init(profile:history:)`:
[dynamic profiles](#dynamic-profiles), which mirror Foundation Models 27 on
earlier deployment targets. `resolvedRequestContext(including:options:)` and
the explicit compatibility lifecycle hooks are AnyLanguageModel extensions for
external model implementations until the package adopts the executor contract.
- `Transcript.Entry.reasoning` and `Transcript.Reasoning`:
[reasoning in the transcript](#reasoning-in-the-transcript),
which follows the Foundation Models 27 API.
Expand Down
27 changes: 27 additions & 0 deletions Sources/AnyLanguageModel/ContextOptions.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
/// Options that configure a model's request context.
///
/// This mirrors the Foundation Models 27 request-context surface while keeping
/// the package available on its existing deployment targets.
public struct ContextOptions: Sendable, Equatable {
/// The amount of reasoning the model should perform for a request.
public enum ReasoningLevel: Sendable, Equatable {
case light
case moderate
case deep
case custom(String)
}

/// Whether a structured-generation schema should also appear in the prompt.
public var includeSchemaInPrompt: Bool?

/// The amount of reasoning the model should perform.
public var reasoningLevel: ReasoningLevel?

public init(
includeSchemaInPrompt: Bool? = nil,
reasoningLevel: ReasoningLevel? = nil
) {
self.includeSchemaInPrompt = includeSchemaInPrompt
self.reasoningLevel = reasoningLevel
}
}
Loading
Loading