Keep recording metadata truthful and repair legacy Library rows

docs/TRUTHFUL_RECORDING_METADATA_V2.md

@@ -0,0 +1,64 @@
+# Truthful recording metadata v2
+
+Status: production migration note.
+
+## Product invariant
+
+`SuperDictateRecording.durationSeconds` means source-audio duration only.
+
+ASR/transcription processing time is operational performance metadata and must never be presented as recording length.
+
+Likewise, `createdAt` is source capture chronology. If an older history row does not know when audio was captured, the product keeps it `nil` rather than substituting migration time, file time, or the current date.
+
+## Legacy behavior
+
+Pre-metadata history rows know:
+
+- transcript text;
+- optional ASR processing duration.
+
+They do not know:
+
+- source recording UUID;
+- capture timestamp;
+- source-audio duration.
+
+Their durable Library projection therefore uses:
+
+- deterministic text + duplicate-occurrence fallback UUID;
+- `createdAt = nil`;
+- `durationSeconds = nil`.
+
+## Metadata-rich runtime rows
+
+The migration contract also accepts optional real runtime metadata:
+
+- `recordingID`;
+- `createdAt`;
+- `sourceAudioDurationSeconds`.
+
+When present, these values are preserved. `transcriptionDurationSeconds` stays separate and is never copied into audio duration.
+
+Explicit runtime UUID rows do not advance the duplicate counter used by fallback legacy IDs. Adding a new metadata-rich row therefore cannot silently renumber existing legacy identities.
+
+## Repair of already-written Library rows
+
+Earlier product builds could persist ASR processing duration in the Library recording-duration field.
+
+The existing agent-owned startup merge performs a narrow repair only when a durable row:
+
+1. has an ID that exactly matches the deterministic legacy text/occurrence identity;
+2. has no capture date;
+3. has a non-nil duration.
+
+Only that duration is reset to unknown. Real runtime UUID rows are excluded. Legacy fallback rows enriched with a real capture date are excluded.
+
+The repair is counted as a merge change so the existing single writer atomically persists the corrected archive even when no new history rows are added.
+
+## Release gate
+
+This slice changes Core migration/repair semantics only. It does not change audio capture, ASR, hotkeys, insertion, TCC or visible UI runtime behavior. Even so, it must pass the same complete pull-request gate as runtime changes: repository checks, unchanged Parakey self-tests, all Core XCTest, release `.app` build, strict codesign verification and real install/uninstall smoke.
+
+## Follow-up
+
+The next runtime slice gives every new successful in-session dictation a real UUID and capture timestamp and carries the actual captured audio duration through history, live projection and the single-writer Library path. Pending crash-recovery audio remains backward-compatible until the versioned journal-header migration lands.

swift/Sources/SuperDictateCore/LegacyHistoryMigration.swift

@@ -1,57 +1,74 @@
 import CryptoKit
 import Foundation
 
+/// Runtime-history projection accepted by the durable Library migrator.
+///
+/// Old history rows only know transcript text + ASR processing duration. Newer
+/// runtime rows may additionally provide a real recording UUID, capture time and
+/// source-audio duration. ASR duration is deliberately kept separate from audio
+/// duration so the product never presents processing time as recording length.
 public struct SuperDictateLegacyHistoryEntry: Equatable, Sendable {
     public var text: String
     public var transcriptionDurationSeconds: Double?
+    public var recordingID: UUID?
+    public var createdAt: Date?
+    public var sourceAudioDurationSeconds: Double?
 
     public init(
         text: String,
-        transcriptionDurationSeconds: Double? = nil
+        transcriptionDurationSeconds: Double? = nil,
+        recordingID: UUID? = nil,
+        createdAt: Date? = nil,
+        sourceAudioDurationSeconds: Double? = nil
     ) {
         self.text = text
-        if let duration = transcriptionDurationSeconds,
-           duration.isFinite,
-           duration >= 0 {
-            self.transcriptionDurationSeconds = duration
-        } else {
-            self.transcriptionDurationSeconds = nil
-        }
+        self.transcriptionDurationSeconds = Self.validDuration(transcriptionDurationSeconds)
+        self.recordingID = recordingID
+        self.createdAt = createdAt
+        self.sourceAudioDurationSeconds = Self.validDuration(sourceAudioDurationSeconds)
+    }
+
+    private static func validDuration(_ value: Double?) -> Double? {
+        guard let value, value.isFinite, value >= 0 else { return nil }
+        return value
     }
 }
 
-/// One-way projection from the pre-Library transcript archive into stable
-/// product identities.
-///
-/// The UUID algorithm intentionally matches the already-shipped temporary
-/// identity in `ProductRuntimeBridge.swift`: normalized transcript text plus
-/// duplicate occurrence ordinal. This prevents a legacy row from changing ID
-/// when it moves from the live bridge snapshot into the persisted Library.
+/// One-way projection from runtime transcript history into durable product rows.
 ///
-/// Legacy capture chronology does not exist. `createdAt` therefore stays nil.
+/// Legacy rows without an explicit recording identity retain the historical
+/// deterministic text+occurrence UUID. Rows with a real runtime UUID preserve it.
+/// Explicit-ID rows do not advance the legacy duplicate occurrence counter, so
+/// adding newer metadata-rich rows cannot silently renumber old legacy IDs.
 public enum SuperDictateLegacyHistoryMigrator {
     public static func recordings(
         from entries: [SuperDictateLegacyHistoryEntry]
     ) -> [SuperDictateRecording] {
-        var occurrences: [String: Int] = [:]
+        var legacyOccurrences: [String: Int] = [:]
         var result: [SuperDictateRecording] = []
         result.reserveCapacity(entries.count)
 
         for entry in entries {
             let text = entry.text.trimmingCharacters(in: .whitespacesAndNewlines)
             guard !text.isEmpty else { continue }
 
-            let occurrence = occurrences[text, default: 0]
-            occurrences[text] = occurrence + 1
+            let occurrence = legacyOccurrences[text, default: 0]
+            let id: UUID
+            if let recordingID = entry.recordingID {
+                id = recordingID
+            } else {
+                id = stableRecordingID(text: text, occurrence: occurrence)
+                legacyOccurrences[text] = occurrence + 1
+            }
 
             result.append(
                 SuperDictateRecording(
-                    id: stableRecordingID(text: text, occurrence: occurrence),
+                    id: id,
                     title: suggestedTitle(from: text),
                     transcript: text,
                     summary: nil,
-                    createdAt: nil,
-                    durationSeconds: entry.transcriptionDurationSeconds,
+                    createdAt: entry.createdAt,
+                    durationSeconds: entry.sourceAudioDurationSeconds,
                     people: [],
                     requiresAttention: false
                 )
@@ -72,8 +89,7 @@ public enum SuperDictateLegacyHistoryMigrator {
         )
     }
 
-    /// Public so the runtime bridge can eventually delegate to the same source
-    /// of truth and delete its temporary duplicate implementation.
+    /// Stable fallback identity for pre-metadata history rows only.
     public static func stableRecordingID(
         text: String,
         occurrence: Int

swift/Sources/SuperDictateCore/LegacyLibraryMerge.swift

@@ -4,31 +4,38 @@ public struct SuperDictateLegacyLibraryMergeResult: Equatable, Sendable {
     public var archive: SuperDictateLibraryArchive
     public var addedRecordingCount: Int
     public var addedDocumentCount: Int
+    public var repairedRecordingMetadataCount: Int
 
     public init(
         archive: SuperDictateLibraryArchive,
         addedRecordingCount: Int,
-        addedDocumentCount: Int
+        addedDocumentCount: Int,
+        repairedRecordingMetadataCount: Int = 0
     ) {
         self.archive = archive
         self.addedRecordingCount = addedRecordingCount
         self.addedDocumentCount = addedDocumentCount
+        self.repairedRecordingMetadataCount = repairedRecordingMetadataCount
     }
 
     public var changed: Bool {
-        addedRecordingCount > 0 || addedDocumentCount > 0
+        addedRecordingCount > 0
+            || addedDocumentCount > 0
+            || repairedRecordingMetadataCount > 0
     }
 }
 
-/// Migration-only merge from the bounded legacy transcript history into the
-/// durable Library. Existing durable objects always win.
+/// Migration-only merge from runtime transcript history into the durable
+/// Library. Existing durable objects always win, except for one narrowly
+/// defined repair of metadata written by the pre-v2 legacy migrator.
 public enum SuperDictateLegacyLibraryMerger {
     public static func merge(
         _ entries: [SuperDictateLegacyHistoryEntry],
         into archive: SuperDictateLibraryArchive
     ) -> SuperDictateLegacyLibraryMergeResult {
+        let repair = repairLegacyDurationPollution(in: archive)
         let migrated = SuperDictateLegacyHistoryMigrator.archive(from: entries)
-        var next = archive
+        var next = repair.archive
         var recordingIDs = Set(next.recordings.map(\.id))
         var documentIDs = Set(next.memoryDocuments.map(\.recordingID))
         var addedRecordings = 0
@@ -49,7 +56,50 @@ public enum SuperDictateLegacyLibraryMerger {
         return SuperDictateLegacyLibraryMergeResult(
             archive: next,
             addedRecordingCount: addedRecordings,
-            addedDocumentCount: addedDocuments
+            addedDocumentCount: addedDocuments,
+            repairedRecordingMetadataCount: repair.repairedCount
         )
     }
+
+    /// Releases written before truthful metadata v2 could store ASR processing
+    /// time in `SuperDictateRecording.durationSeconds`. We can identify those
+    /// rows without guessing because they use the deterministic legacy
+    /// text+occurrence UUID and have no source capture timestamp.
+    ///
+    /// Explicit runtime UUID rows never advance the fallback occurrence counter,
+    /// matching `SuperDictateLegacyHistoryMigrator` semantics.
+    public static func repairLegacyDurationPollution(
+        in archive: SuperDictateLibraryArchive
+    ) -> (archive: SuperDictateLibraryArchive, repairedCount: Int) {
+        var next = archive
+        var legacyOccurrences: [String: Int] = [:]
+        var repairedCount = 0
+
+        for index in next.recordings.indices {
+            let recording = next.recordings[index]
+            let text = recording.transcript.trimmingCharacters(in: .whitespacesAndNewlines)
+            guard !text.isEmpty else { continue }
+
+            let occurrence = legacyOccurrences[text, default: 0]
+            let expectedLegacyID = SuperDictateLegacyHistoryMigrator.stableRecordingID(
+                text: text,
+                occurrence: occurrence
+            )
+            guard recording.id == expectedLegacyID else {
+                // Real runtime UUID rows do not consume a legacy occurrence.
+                continue
+            }
+
+            legacyOccurrences[text] = occurrence + 1
+            guard recording.createdAt == nil,
+                  recording.durationSeconds != nil else {
+                continue
+            }
+
+            next.recordings[index].durationSeconds = nil
+            repairedCount += 1
+        }
+
+        return (next, repairedCount)
+    }
 }

swift/Tests/SuperDictateCoreTests/LegacyHistoryMigrationTests.swift

@@ -29,7 +29,7 @@ final class LegacyHistoryMigrationTests: XCTestCase {
         )
     }
 
-    func testDuplicateRowsReceiveDistinctStableIdentities() {
+    func testDuplicateLegacyRowsReceiveDistinctStableIdentities() {
         let entries = [
             SuperDictateLegacyHistoryEntry(text: "Same words"),
             SuperDictateLegacyHistoryEntry(text: "Same words"),
@@ -48,36 +48,81 @@ final class LegacyHistoryMigrationTests: XCTestCase {
         )
     }
 
-    func testUnknownChronologyStaysUnknownWhileKnownTimingSurvives() throws {
+    func testLegacyASRDurationIsNotPresentedAsRecordingDuration() throws {
         let migrated = try XCTUnwrap(
             SuperDictateLegacyHistoryMigrator.recordings(
                 from: [
                     SuperDictateLegacyHistoryEntry(
-                        text: "Timed transcript",
+                        text: "Timed ASR transcript",
                         transcriptionDurationSeconds: 2.75
                     ),
                 ]
             ).first
         )
 
         XCTAssertNil(migrated.createdAt)
-        XCTAssertEqual(migrated.durationSeconds, 2.75)
+        XCTAssertNil(migrated.durationSeconds)
     }
 
-    func testInvalidDurationsNormalizeToUnknown() {
-        let migrated = SuperDictateLegacyHistoryMigrator.recordings(
+    func testRealRuntimeMetadataSurvivesProjection() throws {
+        let id = UUID()
+        let createdAt = Date(timeIntervalSince1970: 1_700_000_000)
+        let migrated = try XCTUnwrap(
+            SuperDictateLegacyHistoryMigrator.recordings(
+                from: [
+                    SuperDictateLegacyHistoryEntry(
+                        text: "Metadata rich transcript",
+                        transcriptionDurationSeconds: 0.8,
+                        recordingID: id,
+                        createdAt: createdAt,
+                        sourceAudioDurationSeconds: 12.5
+                    ),
+                ]
+            ).first
+        )
+
+        XCTAssertEqual(migrated.id, id)
+        XCTAssertEqual(migrated.createdAt, createdAt)
+        XCTAssertEqual(migrated.durationSeconds, 12.5)
+    }
+
+    func testExplicitIDRowsDoNotRenumberLegacyDuplicateFallbackIDs() {
+        let legacyOnly = SuperDictateLegacyHistoryMigrator.recordings(
             from: [
-                SuperDictateLegacyHistoryEntry(
-                    text: "Negative",
-                    transcriptionDurationSeconds: -1
-                ),
-                SuperDictateLegacyHistoryEntry(
-                    text: "Infinite",
-                    transcriptionDurationSeconds: .infinity
-                ),
+                SuperDictateLegacyHistoryEntry(text: "Same words"),
+                SuperDictateLegacyHistoryEntry(text: "Same words"),
             ]
         )
+        let mixed = SuperDictateLegacyHistoryMigrator.recordings(
+            from: [
+                SuperDictateLegacyHistoryEntry(text: "Same words", recordingID: UUID()),
+                SuperDictateLegacyHistoryEntry(text: "Same words"),
+                SuperDictateLegacyHistoryEntry(text: "Same words"),
+            ]
+        )
+
+        XCTAssertEqual(Array(mixed.dropFirst()).map(\.id), legacyOnly.map(\.id))
+    }
+
+    func testInvalidDurationsNormalizeToUnknown() {
+        let entries = [
+            SuperDictateLegacyHistoryEntry(
+                text: "Negative",
+                transcriptionDurationSeconds: -1,
+                sourceAudioDurationSeconds: -2
+            ),
+            SuperDictateLegacyHistoryEntry(
+                text: "Infinite",
+                transcriptionDurationSeconds: .infinity,
+                sourceAudioDurationSeconds: .infinity
+            ),
+        ]
+        XCTAssertNil(entries[0].transcriptionDurationSeconds)
+        XCTAssertNil(entries[0].sourceAudioDurationSeconds)
+        XCTAssertNil(entries[1].transcriptionDurationSeconds)
+        XCTAssertNil(entries[1].sourceAudioDurationSeconds)
 
+        let migrated = SuperDictateLegacyHistoryMigrator.recordings(from: entries)
         XCTAssertNil(migrated[0].durationSeconds)
         XCTAssertNil(migrated[1].durationSeconds)
     }