diff --git a/docs/user/global-host-profiles.md b/docs/user/global-host-profiles.md index 949d7e8bd..64a5b44ec 100644 --- a/docs/user/global-host-profiles.md +++ b/docs/user/global-host-profiles.md @@ -68,7 +68,7 @@ The profile schema is the existing repository check/test environment surface: | `FKST_DEVLOOP_INTEGRATION_BRANCH` | `github-devloop` | Per-device integration branch. | | `FKST_DEVLOOP_INTAKE_MILESTONE_NUMBERS` | optional | Comma-separated GitHub milestone numbers eligible for an initial issue claim. | | `FKST_DEVLOOP_LOCAL_TEST_COMMAND` | `github-devloop` | Repository-root local verification gate run by implement/fix workers before handoff. | -| `FKST_DEVLOOP_CACHE_PREPARATION_COMMAND` | optional | Trusted-base cache preparation run for each implementation worktree before Codex starts. | +| `FKST_DEVLOOP_CACHE_PREPARATION_COMMAND` | optional | Trusted-base cache preparation run before Codex and before each candidate or detached-base local verification. | Those five keys together are the whole trusted-author allowlist: `FKST_GITHUB_BOT_LOGIN` (a required anchor) ∪ `FKST_DEVLOOP_MANAGED_BOT_LOGINS` ∪ `FKST_GITHUB_AUTHORIZED_LOGINS`, @@ -124,17 +124,32 @@ command shape is not safely preflightable; it does not execute the test suite du ## Implementation cache preparation `FKST_DEVLOOP_CACHE_PREPARATION_COMMAND` optionally names a repository-owned executable or task target -that hydrates build caches in an implementation worktree. `github-devloop` runs it after refreshing -`.fkst/substrate-ref` and before starting the Codex wall-clock deadline. The command runs from the -trusted supervisor project root with the candidate path in -`FKST_DEVLOOP_CACHE_PREPARATION_WORKTREE`, never from candidate-controlled content. The command has -a 10-minute timeout; a nonzero exit fails the implementation attempt loudly. - -The command must be idempotent because redelivery or a later implementation attempt can run it again -for an existing worktree. It must treat the candidate path as untrusted data and must not execute -candidate-controlled build scripts. Persistent cache ownership and reuse remain repository concerns, -so trusted base logic can use the repository's native cache mechanism without teaching -`github-devloop` about `.lake`, `node_modules`, `target`, or other toolchain-specific directories. +that hydrates build caches for a worktree. `github-devloop` runs it after refreshing +`.fkst/substrate-ref` and before starting the Codex wall-clock deadline, then runs it again immediately +before each candidate local verification. A detached raw-base attribution probe receives the same +preparation after its tree has been fully materialized and before its local gate runs. The command +runs from the trusted supervisor project root with the worktree path in +`FKST_DEVLOOP_CACHE_PREPARATION_WORKTREE`, never from candidate-controlled content. The command has a +10-minute timeout; a nonzero exit fails the implementation attempt loudly. + +The command must be idempotent and concurrency-safe because redelivery, repeated verification, or +another worktree can run it concurrently. It must derive artifact reuse and invalidation from the +repository's complete native build action inputs, including the materialized tree, toolchain, +dependencies, configuration, and relevant environment. A repeated identical action may reuse +artifacts; a changed input must invalidate the affected artifacts. It must treat the worktree path as +untrusted data and must not execute candidate-controlled build scripts. Persistent cache ownership +and reuse remain repository concerns, so trusted base logic can use the repository's native cache +mechanism without teaching `github-devloop` about `.lake`, `node_modules`, `target`, or other +toolchain-specific directories. Cache preparation never substitutes a verification verdict: every +candidate and detached-base local gate still executes and produces its own typed result. + +This repository's canonical provider is `scripts/warm_pinned_bin.sh`. For a Cargo worktree, it uses +Git's absolute common-directory identity to connect the ignored worktree `target` path to the source +checkout's `target`. Cargo remains the artifact authority: its native fingerprints invalidate source, +dependency, toolchain, configuration, and relevant environment changes, and its target lock provides +concurrent single-flight compilation. The provider does not invoke Cargo against candidate content. +For a non-Cargo worktree that needs the pinned framework binary, it reads `.fkst/substrate-ref` from +the trusted project root, never from the candidate worktree, before using the pinned binary cache. ## Repository checks and tests diff --git a/docs/user/host-profile.env.example b/docs/user/host-profile.env.example index 56b98d680..212e45bfe 100644 --- a/docs/user/host-profile.env.example +++ b/docs/user/host-profile.env.example @@ -34,9 +34,10 @@ FKST_DEVLOOP_ROLLUP_MERGE=auto # default is scripts/run.sh test-affected; configure the repository's local CI-equivalent gate # when that executable is absent. Multi-step logic belongs inside the target, not this value. # export FKST_DEVLOOP_LOCAL_TEST_COMMAND='make preflight' -# Optional idempotent cache hydration run from trusted base content before Codex starts. -# The candidate path is available as FKST_DEVLOOP_CACHE_PREPARATION_WORKTREE. -# export FKST_DEVLOOP_CACHE_PREPARATION_COMMAND='make prepare-cache' +# Optional idempotent, concurrency-safe cache hydration run from trusted base content before Codex +# and before each candidate or detached-base local gate. The current worktree path is available as +# FKST_DEVLOOP_CACHE_PREPARATION_WORKTREE; cache identity must cover the complete native build action. +# export FKST_DEVLOOP_CACHE_PREPARATION_COMMAND='scripts/warm_pinned_bin.sh' # Optional: skip local BIN freshness builds when another process manages the build. # FKST_NO_AUTOBUILD=1 diff --git a/packages/github-devloop/departments/implement/harvest.lua b/packages/github-devloop/departments/implement/harvest.lua index 110790f36..e2e396f0d 100644 --- a/packages/github-devloop/departments/implement/harvest.lua +++ b/packages/github-devloop/departments/implement/harvest.lua @@ -3,6 +3,7 @@ local devloop_commands = require("devloop.commands") local config = require("devloop.config") local payloads_builders = require("devloop.payloads.builders") local branch_progress = require("departments.implement.branch_progress") +local cache_preparation = require("departments.implement.cache_preparation") local substrate_pin = require("departments.implement.substrate_pin") local local_iteration_result = require("departments.implement.local_iteration_result") local local_iteration_verdict = require("departments.implement.local_iteration_verdict") @@ -180,7 +181,8 @@ function M.implementation_refusal_outcome(ready, receipt, attempt, started_at, e } end -local function execute_local_iteration_check(worktree, base_head, observe_worktree, exec) +local function execute_local_iteration_check(worktree, base_head, observe_worktree, exec, prepare_cache) + (prepare_cache or cache_preparation.run)(worktree) local quoted_worktree = devloop_base._shell_single_quote(worktree) local command = "cd " .. quoted_worktree if observe_worktree then @@ -208,7 +210,8 @@ function M.local_iteration_check(worktree, base_head, deps) if base_head == nil or tostring(base_head) == "" then error("github-devloop: local-iteration-base-missing: candidate base head is required") end - return execute_local_iteration_check(worktree, base_head, true, deps and deps.exec or nil) + return execute_local_iteration_check( + worktree, base_head, true, deps and deps.exec or nil, deps and deps.prepare_cache or nil) end local function worktree_unavailability_from_command(result) @@ -467,6 +470,7 @@ function M.clean_branch_head(base_head, branch) end function M.commit_dirty_worktree(repo, issue_number, ready, worktree, branch) + cache_preparation.run(worktree) local add_result = devloop_commands.git_add_all(worktree, 30) if add_result.exit_code ~= 0 then error("github-devloop: git-add-failed: git add failed: " .. tostring(add_result.stderr)) diff --git a/packages/github-devloop/tests/implement_local_gate_timeout_test.lua b/packages/github-devloop/tests/implement_local_gate_timeout_test.lua index ab991f53f..a0f44931b 100644 --- a/packages/github-devloop/tests/implement_local_gate_timeout_test.lua +++ b/packages/github-devloop/tests/implement_local_gate_timeout_test.lua @@ -84,4 +84,23 @@ return { t.is_true(calls[1].cmd:find("export BASE='abc123' && make preflight", 1, true) ~= nil) end, + + test_local_gate_prepares_the_exact_worktree_before_verification = function() + local sequence = {} + mock_local_test_command() + + harvest.local_iteration_check("/tmp/fkst worktree", "abc123", { + prepare_cache = function(worktree) + table.insert(sequence, "prepare:" .. worktree) + end, + exec = function() + table.insert(sequence, "verify") + return { stdout = "", stderr = "FKST_LOCAL_ITERATION_RESULT:v2:PASS:NONE\n", exit_code = 0 } + end, + }) + + t.eq(#sequence, 2) + t.eq(sequence[1], "prepare:/tmp/fkst worktree") + t.eq(sequence[2], "verify") + end, } diff --git a/packages/github-devloop/tests/integration_implement_cache_preparation_test.lua b/packages/github-devloop/tests/integration_implement_cache_preparation_test.lua index 6f62dfe93..ad285cc6d 100644 --- a/packages/github-devloop/tests/integration_implement_cache_preparation_test.lua +++ b/packages/github-devloop/tests/integration_implement_cache_preparation_test.lua @@ -1,4 +1,5 @@ local h = require("tests.devloop_helpers") +local codex_jsonl = require("testkit_internal.codex_jsonl") local t = h.t local ready = h.ready local opts = h.opts @@ -12,31 +13,34 @@ local mock_git_status = h.mock_git_status local mock_git_commit = h.mock_git_commit local mock_result_checkpoint = h.mock_result_checkpoint local mock_branch_diff_paths = h.mock_branch_diff_paths +local mock_force_clean = h.mock_force_clean local count_calls = h.count_calls -local cache_command = "make prepare-cache" +local cache_command = "scripts/warm_pinned_bin.sh" local cache_command_env = 'printf %s "$FKST_DEVLOOP_CACHE_PREPARATION_COMMAND"' local project_root_env = 'printf %s "$FKST_PROJECT_ROOT"' local trusted_repository_root = "/trusted/repository" local current_base_pin = "2222222222222222222222222222222222222222" local stale_branch_pin = "1111111111111111111111111111111111111111" -local function mock_cache_command(result) - t.mock_command(cache_command_env, { - stdout = cache_command, - stderr = "", - exit_code = 0, - }) - t.mock_command(cache_command, result or { - stdout = "cache ready\n", - stderr = "", - exit_code = 0, - }) - t.mock_command(project_root_env, { - stdout = trusted_repository_root, - stderr = "", - exit_code = 0, - }) +local function mock_cache_command(result, invocation_count) + for _ = 1, invocation_count or 1 do + t.mock_command(cache_command_env, { + stdout = cache_command, + stderr = "", + exit_code = 0, + }) + t.mock_command(cache_command, result or { + stdout = "cache ready\n", + stderr = "", + exit_code = 0, + }) + t.mock_command(project_root_env, { + stdout = trusted_repository_root, + stderr = "", + exit_code = 0, + }) + end end local function mock_cache_command_unset() @@ -56,13 +60,14 @@ local function command_index(needle) return nil end -local function command_call(needle) - for _, call in ipairs(t.command_calls()) do +local function command_indices(needle) + local indices = {} + for index, call in ipairs(t.command_calls()) do if tostring(call.rendered or ""):find(needle, 1, true) ~= nil then - return call + table.insert(indices, index) end end - return nil + return indices end local function command_env(call, name) @@ -81,29 +86,78 @@ local function mock_successful_candidate(event) mock_git_commit("def456", branch) end +local function mock_candidate_local_red() + t.mock_command("FKST_IMPLEMENTATION_WORKTREE_RESULT:v1:ENTERED", { + stdout = "", + stderr = "FKST_LOCAL_ITERATION_RESULT:v2:FAIL:SEMANTIC\ncandidate failed\n", + exit_code = 1, + }) +end + +local function mock_codex_success_without_local_iteration(message) + t.mock_command("codex exec", { + stdout = codex_jsonl.final_message(message), + stderr = "", + exit_code = 0, + }) +end + +local function mock_base_probe(worktree) + local base_probe = worktree .. "-base-probe" + for _ = 1, 2 do + mock_force_clean(base_probe) + end + t.mock_command("mkdir -p", { stdout = "", stderr = "", exit_code = 0 }) + t.mock_command("git worktree add --detach", { + stdout = "Preparing worktree (detached HEAD abc123)\n", + stderr = "", + exit_code = 0, + }) + t.mock_command("rev-parse HEAD", { stdout = "abc123\n", stderr = "", exit_code = 0 }) + t.mock_command("status --porcelain", { stdout = "", stderr = "", exit_code = 0 }) + t.mock_command("ls-files", { stdout = "", stderr = "", exit_code = 0 }) + t.mock_command("ls-tree", { stdout = "", stderr = "", exit_code = 0 }) + t.mock_command("FKST_IMPLEMENTATION_WORKTREE_RESULT:v1:ENTERED", { + stdout = "", + stderr = "FKST_LOCAL_ITERATION_RESULT:v2:PASS:NONE\n", + exit_code = 0, + }) +end + return { test_cache_preparation_runs_after_substrate_refresh_and_before_codex = function() local event = ready() mock_issue_implement({ "fkst-dev:ready", "fkst-dev:thinking" }) local worktree = mock_fresh_implement_worktree(nil, current_base_pin, stale_branch_pin) - mock_cache_command() + mock_cache_command(nil, 2) mock_successful_candidate(event) local result = run_implement(event, opts("implement-cache-preparation-order")) t.eq(result.exit_code, 0) local pin_refresh = command_index("commit -m 'chore: refresh fkst-substrate pin'") - local preparation = command_index(cache_command) + local preparations = command_indices(cache_command) local codex = command_index("codex exec") + local verification = command_index("scripts/run.sh test-affected") t.is_true(pin_refresh ~= nil) - t.is_true(preparation ~= nil) + t.eq(#preparations, 2) t.is_true(codex ~= nil) - t.is_true(pin_refresh < preparation) - t.is_true(preparation < codex) - t.eq(count_calls(cache_command), 1) - local preparation_call = command_call(cache_command) - t.eq(preparation_call.cwd, trusted_repository_root) - t.eq(command_env(preparation_call, "FKST_DEVLOOP_CACHE_PREPARATION_WORKTREE"), worktree) + t.is_true(verification ~= nil) + t.is_true(pin_refresh < preparations[1]) + t.is_true(preparations[1] < codex) + t.is_true(codex < preparations[2]) + t.is_true(preparations[2] < verification) + t.eq(count_calls(cache_command), 2) + local preparation_calls = {} + for _, call in ipairs(t.command_calls()) do + if tostring(call.rendered or ""):find(cache_command, 1, true) ~= nil then + table.insert(preparation_calls, call) + end + end + for _, preparation_call in ipairs(preparation_calls) do + t.eq(preparation_call.cwd, trusted_repository_root) + t.eq(command_env(preparation_call, "FKST_DEVLOOP_CACHE_PREPARATION_WORKTREE"), worktree) + end end, test_cache_preparation_unset_preserves_implementation_flow = function() @@ -139,12 +193,35 @@ return { t.is_true(tostring(result.error):find("cache-preparation-failed", 1, true) ~= nil) end, + test_cache_preparation_failure_after_codex_stops_before_staging = function() + local event = ready() + mock_issue_implement({ "fkst-dev:ready", "fkst-dev:thinking" }) + mock_fresh_implement_worktree() + mock_cache_command(nil, 1) + mock_cache_command({ + stdout = "", + stderr = "target is not ignored", + exit_code = 1, + }) + mock_implement_codex(0, "implementation changed cache tracking") + mock_git_status(" M .gitignore\n") + + local result = run_implement(event, opts("implement-cache-preparation-before-staging")) + + t.eq(result.exit_code, 1) + t.eq(count_calls(cache_command), 2) + t.eq(count_calls("add -A"), 1) + t.eq(count_calls("commit -m 'Implement github-devloop ready state'"), 0) + t.eq(count_calls("codex exec"), 1) + t.is_true(tostring(result.error):find("cache-preparation-failed", 1, true) ~= nil) + end, + test_cache_preparation_runs_for_reused_worktree_cache = function() local event = ready() local branch = deterministic_branch_for(event) mock_issue_implement({ "fkst-dev:ready" }) local worktree = mock_existing_empty_implement_worktree_reuse(nil, branch, "1") - mock_cache_command() + mock_cache_command(nil, 2) mock_implement_codex(0, "committed implementation from warm cache") mock_git_status("") mock_branch_diff_paths("packages/github-devloop/core.lua\n") @@ -164,11 +241,56 @@ return { t.eq(result.exit_code, 0) t.eq(count_calls("git worktree add"), 0) - t.eq(count_calls(cache_command), 1) + t.eq(count_calls(cache_command), 2) t.eq(count_calls("codex exec"), 1) - t.is_true(command_index(cache_command) < command_index("codex exec")) - local preparation_call = command_call(cache_command) - t.eq(preparation_call.cwd, trusted_repository_root) - t.eq(command_env(preparation_call, "FKST_DEVLOOP_CACHE_PREPARATION_WORKTREE"), worktree) + local preparations = command_indices(cache_command) + t.is_true(preparations[1] < command_index("codex exec")) + t.is_true(command_index("codex exec") < preparations[2]) + t.is_true(preparations[2] < command_index("scripts/run.sh test-affected")) + for _, call in ipairs(t.command_calls()) do + if tostring(call.rendered or ""):find(cache_command, 1, true) ~= nil then + t.eq(call.cwd, trusted_repository_root) + t.eq(command_env(call, "FKST_DEVLOOP_CACHE_PREPARATION_WORKTREE"), worktree) + end + end + end, + + test_cache_preparation_covers_candidate_and_detached_base_without_reusing_verdicts = function() + local event = ready() + local branch = deterministic_branch_for(event) + mock_issue_implement({ "fkst-dev:ready", "fkst-dev:thinking" }) + local worktree = mock_fresh_implement_worktree() + mock_cache_command(nil, 4) + mock_codex_success_without_local_iteration("implemented with a semantic regression") + mock_git_status(" M packages/github-devloop/core.lua\n") + mock_git_commit("def456", branch) + mock_candidate_local_red() + mock_base_probe(worktree) + + local result = run_implement(event, opts("implement-cache-preparation-base-probe")) + + t.eq(result.exit_code, 0) + t.eq(count_calls(cache_command), 4) + t.eq(count_calls("scripts/run.sh test-affected"), 2) + local preparations = command_indices(cache_command) + local verifications = command_indices("scripts/run.sh test-affected") + t.is_true(preparations[1] < command_index("codex exec")) + t.is_true(command_index("codex exec") < preparations[2]) + t.is_true(preparations[2] < preparations[3]) + t.is_true(preparations[3] < verifications[1]) + t.is_true(verifications[1] < preparations[4]) + t.is_true(preparations[4] < verifications[2]) + + local preparation_worktrees = {} + for _, call in ipairs(t.command_calls()) do + if tostring(call.rendered or ""):find(cache_command, 1, true) ~= nil then + table.insert(preparation_worktrees, + command_env(call, "FKST_DEVLOOP_CACHE_PREPARATION_WORKTREE")) + end + end + t.eq(preparation_worktrees[1], worktree) + t.eq(preparation_worktrees[2], worktree) + t.eq(preparation_worktrees[3], worktree) + t.is_true(tostring(preparation_worktrees[4]):find(worktree .. "-base-probe-", 1, true) ~= nil) end, } diff --git a/scripts/host_profile_scaffold_test.py b/scripts/host_profile_scaffold_test.py index 59459bbaf..adb7d897a 100644 --- a/scripts/host_profile_scaffold_test.py +++ b/scripts/host_profile_scaffold_test.py @@ -95,8 +95,9 @@ def test_devloop_cache_preparation_is_documented_as_an_optional_host_contract(se self.assertIn("must be idempotent", doc) self.assertIn("trusted supervisor project root", doc) self.assertIn("`FKST_DEVLOOP_CACHE_PREPARATION_WORKTREE`", doc) + self.assertIn("Cargo remains the artifact authority", doc) self.assertIn("FKST_DEVLOOP_CACHE_PREPARATION_COMMAND", scaffold) - self.assertIn("make prepare-cache", scaffold) + self.assertIn("scripts/warm_pinned_bin.sh", scaffold) if __name__ == "__main__": diff --git a/scripts/run.sh b/scripts/run.sh index 5298f2b7b..99a6873d5 100755 --- a/scripts/run.sh +++ b/scripts/run.sh @@ -198,6 +198,7 @@ cmd_check() { 'python3 -B "$ROOT/scripts/check_repo_restart_preflight_test.py"' 'python3 -B "$ROOT/scripts/bin_cache_test.py"' 'python3 -B "$ROOT/scripts/bin_bootstrap_test.py"' + 'python3 -B "$ROOT/scripts/warm_pinned_bin_test.py"' 'python3 -B "$ROOT/scripts/host_entry_test.py"' 'python3 -B "$ROOT/scripts/run_bin_test.py"' 'python3 -B "$ROOT/scripts/host_profile_scaffold_test.py"' diff --git a/scripts/run_script_contract_test.py b/scripts/run_script_contract_test.py index 4dba8f11f..295df31f2 100644 --- a/scripts/run_script_contract_test.py +++ b/scripts/run_script_contract_test.py @@ -242,7 +242,7 @@ def test_full_test_fails_on_g1_before_bin_resolution(self) -> None: shutil.copy2(root / "scripts" / name, scripts / name) shutil.copy2(root / "scripts/check_repo_restart_preflight.py", scripts / "check_repo_restart_preflight.py") shutil.copy2(root / "libraries/contract/error_facts.lua", contract / "error_facts.lua") - for name in ("check_repo_coverage_test.py", "check_repo_integration_coverage_test.py", "check_repo_intake_default_surface_test.py", "check_repo_dead_letter_test.py", "check_repo_dead_locals_test.py", "check_repo_dedup_test.py", "check_repo_content_truncation_test.py", "check_repo_bot_login_mediation_test.py", "check_repo_fanout_only_test.py", "check_repo_codex_timeout_test.py", "check_repo_dependency_cycle_test.py", "check_repo_producer_liveness_test.py", "check_repo_monotone_gate_test.py", "check_repo_hidden_state_test.py", "check_repo_test_graphql.py", "check_repo_interface_test.py", "lua_coverage_to_lcov_test.py", "check_repo_test.py", "check_repo_github_content_ingress_test.py", "check_repo_error_class_test.py", "check_repo_library_error_class_test.py", "check_repo_library_layering_test.py", "check_repo_std_dependency_model_test.py", "check_repo_devloop_installer_test.py", "check_repo_gh_egress_test.py", "check_repo_gh_handle_construction_test.py", "check_repo_restart_lifecycle_test.py", "check_repo_saga_head_test.py", "check_repo_namespaced_queue_test.py", "check_repo_shell_out_to_self_test.py", "check_repo_fkst_layout.py", "check_repo_fkst_layout_test.py", "bin_cache_test.py", "bin_bootstrap_test.py", "host_entry_test.py", "run_sh_coverage_test.py", "run_sh_test_affected_test.py", "composed_manifest_test.py", "board_test.py", "lifecycle_board_fact_test.py", "doctor_test.py", "ratchet_migration_slicer_test.py", "run_script_contract_test.py", "ratchet_base_test.py", "competence_gate_test.py", "test_parallel_test.py"): + for name in ("check_repo_coverage_test.py", "check_repo_integration_coverage_test.py", "check_repo_intake_default_surface_test.py", "check_repo_dead_letter_test.py", "check_repo_dead_locals_test.py", "check_repo_dedup_test.py", "check_repo_content_truncation_test.py", "check_repo_bot_login_mediation_test.py", "check_repo_fanout_only_test.py", "check_repo_codex_timeout_test.py", "check_repo_dependency_cycle_test.py", "check_repo_producer_liveness_test.py", "check_repo_monotone_gate_test.py", "check_repo_hidden_state_test.py", "check_repo_test_graphql.py", "check_repo_interface_test.py", "lua_coverage_to_lcov_test.py", "check_repo_test.py", "check_repo_github_content_ingress_test.py", "check_repo_error_class_test.py", "check_repo_library_error_class_test.py", "check_repo_library_layering_test.py", "check_repo_std_dependency_model_test.py", "check_repo_devloop_installer_test.py", "check_repo_gh_egress_test.py", "check_repo_gh_handle_construction_test.py", "check_repo_restart_lifecycle_test.py", "check_repo_saga_head_test.py", "check_repo_namespaced_queue_test.py", "check_repo_shell_out_to_self_test.py", "check_repo_fkst_layout.py", "check_repo_fkst_layout_test.py", "bin_cache_test.py", "bin_bootstrap_test.py", "warm_pinned_bin_test.py", "host_entry_test.py", "run_sh_coverage_test.py", "run_sh_test_affected_test.py", "composed_manifest_test.py", "board_test.py", "lifecycle_board_fact_test.py", "doctor_test.py", "ratchet_migration_slicer_test.py", "run_script_contract_test.py", "ratchet_base_test.py", "competence_gate_test.py", "test_parallel_test.py"): (scripts / name).write_text("#!/usr/bin/env python3\nraise SystemExit(0)\n", encoding="utf-8") (scripts / "check_repo_restart_preflight_test.py").write_text( "#!/usr/bin/env python3\nraise SystemExit(0)\n", encoding="utf-8" diff --git a/scripts/warm_pinned_bin.sh b/scripts/warm_pinned_bin.sh index 7106cf4c0..c1f46cf3e 100755 --- a/scripts/warm_pinned_bin.sh +++ b/scripts/warm_pinned_bin.sh @@ -12,15 +12,75 @@ case "$worktree" in *) bootstrap_die "warm-pinned-bin-invalid-worktree: FKST_DEVLOOP_CACHE_PREPARATION_WORKTREE must be absolute" ;; esac -# A repository that declares no substrate pin has nothing to warm. fkst-substrate is the engine -# itself and carries no `.fkst/substrate-ref`, so treat absence as not-applicable rather than as a -# failure: erroring here fails the cache-preparation hook, which fails the whole implement attempt. -if [ ! -f "$worktree/.fkst/substrate-ref" ]; then - printf 'warm-pinned-bin pin=none result=not-applicable\n' - exit 0 -fi +prepare_shared_cargo_target() { + local candidate_target="$worktree/target" common_git_dir shared_target linked_target + CARGO_TARGET_RESULT="not-applicable" + [ -f "$worktree/Cargo.toml" ] || return 0 + + # A target symlink must remain an ignored build artifact, never candidate source. + if git -C "$worktree" ls-files --error-unmatch -- target >/dev/null 2>&1; then + bootstrap_die "warm-pinned-bin-target-tracked: target must remain an untracked cache link" + fi + if ! git -C "$worktree" check-ignore -q -- target; then + bootstrap_die "warm-pinned-bin-target-not-ignored: target must remain ignored" + fi + + common_git_dir="$(git -C "$worktree" rev-parse --path-format=absolute --git-common-dir)" \ + || bootstrap_die "warm-pinned-bin-git-identity-unavailable: cannot resolve worktree common git directory" + common_git_dir="${common_git_dir%/}" + case "$common_git_dir" in + /*/.git) ;; + *) bootstrap_die "warm-pinned-bin-git-identity-invalid: expected an absolute .git common directory" ;; + esac + [ -d "$common_git_dir" ] \ + || bootstrap_die "warm-pinned-bin-git-identity-missing: common git directory does not exist" + + shared_target="${common_git_dir%/.git}/target" + if [ "$candidate_target" = "$shared_target" ]; then + CARGO_TARGET_RESULT="repository-target" + return 0 + fi + mkdir -p "$shared_target" -pin="$(bootstrap_read_pin "$worktree")" -bootstrap_result="" -bootstrap_bin_on_total_miss "$worktree" bootstrap_result >/dev/null -printf 'warm-pinned-bin pin=%s result=%s\n' "$pin" "$bootstrap_result" + if [ -L "$candidate_target" ]; then + linked_target="$(readlink "$candidate_target")" + [ "$linked_target" = "$shared_target" ] \ + || bootstrap_die "warm-pinned-bin-target-conflict: target symlink does not select the repository cache" + CARGO_TARGET_RESULT="hit" + return 0 + fi + if [ -e "$candidate_target" ]; then + CARGO_TARGET_RESULT="local-target" + return 0 + fi + if ln -s "$shared_target" "$candidate_target"; then + CARGO_TARGET_RESULT="linked" + return 0 + fi + + # A concurrent preparer may have created the same link after the absence check. + if [ -L "$candidate_target" ] && [ "$(readlink "$candidate_target")" = "$shared_target" ]; then + CARGO_TARGET_RESULT="hit" + return 0 + fi + bootstrap_die "warm-pinned-bin-target-link-failed: cannot connect worktree to repository Cargo target" +} + +prepare_shared_cargo_target + +# The command runs from trusted project-root content. Candidate pins are data under review and must +# not select source that this host-side preparation process clones and builds. +trusted_root="$PWD" +if [ -f "$trusted_root/.fkst/substrate-ref" ]; then + pin="$(bootstrap_read_pin "$trusted_root")" + bootstrap_result="" + bootstrap_bin_on_total_miss "$trusted_root" bootstrap_result >/dev/null + if [ "$CARGO_TARGET_RESULT" = "not-applicable" ]; then + printf 'warm-pinned-bin pin=%s result=%s\n' "$pin" "$bootstrap_result" + else + printf 'warm-pinned-bin pin=%s result=%s cargo_target=%s\n' \ + "$pin" "$bootstrap_result" "$CARGO_TARGET_RESULT" + fi +else + printf 'warm-pinned-bin pin=none result=%s\n' "$CARGO_TARGET_RESULT" +fi diff --git a/scripts/warm_pinned_bin_test.py b/scripts/warm_pinned_bin_test.py index 3984af5a2..2ad4937d4 100644 --- a/scripts/warm_pinned_bin_test.py +++ b/scripts/warm_pinned_bin_test.py @@ -3,7 +3,9 @@ from __future__ import annotations +import concurrent.futures import os +import shutil import stat import subprocess import sys @@ -50,6 +52,11 @@ def __init__(self, project_pin: str, worktree_pin: str) -> None: def close(self) -> None: self.tmp.cleanup() + def create_worktree(self, name: str, pin: str) -> Path: + worktree = self.worktree.parent / name + self._create_repo(worktree, pin) + return worktree + @staticmethod def _create_repo(root: Path, pin: str) -> None: (root / ".fkst").mkdir(parents=True) @@ -82,6 +89,9 @@ def _install_fake_tools(self) -> None: """\ #!/usr/bin/env sh echo "cargo $*" >> "$FKST_TEST_COMMAND_LOG" + if [ -n "${FKST_TEST_CARGO_DELAY_SECONDS:-}" ]; then + sleep "$FKST_TEST_CARGO_DELAY_SECONDS" + fi while [ "$#" -gt 0 ]; do if [ "$1" = "--manifest-path" ]; then checkout="${2%/Cargo.toml}" @@ -136,9 +146,74 @@ def calls(self) -> str: class WarmPinnedBinTest(unittest.TestCase): + def test_identical_trusted_pin_reuses_binary_across_worktrees(self) -> None: + pin = "ProjectOwner/project-substrate@project-ref" + h = WarmPinnedBinHarness(pin, "CandidateOwner/candidate-substrate@candidate-ref") + try: + detached_base = h.create_worktree( + "detached-base", "BaseOwner/base-substrate@base-ref" + ) + + candidate_result = h.run(str(h.worktree)) + base_result = h.run(str(detached_base)) + + self.assertEqual(candidate_result.returncode, 0, candidate_result.stderr) + self.assertEqual(base_result.returncode, 0, base_result.stderr) + self.assertIn("result=build", candidate_result.stdout) + self.assertIn("result=hit", base_result.stdout) + self.assertEqual(h.calls().count("cargo build --manifest-path"), 1) + finally: + h.close() + + def test_changed_trusted_pin_uses_a_distinct_binary(self) -> None: + first_pin = "SharedOwner/shared-substrate@first-ref" + second_pin = "SharedOwner/shared-substrate@second-ref" + h = WarmPinnedBinHarness(first_pin, "CandidateOwner/candidate-substrate@candidate-ref") + try: + first_result = h.run(str(h.worktree)) + (h.project_root / ".fkst" / "substrate-ref").write_text( + second_pin + "\n", encoding="utf-8" + ) + second_result = h.run(str(h.worktree)) + + self.assertEqual(first_result.returncode, 0, first_result.stderr) + self.assertEqual(second_result.returncode, 0, second_result.stderr) + self.assertIn("result=build", first_result.stdout) + self.assertIn("result=build", second_result.stdout) + calls = h.calls() + self.assertEqual(calls.count("cargo build --manifest-path"), 2) + self.assertIn(" checkout --detach first-ref", calls) + self.assertIn(" checkout --detach second-ref", calls) + finally: + h.close() + + def test_concurrent_identical_trusted_pins_share_one_build(self) -> None: + pin = "SharedOwner/shared-substrate@concurrent-ref" + h = WarmPinnedBinHarness(pin, "CandidateOwner/candidate-substrate@candidate-ref") + try: + peer_worktree = h.create_worktree( + "concurrent-peer", "PeerOwner/peer-substrate@peer-ref" + ) + h.env["FKST_TEST_CARGO_DELAY_SECONDS"] = "0.2" + + with concurrent.futures.ThreadPoolExecutor(max_workers=2) as executor: + futures = [ + executor.submit(h.run, str(h.worktree)), + executor.submit(h.run, str(peer_worktree)), + ] + results = [future.result() for future in futures] + + for result in results: + self.assertEqual(result.returncode, 0, result.stderr) + outcomes = {result.stdout.strip().rsplit("=", 1)[-1] for result in results} + self.assertEqual(outcomes, {"build", "hit"}) + self.assertEqual(h.calls().count("cargo build --manifest-path"), 1) + finally: + h.close() + def test_warm_exact_pin_cache_exits_without_git_or_cargo(self) -> None: pin = "WarmOwner/warm-substrate@warm-ref" - h = WarmPinnedBinHarness("ProjectOwner/project-substrate@project-ref", pin) + h = WarmPinnedBinHarness(pin, "CandidateOwner/candidate-substrate@candidate-ref") try: cached_bin = h.cache_bin("WarmOwner", "warm-substrate", "warm-ref") cached_bin.parent.mkdir(parents=True) @@ -152,7 +227,7 @@ def test_warm_exact_pin_cache_exits_without_git_or_cargo(self) -> None: finally: h.close() - def test_cold_cache_builds_pin_read_from_worktree(self) -> None: + def test_cold_cache_builds_pin_read_from_trusted_project(self) -> None: project_pin = "ProjectOwner/project-substrate@project-ref" worktree_pin = "WorktreeOwner/worktree-substrate@worktree-ref" h = WarmPinnedBinHarness(project_pin, worktree_pin) @@ -160,31 +235,26 @@ def test_cold_cache_builds_pin_read_from_worktree(self) -> None: result = h.run(str(h.worktree)) self.assertEqual(result.returncode, 0, result.stderr) - self.assertEqual(result.stdout.strip(), f"warm-pinned-bin pin={worktree_pin} result=build") + self.assertEqual(result.stdout.strip(), f"warm-pinned-bin pin={project_pin} result=build") calls = h.calls() self.assertIn( - "git clone --no-checkout https://github.com/WorktreeOwner/worktree-substrate.git", + "git clone --no-checkout https://github.com/ProjectOwner/project-substrate.git", calls, ) - self.assertIn(" checkout --detach worktree-ref", calls) + self.assertIn(" checkout --detach project-ref", calls) self.assertIn("cargo build --manifest-path", calls) - self.assertNotIn("project-ref", calls) - self.assertNotIn("ProjectOwner/project-substrate", calls) + self.assertNotIn("worktree-ref", calls) + self.assertNotIn("WorktreeOwner/worktree-substrate", calls) finally: h.close() - def test_worktree_without_a_substrate_pin_is_not_applicable(self) -> None: - """fkst-substrate is the engine and carries no `.fkst/substrate-ref`. - - Absence of a pin means there is nothing to warm, not that preparation failed. Erroring here - fails the cache-preparation hook, which fails the whole implement attempt: that regression - produced 18 dead-letters on the substrate target before it was caught. - """ + def test_non_cargo_worktree_without_a_substrate_pin_is_not_applicable(self) -> None: h = WarmPinnedBinHarness( "ProjectOwner/project-substrate@project-ref", "WorktreeOwner/worktree-substrate@worktree-ref", ) try: + (h.project_root / ".fkst" / "substrate-ref").unlink() (h.worktree / ".fkst" / "substrate-ref").unlink() result = h.run(str(h.worktree)) self.assertEqual(result.returncode, 0, result.stderr) @@ -209,5 +279,239 @@ def test_missing_or_relative_worktree_fails_with_narrow_error(self) -> None: h.close() +class SharedCargoTargetHarness: + def __init__(self) -> None: + cargo = shutil.which("cargo") + if cargo is None: + raise RuntimeError("cargo is required for shared Cargo target behavior tests") + + self.tmp = tempfile.TemporaryDirectory() + tmp_root = Path(self.tmp.name) + self.project_root = tmp_root / "trusted-project" + self.checkout = tmp_root / "source-checkout" + self.common_git_dir = self.checkout / ".git" + self.shared_dependency = tmp_root / "shared-dependency" + self.candidate = tmp_root / "candidate-worktree" + self.detached_base = tmp_root / "detached-base-worktree" + self.rustc_log = tmp_root / "rustc.log" + self.project_root.mkdir() + self._write_shared_dependency(1) + self._create_repository() + self.rustc_wrapper = tmp_root / "rustc-wrapper.sh" + write_executable( + self.rustc_wrapper, + textwrap.dedent( + """\ + #!/usr/bin/env sh + printf '%s\\n' "$*" >> "$FKST_TEST_RUSTC_LOG" + exec "$@" + """ + ), + ) + + self.env = os.environ.copy() + self.env.pop("CARGO_TARGET_DIR", None) + self.env.update( + { + "FKST_TEST_RUSTC_LOG": str(self.rustc_log), + "RUSTC_WRAPPER": str(self.rustc_wrapper), + } + ) + + def close(self) -> None: + self.tmp.cleanup() + + def _write_shared_dependency(self, value: int) -> None: + (self.shared_dependency / "src").mkdir(parents=True, exist_ok=True) + (self.shared_dependency / "Cargo.toml").write_text( + textwrap.dedent( + """\ + [package] + name = "shared_dependency" + version = "0.1.0" + edition = "2021" + """ + ), + encoding="utf-8", + ) + (self.shared_dependency / "src" / "lib.rs").write_text( + f"pub fn value() -> u8 {{ {value} }}\n", encoding="utf-8" + ) + + def _create_repository(self) -> None: + self.checkout.mkdir() + self._run_git("init", str(self.checkout)) + self._write_project(self.checkout) + self._run_git("-C", str(self.checkout), "add", ".") + self._run_git( + "-C", + str(self.checkout), + "-c", + "user.name=Cache Test", + "-c", + "user.email=cache-test@example.invalid", + "commit", + "-m", + "fixture", + ) + for worktree in (self.candidate, self.detached_base): + self._run_git( + "-C", str(self.checkout), "worktree", "add", "--detach", str(worktree), "HEAD" + ) + + def _write_project(self, root: Path) -> None: + (root / "src").mkdir(parents=True) + dependency_path = str(self.shared_dependency).replace("\\", "\\\\") + (root / ".gitignore").write_text("/target\n", encoding="utf-8") + (root / "Cargo.toml").write_text( + textwrap.dedent( + f"""\ + [package] + name = "cache_probe" + version = "0.1.0" + edition = "2021" + + [dependencies] + shared_dependency = {{ path = "{dependency_path}" }} + """ + ), + encoding="utf-8", + ) + (root / "src" / "main.rs").write_text( + 'fn main() { println!("{}", shared_dependency::value()); }\n', + encoding="utf-8", + ) + + @staticmethod + def _run_git(*args: str) -> None: + subprocess.run( + ["git", *args], + text=True, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + check=True, + ) + + def prepare(self, worktree: Path) -> subprocess.CompletedProcess[str]: + env = self.env.copy() + env["FKST_DEVLOOP_CACHE_PREPARATION_WORKTREE"] = str(worktree) + return subprocess.run( + ["/bin/bash", str(WARM_SCRIPT)], + cwd=self.project_root, + env=env, + text=True, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + check=False, + ) + + def build(self, worktree: Path) -> subprocess.CompletedProcess[str]: + return subprocess.run( + ["cargo", "build", "--manifest-path", str(worktree / "Cargo.toml")], + cwd=worktree, + env=self.env, + text=True, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + check=False, + ) + + def shared_dependency_compiles(self) -> int: + if not self.rustc_log.exists(): + return 0 + return self.rustc_log.read_text(encoding="utf-8").count( + "--crate-name shared_dependency" + ) + + +class SharedCargoTargetTest(unittest.TestCase): + def test_non_ignored_target_fails_closed(self) -> None: + h = SharedCargoTargetHarness() + try: + prepared = h.prepare(h.candidate) + self.assertEqual(prepared.returncode, 0, prepared.stderr) + self.assertTrue(h.candidate.joinpath("target").is_symlink()) + + h.candidate.joinpath(".gitignore").write_text("", encoding="utf-8") + rejected = h.prepare(h.candidate) + + self.assertNotEqual(rejected.returncode, 0) + self.assertIn("warm-pinned-bin-target-not-ignored", rejected.stderr) + finally: + h.close() + + def test_tracked_target_fails_closed(self) -> None: + h = SharedCargoTargetHarness() + try: + prepared = h.prepare(h.candidate) + self.assertEqual(prepared.returncode, 0, prepared.stderr) + h._run_git("-C", str(h.candidate), "add", "-f", "target") + + rejected = h.prepare(h.candidate) + + self.assertNotEqual(rejected.returncode, 0) + self.assertIn("warm-pinned-bin-target-tracked", rejected.stderr) + finally: + h.close() + + def test_identical_native_action_reuses_artifact_across_worktrees(self) -> None: + h = SharedCargoTargetHarness() + try: + candidate_prepare = h.prepare(h.candidate) + base_prepare = h.prepare(h.detached_base) + + self.assertEqual(candidate_prepare.returncode, 0, candidate_prepare.stderr) + self.assertEqual(base_prepare.returncode, 0, base_prepare.stderr) + shared_target = h.checkout.joinpath("target").resolve() + self.assertEqual(h.candidate.joinpath("target").resolve(), shared_target) + self.assertEqual(h.detached_base.joinpath("target").resolve(), shared_target) + self.assertEqual(h.shared_dependency_compiles(), 0) + + candidate_build = h.build(h.candidate) + base_build = h.build(h.detached_base) + + self.assertEqual(candidate_build.returncode, 0, candidate_build.stderr) + self.assertEqual(base_build.returncode, 0, base_build.stderr) + self.assertEqual(h.shared_dependency_compiles(), 1) + finally: + h.close() + + def test_changed_native_source_input_invalidates_artifact(self) -> None: + h = SharedCargoTargetHarness() + try: + for worktree in (h.candidate, h.detached_base): + prepared = h.prepare(worktree) + self.assertEqual(prepared.returncode, 0, prepared.stderr) + + first_build = h.build(h.candidate) + h._write_shared_dependency(2) + second_build = h.build(h.detached_base) + + self.assertEqual(first_build.returncode, 0, first_build.stderr) + self.assertEqual(second_build.returncode, 0, second_build.stderr) + self.assertEqual(h.shared_dependency_compiles(), 2) + finally: + h.close() + + def test_concurrent_native_actions_share_one_artifact_build(self) -> None: + h = SharedCargoTargetHarness() + try: + for worktree in (h.candidate, h.detached_base): + prepared = h.prepare(worktree) + self.assertEqual(prepared.returncode, 0, prepared.stderr) + + with concurrent.futures.ThreadPoolExecutor(max_workers=2) as executor: + futures = [ + executor.submit(h.build, h.candidate), + executor.submit(h.build, h.detached_base), + ] + results = [future.result() for future in futures] + + for result in results: + self.assertEqual(result.returncode, 0, result.stderr) + self.assertEqual(h.shared_dependency_compiles(), 1) + finally: + h.close() + if __name__ == "__main__": unittest.main()