Skip to content

improve model debugging workflow - #611

Open
ethernet8023 wants to merge 8 commits into
mainfrom
train-refactor
Open

improve model debugging workflow#611
ethernet8023 wants to merge 8 commits into
mainfrom
train-refactor

modeling: work around noexec tmp dirs for triton cache

2be4669
Select commit
Loading
Failed to load commit list.
Garnix CI / check validate-psyche-http-mainnet-caddyfile [x86_64-linux] succeeded Mar 18, 2026 in 4m 27s

check validate-psyche-http-mainnet-caddyfile [x86_64-linux]

check validate-psyche-http-mainnet-caddyfile [x86_64-linux] succeeded

Details

Last 100 lines of logs:

these 15 derivations will be built:
  /nix/store/ajlsan68rj3ipdsyqrf0vaqdp371ammk-vendor-registry.drv
  /nix/store/21djzpmbwb62ns9y98xyf58nyhhvwlqb-vendor-cargo-deps.drv
  /nix/store/0ijvfji3kl5a98l86y0363cm30vlqhrm-solana-mining-pool-idl-0.drv
  /nix/store/chh26jmfhjf7skjiry4v0yzg19ix5m30-linkLockedDeps.drv
  /nix/store/rvyhccm0j9069w6klvgkya9j5q7hcfb2-linkLockedDeps.drv
  /nix/store/z8g8mrzqwlijwrmnhwi2p4m84wqvdgjz-vendor-registry.drv
  /nix/store/3m3gy4lnh4l8xa6031fk1b6wrwg8jx19-vendor-cargo-deps.drv
  /nix/store/jpqw6fjrx4110rl27v6sgf45kwvvx6ym-solana-coordinator-idl-0.drv
  /nix/store/fzdk67vyr38xpraw64ixw54i2zs87ag2-psyche-deserialize-zerocopy-wasm-0.2.0.drv
  /nix/store/pmaxx5y32ywmqwzsav2h4df4szh3054w-psyche-website-wasm.drv
  /nix/store/aa5qw6dbgbiznwr6rdigxdy1300v89s3-website-shared-1.0.0.drv
  /nix/store/cxnmxk2fxfl36z0fczmapspgmj960278-website-frontend-1.0.0.drv
  /nix/store/4fv4nf96hs8sisg7gdizl32chb511knr-Caddyfile.drv
  /nix/store/92760ljlgz90w0brffgzqc9909sw4fw1-Caddyfile-formatted.drv
  /nix/store/p5q7npmp3f528970ppsp06d5w1dilndc-validate-psyche-http-mainnet-caddyfile.drv
these 57 paths will be fetched (1238.80 MiB download, 1638.02 MiB unpacked):
  /nix/store/a806cxkfh97ss5c5ci99p1xpqvrpjvpa-bubblewrap-0.11.0-dev
  /nix/store/4rcqlgh703y3is050jbqdn9j7j3nxmn9-cargo-package-anyhow-1.0.98
  /nix/store/f31jgrf5l2ndpsd84n0vifgjazwgnz0n-cargo-package-borsh-1.5.3
  /nix/store/i2ayngs33nm86a7xalq5iim97xnn1z9z-cargo-package-borsh-derive-1.5.3
  /nix/store/564kag32rfc0yim0sfl519g55pjxccjp-cargo-package-bytemuck_derive-1.8.0
  /nix/store/3cd1kivcigxqvg5a4s3y1k02fgwzn691-cargo-package-chrono-0.4.38
  /nix/store/cn86im19f3pgjvzd34s1b9s5ssqb45m9-cargo-package-crunchy-0.2.2
  /nix/store/0z1ln61ppr426f97dxjh10ypyv736jm1-cargo-package-darling_core-0.20.10
  /nix/store/0v1k9psdkml61xypmfnxy3lwfvr35cr8-cargo-package-darling_macro-0.20.10
  /nix/store/78qaf8lzqd3c0nwkl1zza0m5fq1a0lgv-cargo-package-deranged-0.3.11
  /nix/store/wrz8rnl8pn6q2i7bs3v60nlnql2lj2nl-cargo-package-either-1.13.0
  /nix/store/r7979wlzhm42jw32va4iqkhj8hsyfxkc-cargo-package-getrandom-0.2.15
  /nix/store/cb0wvbvrkb5aqzmp6d1icdzg3sfz2a6a-cargo-package-iana-time-zone-0.1.61
  /nix/store/jsvyxg7qgbjrsax5xnaydnkf6123229s-cargo-package-indexmap-2.7.0
  /nix/store/k0d2430l9wdifhai4bv5a4lcykjwm5jc-cargo-package-js-sys-0.3.74
  /nix/store/byvpk6rhd6vp8rd1r4cyqzi3aw7j0ka1-cargo-package-once_cell-1.20.2
  /nix/store/p6snn2bpxknim84ll6h7pw8j8a8073ld-cargo-package-openssl-0.10.68
  /nix/store/87n9nq8a41gg7qfcdagh16p3c8dwzm36-cargo-package-openssl-sys-0.9.104
  /nix/store/jljxikxki5f717h00c1hsl6gzmkhyfk1-cargo-package-parking_lot-0.12.3
  /nix/store/lcn5a9cfvkc3xsbgv5fbz1ab7wdvas2d-cargo-package-percent-encoding-2.3.1
  /nix/store/1qn39fw0pjvyy29mgdwm2538jn4rd0cf-cargo-package-postcard-1.1.1
  /nix/store/mr711l4ypbmck2fin3zbzzyy9m644iq3-cargo-package-proc-macro2-1.0.92
  /nix/store/s1lzsc175mm29fyyh5q26s96l9ib2664-cargo-package-quote-1.0.37
  /nix/store/cmpc17djgvwb8l30q1ig8pxrmvvbpmgk-cargo-package-redox_syscall-0.5.7
  /nix/store/7f31ir36hjrngahz1sfybr83d2ygslfs-cargo-package-regex-1.11.1
  /nix/store/hwz2s54rlj27l98dib338ms1s0zy3a2r-cargo-package-regex-automata-0.4.9
  /nix/store/9gaxp27rrqjjphr2r47a91q8c767fivq-cargo-package-semver-1.0.23
  /nix/store/ymsc79zirfryw5c7wq5kz67fpmclxxca-cargo-package-serde_derive-1.0.217
  /nix/store/4bnbkg4v6fqgr5wjjvqgxpk0pks6jfqv-cargo-package-serde_with-3.11.0
  /nix/store/4jc2fphf5v95fjj32ryqpvzrwn6akw1y-cargo-package-serde_with_macros-3.11.0
  /nix/store/igds7wznpl3bwgs3rw3yk836k05j80j2-cargo-package-spl-token-confidential-transfer-ciphertext-arithmetic-0.2.0
  /nix/store/9s9hpl9a3689d1wbvm15f68wff5mc5zg-cargo-package-thiserror-2.0.12
  /nix/store/4mks2kcjhkvx1sm5gm3x9ml08cwzbqdb-cargo-package-time-0.3.37
  /nix/store/z7svw8jb3azvj5xjxblg9kkcy75ls9wl-cargo-package-time-macros-0.2.19
  /nix/store/rqpci5bfvw42pcadb903wrblvkk1xbx4-cargo-package-toml_datetime-0.6.8
  /nix/store/lhn4x5dfg3xx582r7wfinv947q1q96ld-cargo-package-toml_edit-0.22.22
  /nix/store/fjlrlwz10lrms3jz1j0wbxp256ni41ba-cargo-package-typenum-1.17.0
  /nix/store/7v3kqgqhzqprix0fmbkkggjajzrmpc8s-cargo-package-unicode-ident-1.0.14
  /nix/store/ca1mgd8x11imif7mjrfqnyh9hhz8wgh4-cargo-package-wasm-bindgen-backend-0.2.97
  /nix/store/6znrlaz32clbbnlh4nabgi7cdm3zz59w-cargo-package-wasm-bindgen-macro-0.2.97
  /nix/store/xf1nvzzwabmppn7c6gk0qgr8g1n5mxpg-cargo-package-wasm-bindgen-shared-0.2.97
  /nix/store/69zz2fab2y5zw58yiq4pykhlaa7inq4m-cargo-package-web-sys-0.3.74
  /nix/store/61df3vwy9vnx16ngjppgxarn9bcpsvbg-cargo-package-zerocopy-0.7.35
  /nix/store/p60gfq2vfqikc574w644p63a7wh9804k-cargo-package-zeroize-1.8.1
  /nix/store/99xld92bxzwslkzzkzxh3d6a3cg5xh0a-linkLockedDeps
  /nix/store/a5nyfkwmfyhw3hnqr1c1nb5j9ppqfrdv-linkLockedDeps
  /nix/store/nq5rjmqv9ri7l229djf969cf76xv7rxi-psyche-deserialize-zerocopy-wasm-deps-0.2.0
  /nix/store/ia49z9knmqcjh5mbzwjdsd3ghzf85kbb-python3.12-torch-2.9.1
  /nix/store/59a3nrbzwscj1lqrwqwx629n1v4jhcmb-python3.12-torch-2.9.1-dev
  /nix/store/jn3imm7vabrk494jd26wan35dyfbb02x-python3.12-triton-3.5.1
  /nix/store/id7si7zq66vss34gyfccnhym1im7lhwr-solana-idl-solana-coordinator-deps-0.2.0
  /nix/store/fcpmnxc6g19yw8r7niabqwh9qmfiazh8-solana-idl-solana-mining-pool-deps-0.2.0
  /nix/store/58mrhq7b0ljw9njr8k4vd42k21si75nz-vendor-cargo-deps
  /nix/store/ibln4529b3f2bidl818k673jyjswswj6-vendor-cargo-deps
  /nix/store/c2yxdw514d0bkwrn7njzxrg5485sjs5p-vendor-registry
  /nix/store/n58z3icgarzly752k7yifnhgnvx47j08-vendor-registry
  /nix/store/361j7m41s23najg5wr7lwbkgwp6xjbb7-wasm-bindgen-cli-0.2.108
validate-psyche-http-mainnet-caddyfile> 2026/03/18 22:11:03.254	INFO	using config from file	{"file": "./Caddyfile"}
validate-psyche-http-mainnet-caddyfile> 2026/03/18 22:11:03.257	INFO	adapted config to JSON	{"adapter": "caddyfile"}
validate-psyche-http-mainnet-caddyfile> Valid configuration