Add parallelism_auto flag to automatically set dp, tp and micro batch size - #516
Closed
pefontana wants to merge 28 commits into
Closed
Add parallelism_auto flag to automatically set dp, tp and micro batch size#516pefontana wants to merge 28 commits into
parallelism_auto flag to automatically set dp, tp and micro batch size#516pefontana wants to merge 28 commits into
Garnix CI / check validate-all-configs [x86_64-linux]
succeeded
Mar 3, 2026 in 6m 58s
check validate-all-configs [x86_64-linux]
check validate-all-configs [x86_64-linux] succeeded
Details
Last 100 lines of logs:
these 4 derivations will be built:
/nix/store/g5vws575wskdwwlky1ds0hgw23cdamhx-source.drv
/nix/store/lms9xsdz4q17wnjbcv0nfjbljzp2qvdl-psyche-workspace-deps-0.2.0.drv
/nix/store/8yg4lbac4cfd8d4x1zsfmvqf4dv3k1y4-psyche-centralized-server-0.2.0.drv
/nix/store/bklrlylrgr48c37v8qjn7k9raby0qq3z-validate-configs.drv
these 16 paths will be fetched (0.01 MiB download, 0.01 MiB unpacked):
/nix/store/18d78g9khdvdfrs048nz2jppisp4w2h9-Cargo.toml
/nix/store/6zn7hw4nb7c4y6cimm5c7jzm6q9rbk2p-Cargo.toml
/nix/store/763wzmdncn2ga39cihir06yw6hnqiday-Cargo.toml
/nix/store/8hz9g0aayj79wbfyvcb5vzlnwgkb35h2-Cargo.toml
/nix/store/9ihc790hagwyw3da2226a53l2smjl1b0-Cargo.toml
/nix/store/acbjlf7fds8qcm3cbng25lgmvvcpx8bv-Cargo.toml
/nix/store/an4s0cb8j6hrvahbsykdkin170p9nnly-Cargo.toml
/nix/store/bjrpxczq8dsdrwcba0qpys18viz2bmd2-Cargo.toml
/nix/store/bzx24w3x9la8zqgmkmg9fdgd3yfj9qp6-Cargo.toml
/nix/store/c80ngpp84anvw28gih9q8nbyb56kd9f1-Cargo.toml
/nix/store/cnrny6xbnfsz4z3g4n9ahg3iwd8lif5k-Cargo.toml
/nix/store/hqy4bvj5m83l67q567zw45i7n9d71px4-Cargo.toml
/nix/store/p6y2dpcxz6h28cplhd0y3jga73dn282y-Cargo.toml
/nix/store/r4h38rk70ajs55vc0w1pfasg2s70fsfv-Cargo.toml
/nix/store/sx1h3yq5d0hxs1f4gmfzdj6lnhz6pnh6-Cargo.toml
/nix/store/y2460wwjnwqdlcarr6s0g1z4qllvjhpk-Cargo.toml
validate-configs> Note: client has no state.toml, skipping validation
validate-configs> checking consilience/state.toml
validate-configs> 2026-03-03T19:19:20.629760Z INFO psyche_centralized_server: Configs are OK!
validate-configs> ok!
validate-configs> ccecking consilience-match-llama2-20m-fineweb-pretrain-dev/data.toml and consilience-match-llama2-20m-fineweb-pretrain-dev/state.toml
validate-configs> 2026-03-03T19:19:20.922853Z INFO psyche_centralized_server: Configs are OK!
validate-configs> ok!
validate-configs> ccecking kill-test-short-epoch-checkpoint/data.toml and kill-test-short-epoch-checkpoint/state.toml
validate-configs> 2026-03-03T19:19:21.224959Z INFO psyche_centralized_server: Configs are OK!
validate-configs> ok!
validate-configs> ccecking llama2-20m-dolma-noverify-first-checkpointer/data.toml and llama2-20m-dolma-noverify-first-checkpointer/state.toml
validate-configs> 2026-03-03T19:19:21.527913Z INFO psyche_centralized_server: Configs are OK!
validate-configs> ok!
validate-configs> ccecking llama2-20m-dolma-noverify-no-checkpointer/data.toml and llama2-20m-dolma-noverify-no-checkpointer/state.toml
validate-configs> 2026-03-03T19:19:21.805236Z INFO psyche_centralized_server: Configs are OK!
validate-configs> ok!
validate-configs> ccecking llama2-20m-dolma-verify-first-checkpointer/data.toml and llama2-20m-dolma-verify-first-checkpointer/state.toml
validate-configs> 2026-03-03T19:19:22.140941Z INFO psyche_centralized_server: Configs are OK!
validate-configs> ok!
validate-configs> Note: solana-test has no state.toml, skipping validation
validate-configs> checking test/state.toml
validate-configs> 2026-03-03T19:19:22.462589Z INFO psyche_centralized_server: Configs are OK!
validate-configs> ok!
validate-configs> all configs ok!
Loading