Skip to content

pull all non-essential code out of the on-chain stuff - #640

Open
ethernet8023 wants to merge 1 commit into
devfrom
split-coord
Open

pull all non-essential code out of the on-chain stuff#640
ethernet8023 wants to merge 1 commit into
devfrom
split-coord

refactor: move all non-essential code out of coordinator

5d46751
Select commit
Loading
Failed to load commit list.
Garnix CI / check workspace-test-ci [x86_64-linux] failed Mar 24, 2026 in 11m 21s

check workspace-test-ci [x86_64-linux]

check workspace-test-ci [x86_64-linux] failed

Details

Last 100 lines of logs:

psyche-workspace-nextest-0.2.0>       22:     0x555555ecfda2 - <std[7feda60b202149c0]::thread::lifecycle::spawn_unchecked<test[74134816450ea074]::run_test::{closure#1}, ()>::{closure#1} as core[87884960dd474d61]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
psyche-workspace-nextest-0.2.0>       23:     0x555556af0a0f - <std[7feda60b202149c0]::sys::thread::unix::Thread>::new::thread_start
psyche-workspace-nextest-0.2.0>       24:     0x7ffff7d36e3c - start_thread
psyche-workspace-nextest-0.2.0>       25:     0x7ffff7dbf6cc - __GI___clone3
psyche-workspace-nextest-0.2.0>       26:                0x0 - <unknown>
psyche-workspace-nextest-0.2.0> 
psyche-workspace-nextest-0.2.0>         PASS [   0.860s] (183/241) psyche-network download::scheduler::tests::test_distro_retries_dont_consume_capacity
psyche-workspace-nextest-0.2.0>         PASS [   0.963s] (184/241) psyche-modeling distro::tests::test_idct
psyche-workspace-nextest-0.2.0>         PASS [   0.852s] (185/241) psyche-network download::scheduler::tests::test_model_sharing_never_exceeds_max_retries
psyche-workspace-nextest-0.2.0>         PASS [   1.536s] (186/241) psyche-data-provider weighted::http::tests::test_http_multi_data_provider
psyche-workspace-nextest-0.2.0>         PASS [   1.541s] (187/241) psyche-data-provider::http test_http_data_provider_shuffled
psyche-workspace-nextest-0.2.0>         PASS [   0.735s] (188/241) psyche-network serializable_tensor::tests::test_roundtrip_bool_tensor_non_divisible_by_8
psyche-workspace-nextest-0.2.0>         PASS [   0.879s] (189/241) psyche-network download::scheduler::tests::test_parameter_retries_are_immediate
psyche-workspace-nextest-0.2.0>         PASS [   0.839s] (190/241) psyche-network download::scheduler::tests::test_wait_for_capacity_errors_on_actor_shutdown
psyche-workspace-nextest-0.2.0>         PASS [   1.017s] (191/241) psyche-network download::scheduler::tests::test_config_retries_dont_consume_capacity
psyche-workspace-nextest-0.2.0>         PASS [   1.075s] (192/241) psyche-modeling distro::tests::test_large_numbers
psyche-workspace-nextest-0.2.0>         PASS [   0.841s] (193/241) psyche-network serializable_tensor::tests::test_roundtrip_bool_tensor2d
psyche-workspace-nextest-0.2.0>         PASS [   1.065s] (194/241) psyche-network download::scheduler::tests::test_distro_max_retries_exceeded
psyche-workspace-nextest-0.2.0>         PASS [   0.005s] (195/241) psyche-solana-treasurer test_id
psyche-workspace-nextest-0.2.0>         PASS [   0.904s] (196/241) psyche-network router::tests::test_shutdown
psyche-workspace-nextest-0.2.0>         PASS [   0.889s] (197/241) psyche-network serializable_tensor::tests::test_roundtrip_bool_tensor1d
psyche-workspace-nextest-0.2.0>         PASS [   0.987s] (198/241) psyche-network download::scheduler::tests::test_waiters_are_served_fifo
psyche-workspace-nextest-0.2.0>         PASS [   1.771s] (199/241) psyche-centralized-testing::integration_tests connect_single_node
psyche-workspace-nextest-0.2.0>         PASS [   1.008s] (200/241) psyche-network download::scheduler::tests::test_remove_retry
psyche-workspace-nextest-0.2.0>         PASS [   0.838s] (201/241) psyche-network serializable_tensor::tests::test_roundtrip_tensor2d
psyche-workspace-nextest-0.2.0>         PASS [   0.354s] (202/241) psyche-solana-tooling::mod suites::memnet_coordinator_init_free::run
psyche-workspace-nextest-0.2.0>         PASS [   0.246s] (203/241) psyche-solana-tooling::mod suites::memnet_treasurer_create_claim::run
psyche-workspace-nextest-0.2.0>         PASS [   0.423s] (204/241) psyche-solana-tooling::mod suites::memnet_authorizer_full_cycle::run
psyche-workspace-nextest-0.2.0>         PASS [   0.891s] (205/241) psyche-network serializable_tensor::tests::test_roundtrip_tensor_manyd
psyche-workspace-nextest-0.2.0>         PASS [   1.001s] (206/241) psyche-network serializable_tensor::tests::test_roundtrip_bool_tensor_single_element
psyche-workspace-nextest-0.2.0>         PASS [   0.317s] (207/241) psyche-solana-tooling::mod suites::memnet_mining_pool_cap_deposits::run
psyche-workspace-nextest-0.2.0>         PASS [   0.957s] (208/241) psyche-network serializable_tensor::tests::test_roundtrip_tensor1d
psyche-workspace-nextest-0.2.0>         PASS [   0.291s] (209/241) psyche-solana-tooling::mod suites::memnet_mining_pool_live_freeze::run
psyche-workspace-nextest-0.2.0>         PASS [   0.324s] (210/241) psyche-solana-tooling::mod suites::memnet_treasurer_create_update::run
psyche-workspace-nextest-0.2.0>         PASS [   1.052s] (211/241) psyche-network serializable_tensor::tests::test_roundtrip_bool_tensor_unusual_shape
psyche-workspace-nextest-0.2.0>         PASS [   0.983s] (212/241) psyche-network serialized_distro::tests::test_roundtrip_distro_result_1bit
psyche-workspace-nextest-0.2.0>         PASS [   0.406s] (213/241) psyche-solana-tooling::mod suites::memnet_mining_pool_full_cycle::run
psyche-workspace-nextest-0.2.0>         PASS [   1.010s] (214/241) psyche-network tcp::tests::test_challenge_rejects_invalid_signature
psyche-workspace-nextest-0.2.0>         PASS [   0.761s] (215/241) psyche-network tcp::tests::test_tcp_handshake_uses_only_iroh_key
psyche-workspace-nextest-0.2.0>         PASS [   1.001s] (216/241) psyche-network tcp::tests::test_message_exchange_after_iroh_handshake
psyche-workspace-nextest-0.2.0>         PASS [   1.278s] (217/241) psyche-network serializable_tensor::tests::test_roundtrip_bool_tensor_manyd
psyche-workspace-nextest-0.2.0>         PASS [   0.691s] (218/241) psyche-solana-tooling::mod suites::memnet_coordinator_full_round::run
psyche-workspace-nextest-0.2.0>         PASS [   1.007s] (219/241) psyche-network tcp::tests::test_multiple_clients
psyche-workspace-nextest-0.2.0>         PASS [   0.712s] (220/241) psyche-solana-tooling::mod suites::memnet_distributor_many_claims::run
psyche-workspace-nextest-0.2.0>         PASS [   0.754s] (221/241) psyche-solana-tooling::mod suites::memnet_distributor_claim_vesting::run
psyche-workspace-nextest-0.2.0>         PASS [   2.328s] (222/241) psyche-modeling distro::tests::test_artifical_distro_results_roundtrip
psyche-workspace-nextest-0.2.0>         PASS [   0.709s] (223/241) psyche-solana-tooling::mod suites::memnet_treasurer_full_epoch::run
psyche-workspace-nextest-0.2.0>         PASS [   0.968s] (224/241) psyche-solana-tooling::mod suites::memnet_coordinator_rewards::run
psyche-workspace-nextest-0.2.0>         PASS [   1.106s] (225/241) psyche-solana-tooling::mod suites::memnet_distributor_single_claim::run
psyche-workspace-nextest-0.2.0>         PASS [   2.772s] (226/241) psyche-centralized-testing::integration_tests state_change_waiting_for_members_to_warmup
psyche-workspace-nextest-0.2.0>         PASS [   2.904s] (227/241) psyche-inference vllm::tests::test_list_engines
psyche-workspace-nextest-0.2.0>         PASS [   3.181s] (228/241) psyche-centralized-testing::integration_tests state_change_shutdown_node_in_warmup
psyche-workspace-nextest-0.2.0>         PASS [   3.352s] (229/241) psyche-centralized-testing::integration_tests connect_multiple_nodes
psyche-workspace-nextest-0.2.0>         PASS [   4.018s] (230/241) psyche-network router::tests::test_allowlist
psyche-workspace-nextest-0.2.0>         PASS [  16.606s] (231/241) psyche-centralized-testing::integration_tests state_change_waiting_for_members_to_round_witness
psyche-workspace-nextest-0.2.0>         PASS [  19.637s] (232/241) psyche-centralized-testing::integration_tests replace_node_and_complete_round
psyche-workspace-nextest-0.2.0>         PASS [  19.336s] (233/241) psyche-network test::test_retry_connection
psyche-workspace-nextest-0.2.0>         PASS [  21.712s] (234/241) psyche-centralized-testing::integration_tests shutdown_node_in_training_and_complete_round
psyche-workspace-nextest-0.2.0>         PASS [  22.079s] (235/241) psyche-centralized-testing::integration_tests state_change_waiting_for_members_to_round_train
psyche-workspace-nextest-0.2.0>         PASS [  22.675s] (236/241) psyche-network test::test_retry_connection_mid_download
psyche-workspace-nextest-0.2.0>         PASS [  31.663s] (237/241) psyche-centralized-testing::integration_tests validate_all_clients_participate_in_witness_bloom
psyche-workspace-nextest-0.2.0>         PASS [  41.721s] (238/241) psyche-centralized-testing::integration_tests finish_epoch
psyche-workspace-nextest-0.2.0>         PASS [  41.840s] (239/241) psyche-centralized-testing::integration_tests client_join_in_training
psyche-workspace-nextest-0.2.0>         SLOW [> 60.000s] (-------) psyche-centralized-testing::integration_tests client_join_in_training_and_get_model_using_p2p
psyche-workspace-nextest-0.2.0>         SLOW [> 60.000s] (-------) psyche-centralized-testing::integration_tests two_clients_join_in_training_and_get_model_using_p2p
psyche-workspace-nextest-0.2.0>         PASS [  72.220s] (240/241) psyche-centralized-testing::integration_tests client_join_in_training_and_get_model_using_p2p
psyche-workspace-nextest-0.2.0>         PASS [  83.868s] (241/241) psyche-centralized-testing::integration_tests two_clients_join_in_training_and_get_model_using_p2p
psyche-workspace-nextest-0.2.0> ------------
psyche-workspace-nextest-0.2.0>      Summary [  83.871s] 241 tests run: 240 passed (2 slow), 1 failed, 2 skipped
psyche-workspace-nextest-0.2.0>         FAIL [   0.033s] (182/241) psyche-solana-tooling::mod suites::memnet_coordinator_data_layout::run
psyche-workspace-nextest-0.2.0> error: test run failed
error: build of '/nix/store/dvc0kgyary1m8rsfhfk15ynf4kpcbmy1-psyche-workspace-nextest-0.2.0.drv' on 'ssh-ng://nix-ssh@garnix9' failed: builder for '/nix/store/dvc0kgyary1m8rsfhfk15ynf4kpcbmy1-psyche-workspace-nextest-0.2.0.drv' failed with exit code 100;
       last 25 log lines:
       >         PASS [   0.709s] (223/241) psyche-solana-tooling::mod suites::memnet_treasurer_full_epoch::run
       >         PASS [   0.968s] (224/241) psyche-solana-tooling::mod suites::memnet_coordinator_rewards::run
       >         PASS [   1.106s] (225/241) psyche-solana-tooling::mod suites::memnet_distributor_single_claim::run
       >         PASS [   2.772s] (226/241) psyche-centralized-testing::integration_tests state_change_waiting_for_members_to_warmup
       >         PASS [   2.904s] (227/241) psyche-inference vllm::tests::test_list_engines
       >         PASS [   3.181s] (228/241) psyche-centralized-testing::integration_tests state_change_shutdown_node_in_warmup
       >         PASS [   3.352s] (229/241) psyche-centralized-testing::integration_tests connect_multiple_nodes
       >         PASS [   4.018s] (230/241) psyche-network router::tests::test_allowlist
       >         PASS [  16.606s] (231/241) psyche-centralized-testing::integration_tests state_change_waiting_for_members_to_round_witness
       >         PASS [  19.637s] (232/241) psyche-centralized-testing::integration_tests replace_node_and_complete_round
       >         PASS [  19.336s] (233/241) psyche-network test::test_retry_connection
       >         PASS [  21.712s] (234/241) psyche-centralized-testing::integration_tests shutdown_node_in_training_and_complete_round
       >         PASS [  22.079s] (235/241) psyche-centralized-testing::integration_tests state_change_waiting_for_members_to_round_train
       >         PASS [  22.675s] (236/241) psyche-network test::test_retry_connection_mid_download
       >         PASS [  31.663s] (237/241) psyche-centralized-testing::integration_tests validate_all_clients_participate_in_witness_bloom
       >         PASS [  41.721s] (238/241) psyche-centralized-testing::integration_tests finish_epoch
       >         PASS [  41.840s] (239/241) psyche-centralized-testing::integration_tests client_join_in_training
       >         SLOW [> 60.000s] (-------) psyche-centralized-testing::integration_tests client_join_in_training_and_get_model_using_p2p
       >         SLOW [> 60.000s] (-------) psyche-centralized-testing::integration_tests two_clients_join_in_training_and_get_model_using_p2p
       >         PASS [  72.220s] (240/241) psyche-centralized-testing::integration_tests client_join_in_training_and_get_model_using_p2p
       >         PASS [  83.868s] (241/241) psyche-centralized-testing::integration_tests two_clients_join_in_training_and_get_model_using_p2p
       > ------------
       >      Summary [  83.871s] 241 tests run: 240 passed (2 slow), 1 failed, 2 skipped
       >         FAIL [   0.033s] (182/241) psyche-solana-tooling::mod suites::memnet_coordinator_data_layout::run
       > error: test run failed
       For full logs, run 'nix log /nix/store/dvc0kgyary1m8rsfhfk15ynf4kpcbmy1-psyche-workspace-nextest-0.2.0.drv'.
error: builder for '/nix/store/dvc0kgyary1m8rsfhfk15ynf4kpcbmy1-psyche-workspace-nextest-0.2.0.drv' failed with exit code 1