fix: hubrepo no longer bytemucks weird - #636
Open
ethernet8023 wants to merge 1 commit into
Open
Garnix CI / check workspace-test-ci [x86_64-linux]
failed
Mar 17, 2026 in 4m 21s
check workspace-test-ci [x86_64-linux]
check workspace-test-ci [x86_64-linux] failed
Details
Last 100 lines of logs:
psyche-workspace-nextest-0.2.0> 22: 0x555555ec0ff4 - std[7feda60b202149c0]::sys::backtrace::__rust_begin_short_backtrace::<test[74134816450ea074]::run_test::{closure#1}, ()>
psyche-workspace-nextest-0.2.0> 23: 0x555555ec9632 - <std[7feda60b202149c0]::thread::lifecycle::spawn_unchecked<test[74134816450ea074]::run_test::{closure#1}, ()>::{closure#1} as core[87884960dd474d61]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
psyche-workspace-nextest-0.2.0> 24: 0x555556aef1cf - <std[7feda60b202149c0]::sys::thread::unix::Thread>::new::thread_start
psyche-workspace-nextest-0.2.0> 25: 0x7ffff7d36e3c - start_thread
psyche-workspace-nextest-0.2.0> 26: 0x7ffff7dbf6cc - __GI___clone3
psyche-workspace-nextest-0.2.0> 27: 0x0 - <unknown>
psyche-workspace-nextest-0.2.0>
psyche-workspace-nextest-0.2.0> PASS [ 0.772s] (165/222) psyche-modeling distro::tests::test_signed_vals_reconstructs_original_sign
psyche-workspace-nextest-0.2.0> PASS [ 0.750s] (166/222) psyche-network download::scheduler::tests::test_parameter_retries_are_immediate
psyche-workspace-nextest-0.2.0> PASS [ 0.830s] (167/222) psyche-network download::scheduler::tests::test_model_sharing_retry_respects_capacity
psyche-workspace-nextest-0.2.0> PASS [ 0.781s] (168/222) psyche-network router::tests::test_shutdown
psyche-workspace-nextest-0.2.0> PASS [ 0.829s] (169/222) psyche-network download::scheduler::tests::test_remove_retry
psyche-workspace-nextest-0.2.0> PASS [ 0.917s] (170/222) psyche-network download::scheduler::tests::test_distro_retries_dont_consume_capacity
psyche-workspace-nextest-0.2.0> PASS [ 0.915s] (171/222) psyche-network download::scheduler::tests::test_distro_retry_not_immediately_due
psyche-workspace-nextest-0.2.0> PASS [ 0.785s] (172/222) psyche-network serializable_tensor::tests::test_roundtrip_bool_tensor1d
psyche-workspace-nextest-0.2.0> PASS [ 0.914s] (173/222) psyche-network download::scheduler::tests::test_release_unblocks_waiter
psyche-workspace-nextest-0.2.0> PASS [ 0.944s] (174/222) psyche-network download::scheduler::tests::test_config_retries_dont_consume_capacity
psyche-workspace-nextest-0.2.0> PASS [ 0.007s] (175/222) psyche-solana-treasurer test_id
psyche-workspace-nextest-0.2.0> PASS [ 0.934s] (176/222) psyche-network download::scheduler::tests::test_wait_for_capacity_errors_on_actor_shutdown
psyche-workspace-nextest-0.2.0> PASS [ 0.823s] (177/222) psyche-network serializable_tensor::tests::test_roundtrip_bool_tensor2d
psyche-workspace-nextest-0.2.0> PASS [ 0.839s] (178/222) psyche-network serializable_tensor::tests::test_roundtrip_bool_tensor_non_divisible_by_8
psyche-workspace-nextest-0.2.0> PASS [ 0.843s] (179/222) psyche-network serializable_tensor::tests::test_roundtrip_bool_tensor_single_element
psyche-workspace-nextest-0.2.0> PASS [ 1.025s] (180/222) psyche-network download::scheduler::tests::test_waiters_are_served_fifo
psyche-workspace-nextest-0.2.0> PASS [ 0.878s] (181/222) psyche-network serializable_tensor::tests::test_roundtrip_tensor1d
psyche-workspace-nextest-0.2.0> PASS [ 0.889s] (182/222) psyche-network serializable_tensor::tests::test_roundtrip_tensor2d
psyche-workspace-nextest-0.2.0> PASS [ 1.146s] (183/222) psyche-network download::scheduler::tests::test_distro_retries_returned_and_removed
psyche-workspace-nextest-0.2.0> PASS [ 0.242s] (184/222) psyche-solana-tooling::mod suites::memnet_treasurer_create_update::run
psyche-workspace-nextest-0.2.0> PASS [ 0.985s] (185/222) psyche-network serializable_tensor::tests::test_roundtrip_bool_tensor_unusual_shape
psyche-workspace-nextest-0.2.0> PASS [ 0.260s] (186/222) psyche-solana-tooling::mod suites::memnet_mining_pool_live_freeze::run
psyche-workspace-nextest-0.2.0> PASS [ 0.352s] (187/222) psyche-solana-tooling::mod suites::memnet_mining_pool_full_cycle::run
psyche-workspace-nextest-0.2.0> PASS [ 0.274s] (188/222) psyche-solana-tooling::mod suites::memnet_treasurer_create_claim::run
psyche-workspace-nextest-0.2.0> PASS [ 0.470s] (189/222) psyche-solana-tooling::mod suites::memnet_coordinator_init_free::run
psyche-workspace-nextest-0.2.0> PASS [ 0.512s] (190/222) psyche-solana-tooling::mod suites::memnet_authorizer_full_cycle::run
psyche-workspace-nextest-0.2.0> PASS [ 1.028s] (191/222) psyche-network serialized_distro::tests::test_roundtrip_distro_result_1bit
psyche-workspace-nextest-0.2.0> PASS [ 0.813s] (192/222) psyche-network tcp::tests::test_multiple_clients
psyche-workspace-nextest-0.2.0> PASS [ 1.923s] (193/222) psyche-centralized-testing::integration_tests connect_single_node
psyche-workspace-nextest-0.2.0> PASS [ 0.450s] (194/222) psyche-solana-tooling::mod suites::memnet_mining_pool_cap_deposits::run
psyche-workspace-nextest-0.2.0> PASS [ 0.886s] (195/222) psyche-network tcp::tests::test_message_exchange_after_iroh_handshake
psyche-workspace-nextest-0.2.0> PASS [ 1.198s] (196/222) psyche-network serializable_tensor::tests::test_roundtrip_bool_tensor_manyd
psyche-workspace-nextest-0.2.0> PASS [ 0.804s] (197/222) psyche-network tcp::tests::test_tcp_handshake_uses_only_iroh_key
psyche-workspace-nextest-0.2.0> PASS [ 1.051s] (198/222) psyche-network tcp::tests::test_challenge_rejects_invalid_signature
psyche-workspace-nextest-0.2.0> PASS [ 1.141s] (199/222) psyche-network serializable_tensor::tests::test_roundtrip_tensor_manyd
psyche-workspace-nextest-0.2.0> PASS [ 0.772s] (200/222) psyche-solana-tooling::mod suites::memnet_distributor_claim_vesting::run
psyche-workspace-nextest-0.2.0> PASS [ 0.778s] (201/222) psyche-solana-tooling::mod suites::memnet_distributor_many_claims::run
psyche-workspace-nextest-0.2.0> PASS [ 0.857s] (202/222) psyche-solana-tooling::mod suites::memnet_coordinator_full_round::run
psyche-workspace-nextest-0.2.0> PASS [ 2.229s] (203/222) psyche-modeling distro::tests::test_artifical_distro_results_roundtrip
psyche-workspace-nextest-0.2.0> PASS [ 2.399s] (204/222) psyche-centralized-testing::integration_tests state_change_waiting_for_members_to_warmup
psyche-workspace-nextest-0.2.0> PASS [ 1.114s] (205/222) psyche-solana-tooling::mod suites::memnet_distributor_single_claim::run
psyche-workspace-nextest-0.2.0> PASS [ 1.035s] (206/222) psyche-solana-tooling::mod suites::memnet_treasurer_full_epoch::run
psyche-workspace-nextest-0.2.0> PASS [ 2.748s] (207/222) psyche-centralized-testing::integration_tests state_change_shutdown_node_in_warmup
psyche-workspace-nextest-0.2.0> PASS [ 2.752s] (208/222) psyche-inference vllm::tests::test_list_engines
psyche-workspace-nextest-0.2.0> PASS [ 2.136s] (209/222) psyche-network router::tests::test_allowlist
psyche-workspace-nextest-0.2.0> PASS [ 1.452s] (210/222) psyche-solana-tooling::mod suites::memnet_coordinator_rewards::run
psyche-workspace-nextest-0.2.0> PASS [ 3.212s] (211/222) psyche-centralized-testing::integration_tests connect_multiple_nodes
psyche-workspace-nextest-0.2.0> PASS [ 16.793s] (212/222) psyche-centralized-testing::integration_tests state_change_waiting_for_members_to_round_witness
psyche-workspace-nextest-0.2.0> PASS [ 19.104s] (213/222) psyche-centralized-testing::integration_tests replace_node_and_complete_round
psyche-workspace-nextest-0.2.0> PASS [ 19.252s] (214/222) psyche-network test::test_retry_connection
psyche-workspace-nextest-0.2.0> PASS [ 21.648s] (215/222) psyche-centralized-testing::integration_tests state_change_waiting_for_members_to_round_train
psyche-workspace-nextest-0.2.0> PASS [ 22.959s] (216/222) psyche-network test::test_retry_connection_mid_download
psyche-workspace-nextest-0.2.0> PASS [ 25.139s] (217/222) psyche-centralized-testing::integration_tests shutdown_node_in_training_and_complete_round
psyche-workspace-nextest-0.2.0> PASS [ 31.010s] (218/222) psyche-centralized-testing::integration_tests validate_all_clients_participate_in_witness_bloom
psyche-workspace-nextest-0.2.0> PASS [ 41.576s] (219/222) psyche-centralized-testing::integration_tests finish_epoch
psyche-workspace-nextest-0.2.0> PASS [ 44.628s] (220/222) psyche-centralized-testing::integration_tests client_join_in_training
psyche-workspace-nextest-0.2.0> SLOW [> 60.000s] (-------) psyche-centralized-testing::integration_tests two_clients_join_in_training_and_get_model_using_p2p
psyche-workspace-nextest-0.2.0> SLOW [> 60.000s] (-------) psyche-centralized-testing::integration_tests client_join_in_training_and_get_model_using_p2p
psyche-workspace-nextest-0.2.0> PASS [ 86.343s] (221/222) psyche-centralized-testing::integration_tests two_clients_join_in_training_and_get_model_using_p2p
psyche-workspace-nextest-0.2.0> PASS [ 86.857s] (222/222) psyche-centralized-testing::integration_tests client_join_in_training_and_get_model_using_p2p
psyche-workspace-nextest-0.2.0> ------------
psyche-workspace-nextest-0.2.0> Summary [ 86.857s] 222 tests run: 221 passed (2 slow), 1 failed, 2 skipped
psyche-workspace-nextest-0.2.0> FAIL [ 0.030s] (164/222) psyche-solana-tooling::mod suites::memnet_coordinator_data_layout::run
psyche-workspace-nextest-0.2.0> error: test run failed
error: build of '/nix/store/1ybrb3vj9w6gx0sgpzaj7i9zjk66ja07-psyche-workspace-nextest-0.2.0.drv' on 'ssh-ng://nix-ssh@garnix8' failed: builder for '/nix/store/1ybrb3vj9w6gx0sgpzaj7i9zjk66ja07-psyche-workspace-nextest-0.2.0.drv' failed with exit code 100;
last 25 log lines:
> PASS [ 2.399s] (204/222) psyche-centralized-testing::integration_tests state_change_waiting_for_members_to_warmup
> PASS [ 1.114s] (205/222) psyche-solana-tooling::mod suites::memnet_distributor_single_claim::run
> PASS [ 1.035s] (206/222) psyche-solana-tooling::mod suites::memnet_treasurer_full_epoch::run
> PASS [ 2.748s] (207/222) psyche-centralized-testing::integration_tests state_change_shutdown_node_in_warmup
> PASS [ 2.752s] (208/222) psyche-inference vllm::tests::test_list_engines
> PASS [ 2.136s] (209/222) psyche-network router::tests::test_allowlist
> PASS [ 1.452s] (210/222) psyche-solana-tooling::mod suites::memnet_coordinator_rewards::run
> PASS [ 3.212s] (211/222) psyche-centralized-testing::integration_tests connect_multiple_nodes
> PASS [ 16.793s] (212/222) psyche-centralized-testing::integration_tests state_change_waiting_for_members_to_round_witness
> PASS [ 19.104s] (213/222) psyche-centralized-testing::integration_tests replace_node_and_complete_round
> PASS [ 19.252s] (214/222) psyche-network test::test_retry_connection
> PASS [ 21.648s] (215/222) psyche-centralized-testing::integration_tests state_change_waiting_for_members_to_round_train
> PASS [ 22.959s] (216/222) psyche-network test::test_retry_connection_mid_download
> PASS [ 25.139s] (217/222) psyche-centralized-testing::integration_tests shutdown_node_in_training_and_complete_round
> PASS [ 31.010s] (218/222) psyche-centralized-testing::integration_tests validate_all_clients_participate_in_witness_bloom
> PASS [ 41.576s] (219/222) psyche-centralized-testing::integration_tests finish_epoch
> PASS [ 44.628s] (220/222) psyche-centralized-testing::integration_tests client_join_in_training
> SLOW [> 60.000s] (-------) psyche-centralized-testing::integration_tests two_clients_join_in_training_and_get_model_using_p2p
> SLOW [> 60.000s] (-------) psyche-centralized-testing::integration_tests client_join_in_training_and_get_model_using_p2p
> PASS [ 86.343s] (221/222) psyche-centralized-testing::integration_tests two_clients_join_in_training_and_get_model_using_p2p
> PASS [ 86.857s] (222/222) psyche-centralized-testing::integration_tests client_join_in_training_and_get_model_using_p2p
> ------------
> Summary [ 86.857s] 222 tests run: 221 passed (2 slow), 1 failed, 2 skipped
> FAIL [ 0.030s] (164/222) psyche-solana-tooling::mod suites::memnet_coordinator_data_layout::run
> error: test run failed
For full logs, run 'nix log /nix/store/1ybrb3vj9w6gx0sgpzaj7i9zjk66ja07-psyche-workspace-nextest-0.2.0.drv'.
error: builder for '/nix/store/1ybrb3vj9w6gx0sgpzaj7i9zjk66ja07-psyche-workspace-nextest-0.2.0.drv' failed with exit code 1
Loading