Can we benefit from sparse sage attention (spargeattn)?

[dsouzaankit]

Ref: https://github.com/thu-ml/SpargeAttn

[dsouzaankit]

Nice work! I see you updated Comfy gui too.
[sparge_sa2, 3bNvfp4] is really sparge_sa3 (Blackwell FP4), right?
I am thinking we need full Cuda Kernels for Torch.compile + sparse sageattn3?
We can test on cloud B200/B300 hourly instances for economy prices here.
B300 build needs code changes (compute capability level related) here though. (For now, thinking to replace sm_100 with sm_103 to test)

[brumskysb]

Is it possible you could share the NVFP4 3b model? Would it be possible to create a NVFP4 7b model for educational purposes?

Edit:
I did find this link, which has a number of Q based models. https://huggingface.co/cmeka/SeedVR2-GGUF/tree/main

Also, this one which has some NVFP4 models. https://huggingface.co/Nexus24/vaeGGUF/tree/main
I previously copied the wrong link for the NVFP4 models I had found. Which were naxci1's models haha.

I've downloaded them and placed them into the seedvr2 folder under models. However, they are not showing up. Any idea how I can get them to be available in the drop down list?

Edit2:
Nevermind, I had to refresh the damn browser. I can see the models now.

[brumskysb]

The NVFP4 models in the links you provided aren't even there; I have all of those in my repository. But to use them, the code needs to support them. These are the models in the links.

https://huggingface.co/Nexus24/vaeGGUF/tree/main

Yeah turns out I found your repo and downloaded your models. I copied the wrong link in my reply earlier. Thanks for the models and your right the NVFP4 models aren't worth it at this time.

[skv89]

The 1st time I tested the NVFP4_3b_fast_nvfp4.safetensors in the DIT node with the ema_vae_fp16.safetensors in the VAE node at 2x, the quality was worst than seedvr2_ema_3b-Q8_0.gguf in the DIT node and speed about the same. The 2nd time I tested it on another test clip, the quality was near identical to seedvr2_ema_3b-Q8_0.gguf but slightly slower. I'm not sure what changed because I've been non-stop tweaking settings and updating dependencies. Do we also need to select a corresponding blackwell safetensors in the VAE node? I was on SEEDVR2 official branch v2.5.24 for these tests with Pytorch 2.7.1 stack.

[noland81]

https://x.com/PavloMolchanov/status/2016577252746785155

@naxci1 Have you tried this yet?

[brumskysb]

You can follow the steps via this link and test it yourself.

naxci1/ComfyUI-SeedVR2.5_new#1

Thank you for working on this. I can't wait to see the results.

[brumskysb]

@naxci1 how's the NVFP4 work coming along?