Training Support for FP8

[Banaxi-Tech]

Feature request

Add Training Support for the QuantizationMethod.FP8

Motivation

I always want to finetune in FP8 but then get ValueError: The model you are trying to fine-tune is quantized with QuantizationMethod.FP8 but that quantization method do not support training. Please open an issue on GitHub: https://github.com/huggingface/transformers to request the support for training support for QuantizationMethod.FP8

Your contribution

No

[sirzechs66]

Fine-Tuning Existing Models

If you want to fine-tune an existing model, use Unsloth quantized models directly inside the Hugging Face ecosystem. It works natively with the HF Trainer and SFTTrainer classes and supports:

• PEFT / LoRA: For ultra-fast, memory-efficient adapter training.
• GRPO / DPO: For reinforcement learning and training reasoning models.

Changing the Architecture Completely

If you want to modify the model structure itself (e.g., changing layer configurations or adding custom attention blocks), Unsloth will not work. Unsloth is strictly an acceleration patcher for standard, pre-existing models. For complete structural modifications, you should bypass Unsloth entirely and use PyTorch TorchAO (Architecture Optimization).

if you want the custom arch on fp8 completely then i would suggest you use the Pytorch TorchAO