wip

[haoyangli0109]

1. Currently, Atom does not support loading or inference for minimax-m3-mxfp8; this PR only implements online quantization for minimax-m3-mxfp8.
2. mxfp4 and ptpc_fp8 quantized inference both run smoothly, but ptpc_fp8 inference encounters a kernel error in aiter.

HIP_VISIBLE_DEVICES=4,5,6,7 \
python -m atom.entrypoints.openai_server  --model /shareddata/MiniMaxAI/MiniMax-M3-MXFP8   -tp 4   --trust-remote-code   --block-size 128   --server-port 01345   --online_quant_config '{"global_quant_config": "ptpc_fp8", "layer_quant_config": {"*expert*": "mxfp4"}, "exclude_layer": ["lm_head", "model.embed_tokens", "vision_tower", "multi_modal_projector", "patch_merge_mlp", "*block_sparse_moe.gate"]}'


lm_eval --model local-completions \
  --model_args "model=/shareddata/MiniMaxAI/MiniMax-M3-MXFP8,base_url=http://localhost:01345/v1/completions,num_concurrent=65,max_retries=3,tokenized_requests=False,trust_remote_code=True" \
  --tasks gsm8k \
  --num_fewshot 5

Tasks	Version	Filter	n-shot	Metric		Value		Stderr
gsm8k	3	flexible-extract	5	exact_match	↑	0.8810	±	0.0089
		strict-match	5	exact_match	↑	0.8787	±	0.0090