The size of .npz weight file is twice larger than .pth file?

[Lance729]

I use convert.py to convert llama's pth file into npz file, but the size goes from 13GB to 26GB.

I don't know if it is correct.

[shayanjl]

A 2x increase from 13 GB to 26 GB usually means the converted weights were saved in a larger dtype, most likely float32 instead of float16.

So this may still be correct.

The main things to check are:

the --dtype used during conversion
whether you converted to a non-quantized MLX model
whether the original .pth and output .npz are being stored in different formats

If the output was saved as float32, then the size doubling is expected. If you want a smaller file, try converting with --dtype float16 or using quantization.

[yentur]

It is dtype, and you can pin it down exactly rather than guessing.

.npz written by mx.savez is an uncompressed zip of .npy members, so the file is just param_count * itemsize plus a few hundred bytes of header. A 4M-element array on mlx 0.32.0:

     dtype   file bytes  bytes/param  roundtrip dtype
   float32     16000210       4.0001 mlx.core.float32
   float16      8000210       2.0001 mlx.core.float16
  bfloat16      8000210       2.0001 mlx.core.bfloat16
zip member: w.npy compress_type 0 raw 8000082 stored 8000082

compress_type 0 is ZIP_STORED, and bfloat16 does survive the round trip at 2 bytes. So there is no packing or compression effect available to explain a 2x. 13 GB to 26 GB is 2 bytes/param becoming 4 bytes/param: an fp16 source written out as fp32.

In that script the output dtype is decided entirely by --dtype, because torch_to_mx casts unconditionally (llms/llama/convert.py:19-22):

def torch_to_mx(a: torch.Tensor, *, dtype: str) -> mx.array:
    # bfloat16 is not numpy convertible. Upcast to float32 to avoid precision loss
    a = a.to(torch.float32) if dtype == "bfloat16" else a.to(getattr(torch, dtype))
    return mx.array(a.numpy(), getattr(mx, dtype))

The default is float16 (line 211), so a doubled file means --dtype float32 was passed. I ran the actual script against a synthetic 67.8M-param llama checkpoint saved as fp16:

params    : 67,773,472
pth dtype : torch.float16
pth bytes : 135,560,091  (2.000 bytes/param)

135560091  consolidated.00.pth
135556764  weights.npz  (default)          1.00x
271103708  weights.npz  (--dtype float32)  2.00x

Check your own file without re-converting:

import mlx.core as mx
print({str(v.dtype) for v in mx.load("weights.npz").values()})

Two notes if you are hitting this now rather than back then. llms/llama/convert.py is the old mlx-examples path, and mlx-lm replaced it: it does not write .npz at all, it writes safetensors (mlx_lm/utils.py:756, mx.save_safetensors). And mlx_lm.convert's --dtype defaults to config.json's torch_dtype instead of a hardcoded value (mlx_lm/convert.py:129-135), so the source precision is kept rather than upcast:

source (HF)            mlx.core.bfloat16   269,060,552 bytes  2.000 B/param
mlx default            mlx.core.bfloat16   269,060,381 bytes  2.000 B/param
mlx --dtype float32    mlx.core.float32    538,090,259 bytes  4.000 B/param

SmolLM2-135M, mlx-lm 0.31.3.