Skip to content

fix(pytorch): load saved models with weights_only=False (#5365)#5630

Open
Bharath-970 wants to merge 1 commit into
bentoml:mainfrom
Bharath-970:fix/pytorch-load-weights-only
Open

fix(pytorch): load saved models with weights_only=False (#5365)#5630
Bharath-970 wants to merge 1 commit into
bentoml:mainfrom
Bharath-970:fix/pytorch-load-weights-only

Conversation

@Bharath-970

Copy link
Copy Markdown

Fixes loading of models saved with the bentoml.pytorch framework on PyTorch >= 2.6.

Problem

bentoml.pytorch.save_model serializes the whole model object (not a state dict) via torch.save(model, file, pickle_module=cloudpickle). Loading such a file requires torch.load(..., weights_only=False).

PyTorch 2.6 changed the default of torch.load from weights_only=False to weights_only=True. weights_only=True cannot unpickle arbitrary classes, so loading any model saved with bentoml.pytorch now fails:

_pickle.UnpicklingError: Weights only load failed. ...
WeightsUnpickler error: Unsupported global: GLOBAL <YourModel> was not an allowed global by default.

This breaks both bentoml.pytorch.load_model(...) and the legacy runner serving path (_internal/frameworks/common/pytorch.py), which calls load_model with no way for the user to pass torch_load_args. Reported in #5365.

Fix

Default weights_only to False in load_model via setdefault, since the model store is a trusted, BentoML-produced artifact (consistent with torch's guidance to only disable weights_only for trusted sources). Callers can still override it through **torch_load_args.

torch_load_args.setdefault("weights_only", False)
model = torch.load(file, map_location=device_id, **torch_load_args)

Only pytorch.py is affected; pytorch_lightning and torchscript use torch.jit.load, which is unaffected.

Test

Adds test_load_model_defaults_to_weights_only_false (isolated temp model store):

  • saving + loading a custom nn.Module succeeds by default
  • an explicit weights_only=True override is still honored

Verified as a true regression guard: the test fails without the fix (UnpicklingError) and passes with it. Full file:

tests/integration/frameworks/test_pytorch_unit.py ...  3 passed

Closes #5365.

`save_model` serializes the whole model object via `torch.load`'s pickle
path, so it must be loaded with `weights_only=False`. PyTorch >= 2.6 flipped
the default to `weights_only=True`, which cannot unpickle arbitrary classes
and breaks loading every model saved with `bentoml.pytorch` (both direct
`load_model` and the runner path, which has no way to pass torch load args).

The model store is a trusted, BentoML-produced artifact, so default
`weights_only` to False via `setdefault` while still allowing callers to
override it through `torch_load_args`. Add a regression test.
@Bharath-970 Bharath-970 requested a review from a team as a code owner June 2, 2026 11:07
@Bharath-970 Bharath-970 requested review from bojiang and removed request for a team June 2, 2026 11:07
@Bharath-970

Copy link
Copy Markdown
Author

gentle bump on this one. it bites anyone on torch 2.6+ since the weights_only default flipped — every model saved via bentoml.pytorch fails to load with the UnpicklingError. fix is a one-liner (setdefault so callers can still override) plus a regression test, CI's green.

@frostming @aarnphm mind taking a look when you get a sec? happy to tweak if you'd rather gate it differently.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: Bentoml Pytorch model serve bug

1 participant