fix(pytorch): load saved models with weights_only=False (#5365)#5630
Open
Bharath-970 wants to merge 1 commit into
Open
fix(pytorch): load saved models with weights_only=False (#5365)#5630Bharath-970 wants to merge 1 commit into
Bharath-970 wants to merge 1 commit into
Conversation
`save_model` serializes the whole model object via `torch.load`'s pickle path, so it must be loaded with `weights_only=False`. PyTorch >= 2.6 flipped the default to `weights_only=True`, which cannot unpickle arbitrary classes and breaks loading every model saved with `bentoml.pytorch` (both direct `load_model` and the runner path, which has no way to pass torch load args). The model store is a trusted, BentoML-produced artifact, so default `weights_only` to False via `setdefault` while still allowing callers to override it through `torch_load_args`. Add a regression test.
Author
|
gentle bump on this one. it bites anyone on torch 2.6+ since the weights_only default flipped — every model saved via bentoml.pytorch fails to load with the UnpicklingError. fix is a one-liner (setdefault so callers can still override) plus a regression test, CI's green. @frostming @aarnphm mind taking a look when you get a sec? happy to tweak if you'd rather gate it differently. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes loading of models saved with the
bentoml.pytorchframework on PyTorch >= 2.6.Problem
bentoml.pytorch.save_modelserializes the whole model object (not a state dict) viatorch.save(model, file, pickle_module=cloudpickle). Loading such a file requirestorch.load(..., weights_only=False).PyTorch 2.6 changed the default of
torch.loadfromweights_only=Falsetoweights_only=True.weights_only=Truecannot unpickle arbitrary classes, so loading any model saved withbentoml.pytorchnow fails:This breaks both
bentoml.pytorch.load_model(...)and the legacy runner serving path (_internal/frameworks/common/pytorch.py), which callsload_modelwith no way for the user to passtorch_load_args. Reported in #5365.Fix
Default
weights_onlytoFalseinload_modelviasetdefault, since the model store is a trusted, BentoML-produced artifact (consistent with torch's guidance to only disableweights_onlyfor trusted sources). Callers can still override it through**torch_load_args.Only
pytorch.pyis affected;pytorch_lightningandtorchscriptusetorch.jit.load, which is unaffected.Test
Adds
test_load_model_defaults_to_weights_only_false(isolated temp model store):nn.Modulesucceeds by defaultweights_only=Trueoverride is still honoredVerified as a true regression guard: the test fails without the fix (
UnpicklingError) and passes with it. Full file:Closes #5365.