Ship ag_args as JSON instead of pickle - #263
Merged
Merged
Conversation
The remote-training config (predictor init/fit args) was serialized with pickle and unpickled inside the SageMaker/Ray container, so a tampered ag_args file would be arbitrary-code execution on load. Switch to JSON. Non-JSON-serializable inputs (search spaces, custom metric objects, classes/callables in hyperparameters) now raise an informative error that names the offending user argument. A future 'extra' cloudpickle channel can carry such objects opt-in. - backend.dumps_ag_args(): serialize-or-raise helper, used by prepare_args - flip the three read sites to json.load; rename ag_args.pkl -> ag_args.json - add unit tests for the serialization error paths
- dumps_ag_args: reuse the original TypeError on the fallback path instead of re-serializing the config just to re-raise. - Raise an informative error when image_column is passed for a tabular predictor; tabular image support requires autogluon.multimodal, which is being deprecated. MultiModal predictors (string image_column) are unaffected.
test_tabular_tabular_text_image trained a tabular predictor with an image column (AG_IMAGE_NN/AG_TEXT_NN + a live FeatureMetadata object) — a workflow that requires autogluon.multimodal and no longer works over JSON ag_args. Drop it along with the now-deprecated image_column docstring entry.
|
Job PR-263-cf99774 is done. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The remote-training config (predictor init/fit args) was serialized with
pickleand unpickled inside the SageMaker/Ray container, so a tamperedag_argsfile would be arbitrary-code execution on load. This switches the on-the-wire format to JSON.Non-JSON-serializable inputs (search spaces, custom metric objects, classes/callables in
hyperparameters) now raise an informative error that names the offending user argument. A futureextracloudpickle channel can carry such objects opt-in.Changes
backend.dumps_ag_args(): serialize-or-raise helper, used byprepare_argsjson.load; renameag_args.pkl→ag_args.json