Cem maf vuln - #208
Closed
rluss wants to merge 6 commits into
Closed
Conversation
…nloader
Addresses two security advisories against the CEM_MAF explainer:
* CVSS 9.2 (CWE-502) — pickle.load + exec(build_module_src) RCE in the
Karras PGG celebA-HQ GAN loader.
* CVSS 8.5 (CWE-22 + CWE-502) — path-traversal / pickle deserialization
in the unreachable load_AE helper.
Changes:
- GAN loader (CEM_MAF_aen_PN.py): replace pickle.load + exec of the
embedded build_module_src with a safetensors weights file plus a JSON
static-kwargs sidecar. The trusted G_paper build function still comes
from progressive_growing_of_gans/networks.py; no untrusted source is
exec'd at runtime.
- load_AE (CEM_MAF_utils.py): deleted. No caller reachable in-tree.
- Downloader (dwnld_CEM_MAF_celebA.py): rewrite to fetch from two
Hugging Face repos over HTTPS with SHA-256 verification. 78 files
pinned; every download streams to <dest>.part, hashes on the fly, and
is os.replace()-moved into place only on match. Unknown names are
rejected. Path traversal is impossible: filenames come from the
caller-provided list, dest paths use os.path.join with basename-only
components.
- attr_classifier_device / gan_device kwargs (CEM_MAF.py,
CEM_MAF_aen_PN.py, CEM_MAF_aen_PP.py): pin the Karras GAN and the
per-attribute classifiers to /cpu:0 on Ampere/Hopper to dodge a
TF1.15 + bundled-cuBLAS-10 SGEMM bug that NaN-corrupts / silently
non-determinises the forward pass on A100/H100. Default None
preserves pre-existing behavior on pre-Ampere GPUs.
- Notebook (CEM-MAF-CelebA.ipynb): remove manual Google-Drive-pickle
setup instructions; fetch the GAN artifacts via dwnld_celebA_gan()
alongside the other automatic downloads; pass '/cpu:0' for both
device knobs.
- .gitignore: exclude runtime-fetched model + data files (they now come
from Hugging Face on demand) and other local-only artifacts.
Verified end-to-end on a fresh working tree with all files re-downloaded
from Hugging Face: notebook produces Added High_Cheekbones (class 4→6)
for image 15, matching pre-rewrite results bit-for-bit.
Signed-off-by: Ronny Luss <rluss@us.ibm.com>
The Karras et al. 2018 progressive-growing GAN weights bundled via Hugging Face are released under CC-BY-NC 4.0, distinct from AIX360's Apache-2.0 license. Add the full CC-BY-NC 4.0 text under `supplementary license/`, note the attribution in the licenses README, and mention it in the CEM-MAF-CelebA notebook where users encounter the GAN. Signed-off-by: Ronny Luss <rluss@us.ibm.com>
The CEM_MAF vulnerability fix (d072a36) replaced pickle.load of the Karras GAN with safetensors, but did not add safetensors to setup.py's contrastive extras. Pin <0.4 so Py3.6 wheels remain available under the tensorflow==1.14 / keras==2.3.1 environment. Signed-off-by: Ronny Luss <rluss@us.ibm.com>
The CEM_MAF vulnerability fix (d072a36) .gitignore'd the whole progressive_growing_of_gans/ shadow directory as "Karras PGG upstream (Google Drive pickle, no longer used at runtime)". That was too broad: CEM_MAF_aen_PN.py top-level imports G_paper from networks.py inside that directory, so a fresh clone would ImportError. Copy the code CEM_MAF actually needs -- G_paper plus the twelve helpers it transitively calls -- into a new gan_networks.py sibling module. Swap the import to match. The rest of the upstream directory (D_paper discriminator, NVIDIA training scripts, sample images, .pkl weights) stays untracked; those are neither imported nor safe to distribute. The vendored code is licensed CC-BY-NC 4.0; attribution and modification notes are in the file header, and the full license text was already committed under supplementary license/LICENSE-CC.txt in 5b286a7. Signed-off-by: Ronny Luss <rluss@us.ibm.com>
The safetensors dependency added in 15a1e26 has no Py3.6 wheels and its sdist requires PyO3 >= 3.7, breaking pip install .[contrastive] on the CEM CI job. Bump the job to Python 3.7: - .github/workflows/Build.yml: rename build-cem-on-py36 to build-cem-on-py37 and set python-version to ["3.7"]. - README.md: update the contrastive row's Python column to 3.7. profwt and shap rows keep 3.6 -- their extras don't include safetensors, so nothing forces a bump there. Also pin protobuf<3.20 in the contrastive extras. On Py3.6 pip happened to resolve to protobuf 3.19.x (last cp36 wheel) which TF 1.14 tolerates; on Py3.7 pip resolves to protobuf 4.x, whose descriptor API refactor makes TF 1.14's generated _pb2.py files refuse to import. The pin makes the resolution explicit rather than accidental. Correct the safetensors comment while we're here: the <0.4 bound is about 0.3.x being the last series with Py3.7 wheels, not about Py3.6. Signed-off-by: Ronny Luss <rluss@us.ibm.com>
http://yann.lecun.com/exdb/mnist/ has returned 404 for the raw .gz files since ~2022, silently breaking any test or notebook that constructs MNISTDataset() without pre-populated files. Surfaced on the CEM CI job now that its install path resolves cleanly and the test actually runs to the download step. Switch to https://ossci-datasets.s3.amazonaws.com/mnist/, the S3 bucket the PyTorch team stood up as the community successor mirror after LeCun's copy died. Serves byte-identical files at the same paths; also upgrades the transport from http to https. Signed-off-by: Ronny Luss <rluss@us.ibm.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixing links in README.md