Skip to content

Unskip integ tests mlops#5887

Merged
lucasjia-aws merged 2 commits into
aws:masterfrom
lucasjia-aws:unskip-integ-tests-mlops
May 21, 2026
Merged

Unskip integ tests mlops#5887
lucasjia-aws merged 2 commits into
aws:masterfrom
lucasjia-aws:unskip-integ-tests-mlops

Conversation

@lucasjia-aws
Copy link
Copy Markdown
Collaborator

@lucasjia-aws lucasjia-aws commented May 21, 2026

Summary

5 integ tests in sagemaker-mlops were conditionally skipped via @pytest.mark.skipif because they require SageMaker Spark container images, which only support Python 3.9 and 3.12. Since CI runs integ tests on Python 3.10, these tests were never executed.

This PR introduces a spark_py312 marker to split mlops integ tests into two groups:

  • py310: runs all integ tests excluding spark_py312
  • py312: runs only spark_py312 marked tests (Spark/Feature Processor tests that depend on SageMaker Spark images)

This allows both groups to run in CI without version conflicts. The corresponding buildspec change to add the py312 step will be done separately in the CI infra (SageMakerMLFPySDKInfraCDK).

Changes

  • Register spark_py312 marker in sagemaker-mlops/tox.ini
  • Replace @pytest.mark.skipif with @pytest.mark.spark_py312 on 4 tests in test_feature_processor_integ.py
  • Add @pytest.mark.spark_py312 to 1 test in test_feature_processor_spark_compat.py
  • test_to_pipeline_and_execute_with_lake_formation remains skipped (@pytest.mark.skip) due to unresolved Lake Formation environment configuration in CI (tracked separately)

Testing

Validated via CodeBuild in Prod:

  • py310 with -m "not spark_py312": all tests pass, spark tests correctly excluded
  • py312 with -m spark_py312: 4/5 tests pass, 1 skipped (Lake Formation)

@lucasjia-aws lucasjia-aws merged commit d18005d into aws:master May 21, 2026
11 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants