fix: Fix CVE-2023-44487 in mmlspark/release Docker image#2520
Conversation
|
Hey @BrendanWalsh 👋! We use semantic commit messages to streamline the release process. Examples of commit messages with semantic prefixes:
To test your commit locally, please follow our guild on building from source. |
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Snapshot WarningsEnsure that dependencies are being submitted on PR branches and consider enabling retry-on-snapshot-warnings. See the documentation for more information and troubleshooting advice. Scanned FilesNone |
There was a problem hiding this comment.
Pull request overview
Updates the Docker build for the mcr.microsoft.com/mmlspark/release image (built from tools/docker/demo/Dockerfile) to remove vulnerable Jetty components by upgrading the base OS and Spark distribution, and to address additional Python dependency CVEs.
Changes:
- Upgrade base image from Ubuntu 20.04 to 22.04.
- Bump Spark from 3.5.0 to 3.5.4 (per PR description, removes Jetty JARs from the Spark distro).
- Update Miniconda installer source/TOS handling and upgrade PyJWT to
>=2.12.0.
Comments suppressed due to low confidence (1)
tools/docker/demo/Dockerfile:33
conda update condais invoked without a non-interactive flag. In Docker builds this commonly prompts for confirmation and can hang/fail. Use-y/--yes(or setalways_yes) to make the build deterministic.
&& conda update conda \
&& conda install -y python=3 jupyter pyspark \
| RUN curl -sSL https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -o /tmp/miniconda.sh \ | ||
| && bash /tmp/miniconda.sh -bfp /usr/local \ | ||
| && rm -rf /tmp/miniconda.sh \ | ||
| && conda config --set auto_activate_base false \ | ||
| && conda tos accept --override-channels --channel https://repo.anaconda.com/pkgs/main \ |
There was a problem hiding this comment.
Miniconda is installed under /usr/local (-p /usr/local), but later the image sets PATH to prefer /opt/conda/bin. Since /opt/conda isn’t created here, this is confusing and makes it easier for future edits to accidentally reference a non-existent conda prefix. Consider aligning the install prefix and PATH (either install to /opt/conda or update PATH to /usr/local/bin).
This issue also appears on line 32 of the same file.
There was a problem hiding this comment.
Fixed in f304569 — changed PATH from /opt/conda/bin to /usr/local/bin to match the install prefix. Also added -y to conda update for non-interactive builds.
| @@ -1,9 +1,9 @@ | |||
| FROM mcr.microsoft.com/oss/mirror/docker.io/library/ubuntu:20.04 | |||
| FROM ubuntu:22.04 | |||
There was a problem hiding this comment.
The base image was switched from the repo’s standard MCR OSS mirror to ubuntu:22.04 from Docker Hub. This can introduce Docker Hub rate-limits/supply-chain variability and is inconsistent with the other Docker image in this repo (e.g., tools/docker/minimal/Dockerfile still uses mcr.microsoft.com/oss/mirror/...). Consider using the MCR mirror for Ubuntu 22.04 and/or pinning the image by digest for reproducible builds.
| FROM ubuntu:22.04 | |
| FROM mcr.microsoft.com/oss/mirror/docker/library/ubuntu:22.04 |
There was a problem hiding this comment.
This was already addressed before the review posted — the image uses mcr.microsoft.com/mirror/docker/library/ubuntu:22.04 (MCR mirror), not Docker Hub. The suggested path mcr.microsoft.com/oss/mirror/docker/library/ doesn't carry 22.04. The tools/docker/minimal/Dockerfile still references the old oss/mirror path with 20.04 but that image hasn't been built or published in years.
f7f6c9f to
f304569
Compare
0bf6a0d to
5ab9127
Compare
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #2520 +/- ##
==========================================
- Coverage 84.61% 80.91% -3.70%
==========================================
Files 335 335
Lines 17708 17708
Branches 1612 1612
==========================================
- Hits 14984 14329 -655
- Misses 2724 3379 +655 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
- Upgrade base from Ubuntu 20.04 to 22.04 (MCR mirror) - Bump Spark from 3.5.0 to 3.5.4 (eliminates vulnerable jetty-io) - Update conda installer URL and add TOS acceptance - Add -y flag to conda update for non-interactive builds - Fix PATH to match conda install prefix (/usr/local) - Upgrade PyJWT >= 2.12.0 (CVE-2026-32597) - Fix minimal Dockerfile with same security updates - Restructure pipeline: build always, push only on release Resolves IcM incident 31000000570827 / MSRC case 110886
5ab9127 to
d9ad6a1
Compare
Summary
Fixes the vulnerable
org.eclipse.jetty:jetty-iopackage in themcr.microsoft.com/mmlspark/releaseDocker image, addressing MSRC case 110886 (IcM incident 31000000570827).Changes
Dockerfile (
tools/docker/demo/Dockerfile):-yflag for non-interactive builds/usr/local/bin)Pipeline (
pipeline.yaml):build-minimalimage build/push steps (unused image, broken Dockerfile, 0 active consumers)Verification
Built and scanned locally with Trivy:
jetty-io < 9.4.53) — resolved (jetty-io no longer present)PyJWT < 2.12.0) — resolvedContext
An external researcher reported that their Jenkins compliance scan fails when building on
mcr.microsoft.com/mmlspark/releasedue to vulnerablejetty-ioversions (9.4.43 and 9.4.50). Spark 3.5.4's binary distribution no longer includes jetty JARs, fully eliminating the vulnerability.The
PublishDockerpipeline job was commented out in June 2024 (PR #2243) and has not run since. Thebuild-minimalimage used a separate broken Dockerfile that nobody maintained — removing it avoids a guaranteed build failure when re-enabling Docker publishing.Publish Steps
After merge:
publishDockerImages: true(no tag) — validates auth and buildsbuild-demov1.1.3) and run again — publishesmmlspark/release