Repository navigation
perf(playwright): bake chromium into the image instead of fetching it per lane - #22
Merged
Merged
Conversation
… per lane The image installed Chromium's ~200 OS-level dependencies but deliberately not the browser binary, leaving every consuming lane to fetch ~170 MB at pipeline time. In enterprise-ui that is paid up to 16 times concurrently, all against the same MinIO host that is simultaneously serving the build artifact and the docker image cache. Measured there: a single pipeline pulls 39.6 GiB from that host, ~70% of it static content that never changes between runs, and the host tops out around 200-250 MiB/s in aggregate — so a transfer that runs at ~100 MiB/s alone collapses to ~22 MiB/s once the fan-out is in flight. The browsers are the textbook case: identical bytes, every run, every lane. Downloading them once at image build time removes that entirely. The bytes ship in a layer the agents already cache. Installed to the default location for this image's user (root, HOME unset, so /root/.cache/ms-playwright). That path is deliberate, not incidental: it is what enterprise-ui's .ci/scripts/restore-playwright-cache.sh and the install guard in .ci/scripts/playwright-setup.sh both probe, so a baked image short-circuits both the MinIO restore and `npx playwright install` with no change required on the consumer side. PLAYWRIGHT_BROWSERS_PATH is intentionally left unset — setting it would move the browsers away from those probes and silently reintroduce the download. PLAYWRIGHT_VERSION is a build arg pinned to 1.57.0 to match the @playwright/test version in enterprise-ui's package-lock.json, because Playwright resolves a browser by revision. If a consumer moves ahead of it the revision will not match and the runtime path falls back to the MinIO cache and then to `npx playwright install` — a slow pipeline, not a broken one, and the signal to rebuild here. install-deps is pinned to the same version so the OS packages and the browser cannot drift apart. The build now asserts the browser directory exists and is non-empty. Without that an empty tree would ship silently: every consumer probe only checks for a non-empty path, so a half-installed tree reads as "baked" and the failure surfaces much later as a missing executable inside a test run. Signed-off-by: slayerjain <shubham@keploy.io>
This was referenced Sep 2, 2026
Merged
slayerjain
added a commit
that referenced
this pull request
Sep 4, 2026
* perf(ci): ship zstd and pigz in the images that compress CI artifacts
The enterprise pipelines move large objects through MinIO on every PR — a
~198 MiB binary fetched by ~59 steps, a ~275 MiB docker save tar fetched by
21, and Go build-cache tarballs that had reached 4.5 GiB (prepare-and-run)
and 6.89 GiB (go-test). All of it goes through single-threaded gzip today,
and that is measurably the slow part rather than the LAN transfer.
Measured on a real 2.1 GB Go build cache, 8 cores:
tar + gzip -1 14.6 s compress 7.8 s decompress 0.54 GiB
tar + zstd -3 -T0 1.3 s compress 1.3 s decompress 0.45 GiB
and on the 197.6 MiB enterprise binary, gzip -6 took 4.0 s where pigz -6 -p8
took 0.6 s for the same output size.
keploy-ci already had zstd; it gains pigz so the existing gzip call sites get
the multi-threaded drop-in without changing format. keploy-ci-slim runs the
download-artifacts steps and keploy-ci-go-build runs both the gate build and
go-test's build, so those are the two that actually pay the compression cost —
they gain both.
No call site changes here: this only makes the tools available so the
consuming pipelines can switch.
Signed-off-by: slayerjain <shubhamkjain@outlook.com>
* perf(playwright): bake kind, kubectl and helm instead of fetching them per lane
Eight enterprise lanes — daemonset-{arm-race,crossapp-contam,
go-timefreeze-nocommand,jsse-execrace,overlay-mark,podcache-arm,
tls-record-replay} and selfhosted-cloud-replay — each pull these three binaries
from the public internet on every run. That is ~73 MiB per lane and ~580 MiB per
PR pipeline, sitting on each lane's critical path, and it makes eight lanes
depend on dl.k8s.io / kind.sigs.k8s.io / get.helm.sh being reachable and fast.
Same shape as the chromium bake in #22/#24. Every one of those lanes already
wraps the fetch in `if ! command -v <tool>`, so this needs no pipeline change to
take effect and degrades safely — a lane running against an older image just
downloads them as it does today.
Versions match what the lanes pin (kind v0.24.0, kubectl v1.31.0, helm v3.16.4),
so behaviour is unchanged; verified by building this layer on
playwright-1.2.25 and checking all three report those exact versions. They are
build ARGs so the pair can be moved together when either side bumps.
Signed-off-by: slayerjain <shubhamkjain@outlook.com>
---------
Signed-off-by: slayerjain <shubhamkjain@outlook.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
keploy-ci-playwrightinstalled Chromium's ~200 OS-level dependencies but deliberately not the browser binary — the header comment said so explicitly: "Node modules and the Playwright browser binary are cached via MinIO at pipeline runtime."This bakes the browser in.
Why
Every consuming lane fetched ~170 MB at pipeline time. In
enterprise-uithat is paid up to 16 times concurrently, all against the same MinIO host that is simultaneously serving the build artifact and the docker image cache.Measured on that repo:
The browsers are the textbook case for baking: identical bytes, every run, every lane. Downloading once at image build time removes the pipeline-time cost entirely — the bytes ship in a layer the agents already cache.
How it works on the consumer side — no changes needed there
Installed to the default location for this image's user (root,
HOMEunset →/root/.cache/ms-playwright). That path is deliberate: it is exactly whatenterprise-ui's.ci/scripts/restore-playwright-cache.shand the install guard in.ci/scripts/playwright-setup.shalready probe. A baked image therefore short-circuits both the MinIO restore andnpx playwright installwith no consumer change.PLAYWRIGHT_BROWSERS_PATHis intentionally left unset — setting it would move the browsers away from those probes and silently reintroduce the download.Version coupling
PLAYWRIGHT_VERSIONis a build arg pinned to 1.57.0, matching@playwright/testinenterprise-ui'spackage-lock.json, because Playwright resolves a browser by revision.install-depsis pinned to the same version so the OS packages and the browser cannot drift apart.If a consumer moves ahead of this pin the revision will not match and the runtime path falls back to the MinIO cache, then to
npx playwright install— a slow pipeline, not a broken one, and the signal to rebuild this image.Safety
The build now asserts the browser directory exists and is non-empty. Without that an empty tree would ship silently: every consumer probe only checks for a non-empty path, so a half-installed tree reads as "baked" and the failure surfaces much later as a missing-executable error inside a test run.
Follow-up (not in this PR)
Once this ships as
playwright-1.2.33,enterprise-uirepins its 22playwright-*image references. Repinning before the image is verified would break CI, so it is deliberately separate.Testing
RUNblock extracted andsh -nclean.HOMEconfirmed from the published registry config (Userunset → root,HOMEunset →/root).save-playwright-cache.shtars~/.cache/ms-playwrightafternpx playwright installin this same image family.