Skip to content

fix: stream compose logs in post-job cleanup - #298

Merged
neilime merged 6 commits into
mainfrom
copilot/fix-docker-compose-logs-issue
Sep 17, 2026
Merged

neilime merged 6 commits into
mainfrom
copilot/fix-docker-compose-logs-issue

Conversation

Copilot AI commented Aug 7, 2026 •

Copy link
Copy Markdown
Contributor

Large docker compose logs output could crash the post-job hook before cleanup completed because log collection buffered the full command output in memory. As a result, docker compose down could be skipped when service logs were very large.

  • Post-job log collection

    • Replace the docker-compose package logs() helper with a direct docker compose logs process spawn.
    • Stream stdout/stderr chunks to the configured service logger instead of assembling a single in-memory string.
    • Remove the extra full-output re-log in post-runner, since logs are now emitted incrementally.
  • Cleanup resilience

    • Treat log collection failures as non-fatal in the post hook.
    • Continue to docker compose down even when log collection exits non-zero, is terminated by signal, or cannot be spawned.
  • Failure reporting

    • Return concise diagnostic messages for abnormal log-command termination without reintroducing output buffering.
    • Handle edge cases where the child process cannot provide output streams.
const childProcess = spawn("docker", commandArgs, { cwd });

childProcess.stdout.on("data", (chunk) => {
  serviceLogger(chunk.toString());
});

childProcess.stderr.on("data", (chunk) => {
  serviceLogger(chunk.toString());
});

Copilot AI linked an issue Aug 7, 2026 that may be closed by this pull request
5 tasks
Copilot AI changed the title [WIP] Fix issue with collecting docker compose logs output fix: stream compose logs in post-job cleanup Aug 7, 2026
Copilot AI requested a review from neilime August 7, 2026 09:28

@neilime neilime left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot handle feedbacks

Copilot AI commented Aug 16, 2026

Copy link
Copy Markdown
Contributor Author

@copilot handle feedbacks

Addressed in 6100198. I restored the explicit docker compose logs: debug message in post-runner without bringing back buffered log aggregation, updated the related test, and revalidated with lint, build, test:ci, package, review, and CodeQL.

Copilot AI requested a review from neilime August 16, 2026 09:51
@neilime
neilime force-pushed the copilot/fix-docker-compose-logs-issue branch from 6100198 to 6f7e04a Compare August 18, 2026 13:03
@github-actions

Copy link
Copy Markdown
Contributor

Super-linter summary

Language Validation result
BIOME_FORMAT Fail ❌
BIOME_LINT Pass ✅
CHECKOV Pass ✅
GITLEAKS Pass ✅
GIT_COMMITLINT Pass ✅
GIT_MERGE_CONFLICT_MARKERS Pass ✅
JSCPD Pass ✅
PRE_COMMIT Pass ✅
SPELL_CODESPELL Fail ❌
TRIVY Pass ✅

Super-linter detected linting errors

For more information, see the GitHub Actions workflow run

Powered by Super-linter

BIOME_FORMAT
Checked 26 files in 179ms. No fixes applied.
Found 2 errors.src/services/docker-compose.service.test.ts format ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

  × Formatter would have printed the following content:

    381 381 │         const logsPromise = service.logs(logsInputs);
    382 382 │
    383     │ - ······expect(spawnMock).toHaveBeenCalledWith("docker",·[
    384     │ - ········"compose",
    385     │ - ········"-f",
    386     │ - ········"docker-compose.yml",
    387     │ - ········"logs",
    388     │ - ········"helloworld2",
    389     │ - ········"helloworld3",
    390     │ - ······],·{
    391     │ - ········cwd:·"/current/working/dir",
    392     │ - ······});
        383 │ + ······expect(spawnMock).toHaveBeenCalledWith(
        384 │ + ········"docker",
        385 │ + ········[
        386 │ + ··········"compose",
        387 │ + ··········"-f",
        388 │ + ··········"docker-compose.yml",
        389 │ + ··········"logs",
        390 │ + ··········"helloworld2",
        391 │ + ··········"helloworld3",
        392 │ + ········],
        393 │ + ········{
        394 │ + ··········cwd:·"/current/working/dir",
        395 │ + ········},
        396 │ + ······);
    393 397 │
    394 398 │         stdout.emit("data", Buffer.from("logs"));
    ······· │
    421 425 │         const logsPromise = service.logs(logsInputs);
    422 426 │
    423     │ - ······expect(spawnMock).toHaveBeenCalledWith("docker",·[
    424     │ - ········"--context",
    425     │ - ········"dev",
    426     │ - ········"compose",
    427     │ - ········"--profile",
    428     │ - ········"ci",
    429     │ - ········"-f",
    430     │ - ········"docker-compose.yml",
    431     │ - ········"logs",
    432     │ - ······],·{
    433     │ - ········cwd:·"/current/working/dir",
    434     │ - ······});
        427 │ + ······expect(spawnMock).toHaveBeenCalledWith(
        428 │ + ········"docker",
        429 │ + ········[
        430 │ + ··········"--context",
        431 │ + ··········"dev",
        432 │ + ··········"compose",
        433 │ + ··········"--profile",
        434 │ + ··········"ci",
        435 │ + ··········"-f",
        436 │ + ··········"docker-compose.yml",
        437 │ + ··········"logs",
        438 │ + ········],
        439 │ + ········{
        440 │ + ··········cwd:·"/current/working/dir",
        441 │ + ········},
        442 │ + ······);
    435 443 │
    436 444 │         childProcess.emit("close", 1);
    ······· │
    517 525 │
    518 526 │         await expect(service.logs(logsInputs)).resolves.toEqual({
    519     │ - ········error:·"Unable·to·collect·docker·compose·logs:·stdout/stderr·unavailable",
        527 │ + ········error:
        528 │ + ··········"Unable·to·collect·docker·compose·logs:·stdout/stderr·unavailable",
    520 529 │           output: "",
    521 530 │         });


src/services/docker-compose.service.ts format ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

  × Formatter would have printed the following content:

     86  86 │           settled = true;
     87  87 │           resolve({
     88     │ - ··········error:·"Unable·to·collect·docker·compose·logs:·stdout/stderr·unavailable",
         88 │ + ··········error:
         89 │ + ············"Unable·to·collect·docker·compose·logs:·stdout/stderr·unavailable",
     89  90 │             output: "",
     90  91 │           });
    ······· │
    104 105 │           settled = true;
    105 106 │           resolve({
    106     │ - ··········error:
    107     │ - ············signal
    108     │ - ··············?·`Docker·Compose·logs·command·failed·with·signal·${signal}`
    109     │ - ··············:·exitCode·!==·null·&&·exitCode·!==·0
    110     │ - ················?·`Docker·Compose·logs·command·failed·with·exit·code·${exitCode}`
    111     │ - ················:·"",
        107 │ + ··········error:·signal
        108 │ + ············?·`Docker·Compose·logs·command·failed·with·signal·${signal}`
        109 │ + ············:·exitCode·!==·null·&&·exitCode·!==·0
        110 │ + ··············?·`Docker·Compose·logs·command·failed·with·exit·code·${exitCode}`
        111 │ + ··············:·"",
    112 112 │             output: "",
    113 113 │           });


format ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

  × Some errors were emitted while running checks.

SPELL_CODESPELL
/github/workspace/dist/index.js:748: ECT ==> ETC
/github/workspace/dist/index.js:790: TE ==> THE, BE, WE, TO
/github/workspace/dist/index.js:12049: contructor ==> constructor
/github/workspace/dist/index.js:19691: addIn ==> adding, add in, add-on
/github/workspace/dist/index.js:19698: addIn ==> adding, add in, add-on
/github/workspace/dist/index.js:21152: alse ==> also, else, false
/github/workspace/dist/index.js:21625: alse ==> also, else, false
/github/workspace/dist/index.js:22267: addIn ==> adding, add in, add-on
/github/workspace/dist/index.js:22269: addIn ==> adding, add in, add-on
/github/workspace/dist/index.js:28705: fpr ==> for, far, fps
/github/workspace/dist/index.js:28712: fpr ==> for, far, fps
/github/workspace/dist/index.js:30336: boolen ==> boolean
/github/workspace/dist/index.js:30421: collapsable ==> collapsible
/github/workspace/dist/index.js:30424: collapsable ==> collapsible
/github/workspace/dist/post.js:748: ECT ==> ETC
/github/workspace/dist/post.js:790: TE ==> THE, BE, WE, TO
/github/workspace/dist/post.js:12049: contructor ==> constructor
/github/workspace/dist/post.js:19691: addIn ==> adding, add in, add-on
/github/workspace/dist/post.js:19698: addIn ==> adding, add in, add-on
/github/workspace/dist/post.js:21152: alse ==> also, else, false
/github/workspace/dist/post.js:21625: alse ==> also, else, false
/github/workspace/dist/post.js:22267: addIn ==> adding, add in, add-on
/github/workspace/dist/post.js:22269: addIn ==> adding, add in, add-on
/github/workspace/dist/post.js:26885: boolen ==> boolean
/github/workspace/dist/post.js:26970: collapsable ==> collapsible
/github/workspace/dist/post.js:26973: collapsable ==> collapsible

@neilime
neilime force-pushed the copilot/fix-docker-compose-logs-issue branch from 6f7e04a to 05978b5 Compare August 18, 2026 13:14
@github-actions

Copy link
Copy Markdown
Contributor

Super-linter summary

Language Validation result
BIOME_FORMAT Pass ✅
BIOME_LINT Pass ✅
CHECKOV Pass ✅
GITLEAKS Pass ✅
GIT_COMMITLINT Pass ✅
GIT_MERGE_CONFLICT_MARKERS Pass ✅
JSCPD Pass ✅
PRE_COMMIT Pass ✅
SPELL_CODESPELL Pass ✅
TRIVY Pass ✅

All files and directories linted successfully

For more information, see the GitHub Actions workflow run

Powered by Super-linter

@codecov

codecov Bot commented Aug 18, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (e56276d) to head (135b46f).
⚠️ Report is 9 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff            @@
##              main      #298   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files            9         9           
  Lines          173       179    +6     
  Branches        37        38    +1     
=========================================
+ Hits           173       179    +6     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copilot AI and others added 2 commits August 18, 2026 15:39
Co-authored-by: neilime <314088+neilime@users.noreply.github.com>
Co-authored-by: neilime <314088+neilime@users.noreply.github.com>
@neilime
neilime force-pushed the copilot/fix-docker-compose-logs-issue branch from 05978b5 to 5e62815 Compare August 18, 2026 13:39
@github-actions

Copy link
Copy Markdown
Contributor

Super-linter summary

Language Validation result
BIOME_FORMAT Pass ✅
BIOME_LINT Pass ✅
CHECKOV Pass ✅
GITLEAKS Pass ✅
GIT_COMMITLINT Pass ✅
GIT_MERGE_CONFLICT_MARKERS Pass ✅
JSCPD Pass ✅
PRE_COMMIT Pass ✅
SPELL_CODESPELL Pass ✅
TRIVY Pass ✅

All files and directories linted successfully

For more information, see the GitHub Actions workflow run

Powered by Super-linter

Co-authored-by: neilime <314088+neilime@users.noreply.github.com>
Signed-off-by: Emilien Escalle <emilien.escalle@escemi.com>
@neilime
neilime force-pushed the copilot/fix-docker-compose-logs-issue branch from 5e62815 to 6f330cd Compare August 18, 2026 14:29
@github-actions

Copy link
Copy Markdown
Contributor

Super-linter summary

Language Validation result
BIOME_FORMAT Pass ✅
BIOME_LINT Pass ✅
CHECKOV Pass ✅
GITLEAKS Pass ✅
GIT_COMMITLINT Pass ✅
GIT_MERGE_CONFLICT_MARKERS Pass ✅
JSCPD Pass ✅
PRE_COMMIT Pass ✅
SPELL_CODESPELL Pass ✅
TRIVY Pass ✅

All files and directories linted successfully

For more information, see the GitHub Actions workflow run

Powered by Super-linter

@neilime neilime left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot handle feedbacks

Comment thread src/services/docker-compose.service.ts
Co-authored-by: neilime <314088+neilime@users.noreply.github.com>

Copilot AI commented Sep 17, 2026

Copy link
Copy Markdown
Contributor Author

@copilot handle feedbacks

Addressed in 6d8e925. The custom log-spawn path was removed in favor of docker-compose v1.5.0 with maxOutputLength: 0, while keeping post-job log collection non-fatal and regenerating the packaged action output.

@neilime neilime left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot handle feedbacks

Comment thread src/services/docker-compose.service.ts Outdated
Comment on lines +74 to +79
output: out,
};
} catch (error) {
return {
error: this.getDockerComposeErrorMessage(error),
output: "",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO, returning output is useless now

Co-authored-by: neilime <314088+neilime@users.noreply.github.com>

Copilot AI commented Sep 17, 2026

Copy link
Copy Markdown
Contributor Author

@copilot handle feedbacks

Addressed in dd035d0. I removed the unused output value from the internal post-job logs API, updated the related tests, and revalidated with lint, build, test:ci, package, secret scanning, and parallel validation.

Copilot AI requested a review from neilime September 17, 2026 16:18
@github-actions github-actions Bot added the type: fix Fixes a bug label Sep 17, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Super-linter summary

Language Validation result
BIOME_FORMAT Pass ✅
BIOME_LINT Pass ✅
CHECKOV Pass ✅
GITLEAKS Pass ✅
GIT_COMMITLINT Pass ✅
GIT_MERGE_CONFLICT_MARKERS Pass ✅
JSCPD Pass ✅
PRE_COMMIT Pass ✅
SPELL_CODESPELL Pass ✅
TRIVY Pass ✅

All files and directories linted successfully

For more information, see the GitHub Actions workflow run

Powered by Super-linter

@neilime
neilime marked this pull request as ready for review September 17, 2026 16:26

@neilime neilime left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot handle code coverage miss issue: #298 (comment)

Co-authored-by: neilime <314088+neilime@users.noreply.github.com>

Copilot AI commented Sep 17, 2026

Copy link
Copy Markdown
Contributor Author

@copilot handle code coverage miss issue: #298 (comment)

Addressed in 135b46f. I added coverage for the non-Error log collection failure path in post-runner and revalidated with lint, build, test:ci, secret scanning, and parallel validation.

@github-actions

Copy link
Copy Markdown
Contributor

Super-linter summary

Language Validation result
BIOME_FORMAT Pass ✅
BIOME_LINT Pass ✅
CHECKOV Pass ✅
GITLEAKS Pass ✅
GIT_COMMITLINT Pass ✅
GIT_MERGE_CONFLICT_MARKERS Pass ✅
JSCPD Pass ✅
PRE_COMMIT Pass ✅
SPELL_CODESPELL Pass ✅
TRIVY Pass ✅

All files and directories linted successfully

For more information, see the GitHub Actions workflow run

Powered by Super-linter

@neilime
neilime merged commit b659c74 into main Sep 17, 2026
41 checks passed
@neilime
neilime deleted the copilot/fix-docker-compose-logs-issue branch September 17, 2026 16:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type: fix Fixes a bug

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Issue with collecting docker compose logs output in post job cleanup

2 participants