Skip to content

Add roadmap plan for long-running pipeline logging#8

Draft
Copilot wants to merge 2 commits into
mainfrom
copilot/add-more-extensive-logging
Draft

Add roadmap plan for long-running pipeline logging#8
Copilot wants to merge 2 commits into
mainfrom
copilot/add-more-extensive-logging

Conversation

Copilot AI commented Jun 14, 2026

Copy link
Copy Markdown

This change documents a concrete plan for improving observability in long-running graphify operations. It focuses on the current gaps in pipeline and watch-mode progress reporting so future implementation work has clear scope and success criteria.

  • Roadmap addition

    • Added a new ROADMAP.md entry: 3.4a Long-Running Process Observability & Logging
    • Positions the work next to incremental/watch-mode improvements, where the logging need is most visible
  • What the plan covers

    • stage-level timing for detect/extract/build/cluster/analyze/export
    • heartbeat-style progress updates during long extraction loops
    • throughput signals such as files/sec and extraction volume
    • watch-mode diagnostics for debounce, re-detect, merge, cluster, and export phases
    • warning/failure summaries that stay concise by default and expand under --verbose
  • Implementation shape

    • outlines a lightweight timing approach around existing stage boundaries
    • proposes a shared progress-reporting abstraction for PipelineRunner and WatchMode
    • keeps the design compatible with a later move to ILogger / telemetry-backed diagnostics
  • Rollout guidance

    • defines a phased rollout:
      1. add elapsed-time summaries
      2. add periodic heartbeats
      3. centralize structured diagnostic fields

Example target output shape from the documented plan:

[2/6] Extracting code structure...
      412/1204 files processed (34%), 18 skipped, 27.4 files/sec, elapsed 00:00:15

[2b/6] Running AI-enhanced semantic extraction...
      21/87 files processed, 3 warnings, elapsed 00:01:42

[watch] 6 file(s) changed
        debounce: 500ms | extract: 1.2s | merge+cluster: 340ms | export: 220ms

Copilot AI changed the title [WIP] Add more extensive logging for long running processes Add roadmap plan for long-running pipeline logging Jun 14, 2026
Copilot AI requested a review from elbruno June 14, 2026 15:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants