Skip to content

Add verbose Learning Mode diagnostics - #1083

Open
Richie Gomez (richiemsft) wants to merge 9 commits into
user/ramonarjona4/telemetry-05-language-sdksfrom
feature/verboseEventTelemetry
Open

Add verbose Learning Mode diagnostics#1083
Richie Gomez (richiemsft) wants to merge 9 commits into
user/ramonarjona4/telemetry-05-language-sdksfrom
feature/verboseEventTelemetry

Conversation

@richiemsft

@richiemsft Richie Gomez (richiemsft) commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

📖 Description

Adds bounded, sanitized verbose Learning Mode diagnostics alongside the actionable denial artifact. Non-actionable and unsupported events retain a closed exclusion reason for investigation without changing policy recommendations, and telemetry reports verbose-artifact production and ETW write failures.

Also applies the privacy-reviewed Microsoft Products consent disclosure and bumps the consent resource version to require renewed consent for the material wording change.

This PR is stacked on #822.

🔗 References

Related PRs:

🔍 Validation

  • cargo test -p wxc_common telemetry::consent_prompt
  • Node.js telemetry and default consent protocol unit tests
  • .NET MxcTelemetryTests (51 tests)

✅ Checklist

📋 Issue Type

  • Bug fix
  • Feature
  • Task

GitHub Actions runs the PR validation build automatically. The ADO pipeline
(MXC-PR-Build) is the Azure version of the PR pipeline, kept in parity with the GitHub
Actions build; it runs on merge to main, and Microsoft reviewers with write access can trigger it
on a PR with /azp run. See docs/pull-requests.md.

If the dependency-feed-check check fails on a new dependency, the crate must be added to
the feed before the PR can pass. See docs/pull-requests.md
for the steps.

Microsoft Reviewers: Open in CodeFlow

* Unify audit with capture denial routing

Route wxc-exec --audit through captureDenials so native PSEC/V2 remains preferred and guarded WPR provides compatible analysis and ETL retention. Reuse canonical denials for adjusted policy generation and support retainEtl consistently across both capture providers.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 4eff3ee2-323c-494d-99b9-f7b54e495216

* Document unified audit architecture

Update repository guidance for captureDenials-backed audit routing and guarded retainEtl parity.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 4eff3ee2-323c-494d-99b9-f7b54e495216

* Address audit capture review findings

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 4eff3ee2-323c-494d-99b9-f7b54e495216

* Filter guarded WPR traces by process

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 4eff3ee2-323c-494d-99b9-f7b54e495216

* Document signed PLM developer loop

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 4eff3ee2-323c-494d-99b9-f7b54e495216

* Document local PLM signing workflow

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 4eff3ee2-323c-494d-99b9-f7b54e495216

* Link Windows SDK signing tools

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 4eff3ee2-323c-494d-99b9-f7b54e495216

* Address retained trace review findings

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 4eff3ee2-323c-494d-99b9-f7b54e495216

* Keep interactive traces elevated

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 4eff3ee2-323c-494d-99b9-f7b54e495216

* Add Data Loop denial diagnostics

Emit a bounded, username-redacted Data Loop JSON sibling containing canonical denial occurrences and classified decoder outcomes. Keep paired output transactional across native, guarded, and audit capture paths.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 4eff3ee2-323c-494d-99b9-f7b54e495216

* Preserve Learning Mode Data Loop exclusions

Keep excluded namespace evidence uniquely identifiable, retain guarded-trace unknown schemas and brokered capability events, and emit canonical/Data Loop outputs as a transactional pair.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: acb632fc-7b75-4d0e-9be5-f11de51652bf

* Harden guarded decoding and paired rollback

Use targeted broker PID decoding without aborting on unrelated malformed payloads, and make rollback ownership-aware, portable, and no-clobber.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: acb632fc-7b75-4d0e-9be5-f11de51652bf

* Close Learning Mode review gaps

Preserve malformed scoped capability events, retain uncached TDH schemas after the cache cap, and make rollback recovery no-clobber and deletion tolerant.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: acb632fc-7b75-4d0e-9be5-f11de51652bf

* Verify promoted output contents during rollback

Detect in-place overwrites before deleting a promoted output and directly exercise TDH short-circuit and cache-capacity behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: acb632fc-7b75-4d0e-9be5-f11de51652bf

* Complete paired output rollback after cleanup errors

Restore backups even when quarantined output cleanup fails, preserve both errors, and hash promoted files from a known offset.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: acb632fc-7b75-4d0e-9be5-f11de51652bf

* Preserve workload PID on partial capability decode

Recover the brokered payload ProcessId when full event decoding fails after that property, and drop only events that remain unscopable.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: acb632fc-7b75-4d0e-9be5-f11de51652bf

* Preserve rollback diagnostics and PID fallback coverage

Surface quarantine restoration failures with accurate rollback context and codify malformed capability PID fallback scoping.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: acb632fc-7b75-4d0e-9be5-f11de51652bf

* Add path context to paired rollback errors

Identify backup, quarantine, and destination paths in cleanup and restoration failures while preserving strict error propagation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: acb632fc-7b75-4d0e-9be5-f11de51652bf

* Redact complete file paths from Data Loop output

Keep canonical denial resources actionable while replacing file paths in the diagnostic sibling JSON with <REDACTED>.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f9ac49b1-fdfe-4cca-9412-9faa99cef2ab

* Make audit copy fallback race-safe

Acquire fallback destinations with create_new and clean up only files owned by the current relocation attempt.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f9ac49b1-fdfe-4cca-9412-9faa99cef2ab

* Redact path-suffixed Data Loop properties

Treat AppPath, ApplicationPath, and other path-suffixed fields as complete file paths before diagnostic bounding or hashing.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f9ac49b1-fdfe-4cca-9412-9faa99cef2ab

* Make audit artifact relocation transactional

Relocate canonical and Data Loop JSON as an ownership-verified pair and stage ETL copy fallbacks through private no-clobber temporary files.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f9ac49b1-fdfe-4cca-9412-9faa99cef2ab

* Address adversarial review findings

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f9ac49b1-fdfe-4cca-9412-9faa99cef2ab

* Redact absolute paths for non-file events

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f9ac49b1-fdfe-4cca-9412-9faa99cef2ab

* Redact NT filesystem paths in diagnostics

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f9ac49b1-fdfe-4cca-9412-9faa99cef2ab

* Preserve signatures when size fit fails

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f9ac49b1-fdfe-4cca-9412-9faa99cef2ab

* Distinguish DOS paths from device identifiers

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f9ac49b1-fdfe-4cca-9412-9faa99cef2ab

* Decode additional learning mode denial shapes

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f9ac49b1-fdfe-4cca-9412-9faa99cef2ab

* Rename Data Loop to verbose logging

Replace the public and internal Data Loop terminology with verbose logging, including the serialized analysis field and sibling artifact filename.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f9ac49b1-fdfe-4cca-9412-9faa99cef2ab

* Use verbose logging terminology

Replace Data Loop names in APIs, transport fields, artifact paths, diagnostics, tests, and documentation with verbose logging terminology.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f9ac49b1-fdfe-4cca-9412-9faa99cef2ab

* Align fixture event accounting

Route decoded test fixtures directly through the production handler so scope filtering and processing limits use the same single accounting path.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f9ac49b1-fdfe-4cca-9412-9faa99cef2ab

* Harden verbose output relocation

Avoid per-run create-new lock artifacts, preserve committed relocation metadata while surfacing source-cleanup warnings, classify malformed event 28 records by schema name, and update the documented output architecture.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f9ac49b1-fdfe-4cca-9412-9faa99cef2ab

* Address Learning Mode review feedback

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f9ac49b1-fdfe-4cca-9412-9faa99cef2ab

* Harden output backup ownership

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f9ac49b1-fdfe-4cca-9412-9faa99cef2ab

---------

Copilot-Session: 4eff3ee2-323c-494d-99b9-f7b54e495216
Copilot-Session: acb632fc-7b75-4d0e-9be5-f11de51652bf
Copilot-Session: f9ac49b1-fdfe-4cca-9412-9faa99cef2ab
* Keep non-actionable denials in verbose output

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f9ac49b1-fdfe-4cca-9412-9faa99cef2ab

* Address actionable output review comments

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f9ac49b1-fdfe-4cca-9412-9faa99cef2ab

* Fail closed on unknown registry access

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f9ac49b1-fdfe-4cca-9412-9faa99cef2ab

---------

Copilot-Session: f9ac49b1-fdfe-4cca-9412-9faa99cef2ab
Emit sanitized verbose capture artifacts as bounded, independently parseable MXC.Verbose chunks across executor and SDK completion paths. Version the consent disclosure and document the reconstruction and privacy contract.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f9ac49b1-fdfe-4cca-9412-9faa99cef2ab
Propagate EventWriteTransfer status codes through the verbose telemetry path so rejected chunks produce a bounded local diagnostic instead of failing silently.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9e32b4c9-eb47-4231-91de-0da5fde17ab7
Use MXC.VerboseDenials as the ETW identity so the event's Learning Mode denial payload is clear to telemetry consumers.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9e32b4c9-eb47-4231-91de-0da5fde17ab7
Apply the privacy-reviewed Microsoft Products disclosure and bump the consent resource version so existing grants require renewed consent.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ba4f8739-616b-4b42-8eba-24a762602400
@richiemsft
Richie Gomez (richiemsft) requested a review from a team as a code owner September 1, 2026 22:36
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The SDK path regresses non-blocking polling and can suppress or silently discard verbose telemetry failures.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds bounded, sanitized Learning Mode diagnostics, publishes them beside actionable denials, and transports them through consent-gated telemetry. It also updates telemetry consent wording and renews consent via resource version 3.

Changes:

  • Adds verbose denial aggregation, paired artifact handling, and ETW chunking.
  • Extends Windows ETL decoding, filtering, and capture finalization.
  • Updates consent resources, SDK documentation, and tests.
File summaries
File Description
tests/scripts/run_telemetry_etw_smoke_test.ps1 Updates consent expectations.
tests/scripts/run_telemetry_consent_smoke_test.ps1 Updates consent smoke assertions.
src/testing/wxc_e2e_tests/tests/e2e_telemetry_etw.rs Seeds consent version 3.
src/mxc_telemetry/src/lib.rs Adds the verbose-denials ETW event.
src/host/plm/src/stop.rs Handles verbose-output path collisions.
src/host/plm/src/log.rs Updates analysis fixture.
src/host/plm/src/elevated.rs Updates actionable terminology.
src/host/plm/src/analysis.rs Writes paired denial artifacts.
src/host/plm/readme.md Updates PLM guidance.
src/core/wxc/src/main.rs Emits verbose telemetry and cleanup warnings.
src/core/wxc/src/audit.rs Relocates and validates artifact pairs.
src/core/wxc_common/src/telemetry/mod.rs Adds authorized verbose emission.
src/core/wxc_common/src/telemetry/events.rs Defines verbose telemetry records.
src/core/wxc_common/src/telemetry/consent.rs Uses the canonical resource version.
src/core/wxc_common/src/telemetry/consent_prompt.rs Updates consent prompt tests.
src/core/wxc_common/resources/telemetry/consent/en-US.json Introduces consent resource version 3.
src/core/mxc_engine/src/verbose_telemetry.rs Validates, chunks, and emits verbose documents.
src/core/mxc_engine/src/run.rs Adds audit runner resolution.
src/core/mxc_engine/src/lib.rs Integrates verbose SDK telemetry.
src/core/mxc_engine/Cargo.toml Adds hashing and randomness dependencies.
src/core/learning_mode_core/src/verbose_logging.rs Defines bounded verbose diagnostics.
src/core/learning_mode_core/src/model.rs Makes classifications orderable.
src/core/learning_mode_core/src/lib.rs Exports verbose and paired-output APIs.
src/core/learning_mode_core/src/analyze.rs Adds bounded verbose results.
src/core/learning_mode_core/Cargo.toml Adds output-management dependencies.
src/Cargo.toml Adds the shared same-file dependency.
src/Cargo.lock Locks new dependencies.
src/backends/learning_mode/windows/src/ui.rs Updates actionable terminology.
src/backends/learning_mode/windows/src/tdh_decode.rs Adds typed and partial property decoding.
src/backends/learning_mode/windows/src/etl_filter.rs Preserves known-provider events safely.
src/backends/learning_mode/windows/src/capability_dacl.rs Adds sanitized verbose metadata.
src/backends/learning_mode/windows/Cargo.toml Adds hashing support.
src/backends/appcontainer/common/src/guarded_capture.rs Updates paired-output terminology.
src/backends/appcontainer/common/src/capture_output.rs Publishes actionable and verbose artifacts together.
sdk/node/tests/unit/telemetry.test.ts Updates consent title fixture.
sdk/node/tests/unit/default-consent-protocol-runner.test.ts Updates protocol fixture title.
sdk/node/README.md Documents verbose telemetry.
sdk/dotnet/README.md Documents verbose telemetry.
sdk/dotnet/Microsoft.Mxc.Sdk.Tests/MxcTelemetryTests.cs Updates consent expectations.
README.md Documents audit and telemetry changes.
docs/telemetry/telemetry.md Documents the new event and inventory.
docs/telemetry/telemetry-consent-design.md Updates consent disclosure and version.
docs/process-container/os-version-support.md Updates actionable terminology.
docs/learning-mode/capabilities.md Documents verbose artifacts and classifications.
.github/copilot-instructions.md Updates Learning Mode architecture guidance.
Review details

Suppressed comments (1)

docs/telemetry/telemetry.md:309

  • This second review-status statement still calls the consent wording version 2 and says its privacy review is pending, although the resource is version 3 and the PR identifies the disclosure as privacy-reviewed. Remove the consent wording from the pending list while retaining any genuinely pending verbose-event reviews.
The version 2 consent wording, verbose event inventory, WinExt
classification, retention, access, regional processing, deletion, and
localization/accessibility decisions require explicit privacy review.
  • Files reviewed: 47/48 changed files
  • Comments generated: 8
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/core/mxc_engine/src/lib.rs Outdated
Comment thread src/core/mxc_engine/src/lib.rs Outdated
Comment thread src/core/mxc_engine/src/lib.rs Outdated
Comment thread src/mxc_telemetry/src/lib.rs
Comment thread README.md Outdated
Comment thread README.md
Comment thread docs/telemetry/telemetry.md Outdated
Comment thread src/host/plm/readme.md Outdated
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: ba4f8739-616b-4b42-8eba-24a762602400
@microsoft-github-policy-service microsoft-github-policy-service Bot added the Copilot-Instructions PR modifies Copilot instruction files (.github/copilot-instructions.md or .github/instructions/) label Sep 1, 2026
@richiemsft
Richie Gomez (richiemsft) requested a balanced review from Copilot September 1, 2026 23:38

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Cross-platform compilation is broken and the telemetry payload can expose unreviewed workload-controlled values.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details
  • Files reviewed: 48/49 changed files
  • Comments generated: 3
  • Review effort level: Balanced

Comment thread src/core/mxc_engine/src/lib.rs
Comment thread src/core/mxc_engine/src/verbose_telemetry.rs
Comment thread src/core/wxc_common/resources/telemetry/consent/en-US.json
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: ba4f8739-616b-4b42-8eba-24a762602400

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The telemetry projection can emit unvalidated strings, and some new failure diagnostics remain inaccessible or silently discarded.

Review details

Suppressed comments (4)

Previously missed (3) — in code that hasn't changed since the last review.

src/core/learning_mode_core/src/verbose_logging.rs:155

  • When total_occurrences is already u64::MAX, record_overflow saturates it at MAX and the following subtraction reduces it to MAX - 1. Recording another event must never decrease the total; update only the overflow counters here because the total was already incremented before the match.
    src/core/mxc-sdk/src/sandbox.rs:178
  • wait() can both return an error and append the new verbose-telemetry warning (the added engine test constructs exactly this case), but ? consumes and drops self before either warnings or finalized metadata are returned. Consequently mxc_sdk::run and FFI/SDK callers still cannot diagnose an ETW/artifact failure when backend teardown also fails. Preserve these diagnostics in the error path or return a result type that carries them.
    src/core/wxc/src/audit.rs:225
  • If the JSON pair commits but source cleanup produces warnings, and the subsequent ETL move fails, this ? discards those warnings. The operator then sees only the ETL error and is not told that stale actionable/verbose source files may remain. Include accumulated cleanup warnings in the returned error on this branch.

src/core/mxc_engine/src/verbose_telemetry.rs:153

  • Only property values are redacted. This artifact is reopened from a caller-selected path and deserialized without any producer identity/integrity check, so a concurrent replacement can place customer content in properties names or provider_guid; both strings survive this projection and are uploaded in mxc.content. Build a closed telemetry projection (for example, derive the GUID from the provider enum and allowlist/drop property names), or bind the reopened file to the artifact produced during teardown.
  • Files reviewed: 48/49 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

@richiemsft

Copy link
Copy Markdown
Contributor Author

Copilot's suppressed review also identified three issues in inherited base-PR code that are intentionally not addressed by this stacked PR:

  • src/core/learning_mode_core/src/verbose_logging.rs:155 — saturated total_occurrences can decrease in the overflow path.
  • src/core/mxc-sdk/src/sandbox.rs:178wait_with_output() can lose finalized warnings/metadata when wait() also fails.
  • src/core/wxc/src/audit.rs:225 — an ETL relocation failure can discard earlier JSON-pair cleanup warnings.

These findings belong to the underlying telemetry work in #822 (or a follow-up from that branch). This PR remains scoped to the new MXC.VerboseDenials event; its directly related closed-projection finding is being fixed here.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: ba4f8739-616b-4b42-8eba-24a762602400
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Copilot-Instructions PR modifies Copilot instruction files (.github/copilot-instructions.md or .github/instructions/)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants