Skip to content

Annotate two gold answers with existing BCQuality articles - #847

Merged
Wenjie Fan (gggdttt) merged 1 commit into
mainfrom
bench/annotate-existing-articles
Sep 4, 2026
Merged

Annotate two gold answers with existing BCQuality articles#847
Wenjie Fan (gggdttt) merged 1 commit into
mainfrom
bench/annotate-existing-articles

Conversation

@gggdttt

@gggdttt Wenjie Fan (gggdttt) commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

What this does

Adds articles annotations to two code-review gold answers, raising per-article BCQuality coverage from 80/251 (31.9%) to 82/251 (32.7%).

Line Entry Article
110 synthetic__upgrade-obsolete-caption-01 upgrade/obsoletion-requires-reason-and-tag
120 synthetic__sec-internal-not-boundary-01 security/internal-access-is-not-a-security-boundary

Why only two. I set out to annotate all 30 unannotated entries. After reading every candidate article end to end, only these two have an article that faithfully covers the finding. Per the project convention, an entry with no faithful article is left unannotated rather than mis-credited.

Entry 110 — upgrade-obsolete-caption-01

The gold answer requires flagging an action whose caption was changed to '... (Obsolete)' without any of the obsoletion properties being set. upgrade/obsoletion-requires-reason-and-tag states that an obsoleted element must carry ObsoleteState, ObsoleteReason and ObsoleteTag. Direct match.

Entry 120 — sec-internal-not-boundary-01

The gold answer requires flagging code that relies on internal visibility to protect a sensitive value. security/internal-access-is-not-a-security-boundary states exactly that internal is a compile-time visibility modifier and not a security boundary. Direct match.


Finding 1 — a gold answer contradicts a BCQuality article

synthetic__style-this-keyword-01 is a false-positive guard (expected_comments: []). Its patch is a Subtype = Test codeunit that calls its own Initialize() and CreateRequest() procedures as bare identifiers, and the entry expects no finding. Its own description field states the rationale:

BCApps style rejects the this. prefix for self-references in codeunits, so an AA0248-style finding demanding this. on every self-call is a false positive.

BCQuality style/this-keyword-in-codeunits.md states the opposite:

  • Title: Use the this keyword for self-reference inside codeunits (CodeCop AA0248)
  • Best Practice: "Inside a codeunit, prefix calls to procedures ... with this."
  • Anti Pattern: "Calling a codeunit-local procedure as a bare identifier (ValidateCustomer(Customer)) when other readings are possible"
  • bc-version: [25..], and the entry's environment_setup_version is 27.0, so the article applies.

Both sides claim to represent BCApps convention, and they cannot both be right. An agent that follows the article necessarily fails this gold answer; an agent that passes it is necessarily ignoring the article. This needs a human decision — either narrow the article to exempt test codeunits, or retire the guard. No change is made here; this PR only records the conflict.

Finding 2 — 12 human-approved findings have no article behind them

Twelve unannotated entries are not synthetic inventions. Their descriptions identify them as findings the agent produced on real BCApps pull requests that a human reviewer accepted (THUMBS_UP). None of them has a BCQuality article:

Entry Source Subject
upgrade-transferfields-field-collision-01 PR 9663 r3690213000 TransferFields across tables where field 100 has incompatible types
data-modeling-excluded-from-calculation-01 PR 9663 r3691079460 staging buffer omits a flag the live table filters on
data-modeling-blocked-validation-skip-01 PR 9663 r3704042966 Validate() fails for accounts blocked for all transactions
error-handling-silent-skip-01 PR 9663 r3703883994 success-shaped return hides an unresolvable bin
upgrade-broken-hook-copyrows-01 PR 9663 r3704047134 DataTransfer.CopyRows() called outside upgrade/install context
data-modeling-tablerelation-restriction-mismatch-01 PR 9067 r3669143568 unrestricted relation feeds a deliberately narrower one
style-field-repurpose-indent-01 PR 8553 r3535265290 Indent field repurposed to carry an unrelated key
error-handling-drilldown-position-01 PR 8553 r3604298340 row restored by an entry number that is regenerated each refresh
style-tooltip-mismatch-01 PR 8959 ToolTip describes a different action's outcome
error-handling-fieldno-swap-01 accepted feedback copy-pasted trigger passes the wrong FieldNo
error-handling-guiallowed-01 accepted feedback over-broad GuiAllowed() gating skips integrity checks
error-handling-assistedit-cancel-01 accepted feedback unconditional SaveRecord() after a cancellable AssistEdit

These are the strongest candidates for new BCQuality articles: the value has already been demonstrated on production pull requests, and the corpus cannot currently reproduce it.

Finding 3 — recurring false positives with no suppressing article

Six false-positive guards have no article that would suppress them. Two carry explicit occurrence counts in their descriptions:

  • privacy-004"PII in error messages (8 false positives)". GUIDs, SystemIds and document IDs in error messages are not personally identifiable, but no article says so.
  • style-001"caption_false_positive (790 false positives)".
  • style-showmandatory-flowfield-01 — a non-editable FlowField does not need ShowMandatory.
  • error-testfield-enabled-01 — an action's Enabled property already guarantees the field is non-blank, so a demanded TestField is redundant.
  • error-handling-case-unreachable-else-01 — a case that is exhaustive over a non-extensible enum does not require an else.
  • testing-tolerance-clean-01 — a documented +/-1 day timezone tolerance is not an unbounded assertion.

Each of these is a precision cost with no corpus-side defence.

Finding 4 — four entries have stale descriptions

style-001, style-003, style-008 and security-007 carry descriptions that call them clean or false-positive examples, but all four have expect_findings: true and a real positive expected_comment (indentation, Hungarian prefix, non-PascalCase procedure name, HTTP instead of HTTPS). The descriptions appear to be left over from an earlier generation pass. Scoring is unaffected because the loader ignores description, but the field is misleading to anyone reading the dataset. Not changed here.


Articles checked and ruled out

Read in full and rejected, with the reason:

Article Why it does not apply
security/inherent-permissions-minimal-grant covers method-level attributes only, not object-level InherentPermissions = RIX
security/permission-set-avoid-wildcard-grants strictly wildcard * grants
security/do-not-grant-rights-beyond-a-users-entitlement about license-tier ceilings, not least privilege
security/validate-user-configurable-urls about SSRF host/pattern validation, not HTTP to HTTPS
style/tooltip-required-on-page-fields presence of a ToolTip on a field (AA0218), not whether an action's text matches its behaviour
style/this-keyword-in-codeunits contradicts the entry - see Finding 1
data-modeling/check-blocked-in-referencing-code-not-in-master requires TestField(Blocked, false) in consumers; the entry is the opposite case
data-modeling/table-relation-extensions-are-additive-and-top-down about evaluation order, explicitly not about relation width
upgrade/hybrid-migration-codeunits-not-standard-upgrade suppression article telling reviewers not to apply standard upgrade rules
upgrade/datatransfer-skips-triggers-and-subscribers about skipped triggers, not CopyRows() context restrictions
privacy/session-logmessage-requires-dataclassification privacy-clean-02 supplies the classification, but its stated subject is message content, already covered by no-pii-in-telemetry-message-string
testing/ui-handlers-in-tests current main version covers dialog invocation counts, not post-RunModal business assertions

Searched across all 251 articles with no hit: GuiAllowed, SaveRecord, ShowMandatory, UserSecurityId, tolerance, Extensible = false, CopyRows, repurpos, Excluded from calculation.

testing-ui-handler-assert-after-run-01 is deliberately left unannotated pending BCQuality #132, which introduces the article that would cover it.

Coverage gap by domain

172 of 251 articles remain unexercised. The largest gaps are ui (26), events (17), performance (16) and error-handling (15). Four domains are at zero coverage and are small enough to close quickly: query (2), appsource (4), interfaces (5).

Validation

  • uv run ruff format - 366 files unchanged
  • uv run ruff check - clean
  • uv run pytest -q - 871 passed, 2 skipped (matches baseline)
  • Loader reports 145 entries; declared_articles() resolves on both edited entries
  • Coverage report: 82 covered, unknown_articles: []

Both slugs were taken from BCQuality origin/main (2439d5c), so the live-slug e2e gate passes.

Adds the missing BCQuality article annotations to the two unannotated
code-review entries whose findings a published article faithfully covers:

- synthetic__sec-internal-not-boundary-01 ->
  security/internal-access-is-not-a-security-boundary
- synthetic__upgrade-obsolete-caption-01 ->
  upgrade/obsoletion-requires-reason-and-tag

Per-article coverage goes from 80/251 to 82/251. Both slugs already exist
on BCQuality main, so the e2e slug-existence check stays green.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 7a9dff24-8942-4810-8359-18904eaa4da4
@gggdttt Wenjie Fan (gggdttt) changed the title Annotate two code-review gold answers with BCQuality articles Annotate two gold answers with existing BCQuality articles Sep 3, 2026
@gggdttt
Wenjie Fan (gggdttt) merged commit 11b8966 into main Sep 4, 2026
16 checks passed
@gggdttt
Wenjie Fan (gggdttt) deleted the bench/annotate-existing-articles branch September 4, 2026 08:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants