Skip to content

Stop reporting expected startup URL-lookup timeouts to Sentry (BL-16575) - #8082

Draft
hatton wants to merge 1 commit into
masterfrom
BL-16575-sentry-urllookup-timeout-noise
Draft

Stop reporting expected startup URL-lookup timeouts to Sentry (BL-16575)#8082
hatton wants to merge 1 commit into
masterfrom
BL-16575-sentry-urllookup-timeout-noise

Conversation

@hatton

@hatton hatton commented Jul 20, 2026

Copy link
Copy Markdown
Member

Summary

  • TryGetUrlDataFromServer (startup URL lookup, BloomS3Client.DownloadFile) is a deliberate 2.5s/3s best-effort call that already falls back to a cached/default URL on any failure — the user sees nothing.
  • Timeouts/cancellations/network failures there are expected on slow or unreliable connections, so reporting every one to Sentry was pure noise burying real issues: Sentry BLOOM-DESKTOP-ERZ (~889 events / 141 users, live 6.3.2) and -2H2 (older, 6.0.1.0).
  • Added UrlLookup.IsExpectedTransientLookupFailure(Exception), which walks the InnerException chain looking for TimeoutException, OperationCanceledException (covers TaskCanceledException), WebException, SocketException, or HttpRequestException. Those are now logged locally but no longer sent to Sentry. Anything else (e.g. a JSON parse error) is still reported, since that would be a real bug.
  • 4 new NUnit tests pin the classifier's behavior, including the wrapped-exception case.

Fix authored by Claude Opus 4.8; PR opened/landed by Claude Sonnet 5 running the preflight pipeline; rebased and live-verified by Claude Opus 5.

Ref: BL-16575

Test plan

  • build/agent-dotnet.sh build src/BloomExe/BloomExe.csproj — succeeds
  • build/agent-dotnet.sh test src/BloomTests/BloomTests.csproj --filter "FullyQualifiedName~UrlLookupTests" — 8/8 passed (4 existing + 4 new)
  • Light correctness review of the diff — clean, no findings
  • Rebased onto master (2026-07-28). Clean, no conflicts; interdiff empty (rebased patch byte-identical to the original commit). Build + tests re-run on the rebase: 8/8 pass.

Live verification under the debugger (2026-07-28)

Each branch of the classifier was exercised against a real running Bloom with breakpoints on both return statements in IsExpectedTransientLookupFailure, and the production exception shape was read off the actual Sentry events. All four cases behave as intended:

Case How forced Actual exception Result
Production timeout (ERZ + 2H2) real events, read from Sentry AmazonServiceException → inner WebException (timeout) suppressed, matched at depth 1
net8 timeout overall DownloadFile timeout cut to 1 ms bare TaskCanceledException suppressed, depth 0
net8 network/DNS failure S3 ServiceURL pointed at a non-resolvable host HttpRequestException → inner SocketException suppressed, depth 0
Real bug corrupted the JSON handed to DeserializeObject JsonReaderException, no inner reported to Sentry (correct)

Two things worth a reviewer's attention:

  1. The InnerException walk is load-bearing, not defensive. The production events are from the .NET Framework build, where WebExceptionHandler wraps the timeout in an AmazonServiceException; the WebException is only reachable one level down. A flat top-level type check would suppress none of the ~889 events. On current net8 builds the SDK does not wrap — timeouts and DNS failures arrive bare and match at depth 0 — so the walk is what makes the fix cover both generations. Please don't simplify it away.
  2. Deliberately type-based, not message-based. The inner message on the real events is localized ("Le délai d'attente de l'opération a expiré"), so any message-string matching would fail for exactly the non-English users generating most of the noise.

BLOOM-DESKTOP-ERZ and -2H2 were confirmed byte-for-byte identical stacks — one root cause split across two Sentry groups. Both are covered.

Devin's single informational note from the earlier consult (a network failure hidden inside a multi-cause AggregateException would not be found, since the walk follows only the single InnerException link) is now closed empirically as well as by inspection: the real call path uses GetAwaiter().GetResult(), and every live repro produced a bare exception, never an AggregateException.

Devin review


This change is Reviewable

@hatton

hatton commented Jul 20, 2026

Copy link
Copy Markdown
Member Author

[Claude Sonnet 5]

Consulted Devin on 2026-07-20 01:18 UTC up to commit bd04eb0074e7b272f69fb4b658367bd5f64b22d9.

No Bugs, no Investigate flags. 1 Informational note (not posted, low signal): the InnerException chain-walk in IsExpectedTransientLookupFailure only follows the single InnerException link, so a network failure wrapped inside a multi-cause AggregateException could still slip through and get reported (harmless direction -- worst case is a bit of extra noise, never a swallowed real error). Doesn't apply to the actual call path here: BloomS3Client.DownloadFile awaits via GetAwaiter().GetResult(), not .Wait()/.Result, so it never produces an AggregateException in practice. No action needed.

TryGetUrlDataFromServer is a deliberate 2.5s/3s best-effort lookup that
already falls back to a cached/default URL on failure -- the user sees
nothing. Timeouts/cancellations/network failures there are expected on
slow connections, so reporting them to Sentry was pure noise burying
real issues (BLOOM-DESKTOP-ERZ, -2H2: ~889 events/141 users). Add
IsExpectedTransientLookupFailure to classify and suppress just those
expected cases, walking the InnerException chain since the AWS SDK
wraps the underlying WebException/timeout. Genuinely unexpected
failures (e.g. a JSON parse error) are still reported.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@JohnThomson
JohnThomson force-pushed the BL-16575-sentry-urllookup-timeout-noise branch from bd04eb0 to 19fd8b0 Compare July 28, 2026 13:33

@JohnThomson JohnThomson left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested that it fixes the messages we actually want to stop. Maybe there are others we could stop too; I don't think it's worth futher complication.

@JohnThomson made 1 comment.
Reviewable status: 0 of 2 files reviewed, all discussions resolved (waiting on hatton).

@hatton

hatton commented Jul 29, 2026

Copy link
Copy Markdown
Member Author

This is still in draft, but it was ready for Review, so I assume John wants us to look at it.

I'm concerned that we are too readily throwing away the signal here. It's easy to say, "Well, this has no impact on the user," but imagine that the timeout was unreasonably short. Is it actually true that this has no impact on the user? Then why does this URL lookup even exist? So therefore, that's a specious argument that opus is making.

I guess what I want is at least some consideration of whether the current timeout is the appropriate one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants