Skip to content

Publish the external dependencies this implementation requires - #272

Merged
chris-colinsky merged 2 commits into
mainfrom
chore/external-dependency-tracker
Aug 14, 2026
Merged

Publish the external dependencies this implementation requires#272
chris-colinsky merged 2 commits into
mainfrom
chore/external-dependency-tracker

Conversation

@chris-colinsky

Copy link
Copy Markdown
Member

Adds [external_dependencies.langfuse] to conformance.toml: the version range this implementation requires, the version it deliberately verifies against, and the private upstream surface its Langfuse adapter reaches into.

Spec has agreed to render a per-implementation block from these keys on its compatibility page, and will land the renderer once this is on main so the block is populated on first render. Coordinated in discuss-external-dependency-tracking.

Why it lives here rather than on the compatibility page

That page's matrix records what the spec's normative text is written against: one row per dependency, implementation-independent. This records what this implementation requires and has verified, which differs per implementation and moves on a different cadence. A TypeScript implementation would carry entirely different numbers and be equally conforming, so putting them on a spec page would assert as a spec fact something only one implementation is true of.

Two things prompted it. Our declared >=4.6,<5 resolves as far as 4.14.x while the compatibility page records 4.7.1 as verified, and neither artifact let a reader see that gap. And until recently nothing in CI checked the private symbols the shipped adapter depends on: the only guard was the live-account integration test, which CI deselects.

The published list is the enforced list

internals is the source tests/unit/test_langfuse_sdk_internals.py parametrizes over, not a copy of it. That matters because a published record nothing enforces is exactly the failure this is meant to close, and restating the list would have reintroduced it one level up.

The guards, each mutation tested:

  • a declared path that no longer exists upstream fails
  • a path the adapter imports that nobody declared fails, which is the completeness half the per-path checks cannot see
  • verified drifting from the installed version fails
  • a missing section, or one missing a required key, fails by name rather than as a bare KeyError from inside pytest's collector

verified_on is the date the pin was last deliberately moved or re-verified, not the date CI last passed. A column sourced from CI would read "yesterday" on every render and carry nothing. A date sitting well back while requires still admits newer versions is telling the reader something true, and nothing nudges it forward on a run that merely passes.

Note on the count

The spec page names one non-portable internal, the bound-provider read that observability §6's best-effort detection rests on. That sentence is correct and unchanged: it is the internal the normative text depends on. The other nine are how this implementation realizes the mapping — constructing observation objects, back-dating them, reading the credential cache — and belong to the implementation, not the spec.

Losing any of them does not raise to the caller. The graph observer isolates observer errors, so an observation simply stops being emitted and a leak assertion reads clean, which is why they are guarded rather than trusted.

Testing

2099 passed, 504 skipped. Ruff, pyright and check_conformance_manifest.py clean.

Records the version range, the version deliberately verified, and the
private upstream surface the Langfuse adapter reaches into. Spec renders
a per-implementation block from these keys on its compatibility page,
alongside the matrix that records what the normative text is written
against; the two answer different questions and the split is why this
lives here rather than there.

The internals list is the source the guard parametrizes over, not a copy
of it, so the published record and the enforced record are the same
list. A path declared here that no longer exists upstream fails, and one
the adapter imports but nobody declared fails too.

verified_on is the date the pin was last deliberately moved, not the
date CI last passed. A date sitting well back while the range still
admits newer versions is saying something true, so nothing nudges it
forward on a run that merely passes. A new check pins verified to the
installed version so the published number cannot drift from the tested
one.
Copilot AI lite review requested due to automatic review settings August 14, 2026 00:59

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a published, implementation-specific record of the Langfuse SDK dependency and the private SDK internals this Python implementation relies on, then wires the existing Langfuse-internals guard tests to read that record directly so the published surface is also the enforced surface.

Changes:

  • Add an [external_dependencies.langfuse] section to conformance.toml with required range, verified version/date, and a list of private SDK internals depended on.
  • Refactor tests/unit/test_langfuse_sdk_internals.py to parametrize internal-surface checks from conformance.toml and assert the installed SDK version matches the published verified version.
  • Add an adapter-import completeness check to ensure certain private span classes used by the adapter are declared in the published internals list.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
tests/unit/test_langfuse_sdk_internals.py Reads conformance.toml for declared Langfuse internals, verifies existence and version lockstep, and checks adapter import completeness.
conformance.toml Publishes the Langfuse external dependency requirements, verified pin metadata, and guarded private SDK internals list.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread conformance.toml
Comment thread tests/unit/test_langfuse_sdk_internals.py Outdated
Comment thread tests/unit/test_langfuse_sdk_internals.py Outdated
The adapter header claimed validation against 4.7.0 while the same file
cited 4.7.1 behaviour and the manifest published 4.7.1. It no longer
names a version at all: the manifest entry is enforced against what is
installed, a comment is not, so having two claims meant the unenforced
one drifted.

The completeness check now parses the adapter's actual private span
imports instead of testing a hardcoded list of four names against a
substring search. A hardcoded list is not a completeness check, which is
the one job it had: a fifth class added later would have gone unnoticed,
and a name in prose or the same name from another module would both
have matched. It also asserts it found imports at all, so a pattern that
stops matching fails rather than passing having read nothing.

Resolving a declared path now fails by name when an intermediate segment
disappears, rather than raising a bare AttributeError. The message is
this guard's product, since losing one of these paths surfaces nowhere
else.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Suppressed comments (2)

tests/unit/test_langfuse_sdk_internals.py:148

  • This completeness check only extracts imports from langfuse._client.span. Removing any of the published non-span dependencies (for example _resources, _otel_tracer, _create_remote_parent_span, tracer_provider, or _instances) from internals removes its generic parameter while the older dedicated tests still pass because they do not assert membership in the manifest. The suite can therefore stay green while the rendered dependency surface becomes incomplete. Please make the completeness check cover every private Langfuse path used by the adapter, not only span imports.
    adapter_source = inspect.getsource(importlib.import_module("openarmature.observability.langfuse.adapter"))
    imported = set(re.findall(r"from\s+langfuse\._client\.span\s+import\s+(\w+)", adapter_source))

conformance.toml:77

  • requires duplicates the package requirement in pyproject.toml, but no guard compares the two. The new test checks only that this key exists, and its range assertions remain hardcoded as (4, 6) and <5, so either declaration can change independently while CI remains green. Add a synchronization check against the Langfuse optional dependency and use the manifest specifier for the installed-version range check so this published requirement cannot silently become stale.
requires = ">=4.6,<5"

@chris-colinsky
chris-colinsky merged commit 41aa40a into main Aug 14, 2026
7 checks passed
@chris-colinsky
chris-colinsky deleted the chore/external-dependency-tracker branch August 14, 2026 01:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants