Publish the external dependencies this implementation requires - #272
Merged
Conversation
Records the version range, the version deliberately verified, and the private upstream surface the Langfuse adapter reaches into. Spec renders a per-implementation block from these keys on its compatibility page, alongside the matrix that records what the normative text is written against; the two answer different questions and the split is why this lives here rather than there. The internals list is the source the guard parametrizes over, not a copy of it, so the published record and the enforced record are the same list. A path declared here that no longer exists upstream fails, and one the adapter imports but nobody declared fails too. verified_on is the date the pin was last deliberately moved, not the date CI last passed. A date sitting well back while the range still admits newer versions is saying something true, so nothing nudges it forward on a run that merely passes. A new check pins verified to the installed version so the published number cannot drift from the tested one.
There was a problem hiding this comment.
Pull request overview
This PR adds a published, implementation-specific record of the Langfuse SDK dependency and the private SDK internals this Python implementation relies on, then wires the existing Langfuse-internals guard tests to read that record directly so the published surface is also the enforced surface.
Changes:
- Add an
[external_dependencies.langfuse]section toconformance.tomlwith required range, verified version/date, and a list of private SDK internals depended on. - Refactor
tests/unit/test_langfuse_sdk_internals.pyto parametrize internal-surface checks fromconformance.tomland assert the installed SDK version matches the published verified version. - Add an adapter-import completeness check to ensure certain private span classes used by the adapter are declared in the published internals list.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| tests/unit/test_langfuse_sdk_internals.py | Reads conformance.toml for declared Langfuse internals, verifies existence and version lockstep, and checks adapter import completeness. |
| conformance.toml | Publishes the Langfuse external dependency requirements, verified pin metadata, and guarded private SDK internals list. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
The adapter header claimed validation against 4.7.0 while the same file cited 4.7.1 behaviour and the manifest published 4.7.1. It no longer names a version at all: the manifest entry is enforced against what is installed, a comment is not, so having two claims meant the unenforced one drifted. The completeness check now parses the adapter's actual private span imports instead of testing a hardcoded list of four names against a substring search. A hardcoded list is not a completeness check, which is the one job it had: a fifth class added later would have gone unnoticed, and a name in prose or the same name from another module would both have matched. It also asserts it found imports at all, so a pattern that stops matching fails rather than passing having read nothing. Resolving a declared path now fails by name when an intermediate segment disappears, rather than raising a bare AttributeError. The message is this guard's product, since losing one of these paths surfaces nowhere else.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.
Suppressed comments (2)
tests/unit/test_langfuse_sdk_internals.py:148
- This completeness check only extracts imports from
langfuse._client.span. Removing any of the published non-span dependencies (for example_resources,_otel_tracer,_create_remote_parent_span,tracer_provider, or_instances) frominternalsremoves its generic parameter while the older dedicated tests still pass because they do not assert membership in the manifest. The suite can therefore stay green while the rendered dependency surface becomes incomplete. Please make the completeness check cover every private Langfuse path used by the adapter, not only span imports.
adapter_source = inspect.getsource(importlib.import_module("openarmature.observability.langfuse.adapter"))
imported = set(re.findall(r"from\s+langfuse\._client\.span\s+import\s+(\w+)", adapter_source))
conformance.toml:77
requiresduplicates the package requirement inpyproject.toml, but no guard compares the two. The new test checks only that this key exists, and its range assertions remain hardcoded as(4, 6)and<5, so either declaration can change independently while CI remains green. Add a synchronization check against the Langfuse optional dependency and use the manifest specifier for the installed-version range check so this published requirement cannot silently become stale.
requires = ">=4.6,<5"
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
[external_dependencies.langfuse]toconformance.toml: the version range this implementation requires, the version it deliberately verifies against, and the private upstream surface its Langfuse adapter reaches into.Spec has agreed to render a per-implementation block from these keys on its compatibility page, and will land the renderer once this is on
mainso the block is populated on first render. Coordinated indiscuss-external-dependency-tracking.Why it lives here rather than on the compatibility page
That page's matrix records what the spec's normative text is written against: one row per dependency, implementation-independent. This records what this implementation requires and has verified, which differs per implementation and moves on a different cadence. A TypeScript implementation would carry entirely different numbers and be equally conforming, so putting them on a spec page would assert as a spec fact something only one implementation is true of.
Two things prompted it. Our declared
>=4.6,<5resolves as far as 4.14.x while the compatibility page records 4.7.1 as verified, and neither artifact let a reader see that gap. And until recently nothing in CI checked the private symbols the shipped adapter depends on: the only guard was the live-account integration test, which CI deselects.The published list is the enforced list
internalsis the sourcetests/unit/test_langfuse_sdk_internals.pyparametrizes over, not a copy of it. That matters because a published record nothing enforces is exactly the failure this is meant to close, and restating the list would have reintroduced it one level up.The guards, each mutation tested:
verifieddrifting from the installed version failsKeyErrorfrom inside pytest's collectorverified_onis the date the pin was last deliberately moved or re-verified, not the date CI last passed. A column sourced from CI would read "yesterday" on every render and carry nothing. A date sitting well back whilerequiresstill admits newer versions is telling the reader something true, and nothing nudges it forward on a run that merely passes.Note on the count
The spec page names one non-portable internal, the bound-provider read that observability §6's best-effort detection rests on. That sentence is correct and unchanged: it is the internal the normative text depends on. The other nine are how this implementation realizes the mapping — constructing observation objects, back-dating them, reading the credential cache — and belong to the implementation, not the spec.
Losing any of them does not raise to the caller. The graph observer isolates observer errors, so an observation simply stops being emitted and a leak assertion reads clean, which is why they are guarded rather than trusted.
Testing
2099 passed, 504 skipped. Ruff, pyright and
check_conformance_manifest.pyclean.