Affected area
- Rust core runtime
- Python binding
- Node.js binding
- Go binding
- C FFI
- Middleware or guardrails
- Plugins
- Observability or exporters
- Framework integrations
Proposed target release: 0.7
Problem or opportunity
Relay has normalized request and response contracts for LLM calls, but tool execution intercepts exchange only a tool name, JSON arguments, and a raw JSON result. A successful callback result does not say whether the tool semantically succeeded, whether a string is plain content or a serialized envelope, or which schema governs structured output.
Middleware that needs to inspect or safely transform a tool result must either depend on application-specific shapes or infer semantics by recursively inspecting arbitrary JSON. Those heuristics can misclassify ordinary JSON output, lose error state, or handle the same tool differently across managed execution and provider-bound request paths.
Tool implementations remain application-specific. Relay should own only an open, lossless envelope that carries semantics explicitly.
Proposed enhancement
Add a provider-neutral annotated tool-result contract alongside the existing raw result.
The additive contract should include:
- Explicit execution status: succeeded, failed, or unknown.
- Portable content blocks and optional structured content.
- An optional data schema identifying declared custom result payloads.
- Correlation identifiers such as tool-call ID when supplied by the integration.
- Forward-compatible metadata and extra fields.
- The original raw result or a guaranteed lossless relationship to it.
Expose the annotation to tool execution continuations and intercept outcomes, lifecycle events, and supported plugin/binding surfaces. A plugin must never determine that a JSON string is a structured envelope solely because it parses as JSON.
The shared contract should align with portable MCP result concepts such as content, structured content, and error status without making Relay tool execution MCP-specific.
Runtime contract and binding impact
Rust should define the canonical serialized type. Python, Node.js, Go, C FFI, adaptive, worker, and native plugin surfaces should preserve it when they expose tool execution middleware.
The change must be additive. Existing callbacks returning raw JSON and native ABI v1 plugins must continue to work with status set to unknown unless a producer supplies stronger evidence. Any native ABI extension should be capability-detected or introduced through a future ABI version.
Alternatives considered
- Application-specific result structs do not compose across harnesses, providers, or plugins.
- Inferring envelopes from arbitrary JSON is ambiguous and unsafe.
- Using LLM request codecs is too late when a harness has already flattened or discarded execution metadata.
- Standardizing every tool input and output schema in Relay is not practical; only the envelope and schema declaration should be shared.
Acceptance criteria
- Integrations can provide explicit tool-result status and representation without changing application-visible output.
- Unknown/custom data schemas and fields round-trip unchanged.
- Existing raw-JSON interceptors and ABI v1 plugins remain compatible.
- Tool lifecycle events and supported exporters can retain the annotation under existing sanitization policy.
- Tests cover success, failure, unknown status, plain JSON text, declared structured content, custom schemas, and malformed annotations.
- Plain JSON text is never promoted to a structured envelope through content sniffing.
- Documentation distinguishes raw tool output, annotated semantics, and application-owned schemas.
Affected area
Proposed target release: 0.7
Problem or opportunity
Relay has normalized request and response contracts for LLM calls, but tool execution intercepts exchange only a tool name, JSON arguments, and a raw JSON result. A successful callback result does not say whether the tool semantically succeeded, whether a string is plain content or a serialized envelope, or which schema governs structured output.
Middleware that needs to inspect or safely transform a tool result must either depend on application-specific shapes or infer semantics by recursively inspecting arbitrary JSON. Those heuristics can misclassify ordinary JSON output, lose error state, or handle the same tool differently across managed execution and provider-bound request paths.
Tool implementations remain application-specific. Relay should own only an open, lossless envelope that carries semantics explicitly.
Proposed enhancement
Add a provider-neutral annotated tool-result contract alongside the existing raw result.
The additive contract should include:
Expose the annotation to tool execution continuations and intercept outcomes, lifecycle events, and supported plugin/binding surfaces. A plugin must never determine that a JSON string is a structured envelope solely because it parses as JSON.
The shared contract should align with portable MCP result concepts such as content, structured content, and error status without making Relay tool execution MCP-specific.
Runtime contract and binding impact
Rust should define the canonical serialized type. Python, Node.js, Go, C FFI, adaptive, worker, and native plugin surfaces should preserve it when they expose tool execution middleware.
The change must be additive. Existing callbacks returning raw JSON and native ABI v1 plugins must continue to work with status set to unknown unless a producer supplies stronger evidence. Any native ABI extension should be capability-detected or introduced through a future ABI version.
Alternatives considered
Acceptance criteria