Skip to content

feat: implement streaming support for Claude model#1221

Open
serelli wants to merge 1 commit into
google:mainfrom
serelli:feat/claude-streaming-support
Open

feat: implement streaming support for Claude model#1221
serelli wants to merge 1 commit into
google:mainfrom
serelli:feat/claude-streaming-support

Conversation

@serelli
Copy link
Copy Markdown

@serelli serelli commented May 26, 2026

Summary

  • Switch Claude.generateContent() to use the Anthropic SDK's createStreaming() API when stream=true, replacing the previous non-streaming fallback (which had a TODO: Switch to streaming API comment)
  • Text deltas are emitted immediately as partial LlmResponses with partial=true
  • Tool use JSON is accumulated across inputJson deltas and emitted as a function call LlmResponse on contentBlockStop
  • messageStop emits a turnComplete=true sentinel
  • StreamResponse is always closed via Flowable.using() to prevent resource leaks
  • Extracted buildMessageCreateParams() to deduplicate param-building logic shared by both streaming and non-streaming paths

Test plan

  • testStreaming_textChunksEmittedAsPartialResponses — text deltas become partial=true responses in order
  • testStreaming_messageStopEmitsTurnCompletemessageStop event sets turnComplete=true
  • testStreaming_toolCallAccumulatedAndEmittedOnBlockStop — tool JSON accumulates across deltas, emits complete function call on block stop
  • testStreaming_mixedTextAndToolCall — interleaved text + tool call in same stream produces correct ordered responses
  • testStreaming_emptyStream_producesNoResponses — empty event stream produces no output
  • testStreaming_toolCallWithInvalidJson_fallsBackToEmptyArgs — malformed JSON logs a warning and falls back to empty args map
  • All 1284 existing core module tests still pass

@google-cla
Copy link
Copy Markdown

google-cla Bot commented May 26, 2026

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@serelli
Copy link
Copy Markdown
Author

serelli commented May 26, 2026

I have signed the CLA!

1 similar comment
@serelli
Copy link
Copy Markdown
Author

serelli commented May 26, 2026

I have signed the CLA!

Switch Claude.generateContent() to use the Anthropic SDK's
createStreaming() API when stream=true, replacing the previous
non-streaming fallback (which had a TODO: Switch to streaming API
comment).

- Text deltas are emitted immediately as partial LlmResponses
- Tool use JSON is accumulated across inputJson deltas and emitted
  as a function call LlmResponse on contentBlockStop
- messageStop emits a turnComplete=true sentinel
- StreamResponse is always closed via Flowable.using()
- Extracted buildMessageCreateParams() to deduplicate param logic

Add 6 unit tests covering text streaming, tool call accumulation,
mixed text+tool responses, empty streams, and invalid JSON fallback.
@serelli serelli force-pushed the feat/claude-streaming-support branch from d856d37 to a450dee Compare May 26, 2026 00:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant