Skip to content

Navigation Menu

Sign in
Sign up

fix(openai): keep streamed choices separate - #1853

Open
1fanwang wants to merge 5 commits into
langfuse:main from
1fanwang:1fannnw/fix-openai-stream-multiple-choices
Open

fix(openai): keep streamed choices separate #1853
1fanwang wants to merge 5 commits into
langfuse:main from
1fanwang:1fannnw/fix-openai-stream-multiple-choices

Conversation

@1fanwang

@1fanwang 1fanwang commented Sep 3, 2026
edited
Loading

Copy link
Copy Markdown

What does this PR do?

OpenAI streams with n > 1 return separate choices to the application, but Langfuse merges their recorded content and tool arguments. This keeps each choice separate in index order and preserves the single-choice output shape.

The parser accumulates streamed deltas by choice index, which keeps tool-call indexes local to each choice. The sync and async regressions decode the actual exported OTLP payload after consuming an interleaved HTTP stream, including its usage-only final chunk.

Fixes langfuse/langfuse#16998

Type of change

  • Bug fix
  • New feature
  • Breaking change
  • Refactor
  • Documentation update
  • Tooling, CI, or repo maintenance

Verification

The run used Python 3.12.13, Langfuse 4.15.1, OpenAI 2.29.0 and HTTPX 0.28.1. Both endpoints run locally; this is not a live-provider or deployed-server result.

uv run --frozen pytest -q -s tests/unit/test_openai.py::test_streaming_chat_completion_keeps_multiple_choices_separate
Raw logs

Before, on the baseline:

E AssertionError: {'content': 'BA10', 'role': 'assistant', 'tool_calls': [{'function': {'arguments': '{"value":"B{"value":"A0"}1"}', 'name': 'lookup'}, 'id': 'call-0', 'type': 'function'}]}
exit=1

After, both sync and async clients printed this decoded ingestion payload:

{"output":[{"role":"assistant","tool_calls":[{"function":{"name":"lookup","arguments":"{\"value\":\"A0\"}"},"id":"call-0","type":"function"}],"content":"A0"},{"role":"assistant","tool_calls":[{"function":{"name":"lookup","arguments":"{\"value\":\"B1\"}"},"id":"call-1","type":"function"}],"content":"B1"}],"usage":{"completion_tokens":5,"prompt_tokens":7,"total_tokens":12,"completion_tokens_details":null,"prompt_tokens_details":null}}
============================== 2 passed in 3.74s ===============================
exit=0

Checklist

  • I self-reviewed the diff using code_review.md.
  • I added or updated tests for behavior changes.
  • I updated docs, examples, or .env.template if needed.
  • I did not hand-edit generated files; if generated files changed, I used the upstream regeneration path.
  • I did not commit secrets or credentials.

greptile-apps[bot] reacted with thumbs up emoji
Signed-off-by: 1fanwang <1fannnw@gmail.com>

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

CLAassistant commented Sep 3, 2026
edited
Loading

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@claude claude[bot] claude[bot] left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

bug(sdk-python): OpenAI streams merge multiple choices

AltStyle によって変換されたページ (->オリジナル) /