Skip to content

Navigation Menu

Sign in
Sign up

fix(openai): keep n>1 stream choices separate in recorded output - #1864

Open
sailikhithk wants to merge 2 commits into
langfuse:main from
sailikhithk:fix/openai-stream-multi-choice-merging
Open

fix(openai): keep n>1 stream choices separate in recorded output #1864
sailikhithk wants to merge 2 commits into
langfuse:main from
sailikhithk:fix/openai-stream-multi-choice-merging

Conversation

@sailikhithk

@sailikhithk sailikhithk commented Sep 5, 2026
edited
Loading

Copy link
Copy Markdown

What does this PR do?

Fixes langfuse/langfuse#16998

_extract_streamed_openai_response used a single accumulator for every choice in a chat completion stream. An n>1 stream interleaves choices within a single chunk, so content and tool-call arguments from separate choices were merged into one corrupted output (the issue's repro records "AB10" instead of "A0" and "B1").

The extractor now accumulates per choice index and returns one response per choice when the stream carries multiple choices. Single-choice streams keep the existing output shape, so the common n=1 case is unchanged. A stream that only ever yields output for one nonzero choice index (partial consumption) now records that choice instead of None.

Type of change

  • Bug fix
  • New feature
  • Breaking change
  • Refactor
  • Documentation update
  • Tooling, CI, or repo maintenance

Verification

uv run --frozen pytest tests/unit/test_openai.py # 37 passed
uv run --frozen pytest -n auto --dist worksteal tests/unit # 672 passed, 2 skipped (18 pre-existing test_prompt.py fixture errors reproduce on clean main without provider credentials)
uv run --frozen ruff check . # All checks passed!
uv run --frozen ruff format --check . # clean; only pre-existing tests/unit/test_media.py drift on main
uv run --frozen mypy langfuse --no-error-summary # exit 0

Checklist

  • I self-reviewed the diff using code_review.md.
  • I added or updated tests for behavior changes.
  • I updated docs, examples, or .env.template if needed. (no docs impact: internal extractor fix, public output shape for n=1 unchanged)
  • I did not hand-edit generated files; if generated files changed, I used the upstream regeneration path.
  • I did not commit secrets or credentials.

greptile-apps[bot] reacted with thumbs up emoji

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

CLAassistant commented Sep 5, 2026
edited
Loading

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

Comment thread langfuse/openai.py Outdated
_extract_streamed_openai_response used a single accumulator for every
choice in a chat completion stream, so an n>1 stream with interleaved
choices merged content and tool-call arguments from separate choices
into one corrupted output.
Accumulate per choice index and return one response per choice when
the stream carries multiple choices; single-choice streams keep the
existing output shape.
An n>1 stream that only ever yields output for one nonzero choice
index, for example after partial consumption, was read back from
index 0 and recorded as None. Fall back to the single collected
choice regardless of its index.
sailikhithk force-pushed the fix/openai-stream-multi-choice-merging branch from eddd8e8 to aeb8e6b Compare September 5, 2026 18:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@claude claude[bot] claude[bot] left review comments
+1 more reviewer
@greptile-apps greptile-apps[bot] greptile-apps[bot] left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

bug(sdk-python): OpenAI streams merge multiple choices

AltStyle によって変換されたページ (->オリジナル) /