-
Notifications
You must be signed in to change notification settings - Fork 162
fix(llm): Claude provider strips tool calls and results from the conversation #3389
Description
Any GAIA agent running against a Claude model loses its own tool calls on the way out. The
provider rebuilds every outbound message as {"role", "content"}, so tool_calls is
discarded — and an assistant turn that carried tool calls but no narration is dropped
entirely. Tool results are then forwarded with role="tool", which Anthropic does not
accept.
The effect is that the model never sees its own tool use in the conversation. It cannot
tell what it already ran, so it re-runs work, loses the thread on multi-step tasks, and has
no basis for batching calls. None of this errors — the request succeeds with a quietly
truncated history, which is why it has gone unnoticed.
The Lemonade/OpenAI path is unaffected.
🔍 Technical details
src/gaia/llm/providers/claude.py:231, _split_system:
if content is None or content == "": logger.debug("Dropping empty %s message for Claude request", role) continue cleaned.append({"role": role, "content": content})
Two independent losses:
- Whole-turn drop.
_build_assistant_messagesetscontent=Nonewhen the model
returned tool calls and no text. That turn is skipped, tool calls included. - Silent key strip. The
cleaned.appendkeeps onlyroleandcontent. A turn with
both narration and tool calls survives the first check and still loses its
tool_calls. This is the worse of the two because it looks fine.
There is no outbound translation to Anthropic's shape anywhere in the provider. The three
tool_use references (claude.py:59, :358, :411) are all inbound — parsing Claude's
response back into OpenAI form. The reverse direction does not exist.
Anthropic expects:
- assistant turn:
content: [{"type":"tool_use","id":...,"name":...,"input":{...}}] - result turn:
role="user"withcontent: [{"type":"tool_result","tool_use_id":...,"content":...}]
tests/unit/test_claude_provider.py covers the outbound direction only
(test_tool_use_reencoded_as_lemonade_sentinel); no case feeds the provider an
assistant-message-with-tool_calls or a role=tool message, which is why neither loss is
caught.
Found while investigating #3387 (GAIA never emits a parallel tool call). That benchmark
bypassed this provider, so the two are independent — but this one would discard fan-out
even after #3387 is fixed.
Acceptance criteria
- Assistant turns carrying
tool_callsare translated to Anthropictool_useblocks -
role="tool"messages becomerole="user"with atool_resultblock referencing the
originatingtool_use_id - An assistant turn with tool calls and no text is no longer dropped
- Multi-call turns preserve all calls and their ids
- Tests feeding the provider an assistant-with-
tool_callsand arole=toolmessage,
asserting the outbound payload shape - A multi-step agent run on the Claude path shows prior tool use present in the request