-
Couldn't load subscription status.
- Fork 2k
-
One can create a Prompt with tools e.g
Prompt prompt = new Prompt(userText, ToolCallingChatOptions.builder().toolCallbacks(functionCallbacks).build());
or one can pass tools to the ChatClient e.g.
ChatClient client = ...;
client.prompt(text).tools(tools).call()
Is it correct to assume that if tools are passed into the prompt then one can have prompt specific tools and you don't have to pass them separately whereas tools passed to the client directly might be more suitable for prompt independent tools?
Beta Was this translation helpful? Give feedback.
All reactions
Replies: 1 comment 1 reply
-
Hello,
As you mentioned, both
Prompt prompt = new Prompt(userText, ToolCallingChatOptions.builder().toolCallbacks(functionCallbacks).build());
and
ChatClient client = ...; client.prompt(text).tools(tools).call();
work similarly.
However, in DefaultChatClient, if tools are already defined at the ChatClient level, any tools specified in the Prompt are ignored. This behavior can be observed in the prompt(Prompt prompt) method:
@Override public ChatClientRequestSpec prompt(Prompt prompt) { Assert.notNull(prompt, "prompt cannot be null"); DefaultChatClientRequestSpec spec = new DefaultChatClientRequestSpec(this.defaultChatClientRequest); // Options if (prompt.getOptions() != null) { spec.options(prompt.getOptions()); } // Messages if (prompt.getInstructions() != null) { spec.messages(prompt.getInstructions()); } return spec; }
Since tools are not explicitly set from the Prompt in this method, any tools defined at the Prompt level will be ignored if ChatClient already has predefined tools.
This means that:
- If you define tools at the
ChatClientlevel, they will always be used. - If you want prompt-specific tools, you need to ensure that
ChatClientdoes not have globally defined tools, or modify the client’s behavior accordingly.
You can also check this behavior in the AdvisedRequest.java (Lines 176-189)
Beta Was this translation helpful? Give feedback.
All reactions
-
@seungy0 thanks for the reply but I'm not following this comment.
Since tools are not explicitly set from the Prompt in this method, any tools defined at the Prompt level will be ignored if ChatClient already has predefined tools.
The following block suggests that ChatClientRequestSpec will use tools in Prompt (as part of the options).
// Options
if (prompt.getOptions() != null) {
spec.options(prompt.getOptions());
}
I can see that AdvisedRequest seems to ignore the tools in the Prompt IF tools are already set on AdvisedRequest. May be that allows an Advisor to set tools on the AdvisedRequest and override the tools provided via the Prompt.
So we still have the question about what happens with the tools provided on the ChatClient.
Beta Was this translation helpful? Give feedback.