Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

What's the difference between passing tools to the Prompt vs the ChatClient? #2530

Unanswered
rwankar asked this question in Q&A
Discussion options

One can create a Prompt with tools e.g

Prompt prompt = new Prompt(userText, ToolCallingChatOptions.builder().toolCallbacks(functionCallbacks).build());
or one can pass tools to the ChatClient e.g.

ChatClient client = ...;
client.prompt(text).tools(tools).call()

Is it correct to assume that if tools are passed into the prompt then one can have prompt specific tools and you don't have to pass them separately whereas tools passed to the client directly might be more suitable for prompt independent tools?

You must be logged in to vote

Replies: 1 comment 1 reply

Comment options

Hello,

As you mentioned, both

Prompt prompt = new Prompt(userText, ToolCallingChatOptions.builder().toolCallbacks(functionCallbacks).build());

and

ChatClient client = ...;
client.prompt(text).tools(tools).call();

work similarly.

However, in DefaultChatClient, if tools are already defined at the ChatClient level, any tools specified in the Prompt are ignored. This behavior can be observed in the prompt(Prompt prompt) method:

@Override
public ChatClientRequestSpec prompt(Prompt prompt) {
 Assert.notNull(prompt, "prompt cannot be null");
 DefaultChatClientRequestSpec spec = new DefaultChatClientRequestSpec(this.defaultChatClientRequest);
 // Options
 if (prompt.getOptions() != null) {
 spec.options(prompt.getOptions());
 }
 // Messages
 if (prompt.getInstructions() != null) {
 spec.messages(prompt.getInstructions());
 }
 return spec;
}

Since tools are not explicitly set from the Prompt in this method, any tools defined at the Prompt level will be ignored if ChatClient already has predefined tools.

This means that:

  • If you define tools at the ChatClient level, they will always be used.
  • If you want prompt-specific tools, you need to ensure that ChatClient does not have globally defined tools, or modify the client’s behavior accordingly.

You can also check this behavior in the AdvisedRequest.java (Lines 176-189)

You must be logged in to vote
1 reply
Comment options

@seungy0 thanks for the reply but I'm not following this comment.

Since tools are not explicitly set from the Prompt in this method, any tools defined at the Prompt level will be ignored if ChatClient already has predefined tools.

The following block suggests that ChatClientRequestSpec will use tools in Prompt (as part of the options).

 // Options
 if (prompt.getOptions() != null) {
 spec.options(prompt.getOptions());
 }

I can see that AdvisedRequest seems to ignore the tools in the Prompt IF tools are already set on AdvisedRequest. May be that allows an Advisor to set tools on the AdvisedRequest and override the tools provided via the Prompt.

So we still have the question about what happens with the tools provided on the ChatClient.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Q&A
Labels
None yet
2 participants

AltStyle によって変換されたページ (->オリジナル) /