-
Notifications
You must be signed in to change notification settings - Fork 7k
Refactor: How LiteLLMBase Calculate total count #10532
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refactor: How LiteLLMBase Calculate total count #10532
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi, @Woody-Hu,
This is a good idea to uniformly use total_token_count_from_response and remove total_token_count from LiteLLMBase, which should have been deprecated in the last refactoring.
Also, could you help me replace other use of total_token_count with total_token_count_from_response? For example, in Base, total_token_count is simply an alias for total_token_count_from_response. By this way, we can completely clean up the total_token_count.
Cheers.
What problem does this PR solve?
How LiteLLMBase Calculate total count
Type of change