Add support for Azure, OpenAI, Palm, Anthropic, Cohere Models - using litellm #55

ishaan-jaff commented 2023年08月03日 00:16:53 +02:00 (Migrated from github.com)

I'm the maintainer of litellm https://github.com/BerriAI/litellm - a simple & light package to call OpenAI, Azure, Cohere, Anthropic API Endpoints

This PR adds support for models from all the above mentioned providers

Here's a sample of how it's used:

from litellm import completion
## set ENV variables
# ENV variables can be set in .env file, too. Example in .env.example
os.environ["OPENAI_API_KEY"] = "openai key"
os.environ["COHERE_API_KEY"] = "cohere key"
messages = [{ "content": "Hello, how are you?","role": "user"}]
# openai call
response = completion(model="gpt-3.5-turbo", messages=messages)
# cohere call
response = completion("command-nightly", messages)
# anthropic call
response = completion(model="claude-instant-1", messages=messages)

I'm the maintainer of litellm https://github.com/BerriAI/litellm - a simple & light package to call OpenAI, Azure, Cohere, Anthropic API Endpoints This PR adds support for models from all the above mentioned providers Here's a sample of how it's used: ``` from litellm import completion ## set ENV variables # ENV variables can be set in .env file, too. Example in .env.example os.environ["OPENAI_API_KEY"] = "openai key" os.environ["COHERE_API_KEY"] = "cohere key" messages = [{ "content": "Hello, how are you?","role": "user"}] # openai call response = completion(model="gpt-3.5-turbo", messages=messages) # cohere call response = completion("command-nightly", messages) # anthropic call response = completion(model="claude-instant-1", messages=messages) ```

ishaan-jaff commented 2023年08月03日 00:17:12 +02:00 (Migrated from github.com)

cc @jamesturk

cc @jamesturk

jamesturk commented 2023年08月03日 00:49:25 +02:00 (Migrated from github.com)

Hey, this looks like a really great way to do this. I'm curious about other changes that'll need to be made though, presumably the other places that use openai directly (exceptions/etc.) need to be modified to preserve expected behavior.

Glad to take a closer look soon, or if you have time that'd be great too!

Either way, thanks for this!

Hey, this looks like a really great way to do this. I'm curious about other changes that'll need to be made though, presumably the other places that use `openai` directly (exceptions/etc.) need to be modified to preserve expected behavior. Glad to take a closer look soon, or if you have time that'd be great too! Either way, thanks for this!

ishaan-jaff commented 2023年08月03日 01:51:45 +02:00 (Migrated from github.com)

Hi @jamesturk thanks for the prompt response.

As for using litellm - we send the OpenAI exceptions etc back to you. Outputs/Exceptions are exactly what OpenAI sends you

Adding a link to our docs for your convenience: litellm.readthedocs.io/en/latest/

I took a look at scrapeghost afaik it looks like apicall.py is the only place calling chatCompletion.create

Hi @jamesturk thanks for the prompt response. As for using litellm - we send the OpenAI exceptions etc back to you. Outputs/Exceptions are exactly what OpenAI sends you Adding a link to our docs for your convenience: [litellm.readthedocs.io/en/latest/](https://litellm.readthedocs.io/en/latest/) I took a look at scrapeghost afaik it looks like `apicall.py` is the only place calling `chatCompletion.create`

jamesturk commented 2023年10月09日 20:54:54 +02:00 (Migrated from github.com)

As stated: to get this working, we'd need to handle all of the different exceptions appropriately. If someone wants to pick that up I'd be glad to work with them on that, but going to close this PR as it stands as I don't think we'll be using litellm, probably https://llm.datasette.io/en/stable/ instead.

As stated: to get this working, we'd need to handle all of the different exceptions appropriately. If someone wants to pick that up I'd be glad to work with them on that, but going to close this PR as it stands as I don't think we'll be using litellm, probably https://llm.datasette.io/en/stable/ instead.