A lightweight server that translates Gemini API calls into OpenAI API-compatible format.

This project provides an alternative to using Google's /v1beta/openai/ endpoint by addressing its limitations, offering enhanced functionality, and extending support for key features.

While Google does provide a partially OpenAI-compatible API, there are significant limitations:

1. Unsupported Endpoints: Many endpoints, such as /v1/models, are not available.
2. Limited Parameters: Important parameters like "frequency_penalty", "presence_penalty", and "stop" are not supported. When unsupported parameters are included, the API throws an error instead of gracefully ignoring them.
3. Missing Advanced Features: Features like context caching and advanced safety configurations from Gemini API are absent.

This server addresses these issues by acting as a middleware between your application and the Gemini API.

• Download the latest release binary.

1. Create a configuration file (genai.config.json).
2. Run the server. The default port is 4949.

main.exe -c my_owner.config.json

The server uses a JSON-based configuration file. Below is a basic example:

{
 "api_keys": ["sk-xxx"],
 "server": {
 "port": 4949
 }
}

Here is a complete list of configurable parameters:

type Params = {
 api_keys: string[];
 server?: {
 port?: number;
 };
 no_docs?: boolean;
 retry?: {
 enabled?: boolean;
 retries?: number;
 factor?: number;
 minTimeout?: number;
 maxTimeout?: number;
 };
 debug?: {
 stream?: {
 log?: boolean;
 };
 };
 bodyLimit?: number;
 // error catcher settings
 error?: {
 /**
 * throw: 默认模式,直接报错走 fastify 的 error catch
 * json: 将会以 json 形式返回错误
 * str: 将会以 str 形式返回错误,并且只保留 message
 */
 mode?: "throw" | "json" | "str";
 };
};

The server currently supports the following endpoints:

• /v1/models: Retrieve available model list.
• /v1/embeddings: Generate vector embeddings for input text.
• /v1/chat/completions: Chat-based text completions.

Note: /v1/completions is not supported because Gemini models do not support completion functionality, and Google's PaLM model (which does) is likely to be deprecated.

• v1
• 支持配置多个 api key ,并且可以轮询
  • 权重分配
  • 重试/调度/logs
• 支持配置和切换代理

pnpm run build:ci

This project is licensed under the MIT License.