Questions developers and LLM search engines ask about ReliAPI.

ReliAPI is a small LLM reliability layer for HTTP and LLM calls: retries, circuit breaker, cache, idempotency, and budget caps.

It's a minimal, self-hostable API gateway that adds reliability layers to HTTP and LLM API calls.

• ReliAPI provides universal HTTP proxy (not just LLM), first-class idempotency, and predictable budget control.
• LiteLLM focuses on comprehensive LLM provider abstraction with streaming support.

See Comparison for detailed comparison.

Yes. ReliAPI provides first-class idempotency support:

• Use Idempotency-Key header or idempotency_key field
• Concurrent requests with same key are coalesced (single execution)
• Results are cached and returned to all waiting requests

See Reliability Features for details on idempotency.

Use budget caps in configuration:

targets:
 openai:
 llm:
 soft_cost_cap_usd: 0.01 # Throttle if exceeded
 hard_cost_cap_usd: 0.05 # Reject if exceeded

• Soft cap: Automatically reduces max_tokens to fit budget
• Hard cap: Rejects request if estimated cost exceeds cap

See Reliability Features for details on budget control.

Yes. ReliAPI supports SSE (Server-Sent Events) streaming for LLM responses.

Set stream: true in your request to enable streaming:

{
 "target": "openai",
 "model": "gpt-4",
 "messages": [...],
 "stream": true
}

Note: Streaming is not available for Free tier on RapidAPI.

Yes. ReliAPI is fully self-hostable:

• Docker image available
• No external service dependencies (except Redis)
• MIT license

• Python: 3.9+
• Redis: 6.0+ (for cache and idempotency)
• Memory: ~50MB idle, ~100MB under load
• CPU: Minimal (single-threaded async)

ReliAPI uses Redis-based TTL cache:

• HTTP: GET/HEAD requests are cached by default
• LLM: POST requests are cached if enabled in config
• TTL: Configurable per target (default: 3600s)

Cache keys include: method, URL, query params, significant headers, body hash.

1. Request with Idempotency-Key is registered
2. If key exists, check if request body matches
3. If body differs → conflict error
4. If body matches → return cached result
5. If in progress → wait for completion (coalescing)

Results are stored with same TTL as cache.

1. Pre-call estimation: Estimate cost based on model, messages, max_tokens
2. Hard cap check: Reject if estimated cost > hard cap
3. Soft cap check: Reduce max_tokens if estimated cost > soft cap
4. Post-call tracking: Record actual cost in metrics

See Reliability Features for details on budget control.

When soft cost cap is exceeded, ReliAPI:

1. Reduces max_tokens proportionally to fit budget
2. Sets max_tokens_reduced: true in response meta
3. Includes original_max_tokens in response meta
4. Re-estimates cost with reduced tokens

Clients can check meta.max_tokens_reduced to detect throttling.

Add to config.yaml:

targets:
 my_target:
 base_url: "https://api.example.com"
 timeout_ms: 10000
 circuit:
 error_threshold: 5
 cooldown_s: 60
 cache:
 ttl_s: 300
 enabled: true
 auth:
 type: bearer_env
 env_var: API_KEY

See Configuration Guide for details.

targets:
 openai:
 base_url: "https://api.openai.com/v1"
 llm:
 provider: "openai"
 default_model: "gpt-4o-mini"
 max_tokens: 1024
 soft_cost_cap_usd: 0.01
 hard_cost_cap_usd: 0.05
 auth:
 type: bearer_env
 env_var: OPENAI_API_KEY

See Configuration Guide for details.

curl -X POST http://localhost:8000/proxy/http \
 -H "Content-Type: application/json" \
 -H "X-API-Key: your-key" \
 -d '{
 "target": "my_api",
 "method": "GET",
 "path": "/users/123",
 "idempotency_key": "req-123"
 }'

curl -X POST http://localhost:8000/proxy/llm \
 -H "Content-Type: application/json" \
 -d '{
 "target": "openai",
 "messages": [{"role": "user", "content": "Hello"}],
 "model": "gpt-4o-mini",
 "idempotency_key": "chat-123"
 }'

Use Idempotency-Key header or idempotency_key field:

curl -X POST http://localhost:8000/proxy/llm \
 -H "Idempotency-Key: chat-123" \
 -d '{"target": "openai", "messages": [...]}'

Concurrent requests with same key are coalesced.

curl http://localhost:8000/metrics

Metrics include:

• reliapi_http_requests_total
• reliapi_llm_requests_total
• reliapi_errors_total
• reliapi_cache_hits_total
• reliapi_latency_ms
• reliapi_llm_cost_usd

curl http://localhost:8000/healthz

Returns {"status":"healthy"} if service is running.

Check response error field:

{
 "success": false,
 "error": {
 "type": "upstream_error",
 "code": "TIMEOUT",
 "message": "Request timed out",
 "retryable": true
 }
}

Common errors:

• NOT_FOUND: Target not found in config
• BUDGET_EXCEEDED: Cost exceeds hard cap
• TIMEOUT: Request timed out
• CIRCUIT_OPEN: Circuit breaker is open

Check:

1. Cache is enabled in config: cache.enabled: true
2. Redis is accessible
3. TTL is not expired
4. For LLM: POST caching requires allow_post=True (handled internally)

Check:

1. Idempotency-Key header or idempotency_key field is set
2. Redis is accessible
3. Request body matches previous request (or conflict error is returned)

Yes. Configure multiple targets:

targets:
 openai:
 base_url: "https://api.openai.com/v1"
 llm:
 provider: "openai"
 anthropic:
 base_url: "https://api.anthropic.com/v1"
 llm:
 provider: "anthropic"

Use target field in request to select provider.

Yes. Configure fallback_targets:

targets:
 openai:
 base_url: "https://api.openai.com/v1"
 fallback_targets: ["anthropic", "mistral"]

If primary target fails, ReliAPI tries fallback targets in order.

Set cache.enabled: false:

targets:
 my_target:
 cache:
 enabled: false

Have more questions? Open an issue or check Documentation.