🤖

Using a coding agent? Run this to install the Hindsight docs skill:

npx skills add https://github.com/vectorize-io/hindsight --skill hindsight-docs

Extensions

Extensions allow you to customize and extend Hindsight behavior without modifying core code. They enable multi-tenancy, custom authentication, additional HTTP endpoints, and operation hooks.

Available Extensions

TenantExtension

Handles multi-tenancy and API key authentication. Validates incoming requests and determines which PostgreSQL schema to use for database operations, enabling tenant isolation at the database level.

Built-in: ApiKeyTenantExtension

A simple implementation that validates API keys against an environment variable and uses the public schema for all authenticated requests.

HINDSIGHT_API_TENANT_EXTENSION=hindsight_api.extensions.builtin.tenant:ApiKeyTenantExtension
HINDSIGHT_API_TENANT_API_KEY=your-secret-key

Built-in: SupabaseTenantExtension

Validates Supabase JWTs and provides multi-tenant memory isolation. Each authenticated user gets their own PostgreSQL schema ({prefix}_{user_id}), ensuring complete data separation. Performs local JWT verification using JWKS for optimal performance (no network call per request).

HINDSIGHT_API_TENANT_EXTENSION=hindsight_api.extensions.builtin.supabase_tenant:SupabaseTenantExtension
HINDSIGHT_API_TENANT_SUPABASE_URL=https://your-project.supabase.co
# Optional - only needed for legacy HS256 projects or health check
HINDSIGHT_API_TENANT_SUPABASE_SERVICE_KEY=your-service-role-key

See the source code for complete configuration options and implementation details.

For other multi-tenant setups with separate schemas per tenant (e.g., custom JWT-based auth), implement a custom TenantExtension.

HttpExtension

Adds custom HTTP endpoints under the /ext/ path prefix. Useful for adding domain-specific APIs that integrate with Hindsight's memory engine.

Provides two router methods:

get_router(memory) — returns a FastAPI router mounted at /ext/
get_root_router(memory) — returns a FastAPI router mounted at the application root (for well-known endpoints or other paths that must be at specific locations). Returns None by default.

No built-in implementation - implement your own to add custom endpoints.

HINDSIGHT_API_HTTP_EXTENSION=mypackage.ext:MyHttpExtension

OperationValidatorExtension

Hooks into retain/recall/reflect operations for validation and monitoring. Use cases include:

Rate limiting and quota enforcement
Permission checks and content filtering
Audit logging and usage tracking
Custom metrics collection

No built-in implementation - implement your own based on your requirements.

HINDSIGHT_API_OPERATION_VALIDATOR_EXTENSION=mypackage.validators:MyValidator

MCPExtension

Registers additional MCP (Model Context Protocol) tools on the Hindsight MCP server. Enables external packages to add custom tools without modifying core code.

No built-in implementation - implement your own to add custom MCP tools.

HINDSIGHT_API_MCP_EXTENSION=mypackage.mcp:MyMCPExtension

Writing Custom Extensions

Extension Basics

Extensions are Python classes loaded via environment variables:

HINDSIGHT_API_<TYPE>_EXTENSION=mypackage.module:MyExtensionClass

Configuration is passed via prefixed environment variables:

HINDSIGHT_API_<TYPE>_SOME_CONFIG=value
# Extension receives: {"some_config": "value"}

All extensions support lifecycle hooks:

on_startup() - Called when the application starts
on_shutdown() - Called when the application shuts down

Extensions have access to an ExtensionContext that provides:

run_migration(schema) - Run database migrations for a schema
get_memory_engine() - Get the MemoryEngine interface

Example: Custom TenantExtension with JWT

import jwt
from hindsight_api.extensions import TenantExtension, TenantContext, AuthenticationError

classJwtTenantExtension(TenantExtension):
def__init__(self, config:dict[str,str]):
super().__init__(config)
 self.jwt_secret = config.get("jwt_secret")
ifnot self.jwt_secret:
raise ValueError("HINDSIGHT_API_TENANT_JWT_SECRET is required")

asyncdefauthenticate(self, context: RequestContext)-> TenantContext:
 token = context.api_key
ifnot token:
# Optional headers dict is forwarded in HTTP/MCP error responses
raise AuthenticationError("Bearer token required")

try:
 payload = jwt.decode(token, self.jwt_secret, algorithms=["HS256"])
 tenant_id = payload.get("tenant_id")
ifnot tenant_id:
raise AuthenticationError("Missing tenant_id in token")
return TenantContext(schema_name=f"tenant_{tenant_id}")
except jwt.InvalidTokenError as e:
raise AuthenticationError(str(e))

AuthenticationError accepts an optional headers dict that is forwarded in both HTTP and MCP error responses. This is useful for returning custom headers like WWW-Authenticate:

raise AuthenticationError(
"Authorization required",
 headers={"WWW-Authenticate":'Bearer realm="example"'},
)

Example: Custom HttpExtension

from fastapi import APIRouter
from hindsight_api.extensions import HttpExtension

classMyHttpExtension(HttpExtension):
defget_router(self, memory: MemoryEngine)-> APIRouter:
 router = APIRouter()

@router.get("/hello")
asyncdefhello():
return{"message":"Hello from extension!"}

@router.post("/custom/{bank_id}/action")
asyncdefcustom_action(bank_id:str):
# Access memory engine for database operations
 pool =await memory._get_pool()
# ... custom logic
return{"status":"ok"}

return router

defget_root_router(self, memory: MemoryEngine)-> APIRouter |None:
"""Optional: mount routes at the application root (not under /ext/)."""
 router = APIRouter()

@router.get("/.well-known/my-metadata")
asyncdefmetadata():
return{"version":"1.0"}

return router

Routes from get_router are available at /ext/hello, /ext/custom/{bank_id}/action, etc. Routes from get_root_router are mounted at the app root (e.g., /.well-known/my-metadata).

Example: Custom OperationValidatorExtension

from hindsight_api.extensions import(
 OperationValidatorExtension,
 ValidationResult,
 RetainContext,
 RecallContext,
 ReflectContext,
 RetainResult,
)

classMyValidator(OperationValidatorExtension):
# Pre-operation validation (required)
asyncdefvalidate_retain(self, ctx: RetainContext)-> ValidationResult:
# Implement your validation logic
return ValidationResult.accept()
# Or reject: return ValidationResult.reject("Reason")

asyncdefvalidate_recall(self, ctx: RecallContext)-> ValidationResult:
return ValidationResult.accept()

asyncdefvalidate_reflect(self, ctx: ReflectContext)-> ValidationResult:
return ValidationResult.accept()

# Post-operation hooks (optional)
asyncdefon_retain_complete(self, result: RetainResult)->None:
# Log usage, update metrics, send notifications, etc.
pass

Deferring an operation

In addition to accept and reject, a validate_* hook can ask the worker to requeue the operation for a future time by raising DeferOperation. Use this for backpressure (rate-limited upstream, quota window not yet open, dependency warming up) — unlike a retry, it does not increment retry_count or write error_message. The worker sets next_retry_at to your exec_date and the task is invisible to claim queries until that time.

from datetime import datetime, timedelta, timezone

from hindsight_api.extensions import(
 DeferOperation,
 OperationValidatorExtension,
 RetainContext,
 ValidationResult,
)


classQuotaAwareValidator(OperationValidatorExtension):
asyncdefvalidate_retain(self, ctx: RetainContext)-> ValidationResult:
ifnotawait self._quota_available(ctx.bank_id):
raise DeferOperation(
 exec_date=datetime.now(timezone.utc)+ timedelta(minutes=5),
 reason="bank quota window exhausted",
)
return ValidationResult.accept()

DeferOperation is worker-only: do not raise it from validate_recall or validate_reflect in synchronous HTTP request paths — there is no queue to defer to and it will surface as a 500.

Example: Custom MCPExtension

from mcp.server.fastmcp import FastMCP
from hindsight_api.extensions import MCPExtension
from hindsight_api.engine import MemoryEngine

classMyMCPExtension(MCPExtension):
asyncdefregister_tools(self, mcp: FastMCP, memory: MemoryEngine)->None:
@mcp.tool()
asyncdefcustom_search(query:str)->str:
"""Custom MCP tool for specialized search."""
# Access memory engine for operations
 pool =await memory._get_pool()
# ... custom logic
returnf"Results for: {query}"

Deploying Custom Extensions

With Docker

Mount your extension package as a volume and set the environment variable:

# docker-compose.yml
services:
hindsight-api:
image: vectorize/hindsight-api:latest
volumes:
- ./my_extensions:/app/my_extensions
environment:
- HINDSIGHT_API_TENANT_EXTENSION=my_extensions.auth:JwtTenantExtension
- HINDSIGHT_API_TENANT_JWT_SECRET=${JWT_SECRET}
- PYTHONPATH=/app

Or build a custom image with your extensions:

FROM vectorize/hindsight-api:latest
COPY my_extensions /app/my_extensions
ENV PYTHONPATH=/app

Bare Metal

Install your extension package in the same Python environment as Hindsight:

# Install Hindsight
pip install hindsight-api

# Install your extension package
pip install ./my-extensions
# or
pip install my-extensions-package

# Configure
exportHINDSIGHT_API_TENANT_EXTENSION=my_extensions.auth:JwtTenantExtension
exportHINDSIGHT_API_TENANT_JWT_SECRET=your-secret

# Run
hindsight-api

Contributing Extensions

Custom extensions that solve common use cases are welcome contributions to the Hindsight project. If you've built an extension for:

Authentication providers (OAuth, SAML, API gateways)
Rate limiting or quota management
Audit logging integrations
Metrics exporters (Datadog, New Relic, etc.)
Custom HTTP endpoints for specific platforms

Consider contributing it to the hindsight_api.extensions.builtin package. Open an issue or pull request on GitHub to discuss your extension.

Available Extensions​

TenantExtension​

HttpExtension​

OperationValidatorExtension​

MCPExtension​

Writing Custom Extensions​

Extension Basics​

Example: Custom TenantExtension with JWT​

Example: Custom HttpExtension​

Example: Custom OperationValidatorExtension​

Deferring an operation​

Example: Custom MCPExtension​

Deploying Custom Extensions​

With Docker​

Bare Metal​

Contributing Extensions​