Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

kokogo100/sandclaw-memory

Repository files navigation

sandclaw-memory

Self-Growing Tag-Dictionary RAG for Any Device

PyPI Python License Tests


Give your AI long-term memory that grows smarter over time.

No GPU. No vector database. No external dependencies. Just pip install and go.

from sandclaw_memory import BrainMemory
brain = BrainMemory(tag_extractor=my_ai_func)
brain.save("User loves Python and React")
context = brain.recall("what does the user like?")
# -> Returns relevant memories as Markdown, ready for LLM injection

Why sandclaw-memory?

Feature sandclaw-memory Vector DB (Pinecone, Weaviate) mem0
Setup pip install (done) Docker + API keys + config pip install + API key
GPU required No Often yes No
Cost over time Decreases (self-growing) Constant Constant
Search FTS5 + BM25 (proven) Cosine similarity Embedding-based
Privacy 100% local Cloud required Cloud optional
Dependencies 0 (stdlib only) Many Several
Size ~50KB 100MB+ Docker ~5MB

Why tags instead of vectors? Vector embeddings are powerful but opaque. When a wrong result appears, you can't explain why. sandclaw-memory uses AI-powered tag extraction, so the AI understands synonyms and context (e.g., "vehicle" and "car" get the same tag), while every result is traceable to human-readable tags. Same semantic understanding, zero infrastructure, and you can debug it.

The Self-Growing Advantage

Traditional RAG calls AI for every search. sandclaw-memory learns:

Day 1: "Built a page with React" → AI extracts: [react, frontend, page]
 keyword_map registers: "React" → react, "page" → page
Day 30: "React Native app" → keyword_map instant match! No AI needed.
 Cost: 0ドル.00
Day 90: 80% of saves match keyword_map → AI calls reduced by 80%

The more you use it, the cheaper it gets.

Quick Start

Install

pip install sandclaw-memory

3-Line Memory

from sandclaw_memory import BrainMemory
with BrainMemory(tag_extractor=my_tag_func) as brain:
 brain.save("Important: migrate to FastAPI by Q2")
 print(brain.recall("what's the migration plan?"))

With OpenAI

import json, openai
from sandclaw_memory import BrainMemory
def tag_extractor(content: str) -> list[str]:
 resp = openai.chat.completions.create(
 model="gpt-4o-mini",
 messages=[{
 "role": "system",
 "content": "Extract 3-7 keyword tags. Return JSON array only."
 }, {"role": "user", "content": content}],
 temperature=0,
 )
 return json.loads(resp.choices[0].message.content)
with BrainMemory(
 db_path="./my_memory",
 tag_extractor=tag_extractor,
) as brain:
 brain.start_polling() # Background tag extraction every 15s
 brain.save("User prefers Python and TypeScript")
 brain.save("Decided to use PostgreSQL", source="archive")
 context = brain.recall("what tech stack?")
 # Inject 'context' into your LLM's system prompt

With Claude

import json, anthropic
from sandclaw_memory import BrainMemory
client = anthropic.Anthropic()
def tag_extractor(content: str) -> list[str]:
 resp = client.messages.create(
 model="claude-haiku-4-5-20251001",
 max_tokens=200,
 messages=[{"role": "user",
 "content": f"Extract 3-7 tags as JSON array:\n{content}"}],
 )
 return json.loads(resp.content[0].text)
brain = BrainMemory(tag_extractor=tag_extractor)

See examples/ for LangChain, custom callbacks, and scheduling patterns.

Architecture

┌─────────────────────────────────────────────────┐
│ BrainMemory │
│ save() → recall() → start_polling() │
├─────────────────────────────────────────────────┤
│ │
│ ┌─────────┐ ┌──────────┐ ┌────────────────┐ │
│ │ L1 │ │ L2 │ │ L3 │ │
│ │ Session │ │ Summary │ │ Archive │ │
│ │ │ │ │ │ │ │
│ │ 3-day │ │ 30-day │ │ Permanent │ │
│ │ rolling │ │ AI/text │ │ SQLite + FTS5 │ │
│ │ Markdown│ │ summary │ │ + self-growing │ │
│ │ logs │ │ │ │ tag dict │ │
│ └────┬────┘ └────┬─────┘ └───────┬────────┘ │
│ │ │ │ │
│ ┌────┴────────────┴────────────────┴────────┐ │
│ │ Intent Dispatcher │ │
│ │ CASUAL → L1 │ STANDARD → L1+L2 │ │
│ │ │ DEEP → L1+L2+L3 │ │
│ └───────────────┴───────────────────────────┘ │
│ │
│ ┌──────────────────────────────────────────┐ │
│ │ Self-Growing Tag Pipeline │ │
│ │ Stage 1: keyword_map (instant, free) │ │
│ │ Stage 2: AI callback (async, queue) │ │
│ │ → keyword_map grows → Stage 2 shrinks │ │
│ └──────────────────────────────────────────┘ │
└─────────────────────────────────────────────────┘

Three Memory Layers

Layer What Retention Storage Speed
L1 Session Conversation logs 3 days (rolling) Markdown files Instant
L2 Summary Period summaries 30 days In-memory Instant
L3 Archive Important memories Forever SQLite + FTS5 ~1ms

Intent-Based Depth

The dispatcher auto-detects how deep to search:

brain.recall("what time is it?") # → CASUAL (L1 only)
brain.recall("summarize this month") # → STANDARD (L1 + L2)
brain.recall("why did we pick React?") # → DEEP (L1 + L2 + L3)
# Or override manually:
brain.recall("anything", depth="deep")

API Reference

BrainMemory

BrainMemory(
 db_path="./memory", # Where to store files
 tag_extractor=my_func, # REQUIRED: (str) -> list[str]
 promote_checker=None, # Optional: (str) -> bool
 depth_detector=None, # Optional: (str) -> str
 duplicate_checker=None, # Optional: (str, str) -> bool
 conflict_resolver=None, # Optional: (str, str) -> str
 polling_interval=15, # Seconds between maintenance cycles
 rolling_days=3, # L1 retention period
 summary_days=30, # L2 summary period
 max_context_chars=15_000, # Context budget for LLM injection
 encryption_key=None, # Optional SQLCipher encryption
)

Core Methods

Method Description
save(content, source="chat", tags=None) Save to memory. source="archive" forces L3.
recall(query, depth=None) Recall relevant memories as Markdown.
search(query, limit=20) Search L3 archive, returns list[MemoryEntry].
promote(content, tags=None) Manually save to L3. Returns memory ID.
summarize(llm_callback=None) Generate a 30-day summary.
start_polling() Start background maintenance loop.
stop_polling() Stop background maintenance loop.
run_maintenance() Run one maintenance cycle manually.
get_stats() Get stats from all layers.
get_tag_stats() Get tag usage counts.
export_json(path=None) Export all data as JSON.
on(event, callback) Register an event hook.
close() Stop polling and close DB.

The 5 AI Callbacks

Callback Signature Default
tag_extractor (str) -> list[str] REQUIRED
promote_checker (str) -> bool len(content) > 200
depth_detector (str) -> str Keyword matching
duplicate_checker (str, str) -> bool SequenceMatcher > 0.85
conflict_resolver (str, str) -> str Keep newer text

Event Hooks

brain.on("before_save", lambda content, source: ...)
brain.on("after_save", lambda content, source: ...)
brain.on("before_promote", lambda content: ...)
brain.on("after_promote", lambda content, mem_id: ...)
brain.on("after_recall", lambda query, depth, result: ...)
brain.on("after_cycle", lambda stats: ...)

Examples

File Description
basic_usage.py Simplest setup with OpenAI
with_openai.py All 5 callbacks with OpenAI
with_anthropic.py Claude API integration
with_langchain.py LangChain agent + memory
custom_callbacks.py Custom callbacks + hooks
scheduling.py Polling, FastAPI, Celery

Compatibility

  • Python: 3.9, 3.10, 3.11, 3.12, 3.13
  • OS: Windows, macOS, Linux
  • Dependencies: None (Python stdlib only)
  • SQLite: Uses built-in sqlite3 module (FTS5 optional, graceful fallback)

Contributing

See CONTRIBUTING.md for guidelines.

License

MIT License. See LICENSE for details.

Origin

This library was born from the multi-layer RAG engine built for SandClaw — an AI-powered trading IDE. We extracted and open-sourced the memory system so any developer can use it.

AltStyle によって変換されたページ (->オリジナル) /