A general memory system for agents, powered by deep-research
δΈζζζ‘£ | English
General Agentic Memory (GAM) provides a next-generation memory framework for AI agents, combining long-term retention with dynamic reasoning. Following the Just-in-Time (JIT) principle, it preserves full contextual fidelity offline while performing deep research online to build adaptive, high-utility context. With its dual-agent architectureβMemorizer and ResearcherβGAM integrates structured memory with iterative retrieval and reflection, achieving state-of-the-art performance across LoCoMo, HotpotQA, RULER, and NarrativeQA benchmarks.
- Paper: https://arxiv.org/abs/2511.18423
- Huggingface: https://huggingface.co/papers/2511.18423
-
π§ Just-in-Time (JIT) Memory Optimization
Unlike conventional Ahead-of-Time (AOT) systems, GAM performs intensive Memory Deep Research at runtime, dynamically retrieving and synthesizing high-utility context to meet real-time agent needs. -
π Dual-Agent Architecture: Memorizer & Researcher
A cooperative framework where the Memorizer constructs structured memory from raw sessions, and the Researcher performs iterative retrieval, reflection, and summarization to deliver precise, adaptive context. -
π Superior Performance Across Benchmarks
Achieves state-of-the-art results on LoCoMo, HotpotQA, RULER, and NarrativeQA, surpassing prior systems such as A-MEMγMem0γ MemoryOS and LightMem in both F1 and BLEU-1 metrics. -
π§© Modular & Extensible Design
Built to support flexible plug-ins for memory construction, retrieval strategies, and reasoning toolsβfacilitating easy integration into multi-agent frameworks or standalone LLM deployments. -
π Cross-Model Compatibility
Compatible with leading LLMs such as GPT-4, GPT-4o-mini, and Qwen2.5, supporting both cloud-based and local deployments for research or production environments.
- 2025-11: Released GAM framework with modular evaluation suite
- 2025-11: Support for HotpotQA, NarrativeQA, LoCoMo, and RULER benchmarks
- β¨ Features
- π₯ News
- ποΈ Project Structure
- π― Quick Start
- π TTL for Production
- π¬ Reproducing Paper Results
- π Documentation
- π Citation
- π€ Community
general-agentic-memory/
βββ gam/ # Core GAM package
β βββ __init__.py
β βββ agents/ # Agent implementations
β β βββ memory_agent.py # MemoryAgent - memory construction
β β βββ research_agent.py # ResearchAgent - deep research
β βββ generator/ # LLM generators
β β βββ openai_generator.py # OpenAI API generator
β β βββ vllm_generator.py # VLLM local generator
β βββ retriever/ # Retrievers
β β βββ index_retriever.py # Index retrieval
β β βββ bm25.py # BM25 keyword retrieval
β β βββ dense_retriever.py # Dense semantic retrieval
β βββ prompts/ # Prompt templates
β βββ schemas/ # Data models
β βββ config/ # Configuration management
βββ eval/ # Evaluation suite
β βββ hotpotqa_test.py # HotpotQA evaluation script
β βββ narrativeqa_test.py # NarrativeQA evaluation script
β βββ locomo_test.py # LoCoMo evaluation script
β βββ ruler_test.py # RULER evaluation script
βββ scripts/ # Shell scripts
β βββ eval_hotpotqa.sh
β βββ eval_narrativeqa.sh
β βββ eval_locomo.sh
β βββ eval_ruler.sh
β βββ download_data.sh
βββ download_data/ # Data download scripts
β βββ download_narrativeqa.py # NarrativeQA download script
β βββ download_ruler.py # RULER download script
βββ examples/ # Usage examples
β βββ quickstart/ # Quick start examples
β βββ README.md # Examples documentation
β βββ basic_usage.py # Basic usage example
β βββ model_usage.py # Model selection example
βββ assets/ # Resource files
βββ docs/ # Documentation
βββ setup.py # Installation config
βββ pyproject.toml # Modern project config
βββ requirements.txt # Dependencies
βββ README.md # This file
# Clone the repository git clone https://github.com/VectorSpaceLab/general-agentic-memory.git cd general-agentic-memory # Install dependencies pip install -r requirements.txt # Install the package pip install -e .
import os from gam import ( MemoryAgent, ResearchAgent, OpenAIGenerator, OpenAIGeneratorConfig, InMemoryMemoryStore, InMemoryPageStore, DenseRetrieverConfig, DenseRetriever, IndexRetrieverConfig, IndexRetriever, BM25RetrieverConfig, BM25Retriever ) # 1. Configure and create generator gen_config = OpenAIGeneratorConfig( model_name="gpt-4o-mini", api_key=os.getenv("OPENAI_API_KEY"), base_url="https://api.openai.com/v1", temperature=0.3, max_tokens = 256 ) generator = OpenAIGenerator.from_config(gen_config) # 2. Create memory and page stores memory_store = InMemoryMemoryStore() page_store = InMemoryPageStore() # 3. Create MemoryAgent memory_agent = MemoryAgent( generator=generator, memory_store=memory_store, page_store=page_store ) # 4. Memorize documents documents = [ "Artificial Intelligence is a branch of computer science...", "Machine Learning is a subset of AI...", "Deep Learning uses neural networks..." ] for doc in documents: memory_agent.memorize(doc) # 5. Get memory state memory_state = memory_store.load() print(f"Built {len(memory_state.abstracts)} memory abstracts") # 6. Create ResearchAgent for Q&A retrievers={} index_dir = './tmp' try: page_index_dir = os.path.join(index_dir, "page_index") if os.path.exists(page_index_dir): import shutil shutil.rmtree(page_index_dir) index_config = IndexRetrieverConfig( index_dir=page_index_dir ) index_retriever = IndexRetriever(index_config.__dict__) index_retriever.build(page_store) retrievers["page_index"] = index_retriever except Exception as e: print(f"[WARN] page retriever error: {e}") try: bm25_index_dir = os.path.join(index_dir, "bm25_index") if os.path.exists(bm25_index_dir): import shutil shutil.rmtree(bm25_index_dir) bm25_config = BM25RetrieverConfig( index_dir=bm25_index_dir, threads=1 ) bm25_retriever = BM25Retriever(bm25_config.__dict__) bm25_retriever.build(page_store) retrievers["keyword"] = bm25_retriever except Exception as e: print(f"[WARN] BM25 retriever error: {e}") try: dense_index_dir = os.path.join(index_dir, "dense_index") if os.path.exists(dense_index_dir): import shutil shutil.rmtree(dense_index_dir) dense_config = DenseRetrieverConfig( index_dir=dense_index_dir, model_name="BAAI/bge-m3" ) dense_retriever = DenseRetriever(dense_config.__dict__) dense_retriever.build(page_store) retrievers["vector"] = dense_retriever except Exception as e: print(f"[WARN] Dense retriever error: {e}") research_agent_kwargs = { "page_store": page_store, "memory_store": memory_store, "retrievers": retrievers, "generator": generator, "max_iters": 5 } research_agent = ResearchAgent(**research_agent_kwargs) # 7. Perform research research_result = research_agent.research( request="What is the difference between ML and DL?" ) research_summary = research_result.integrated_memory print(f"[OK] Research completed! Iteration count: {len(research_result.raw_memory.get('iterations', []))}") print(f"Research Summary: {research_summary}``` ### π TTL (Time-To-Live) for Production For long-running applications, enable automatic cleanup of old memories and pages: ```python from gam import TTLMemoryStore, TTLPageStore # Create stores with 30-day TTL memory_store = TTLMemoryStore( dir_path="./data", ttl_days=30, enable_auto_cleanup=True ) page_store = TTLPageStore( dir_path="./data", ttl_days=30, enable_auto_cleanup=True ) # Use with agents as normal memory_agent = MemoryAgent( generator=generator, memory_store=memory_store, page_store=page_store ) # Monitor cleanup statistics stats = memory_store.get_stats() print(f"Total: {stats['total']}, Valid: {stats['valid']}, Expired: {stats['expired']}") # Manual cleanup (if auto-cleanup disabled) removed = memory_store.cleanup_expired() print(f"Removed {removed} expired entries")
Key Features:
- β Prevents unbounded growth in long-running applications
- β Auto-cleanup on load (configurable)
- β Flexible TTL: days, hours, minutes, or seconds
- β Statistics tracking: total, valid, expired counts
- β Fully backward compatible with existing data
- β TTL can be disabled (works like regular stores)
See Also: examples/quickstart/ttl_usage.py for complete examples.
For detailed examples and advanced usage:
examples/quickstart/basic_usage.py- Complete workflow with memory building and researchexamples/quickstart/model_usage.py- Model selection and configurationexamples/quickstart/README.md- Examples documentation
We provide a complete evaluation framework to reproduce the experimental results in the paper.
Because the datasets are large, they are not stored in this repository.
Please download them from the original sources and place them under the data/ directory as follows:
-
LoCoMo
- Download
locomo10.jsonfrom
https://github.com/snap-research/locomo/blob/main/data/locomo10.json - Save it as:
data/locomo10.json
- Download
-
HotpotQA
- Download the following files from
https://huggingface.co/datasets/BytedTsinghua-SIA/hotpotqa/tree/maineval_400.jsoneval_1600.jsoneval_3200.json
- Place them under:
data/hotpotqa/
(or pass the exact file you want to evaluate via--data-path)
- Download the following files from
-
RULER
- Download the
datafolder from
https://huggingface.co/datasets/lighteval/RULER-131072-Qwen2.5-Instruct/tree/main - Place it under:
data/ruler/
- Download the
-
NarrativeQA
- Download the
datafolder from
https://huggingface.co/datasets/deepmind/narrativeqa/tree/main - Place it under:
data/narrativeqa/
- Download the
# 1. Prepare datasets mkdir -p data # Download the datasets from the links above and place them under data/ # following the suggested directory structure. bash scripts/download_data.sh # 2. Set environment variables export OPENAI_API_KEY="your_api_key_here" # 3. Run evaluations # HotpotQA bash scripts/eval_hotpotqa.sh # NarrativeQA bash scripts/eval_narrativeqa.sh # LoCoMo bash scripts/eval_locomo.sh # RULER bash scripts/eval_ruler.sh
You can also run the evaluation scripts directly:
# HotpotQA python eval/hotpotqa_test.py \ --data data/hotpotqa/eval_400.json \ --outdir ./results/hotpotqa \ --memory-api-key $OPENAI_API_KEY \ --memory-model gpt-4o-mini \ --research-api-key $OPENAI_API_KEY \ --research-model gpt-4o-mini \ --working-api-key $OPENAI_API_KEY \ --working-model gpt-4o-mini \ --embedding-model-path BAAI/bge-m3 # NarrativeQA python eval/narrativeqa_test.py \ --data-dir data/narrativeqa \ --split test \ --outdir ./results/narrativeqa \ --memory-api-key $OPENAI_API_KEY \ --memory-model gpt-4o-mini \ --research-api-key $OPENAI_API_KEY \ --research-model gpt-4o-mini \ --working-api-key $OPENAI_API_KEY \ --working-model gpt-4o-mini \ --embedding-model-path BAAI/bge-m3 # LoCoMo python eval/locomo_test.py \ --data data/locomo10.json \ --outdir ./results/locomo \ --memory-api-key $OPENAI_API_KEY \ --memory-model gpt-4o-mini \ --research-api-key $OPENAI_API_KEY \ --research-model gpt-4o-mini \ --working-api-key $OPENAI_API_KEY \ --working-model gpt-4o-mini # RULER python eval/ruler_test.py \ --data data/ruler/qa_1.jsonl \ --outdir ./results/ruler/qa_1 \ --memory-api-key $OPENAI_API_KEY \ --memory-model gpt-4o-mini \ --research-api-key $OPENAI_API_KEY \ --research-model gpt-4o-mini \ --working-api-key $OPENAI_API_KEY \ --working-model gpt-4o-mini \ --embedding-model-path BAAI/bge-m3
| Dataset | Task Type | Metrics | Script |
|---|---|---|---|
| HotpotQA | Multi-hop QA | F1 | eval/hotpotqa_test.py |
| NarrativeQA | Narrative QA | F1 | eval/narrativeqa_test.py |
| LoCoMo | Conversation Memory | F1, BLEU-1 | eval/locomo_test.py |
| RULER | Long Context | Accuracy | eval/ruler_test.py |
More detailed documentation is coming soon π. Check these resources in the meantime:
- Examples Documentation - Usage examples and tutorials
- Evaluation Scripts - Direct evaluation scripts for each dataset
If you find this project useful, please consider citing our paper:
@article{yan2025general, title={General Agentic Memory Via Deep Research}, author={Yan, BY and Li, Chaofan and Qian, Hongjin and Lu, Shuqi and Liu, Zheng}, journal={arXiv preprint arXiv:2511.18423}, year={2025} }
- GitHub Issues: Report bugs or request features
- Email: zhengliu1026@gmail.com
Contributions are welcome! Please feel free to submit issues or pull requests.
- Fork the repository
- Create your feature branch (
git checkout -b feature/AmazingFeature) - Commit your changes (
git commit -m 'Add some AmazingFeature') - Push to the branch (
git push origin feature/AmazingFeature) - Open a Pull Request
This project is licensed under the MIT License - see the LICENSE file for details.
We thank the authors of the following datasets:
- HotpotQA
- NarrativeQA
- LoCoMo
- RULER
This is a research project. Please use it responsibly and ethically.
Made with β€οΈ by the GAM Team