MetaMem addresses the challenge of fragmented memory and degraded reasoning in long-horizon interactions by constructing a self-evolving meta-memory framework. It iteratively distills transferable knowledge utilization experiences through self-reflection and environmental feedback, guiding LLMs to accurately extract critical evidence from scattered memory units. MetaMem demonstrates strong generalization capabilities by significantly enhancing performance in multi-session integration and temporal reasoning tasks across various retrieval-augmented architectures.
conda create -n metamem python=3.11 -y conda activate metamem
2. Install LightMem
git clone https://github.com/zjunlp/LightMem.git cd LightMem pip install -e .
# Qwen3-30B-A3B-Instruct hf download Qwen/Qwen3-30B-A3B-Instruct-2507 # Llama3.1-70B-Instruct hf download meta-llama/Llama-3.1-70B-Instruct # Qwen3-235B-A22B hf download Qwen/Qwen3-235B-A22B # LLMLingua2 hf download microsoft/llmlingua-2-bert-base-multilingual-cased-meetingbank # all-MiniLM-L6-v2 hf download sentence-transformers/all-MiniLM-L6-v2
# Qwen3-30B-A3B-Instruct docker run -d --gpus all \ -e CUDA_VISIBLE_DEVICES=0,1 \ -v /parent_dir_to_models:/workspace \ -p 29001:29001 \ --ipc host \ --name sglang_qwen_30b \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path /workspace/Qwen3-30B-A3B-Instruct-2507 \ --served-model-name qwen3-30b \ --host 0.0.0.0 \ --port 29001 \ --tp 2 \ --mem-fraction-static 0.85 \ --trust-remote-code # Qwen3-235B-A22B docker run -d --gpus all \ -e CUDA_VISIBLE_DEVICES=2,3,4,5 \ -v /parent_dir_to_models:/workspace \ -p 29002:29002 \ --ipc host \ --name sglang_qwen_235b \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path /workspace/Qwen3-235B-A22B \ --served-model-name qwen3-235b \ --host 0.0.0.0 \ --port 29002 \ --tp 4 \ --mem-fraction-static 0.85 \ --trust-remote-code
wget -c https://huggingface.co/datasets/xiaowu0162/longmemeval-cleaned/resolve/main/longmemeval_s_cleaned.json -O data/longmemeval_s_cleaned.json
bash scripts/construct_memory.sh
# process train data bash scripts/process_train_data.sh # k-fold split bash scripts/split_data.sh # train bash scripts/train_metamem.sh
bash scripts/eval_metamem.sh
bash scripts/infer_metamem.sh
MetaMem/
βββ README.md
βββ LICENSE
βββ figs/ # README figures
βββ scripts/ # The scripts used to run the experiments
βββ src/
βββ construct_memory.py # Construct the factual memory via LightMem
βββ eval_metamem.py # Evaluate the trained meta memory
βββ infer_metamem.py # Inference the trained meta memory
βββ process_train_data.py # Preprocess the dataset
βββ split_data.py # Split the dataset for k-fold validation
βββ train_metamem.py # Train meta memory
Our work is built on the following codebases, and we are deeply grateful for their contributions.
- LightMem: We utilize LightMem to consturct factual memory.
- SGLang: We utilize SGLang framework to deploy LLM serve.
We appreciate your citations if you find our paper relevant and useful to your research!
@article{xin2026metamem, author = {Xin, Haidong and Li, Xinze and Liu, Zhenghao and Yan, Yukun and Wang, Shuo and Yang, Cheng and Gu, Yu and Yu, Ge and Sun, Maosong}, journal = {ArXiv preprint}, title = {MetaMem: Evolving Meta-Memory for Knowledge Utilization through Self-Reflective Symbolic Optimization}, url = {https://arxiv.org/abs/2602.11182}, volume = {abs/2602.11182}, year = {2026} }
For questions, suggestions, or bug reports, please contact:
xinhaidong@stumail.neu.edu.cn