-
Notifications
You must be signed in to change notification settings - Fork 912
复现PersonaMEM-V2 #216
EverMemOS Profile-only ablation for PersonaMem-v2.
Usage:
uv run python -m evaluation.cli --dataset personamemv2 --system evermemos_only_profile --run-name profile-only
name: "evermemos_only_profile"
version: "1.0"
description: "EverMemOS PersonaMem-v2 profile-only ablation"
adapter: "evermemos"
llm:
provider: "${LLM_PROVIDER:openai}"
model: "${LLM_MODEL:gpt-4.1-mini-2025年04月14日}"
api_key: "${LLM_API_KEY:xxxxxx}"
base_url: "${LLM_BASE_URL:xxxxx}"
temperature: ${LLM_TEMPERATURE:0.3}
max_tokens: ${LLM_MAX_TOKENS:32768}
search:
mode: "agentic"
dataset_overrides:
personamemv2:
add:
enable_profile_extraction: true
enable_clustering: true
enable_event_log_extraction: false
max_concurrent_requests: ${EVALUATION_ADD_MAX_CONCURRENT:4}
profile_extraction_mode: "scene"
profile_scenario: "assistant"
profile_min_confidence: 0.6
profile_min_memcells: 1
profile_life_max_items: 25
answer:
use_profile_in_answer: true
use_episode_in_answer: false
use_profile_classifier: true
按照这个配置跑Locomo-benchmark 分支的PersonaMEM-V2 复现的效果只有38%,请问是需要修改什么配置么?
All reactions
-
👀 1