Skip to content

Navigation Menu

Sign in
Sign up

复现PersonaMEM-V2 #216

Unanswered
YanZiBuGuiCHunShiWan asked this question in Q&A
Discussion options

EverMemOS Profile-only ablation for PersonaMem-v2.

Usage:

uv run python -m evaluation.cli --dataset personamemv2 --system evermemos_only_profile --run-name profile-only

name: "evermemos_only_profile"
version: "1.0"
description: "EverMemOS PersonaMem-v2 profile-only ablation"

adapter: "evermemos"

llm:
provider: "${LLM_PROVIDER:openai}"
model: "${LLM_MODEL:gpt-4.1-mini-2025年04月14日}"
api_key: "${LLM_API_KEY:xxxxxx}"
base_url: "${LLM_BASE_URL:xxxxx}"
temperature: ${LLM_TEMPERATURE:0.3}
max_tokens: ${LLM_MAX_TOKENS:32768}

search:
mode: "agentic"

dataset_overrides:
personamemv2:
add:
enable_profile_extraction: true
enable_clustering: true
enable_event_log_extraction: false
max_concurrent_requests: ${EVALUATION_ADD_MAX_CONCURRENT:4}
profile_extraction_mode: "scene"
profile_scenario: "assistant"
profile_min_confidence: 0.6
profile_min_memcells: 1
profile_life_max_items: 25
answer:
use_profile_in_answer: true
use_episode_in_answer: false
use_profile_classifier: true

按照这个配置跑Locomo-benchmark 分支的PersonaMEM-V2 复现的效果只有38%,请问是需要修改什么配置么?

You must be logged in to vote

Replies: 0 comments

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Q&A
Labels
None yet

AltStyle によって変換されたページ (->オリジナル) /