Person understanding.
Tested through memory and psychology.

All three horizons · native scores

Placement in each horizon · all seven systems

Competitor means are reconstructed from the paper's task/horizon cells using 150 weekly, 150 monthly and 300 quarterly questions. The paper does not publish a pooled all-task leaderboard. Reader and judge models differ; the pinned public release has 6,415 criteria, while the paper reports 7,054. Places are numerical comparisons, not official leaderboard ranks.

Answers used the stored biography, structured memory and retrieved original dialogue, including source-contained forgetting and edit instructions. Source retrieval supplied 32 unique passages per answer. Gold answers, scoring criteria and generator operation labels were excluded from persona construction and answering.

Paper Table 2 ↗ · Four frameworks span 13 memory configurations across Qwen3-32B, GPT-5-mini, DeepSeek-R1 and Gemini-3 Pro. Each framework appears once per task at its best published score. There is no published pooled all-task rank. Synthius leads the cited memory methods on five of seven tasks.

Question coverage. Synthius answered all 1,986 questions, including 446 adversarial questions. The 17 references with verified counts report judged accuracy on the 1,540-question non-adversarial subset; all 15 OmniMemEval backends below use that same subset. Counts are unconfirmed for the cited Hindsight, Backboard.io, mem9 and Supermemory vendor scores. These scopes apply to the cited configurations. Category counts ↗ · Zep scope ↗ · Mem0 scope ↗ · Memori scope ↗. Memtro’s separate full-dataset token-overlap score is shown in its table row. Full-coverage example ↗

Published LoCoMo accuracy references

Shared first at the displayed 94.7%: Synthius, Mi-Memory / MemStack and Zep. LoCoMo places group scores that round to the same one decimal; the underlying values are not exactly equal. One highest cited accuracy configuration is shown per system. Different readers, judges, review procedures and question scopes prevent a controlled leaderboard rank. MemoryLake’s separate composite is shown below.

MemoryLake separately reports 94.0% using a composite of F1, BLEU and model judging; it is not substituted for its accuracy result in the table. Source ↗

Published 32K option-selection references

Best cited option-selection score per memory system, across separate papers and common-harness runs. Backbones and split pins differ. Open-ended scores are excluded. Synthius's full Luna result is not its GPT-4.1-mini OmniMemEval pilot.

Completed cohort, not the full release. The ten persona IDs match the authors’ published comparison: 1, 2, 3, 5, 6, 8, 9, 10, 13 and 14. They represent 10 of 200 released personas (5%). All 1,587 eligible questions were completed; one retired task is excluded under native eligibility rules. No missing answers or grades; no manual score credits. Cohort selection ↗ · Frozen release ↗

Published ten-person comparison · all nine configurations

Highest numerical score among the nine listed configurations; not an official leaderboard placement. Synthius scored 54.74% before rounding: 1.04 percentage points above the best published headline, 3.24 above the best published memory configuration and 5.94 above Mem0. Persona IDs match, but the judge and privacy-input handling differ, and exact published scored-question lists are unavailable. This comparison does not establish statistical superiority. Paper comparison ↗ · Historical results ↗

Mem0 is the dedicated memory product in this table. The two textual-memory entries are model-plus-memory baselines. Long-context and agent-search entries are additional configurations, rather than independent memory products.

Two automated measurements

The additional checks change 65 input records and reduce the aggregate by 2.47 percentage points. They can reject an answer whose wording earns partial rubric credit, for example when a required local write is missing. The published Claude Code and Codex configurations also use final-answer-only grading for agentic tasks; 52.3% is a stricter automated metric, not a manual adjustment.

How the headline is calculated. The native score averages 1,534 normalized metric records: 1,505 ordinary records plus 29 complete before/after lifecycle pairs. Another 24 lifecycle probes were answered but have no opposite phase in the frozen questions, so they cannot form additional pairs. The score is not the percentage of 1,587 questions marked correct, or an equal-weight average of 30 task types.

Observed strengths and gaps

BFI · 20 published actor/evaluator configurations

20 configurations, not 20 independent products. Same BFI dimension-alignment family; actors and evaluators differ. The paper's 80.7 score is 16-Personalities, not Big Five, and is excluded.

2022 PER · two published teams

Numerical position only: our public-training writer evaluation differs from the teams’ held-out essay evaluation. The organizer reports only two teams; no valid official Synthius rank exists.

The organizer also reports LingJing writer-aggregated r = 0.306, a closer aggregation level but a different split. Only two teams entered this personality subtask; extra development variants are not independent competitors. Source ↗

Clean cohort · 57 common people

Computed from the authors’ saved person-level AUCs on the same57 people. Published preprocessing can change effective test rows. Synthius overall63-person AUC is0.5168; the matched comparison uses0.5226. This is the clean full-feature no-time procrastination track, not every study outcome.

Overall 63-person AUC 0.5168; matched 57-person comparison AUC 0.5226. The table and rank use the matched value. Random-forest difference −0.0771 AUC; participant-bootstrap 95% interval [−0.1591, +0.0062].