AIエージェントのメモリは最適量がモデルで異なる、IBMが8モデルで検証

"When we scaled the evaluation to eight models — from a 30B dense model to frontier proprietary systems — one finding stood out:"／"Agentic memory is not a feature you switch on. It's a dose you calibrate to the model."／"We evaluated on AppWorld — 585 multi-step tasks (168 test_normal + 417 test_challenge) across 9 simulated apps (calendars, messaging, payments, and so on). Tasks are scored two ways: whether the agent fully completes each task (TGC — Task Goal Completion) and whether every variant of a scenario passes (SGC — Scenario Goal Completion, a stricter, all-or-nothing bar)."／"SGC — Scenario Goal Completion. A stricter, all-or-nothing metric. Each scenario bundles several variants of the same task (the same request with different data, phrasing, or edge conditions). SGC counts a scenario as passing only if the agent succeeds on every variant."（Results Across the Spectrum 節／Appendix: Understanding the Metrics 節）— IBM Research より

""Memory" here doesn't mean replaying a past transcript. It means a guideline set — strategies that worked, mistakes to avoid, and edge cases — distilled from the agent's own prior trajectories."／"No model weights are updated. The learning loop changes the guidance available to the agent, not the underlying model — which is exactly why it's cheap to adopt and portable across the eight models we tested."／"The agent attempts tasks and produces trajectories. ALTK-Evolve extracts behavioral guidelines from both its successful and unsuccessful runs. It consolidates those guidelines into a reusable set."／"Both memory configurations draw from the same guideline set, mined once (via the loop above) from AppWorld's training split only. What changes between them is only how that one set is delivered — the full guideline set injects all of it every step, while curated retrieval delivers a selected subset — never how the guidelines were produced, and no test-split data ever goes into building it."／"The number of guidelines a model mines depends on its own capability, so we report configurations by strategy — "full guideline set" vs. "curated retrieval" — rather than by raw counts, which aren't comparable across models."（Learning happens around the model, not inside it 節／The three configurations we compare 節）— IBM Research より

"Strong models with headroom want the full guideline set — every guideline, including rare edge-case lessons. They have the capacity to absorb and apply all of it. DeepSeek-V3.2 (671B MoE) climbed +9.5 percentage points in task completion when given its full self-mined guideline set."／"And the effect doesn't disappear at the top of the range: GPT-5.5 and Opus, both near the ceiling on TGC, still gain +7.2 and +7.1pp SGC respectively."／"Memory keeps paying off as long as a model has a remaining failure mode to target."／"Figure 1. Representative models in the three observed patterns. Bars show TGC on AppWorld test_normal for baseline vs. the best-memory configuration"（The Key Insight: Dosage Depends on Capability 節／The three patterns, in one view 節）— IBM Research より

"Smaller or weaker models get drowned by a large guideline set. For these, a tight, high-confidence core plus a handful of task-relevant guidelines retrieved per task works best. gpt-oss-120b (117B MoE) gained +16.1pp with this selective approach — while the full guideline set gained less and cost ~50% more tokens."／"A practical concern: injecting a full guideline set inflates every ReAct step's input, because the guidelines are re-sent each turn."／"Table 1. Average token use per task, accumulated across agent steps, measured against the no-memory baseline."／"Curated retrieval keeps cost near baseline. For weaker models, where selection wins on accuracy, it also wins on cost — the best of both worlds (+16.1pp TGC at only +5% tokens for gpt-oss-120b). Better performance here does not require more inference cost."／"Memory doesn't blow up the reasoning loop. DeepSeek runs about the same number of ReAct steps with memory as without (≈18–19 on average), so the added cost is input-token inflation, not longer trajectories."（The Key Insight: Dosage Depends on Capability 節／The Cheapest Memory Strategy Can Also Be the Best 節）— IBM Research より

"Already-saturated models show no measurable gain. We call this the saturated pattern — the label describes what we observed, not a proven cause. The model may already have been near its ceiling on these tasks, the guidelines may not have addressed its remaining failures, or it may not have applied the guidance effectively. GLM-5 (745B MoE) sat here in our runs."／"What puts a model into one pattern rather than another isn't simply parameter count. Benchmark headroom, context-window size, architecture, guideline quality, and task distribution all appear to shape where a model lands, and separating those factors is ongoing work."／"spending no extra context until their remaining failure modes are better understood"（The Key Insight: Dosage Depends on Capability 節）— IBM Research より

"The real efficiency lever in production is prompt caching: the static portion of the guideline set is identical across steps and can be cached, cutting effective cost substantially. Cache-aware prompt design — keeping the shared guideline-set prefix stable so it stays cacheable — is worth engineering for."（The Cheapest Memory Strategy Can Also Be the Best 節）— IBM Research より