Gemma 4 31Bは金融の検証で有料モデルと同点、費用は約1/40

"A frontier model doing its own retrieval over our content with raw vector search (a good proxy for your typical market data/content MCP) costs roughly 3x more per question than the same model running in our harness with AlphaSense Search."／"the AlphaSense Search-powered answers are preferred more than 2:1 compared to the vector-RAG baseline, all other things being held equal."／"Every configuration, including all models across the vector-RAG baseline and AlphaSense Search alike, runs over the same indexed corpus."／"The gaps reported here are the floor, measuring just the search agent"（冒頭の要点／The Advantage This Benchmark Doesn't Measure 節）— AlphaSense より

"The benchmark is based on 245 challenging, multi-step finance questions modeled on real analyst workflows: time-anchored, cross-source, and deliberately hard."／"Most questions have implicit or explicit date requirements"／"Typos, ticker-only references, mixed languages, and contradictory instructions are part of the workload."／"Company Research"／"Financial Data & Modeling"（Methodology 節／Appendix の Time Sensitivity／設問区分の表の行名）— AlphaSense より

"Today's quality bar; strong quality/cost balance"／"Solid worker but expensive compared to alternatives"／"Sonnet-5 quality at ~40x lower cost"／"Quality scores are medians from our July 2026 harness run, relative to the vector-RAG baseline of 1.0."（per-model 表の Note 列／表の下の注記）— AlphaSense より

"Competitive open-weight option"／"Fast, light"／"Promising fine-tuning base"（Results 節の per-model 表の Note 列）— AlphaSense より

"Costs in the chart are median per-question totals including retrieval-loop tokens, not list token prices."／"Token price does not equal question cost."／"Kimi K3 is a lot cheaper per token than GPT-5.6 Sol, yet more expensive per question, because it spends many more tokens assembling context before it can answer."／"When you buy tokens by the million but deliver answers by the task, tokens-to-completion is the metric that matters."／"Opus 5 scores below its predecessor Opus 4.8 (1.46 vs. 1.60) at several times the per-question cost."（per-model 表の下の注記／A few datapoints deserve commentary 節）— AlphaSense より

"Gemma 4-31B matches Sonnet 5 on answer quality at roughly 40x lower cost."／"where it runs 10–20x faster."／"The combination of good-enough quality for many tasks and very low latency unlocks high-volume and interactive use cases that would be uneconomical on frontier models."（Open weights will play a critical role 節の各文）— AlphaSense より

"For each question, we generate an independent scoring rubric before any model produces an answer."／"Pre-registering rubrics is useful because it prevents graders from drifting toward whatever the models happen to output, and it lets us encode hard requirements up front."／"The generator sees only the question, the as-of date, and two derived signals (time-sensitivity and breadth-sensitivity)."／"A rubric is grading the evidence, not the prose of the output."／"There is a separate rubric that grades the generated answer"／"Retrieval quality and answer quality are graded independently by language-model judges."／"A rubric has 7 axes for each criterion (a question can have 1-60 criteria depending on difficulty)."／"Each criterion carries an integer weight from −5 to +5, never 0."（Methodology 節／What is a rubric? 節／How is a rubric evaluated? 節）— AlphaSense より

"Results are reported as relative scores (vector-RAG baseline = 1.0) and head-to-head preferences."／"Quality scores are medians from our July 2026 harness run, relative to the vector-RAG baseline of 1.0."／"Every configuration, including all models across the vector-RAG baseline and AlphaSense Search alike, runs over the same indexed corpus."／"That isolates the retrieval, harness, and model variables, which is a useful experimental design."（Methodology 節／Results 節の注記／The Advantage This Benchmark Doesn't Measure 節）— AlphaSense より

"Holding the baseline model (Opus 4.8) constant and swapping only the retrieval layer moves preference to 1.7:1"／"Choosing the best frontier model (GPT-5.6 Sol) on top of AlphaSense retrieval pushes preference to 1.8:1."／"Leveraging the optimized AlphaSense harness around that model (i.e., query generation, planning, search score pass-through) gets to 2.1:1"／"routing each task to the best model for that task type reaches 2.8:1"／"No model wins more than a plurality of the benchmark questions."／"Different models win on different question types: metric extraction, screening, long-form synthesis."（Results 節／A few datapoints deserve commentary 節）— AlphaSense より

"Gemma is a great candidate for sub-agent delegation by a frontier lead model."／"Planning, deep multi-step reasoning, coding, data analysis, and Excel/PowerPoint generation stay routed to frontier models"（Open weights will play a critical role 節／What's Next 節）— AlphaSense より

"Quality scores are medians from our July 2026 harness run, relative to the vector-RAG baseline of 1.0."／"Gemma 4-31B matches Sonnet 5 on answer quality at roughly 40x lower cost."（Results 節の注記／Open weights will play a critical role 節）— AlphaSense より