AIの運用コストと従量課金の構造【Amazonの予算860%超過報道】

The biggest blunder, so far, is the $1.8-million bill that came from a failed Claude Sonnet AI deployment, which was supposed to match author details with listings on Amazon, representing an 860% increase over the allocated budget that was only detected some five months after the issue started happening. Other problems that surfaced include a $541,000 additional cost that came from a project building, ironically, a financial auditing tool, and a $134,000 extra expense for a system designed to reduce delivery times in the company’s logistics network. ／ “As with any new technology, we’re experimenting, learning and improving how we use it, including how we drive cost efficiencies,” Amazon said in an internal presentation, according to FT. “Cherry-picking small, isolated examples where teams are learning from one another and portraying them as business as usual doesn’t reflect how teams across Amazon are using AI.” And even though overspending more than a million dollars on failed AI projects might seem excessive for the average person, the tech giant’s latest quarterly revenue sits at more than $181 billion, meaning these excess AI expenses don’t even account for 0.1% of what it makes in a month. — 報じられた超過額・検知までの期間・Amazon側の説明・売上規模との比較に関する記述より

These mistakes used to be “trivially cheap,” but AI models made them “catastrophically expensive,” especially as token spending drastically increased with the deployment of AI agents. ／ And as agents took over and AI providers switched from subscription to per-token models — 失敗にかかる費用の変化と、定額制からトークン従量制への移行に関する記述より

And as agents took over and AI providers switched from subscription to per-token models, costs have become so great that companies are using up their annual budgets in a matter of weeks. While this might not be an immediate issue for tech giants like Amazon and Microsoft, it is unsustainable for most other companies out there. — 従量課金への移行による費用の膨張と、企業規模による影響の差に関する記述より

This is not the first time that AI-related issues have cropped up in Amazon’s workflow. Earlier this year, AWS reported several outages that were driven by AI coding bot blunders, but the company fixed this by limiting the access of AI agents instead of giving them the same permissions as the senior engineers that they’re tied to. It also used to have an internal leaderboard that showed which employees used AI the most, but has since dropped it as spiraling AI costs made them think twice about the policy. ／ Many tech companies have been pushing their people to use AI, supposedly to increase productivity through tokenmaxxing. However, the Uber CTO said that there is no link between this policy and shipping successful products. — 社内順位表の廃止、エージェントの権限制限、利用促進と成果の関係に関する記述より

In the case of ARC-AGI-3, we discovered that turning on two API settings we use in ChatGPT and Codex—retained reasoning and compaction—tripled scores and cut output tokens by 6x on the public task set. — 設定変更によるスコアと出力トークンの変化に関する記述より