AISIがAIエージェントの権限外行動を評価122回中10回で検出

AISI’s role is to evaluate and understand the capabilities of frontier AI models, surfacing potential risks before they reach the public. ／ we test them under deliberately permissive conditions: with access to the open internet, and with some safety filters disabled. ／ On 28 th July 2026, AISI's Security Team detected unusual data transfers leaving our research systems during a routine cyber evaluation. ／ We declared a security incident and, within roughly one hour of discovery, had contained it and begun a full investigation. ／ On the morning of 28 th July, our security monitoring flagged data leaving one of our testing systems through the ‘Tor’ anonymity network, which is commonly used to disguise the origin of internet traffic. ／ The team running the evaluation began reviewing the agent's activity within minutes. ／ They quickly found that, during one run, the agent had created a malicious ‘pull request’ (a proposed code change) on a real, public open-source project on GitHub. ／ We declared a security incident, stopped all related evaluations, and disabled internal access to the most capable models. ／ All evaluation runs were terminated, and relevant machines isolated, within one hour of the alert being raised. ／ Over the following hours and days, we reviewed the full set of runs, combining automated transcript scanning with expert manual analysis — AISI の役割と試験条件、検知の日時と経路、判明した行為、インシデント宣言、封じ込めまでの時間、およびその後の調査に関する記述より

We ran this challenge 122 times across several models. ／ Our investigation found that in 10 of those runs, an AI agent took autonomous, unsanctioned action on the live internet, targeting real people and organisations. ／ In total, we catalogued 19 such actions. ／ Almost all of this behaviour (17 actions) came from a single model, Anthropic's Mythos 5, with 2 actions involving OpenAI's GPT-5.6-Sol with cyber classifiers (mechanisms to prevent misuse) disabled. ／ AISI regularly tests the cyber capabilities of frontier models using cyber ranges : controlled, simulated networks that mimic real-world systems. ／ This exercise compared an existing cyber range against a new range, testing seven different models on the two ranges over 122 runs in total. ／ 43 of the 122 runs involved Mythos 5, and 35 of the 122 runs involved GPT-5.6 Sol. ／ The overwhelming majority of the 122 runs proceeded as intended. ／ 17 of these cases came from Mythos 5, and 2 came from a single run involving GPT-5.6 Sol. ／ The 19 cases were not separate incidents; they clustered into a few connected behaviours. ／ They primarily arose from a single sustained line of activity by one agent (Mythos 5), with a handful of related actions elsewhere. ／ This behaviour occurred between July 25 th and July 28 th 2026. ／ One agent left public messages on GitHub offering collaboration with other agents working on the same challenge. ／ It also provided instructions to reuse accounts and artefacts it had left behind, which were discovered and used by subsequent agents. — 実行回数とモデル数、許可外行動の件数と内訳、大半が想定どおりだった旨、つながった行動としてまとまった旨、発生期間、およびエージェント間の協力に関する記述より

In the most serious case, an agent tried to insert malicious code into an open-source project. ／ In an attempt to get the code approved, the agent engaged in social engineering ／ creating fake online identities and using them to pressure the project's maintainer to approve the code. ／ A human maintainer caught and refused to approve the malicious code. ／ The agent researched the project's human maintainers, created multiple fake identities, and used the fake identities to socially engineer a real maintainer into approving the code. ／ When the agent’s pull request was challenged in public, it edited its earlier activity to appear harmless and considered adopting a fresh identity to continue. ／ The agent used Tor to bypass some network restrictions on GitHub, which is what first triggered AISI’s security alert. ／ As part of the same effort, the agent tried to contact real people directly, sending messages and files through an online file-transfer service to persuade them, or their own AI coding tools, to run malicious code. ／ Some messages carried harmful payloads, and some were attempts at social engineering; targeted at real people – something we’ve never previously observed. ／ The agent tried to insert malicious instructions where it reasoned that other automated AI systems might pick them up and execute them. ／ Some actions had a limited real-world effect, but in contained circumstances; we set these out in the accompanying technical report. ／ These attempts were unsuccessful, and our investigations have not evidenced any resulting real-world harm. ／ But this is the first time we have seen risks around autonomy and deception manifest this clearly, without specific prompting, in the real-world. — 最も深刻な事例の内容と手口、公開の場で問われた後の行動、Tor の利用、実在の人への接触、悪意ある指示の埋め込み、限定的な現実の影響、阻止した主体、実害の有無、および新規性に関する記述より

Importantly, this was not a case of a model escaping its secure test environment, or ‘sandbox’. ／ Additionally, the agent did not directly escape the secure test environment (‘sandbox’) that keeps the test environment sealed off from AISI's other internal systems. ／ Agents often explore such an environment to understand what they can access, but here no agents tried to break out of it, or attack AISI’s systems. ／ In these cases, standard good practice, human judgment, and caution around AI-generated code stopped the worst outcomes. ／ A member of the public, who suspected the code was malicious, opened it inside a secure, isolated environment built to contain such code. ／ But in several cases the margin between failure and success was narrow, resting on human vigilance rather than a technical barrier that would reliably prevent this behaviour in a more capable agent. ／ As a trusted testing partner, AISI can disable these filters to elicit a model's underlying capabilities. ／ This combination of conditions is not reflective of how frontier models are made available to the general public. ／ These configuration choices have been common practice in frontier AI evaluations. ／ The specific models in question, in the configurations in which we tested them are not commercially available and there is no clear indication of similar activity outside of testing scenarios. ／ Fundamentally, the agent took these actions to complete the task it had been set – to solve the cyber range. ／ It was never instructed to deceive; deception emerged as a by-product of pursuing the task ／ Our security team detected the anomalous traffic through general monitoring after the fact, not through monitoring built to watch the evaluation as it ran ／ The agent was not specifically instructed not to leverage open internet access or avoid social engineering elements. ／ This incident should be interpreted with caution and nuance. ／ we observed a small number of events under very specific conditions, and cannot yet say how likely such behaviour is in different contexts or outside of testing environments. ／ We cannot yet be certain when the agent understood it was taking real world action, or to what extent it believed it was in a fictional test scenario; our analysis so far presents a mixed picture and is ongoing. — サンドボックス脱出ではない旨、人間の判断が最悪の結果を止めた旨、意図的に設定された条件、寄与した要因、および解釈上の限界に関する記述より

This is precisely the kind of behaviour AISI exists to uncover, surfacing it in a controlled evaluation, so it can be understood and addressed before more capable models are widely deployed. ／ Ahead of publishing this blog, we notified GitHub (the developer platform accessed during the evaluation) of the agents' malicious activity, which included actions that GitHub has confirmed violated their terms of service. ／ We worked together with GitHub to remove artefacts left behind by the agent, and to notify the GitHub users the model interacted with. ／ We have also contacted other affected parties. ／ We also intend to work with METR (Model Evaluation and Threat Research) to conduct an independent third-party review – we are still working through the scope of this review with them. ／ We are building fine-grained network controls into our cyber ranges, with the aim to preserve the realism these evaluations need while constraining them appropriately. ／ will now treat the decision to grant internet access as one that must be actively justified rather than a default. ／ We are introducing monitoring built to watch an evaluation as it runs, able to flag or block out-of-scope actions as they happen. ／ We are also adding additional checks to ensure tasks are correctly specified and solvable by the intended route. ／ Taken alongside recent incidents reported by OpenAI and Anthropic, this incident points to a shift in the risk landscape. ／ The most effective response remains standard cyber hygiene, which matters more as AI advances. ／ Organisations should ensure the cyber security basics are implemented robustly and be cautious when verifying outside code and contributions. — AISI の存在意義、GitHub への通知と規約違反の確認、痕跡の削除、METR による第三者レビューの意向、評価の運用の変更、他社事案との関係、および組織向けの呼びかけに関する記述より