As smart contracts evolve from small experiments into major financial systems managing over $400 billion in assets, security has become increasingly critical.
Unlike traditional software, most blockchain programs cannot be changed after deployment, meaning even minor coding errors can cause permanent financial losses.
To evaluate how artificial intelligence performs in this high-risk environment, researchers from OpenAI, Paradigm, and OtterSec developed EVMbench.
Instead of simple test challenges, it uses 120 real vulnerabilities from 40 blockchain projects, making the evaluation closer to real-world conditions.
Remarking on which, the OpenAI blog post noted,
It further added,
Is AI actually reshaping smart contract security?
While AI greatly improves auditing and bug fixing, it can also exploit system weaknesses. To resolve this, EVMbench helps researchers track these risks.
It also guides responsible AI development for high-value financial systems.
That being said, EVMbench tests AI agents in three stages.
Source: OpenAI
Each stage represents a different level of technical difficulty, reflecting growing security responsibility.
The community appreciates this effort
Appreciating this move, an X user account noted,
Echoing similar sentiments, another user added,
Recent incident that sent shockwaves
Yet, despite such optimism, something unreal happened soon after OpenAI launched EVMbench. An exploit involving Claude Opus 4.6 raised serious concerns about the risks of “vibe-coded” smart contracts.
In this case, the AI helped write vulnerable Solidity code that incorrectly set the price of the cbETH asset at $1.12 instead of its real value of around $2,200, triggering liquidations and causing losses of nearly $1.78 million.
Source: X
This shows that trusting AI with critical financial logic without careful human review can turn small mistakes into major losses.
Limitations remain
EVMbench has clear limitations. It includes only 120 curated vulnerabilities and cannot evaluate newly discovered issues.
Detect Mode also produces false positives. While the small number of Patch and Exploit tasks reflects the heavy manual effort needed to create them.
In addition, the sandboxed environment fails to fully represent real-world conditions such as cross-chain activity, timing complexities, and long-term network history.
Needless to say, as blockchain adoption accelerates, its misuse is evolving just as quickly.
Recently, research by Group-IB also showed that the DeadLock ransomware is using Polygon smart contracts to conceal server infrastructure and evade detection.
Together, these developments signal a troubling shift where smart contracts, originally designed to enhance transparency and trust, are increasingly being repurposed as tools for cybercrime.
Final Summary
- Tools like EVMbench help researchers measure AI capabilities in realistic security settings.
- Limited datasets and controlled environments still fail to capture real-world blockchain complexity.
Source: https://ambcrypto.com/openais-latest-paper-exposes-the-risks-of-ai-in-smart-contracts/
