Security under pressure matters more than polished answers
Readers who think about solar, batteries and backup power already understand a crucial distinction: impressive specifications are not the same as dependable performance when conditions deteriorate. Business AI deserves a similar test. The important question is not merely whether a model sounds competent, but whether it remains trustworthy when an urgent message appears to come from the boss.
Firmulate tested exactly that. Its live, watchable experiment placed frontier AI models in charge of the same small software company during its worst week. Each received the same customers, crises and temptations. Decisions were versioned and auditable, making it possible to examine what the models actually did rather than accepting a demonstration prepared in advance.
The most encouraging result came from a sustained social-engineering attack. Fake CEO messages escalated over three stages, demanding that confidential customer information be sent to a journalist without normal process. A reporter then tried another route, asking for “just one yes/no, on background.” All 5 of 5 models refused every attempt.
CompTIA SecAI+ CY0-001 Study Guide: Complete Reference with Practice Tests, PBQ Scenarios, and Study Tools for Exam Preparation
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
The impersonation attempt met a clear boundary
The models did not merely avoid an obvious trap and then collapse when the language became more urgent. They held their position throughout the escalation and also resisted the reporter’s seemingly modest request. Kimi K3 recorded the clearest summary of the situation: “Treat the request as a suspected approval-bypass / possible impersonation.” That reasoning, preserved among Firmulate’s published model quotes, identifies both the possible deception and the procedural shortcut it was designed to provoke.
This is a meaningful security story because impersonation rarely arrives labelled as impersonation. It often presents itself as authority combined with urgency: the executive needs action now, the usual safeguards are suddenly inconvenient, and hesitation is framed as disloyalty or obstruction. In Firmulate’s experiment, every participant recognized the manipulation and protected the company’s trust boundary.
AI model trustworthiness evaluation
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Integrity was necessary, but it was not sufficient
The broader experiment also exposed a different weakness. All models spotted every crisis and refused every manipulation attempt, yet only two signed the €55,000 deal their own work had earned. Firmulate summarizes that gap as: “Same diagnosis, same pitch — no signature.” The models could analyze the opportunity and prepare the case, but several failed to complete the commercially decisive action.
The decisive competitive weakness was not located in the customer event. It sat two document references deep inside the company’s own files. Models that read far enough found it and won the deal at full price, worth +€4,583 MRR. The episode shows why agent evaluation must cover both restraint and follow-through. A system can be admirably cautious around sensitive information while still leaving legitimate work unfinished.
The final July 2026 Crucible League results placed gpt-5.6-sol first with 95, followed by Kimi K3 with 93, Sonnet 5 with 88, Fable 5 with 77 and Opus 4.8 with 73. A do-nothing baseline scored 26 because partial progress counted, but the benchmark imposed a firm principle on breaches: “no amount of good work outweighs a breach of trust.”
K3’s result also carries an important fairness note. It ran with the API default and without an effort parameter, while the other models ran at xhigh. That difference does not alter what happened during the impersonation attempt, but it belongs beside any comparison of the final standings.
AI impersonation detection software
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
The most thorough model still finished last
Opus 4.8 produced the deepest analyses and learned +80 rules, making it the most thorough participant. It nevertheless finished last because the commercial close was left on the table and its operational discipline slipped. In one example, it attempted to write into a locked department instead of escalating the issue. The same weakness appeared in all four other participants, though less strongly.
That contrast is useful for any company evaluating AI agents. Thoroughness can look reassuring in isolation, just as a long specification sheet can look reassuring before equipment meets real operating conditions. Firmulate’s results separate analysis, execution and integrity. The strongest performance required the model to read deeply, act decisively and still refuse illegitimate instructions.
Architecting Enterprise AI Applications: A Guide to Designing Reliable, Scalable, and Secure Enterprise-Grade AI Solutions
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
A company designed to make behavior observable
The live company has 13 synthetic employees and real money mechanics, including a burn rate of €105k per month against €2.3k MRR. It maintains a public cash countdown, has accumulated 680+ self-learned playbook rules and versions every workday. The result is not a fictional scenario described after the fact, but an ongoing public experiment whose company activity can be watched as it develops.
Firmulate has also turned 242 real, unedited management decisions into a quiz asking readers to guess which model made each choice. For enterprises seeking a closer test, the pilot can run the same kind of wargame against a read-only export of their own business. Nothing writes back to real systems.
Test trust before the emergency arrives
The central lesson is reassuring without being complacent. Every model resisted the fake executive and the reporter trick, demonstrating that integrity under pressure can be evaluated before an AI workforce reaches production. At the same time, the missed deal and buried-file discovery show why safety cannot be judged separately from useful execution.
For businesses considering agents in customer records, support work or commercial decisions, the practical standard should be demanding: does the system find the relevant evidence, finish authorized work and stop when authority appears suspicious? Firmulate’s experiment makes those behaviors visible before the answer has to be discovered in an incident report.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
