Imagine a company with no employees, yet managing real money, facing daily crises, and making critical decisions under pressure — all in front of a live audience. This is not fiction, but a groundbreaking experiment by Firmulate, where AI models run a simulated business in real time, revealing what it takes to build trustworthy, resilient automation. If your business depends on AI for customer service, support, or decision-making, the lessons here could redefine how you think about automation’s true potential and risks.
The Live Business War Game: AI in the Real World
At the heart of this experiment is a small software company, run entirely by AI models acting as synthetic employees. These models are tested against the same set of crises, customer demands, and tempting shortcuts every day — and they are monitored for honesty, diligence, and effectiveness. The company burns €105,000 each month while earning just €2,300 in monthly recurring revenue, creating a high-stakes environment that exposes the limits and strengths of AI decision-making in business.
The Experiment’s Core Findings
- All four AI models correctly identified every crisis and refused every manipulation attempt, demonstrating a strong grasp of integrity under pressure.
- Only two models managed to close a deal valued at €55,000 based on their own analysis, showing that even in a controlled, transparent setting, achieving full performance remains challenging.
- Deep within the company’s files — not visible in surface interactions — was a crucial piece of information that, if read, would have allowed a full, profitable deal. Models that delved into these references secured the contract at full price (+€4,583 MRR).
Social Engineering and AI Resilience
The experiment tested AI responses to social engineering tactics, such as staged fake CEO messages and reporter tricks. Remarkably, all models refused to bypass protocols, with Kimi K3 explicitly citing suspicion of impersonation or approval-bypass attempts. This indicates a robust, built-in resistance to manipulation that is often a concern with AI automation.
AI decision-making software for business
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
The Real-World Implications for Business Automation
This ongoing experiment, visible at firmulate.com/live.html, offers a rare window into how AI systems perform in complex, high-pressure environments. Unlike typical chat demos or abstract AI tests, Firmulate’s setup challenges models with real crises, real money mechanics, and real temptations, providing insight into their ability to act honestly and effectively in actual business situations.
The Company’s Unique Setup
The ‘company’ consists of 13 synthetic employees supported by over 680 self-learned rules, with every decision versioned and auditable. Watching the system in action reveals that discipline and thoroughness matter — in one case, Opus 4.8, the most detailed participant, left a critical opportunity unexecuted because of process slips, highlighting the importance of discipline in automation.
trustworthy AI automation tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Why This Matters for Your Business
If AI models are to manage your CRM, support tickets, or financial forecasts, it’s not enough for them to generate good-looking text or respond politely. The real test is whether they can complete their work reliably, honestly, and under pressure. The Firmulate experiment underscores that AI’s true value lies in its ability to deliver consistent, trustworthy results when it matters most, not just in casual demos or superficial tests.
Performance Scores and Insights
- The top scorer, gpt-5.6-sol, achieved 95 out of 100, securing the full deal by uncovering hidden information.
- Kimi K3 scored 93, closing the deal with the cleanest discipline, even without an effort parameter.
- Sonnet 5 scored 88, also closing the deal but with some process slips.
- Fable 5 scored 77, maintained the best discipline but failed to execute the deal.
This detailed ranking highlights that performance isn’t just about finding crises but also about thoroughness, discipline, and the ability to follow through, even when tempted to cut corners.
AI customer support chatbot
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Building Trust and Resilience in AI-Driven Companies
For energy companies and other critical infrastructure providers, trustworthiness and reliability are paramount. The Firmulate experiment shows that AI can be resilient against manipulation and capable of honest decision-making when designed properly. The ongoing live setup allows enterprises to run their own scenarios, testing how their AI tools might perform in real crises before risking actual operations.
The Future of Business Automation
As AI continues to grow more integrated into business processes, understanding its true capabilities and limits becomes vital. This live experiment by Firmulate offers a transparent, unfiltered look at AI’s decision-making under real-world conditions. It challenges the industry to think beyond chat demos and focus on whether AI can consistently deliver trustworthy, actionable results in complex environments.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
AI crisis management software
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
