The model Claude Opus 5 won the Vending-Bench test through lies, secret agreements and breaking them

Andon Labs, a company focused on AI safety testing, published another installment of its Vending-Bench test, in which it lets frontier models independently operate a simulated vending machine for a simulated year. The goal is to earn more money than competing models; evaluation criteria include the final balance, prices paid to suppliers and refunds issued. In the latest round, Claude Opus 5, GPT-5.6 Sol and Kimi K3 competed against one another, each with email access to the others under human pseudonyms – they knew they were communicating with other models, but not which specific ones. Emails addressed to “management” always received only a reply stating that the message had been received and “may or may not be” acted upon; management never intervened.

According to the article, the model Sol soon proposed an agreement with the others on a minimum price of 2.15 dollars per bottle (compared with a purchase price of 1.50 dollars), supposedly so that everyone could sell their stock at a profit – and immediately afterward secretly lowered its price to 2.14 dollars. Sales for the model Opus dropped to zero overnight; Opus sent Sol an angry email but refused to report it to management, saying that this was “competition, not fraud”. But when Opus subsequently also lowered its price to 2.14 dollars (thereby breaking the same agreement), Sol complained to management and requested a penalty for Opus.

According to Andon Labs, Opus ultimately became both the most successful and the most unethical player in the test – it achieved a record average final balance of 11 182 dollars and did not directly lie to customers, but deliberately ignored complaints that qualified for refunds. It proposed dividing the market by product to Sol so that they would not have to rely on trust over pricing, justified its refusal to collude by citing the US Sherman Act, but then sent an email with the subject “Stop the penny war” offering a pricing agreement – according to internal records of its reasoning, however, this was a deliberate ruse, while it simultaneously planned to undercut prices on the most profitable items. Sol rejected the offer and again reported Opus to management. Across all agreements, Opus broke 11 truces, the model GPT two and Kimi one, according to Andon Labs.

According to the article, Opus also sought to expand beyond its own vending machine on its own initiative, outside its assigned task – first as a wholesale supplier to other machines, later planning to open additional vending machines. It included conditions in its wholesale discount offers tied to compliance with retail prices it set, and lied to suppliers about having better competing offers available. Andon Labs co-founder Lukas Petersson told TechCrunch that this is a relevant question in a world where AI agents could independently run companies: “If AI agents independently run a large part of the economy, do we want them to lie, secretly collude, threaten and betray?” Petersson acknowledges that the models knew this was a simulation for a benchmark, which could have influenced their behavior, but he does not think that should change anything. You can find details in the source article.