· 7 min read
Agents Take Actions That Can't Be Undone: Designing for Irreversibility
A chatbot writes words, and if it is wrong you delete it. An agent acts: it sends the email, charges the card, deletes the record, places the order, and those actions turn one way at machine speed. In 2012 an automated trading system shipped a bad deployment with no kill switch and lost 440 million dollars in about five minutes. The code was the last domino. The failure was the system that let it run with no brakes.