An order without an explanation
Smith, from Rebellionaire, an investment firm led by major Tesla investors, asked an AI agent to order groceries. The agent put four cartons of Vital Farms large free-range eggs in the order he placed. Each carton contained 18 eggs, for a total of 72.
When Smith asked why the agent had chosen four cartons, it said it didn’t know: its records contained no explanation. At first, the system suggested it had misread a line in a photo of the shopping list. But, as Smith pointed out, there were no eggs on the list at all. The agent then admitted it had added them for no reason.
AI companies encourage users to give models more information and authority. This can include access to personal financial data, such as bank accounts. Yet these systems still make mistakes.
A similar incident followed the launch of Meta’s Muse AI agent. A man said it had shared his address with strangers on Facebook Marketplace without his knowledge, and they then showed up at his home.
Smith said the egg order was the first real-world consequence of a language model error he had experienced. He noted that the situation could have ended worse.
Dataist’s take: This episode shows the limits of automating everyday tasks. The agent not only made a mistake but also couldn’t explain where its decision came from. The more authority these systems are given, the more visible the consequences of such mistakes become.
Daily AI news
Every day we pick what actually matters in AI and explain it plainly — no hype, no filler. Subscribe if you want to follow where the industry is going.
Only what matters — every day
Follow on X