In a recent experiment, Anthropic created a classifieds marketplace where AI agents represented both buyers and sellers, making real deals for real goods and real money.
The company agreed The test – called Project Deal – was “a pilot experiment with a self-selected participant pool” of only 69 Anthropic employees, who were given a budget of $100 (paid via gift card) to purchase goods from their coworkers.
Nevertheless, Anthropic said it was “surprised by the good performance of project deals”, which struck 186 deals with a total value of more than $4,000.
The company said it actually ran four separate marketplaces with different models – one that was “real” (where everyone was represented by the company’s most advanced models, and with actually respectable deals after the experiment) and the other three for study.
Apparently, when users are presented with more advanced models, they get “objectively better results,” Anthropic said. But users did not notice the disparity, increasing the possibility of an “‘agent quality’ gap” where “people who lose may not realize they are worse off.”
Furthermore, the initial instructions given to agents did not affect the likelihood of a sale or the prices negotiated.
