pass^k

Visa's first Agentic Commerce Benchmark put 22 models through 15 shopping tasks. They passed 57.2% of runs, passed all three attempts only 34.2% of the time, and still bought something in about 54% of failed runs.