Commentary: AI will rip off consumers unless they fight back
Published in Op Eds
My great uncle Fred, a former World War II prisoner living in the Cascade Mountains of Oregon, had little to his name but a four-wheel-drive truck. When the treacherous winter storms came, he would drive around, offering to help poor souls who had gotten stuck in the snow. But he was no good Samaritan: He charged exorbitant prices, sussing them out to determine the maximum he could extract. He had all the power and he knew how to use it. Call it siege pricing.
Today’s online markets, with the help of big data and artificial intelligence, are poised to become like digital doppelgängers of my mercenary great uncle — monitoring consumers, seeking weaknesses, extracting the maximum. People need to fight back while they still can.
The practice of surge pricing — charging different amounts depending on conditions — isn’t always bad. Consider raising the prices of plywood or bottled water before a storm. It discourages hoarding and scalping, as well as encouraging sellers to get supplies where they’re needed. In the context of a thriving competitive market, this can work fine even if it doesn’t always seem fair.
So when does reasonable surge pricing become exploitative siege pricing?
As a data scientist who used to work in the travel sector, I have some experience of airline pricing. The normal pattern is to make money on flexibility and perks when business travelers book (typically during the week and especially on Tuesday), and on better routes and sitting together when leisure travelers are buying (usually on the weekends, and especially a few weeks before a long weekend).
But the combination of surveillance and AI can empower sellers to go beyond general categories, sussing out exactly how much money an individual buyer is willing to part with. A person using an expensive Mac laptop, or whose zip code and purchase history suggest affluence, might see a higher price.
Last year, a Federal Trade Commission study found that intermediaries are actively offering retailers tools to achieve this. Another recent investigation found that online grocer Instacart charged significantly different prices for the same items purchased from the same stores at the same time. (Instacart said its differentials resulted from a randomized experiment that it ended.)
Retailers generally say they avoid individualized pricing, for fear of alienating customers. Yet other techniques, perhaps intended to improve the customer’s experience, can have a similar effect. Consider the way online stores such as Amazon rearrange themselves depending on who is shopping. Recently, a friend and I searched for “beanbag chair” on Amazon at the same time. Our first results offered approximately the same product, with a similarly foreign brand name, except mine cost $188 and hers $88. I had to scroll well beyond my first page to find her listing, which I wouldn’t have known existed if she hadn’t told me.
In economic parlance, individualization has great potential to reduce the “consumer surplus” — that is, the difference between what a buyer actually pays and what they’d be willing to pay if pushed to the limit. The British economist Alfred Marshall, who formalized the concept more than a century ago, touted it as a key benefit of free market capitalism: Getting something for less than you would have paid feels great. Yet in a world of personalized price discrimination, this advantage of the free market can break down. And if one seller dominates a market, or if all the sellers use similar technology, consumers have no escape.
In principle, price discrimination could also benefit the poor, granting them access to goods and services they otherwise couldn’t afford. Instead, it’s all too often used to exploit them at their most vulnerable moments. Suppose you desperately need cash. Typing “I need a loan fast” into a search engine — or opening an app provided by your employer — can quickly connect you with lenders eager to charge you an exorbitant interest rate.
The constantly shape-shifting world of online marketing can feel overwhelming, too pervasive to change. Don’t succumb to learned helplessness. Governments have implemented rules to shed light on the pricing of goods and services such as cars, rental apartments and cab rides. Consumers can and should demand better of their online experience, too. Some states are already considering limits on discrimination — such as on what kinds of personal information sellers can use to set prices. The overarching aim should be to establish a better balance of power between sellers and buyers.
My great uncle was a sophisticated marketer. He chose the right moment, analyzed his target and pressed his advantage to the limit. Now some of the world’s biggest companies have the surveillance systems and technology they need to all be my great uncle. Will we let them, or will we resist the siege?
_____
This column reflects the personal views of the author and does not necessarily reflect the opinion of the editorial board or Bloomberg LP and its owners.
Cathy O'Neil is the CEO of ORCAA, an algorithmic auditing company, and the author of "The Shame Machine: Who Profits in the New Age of Humiliation."
_____
©2026 Bloomberg L.P. Visit bloomberg.com/opinion. Distributed by Tribune Content Agency, LLC.






















































Comments