The problem is exploitation which can and does happen in every industry. Sex work is no exception to that rule. Nobody is saying amazon needs to be abolished because of how they treat their workers. But some people see one industry or another as being inherently exploitative.
Plenty of people are calling for Amazon to be stopped, whether it's by being broken up in a trust-busting operation, fined to the point of bankruptcy for various things including illegal exploitation of its employees, or as an extreme example, starving former Amazon employees simply eating Jeff Bezos. Whether or not someone agrees, and whether they think it applies to brothels, multinational mega-corporations, or any other category of business, it's not a particularly controversial take that some kinds of business are inherently too exploitative of their employees, and should therefore be unable to legally exist.