A former friend of mine was heavy into the right wing and worked construction (surprising, I know). He was always complaining about "illegals taking jobs" and how he thought the work they did wasn't good anyway.
One day, I asked him: why doesn't your company stop hiring these people you hate? He said it's because then they wouldn't have enough people. Naturally, this is a contradiction. It didn't matter, of course. His whole personality was built on hating these people.
I think it is that way with a lot of folks. If we penalize employers (like we should, because, you know, the law), then these people can't hate as effectively. That means they might start voting differently.
When they say illegals are taking all the jobs, they mean it's hard to get correct and appropriate pay for their labour because these desperate people will accept much less, which brings down the typical pay for that job. The left blame the employer and the system, the right blame the people accepting those conditions.
And the employers are actually already breaking the law for employing such people. It shouldn't be going beyond that, and yet we never see politicians making that point, because it's apparently a no-no to call out corporations for their actions at this point in American history.
Edit: and also, at least in the case of who I was talking about, they'd never suggest wages were too low across the board. They're secure in their scapegoat. We aren't really disagreeing, I don't think, but this issue runs deeper because there are ideologies at play that do not adhere to logic.