No doubt that's appealing to some (one example is the USA civil rights movement in the 1960s, especially with states conforming to federal laws that mandate desegregation of schools) but I think another advantage for the privileged is the lack of competition for good jobs, study places etc.. If 50% of your peers are kept in slums then just by biological outcomes (lack of nutrition and sleep) the odds are very much in your favor. Throw in the psychological effects of poverty, mass incarceration, addiction and you have a situation like a running race where half the contestants have a broken leg. Fear of a level playing field might be another factor in why the privileged don't want equal rights. BUT, imagine if we had 50% more people working on a cure for cancer etc.
But, but, but.....I am winning everything. Can't let those marginalized communities beat me. I've been told everything is a zero sum game! I say in the most whiny, navel voice. The kind of the voice that makes your soul shiver up and die
For those who you are wondering, the above comment was dripping in sarcasm. Human Rights are not a zero sum game. When marginalized communities prosper, we all better off as a society.