Whiteness, at least from a racist perspective, isn't really about skin color, it's more like a club for 'approved' ethnicities. There's many Italians with darker skin than Mexicans, but Italians are considered 'white' and Mexicans are not. Same for large parts of the Middle East and Asia.
Romani are white skinned Europeans, but they're not 'racist approved', so they make up rumors they're actually from Egypt and omit them from the White Club.
The determination for what counts as white is highly inconsistent. Before the 1700s Germans were not considered white. Before the 1800s Irish were not considered white. For a time in the 1900s Finnish people were considered Asian (while many Finns were striking for better working conditions, what an odd coincidence). Italians weren't considered white until about a hundred years ago. It goes on and on.
Roma people have historically been very persecuted because of racism and ethnocentrism. Case in point: the holocaust killed up to 500,000 Romani people, but the actual figures are not known. Roma people are among the groups that are rarely talked about when the Holocaust is mentioned, despite losing up to 50% of their total population at the time.
Arab and North African folks are usually considered white on the US census but that isn't really an accurate picture.
Race is a social construct that doesn't have clear borders. Racial categories mostly exist as a way of creating division and limiting access to resources, to flatten the diversity of individual cultures represented by a racial category... or to inflict direct and systemic violence. The experience of being a racialized person is entirely the creation of the society that a person lives within; for example, African folks usually don't self-identify as "black," within Africa, but that's an important racialized experience that people can speak to in a place like the US.
Yes, the point I'm trying to make is that people's color is only tangent is racism (but of course it helps to highlight differences between different group of people). That's why Roma, even if white, are still discriminated against. Sorry if I misunderstood your point, or not made mine clear
Okay well if you want to get into that, the concept of whiteness is very selective to racists. Roma, Jews, and (until recently) Russians are not considered white. The very fact that they were targeted the way they were tells us that. They may have white skin but that doesn't matter to the concept because they aren't in the club.
Romani people are Indo-Aryan, more closely related to modern day Indian people than Europeans. They typically have darker skin than Europeans as well. It's not really an American concept either; I've generally seen a lot more anti-Romani sentiment in Europe than the US.
Other people have longer explanations which are great. I just wanted to point out Romani people are not Romanian even though many Romani people have settled in Romania. It's just a coincidence.
The American concept is deceptively complex. At first it's just literally skin color. The Simpsons meme with the cop holding the color swatches is absolutely true. Then it's about stereotypes. So yeah your skin is light, but are you anything they have a stereotype about? Their entire concept of self relies on stereotypes being true. Otherwise they can't be smarter just because they're of pure European descent.