To be fair, the Aiel didn't initially grow up there (and too short for evolution, in my non-scientific brain), but rather migrated there to use it as "a shaping stone to make them, a testing ground to prove their worth, and a punishment for their sin." And it would be too easy on those three reasons if they were perfectly (or mostly) acclimated and suited for the desert genetically.
You're right, and it's only been a few thousand years. I'll still complain to anyone who will listen. If I don't complain how will you know I really enjoy the series?
IIRC, it takes around 100 generations to see a significant shift in skin pigmentation due to evolution. For humans, that would work out to about 1700 years for people that were moved from the Nordic regions to sub-Saharan Africa to develop dark skins (assuming that there were no other factors in play).
Evolution can take what seems like a really long time.
In my homebrew world, elves have a different skin pigment that reflects UV light (whereas melanin absorbs it). Lower amounts of pigmentation mean darker skin, and more means whiter skin
Because of this, deep elves aren't as dark skinned as some humans, with the darkest being an ashy charcoal gray, but desert elves develop extremely white skin with almost no translucence