I think it's rather similar level of maturity and points of interest. Like I may find college students physically attractive, but there's quite a maturity gap between students and people who have been working full time for a couple of years that can be off-putting to me.
You like people you can have a meaningful interactions with, you are more likely to find sexual partners in that group, you fantasize about current or prospective partners, you end up building an attractiveness pattern that matches a certain age group.
Got to be an interaction with our wiring and social constructs. We're mainly wired to get along with our tribe, after all. Makes sense for sexual preferences to naturally hew to social mores.