I would say it is gender neutral to male coded, diminutive in nature.
You call a child "buddy." It is male coded, but because it refers to children it becomes more androgynous in nature.
I've been in social contexts where in the local dialect it was gender neutral and status neutral, but in other contexts it is not so much male coded as it was to belittle.
Even then. One of my best friends says buddy and bud to me and pretty much everyone—his other friends, bartenders, servers, anyone—but I still can't stand it. I just don't say anything because I know he means well but, damn, I find it so annoying. Who knows where he picked that shit up.
I don't know how anyone wouldn't interpret it as an insult, like "boy".