If you are good enough at your job and good enough with people, then you'll get pushed into management again. Realistically it's a compliment, but I understand the desire to weasel your way out of it, since that has been me too at multiple different times.
A good middle manager will change what they can, and hold the line against the things that they can't. They will be a bulwark of protection and understanding for those below them.