Although I don't consider myself an advanced user, I used to daily drive Gentoo a few years back.
Just don't waste your time compiling packages like firefox and libre office. There are binaries available for big packages so make use of those. It won't make much of a difference in performance imo.
Yup optimizations and customizability is what makes gentoo unique. For starters - You can use any init system, replace glibc with different c libraries etc.