I specifically purchase noise cancelling headphones / earbuds now since it makes it easy to listen at MUCH quieter levels. I have done some tests with the noise cancelling disabled and setting the volume and with it on.. It really makes a huge difference as I am not cranking it up to drowned out noise on transit or walking down the street.
Which Sony XM4s are you talking about? Sony is terrible at naming their products, the WH-1000XM4 are noise cancelling overear headphones, while the WF-1000XM4 are in ear ones. If you're talking about the earbuds, try different tips.
Odd I have tested them in stores before and they are typically well rated.
However I was using Bose QC II for a few years and now I use the apple air pod pros.
For in ear seal is really important for nose canceling and apple also provides a test mode / app on apple devices to test that sound isn’t leaking substantially.
Makes no difference in a quiet environment but will easily cancel out a loud air conditioner or most traffic noise from passing cars.