We do use it where necessary; in science, engineering and the military, for example. Some of our imperial units suck, but others, like pounds and feet and inches are superior because they are more intuitive. The reality is that it's a non-issue for most people and because of that we will almost always have some version of a mixed system, as do most of the other Anglophone countries.
What? How is feet and pounds more intuitive? To me metric is more intuitive and imperials are whole another language, because I only learnt metric in school and only remember very few things in imperial (such as some dimensions on specific parts of bicycles).
Converting yards to inches in your head is near damn impossible, but you don't even need to count to know 5.25km is 5250m or 525000cm