I find small breasts the most attractive, but sometimes large breasts are just so eye-catching that you can't not look at them, especially when a large proportion of the surface area is not covered. It's especially annoying to have women offended by this when it's not even my thing.
I feel like making hard eye contact would be creepy so I just don't look at women. Wow that sounds terrible, more I just look at the ground while walking and don't look at anyone. Hmmm, not sure that's an improvement...
I actually do this as well because I've been told so much about guys looking at boobs when talking to women. I make extra effort to keep eye contact. So because of that I've been told "it's intense." That's from people that know me so that is probably the polite version.
In addition to things others have said in reply, maintaining eye contact is a direction given in speech and leadership classes. Looking at someone you are talking to shows interest, engagement, and respect. It would be weird in a different way if you were speaking to someone and just staring at their hands or staring off into space. It's not that there are two options, but if you're having a conversation with someone, it's normal to be looking at them.
While I get the point you're trying to make, the only normal place to look at when talking to someone is their face, which is kinda small, and the two parts of the face that convey emotion are the eyes and mouth.