You know what's weird? What's weird is that, despite your claim that people aren't having discussions with people who get downvoted, my reply to you was part of a huge amount of discussion with someone who was downvoted. And that is true of multiple other discussions in this post.
Look, with nearly 100 comments a day, we realize that no amount of downvotes is going to silence you from commenting in every post. However, for us mere mortals, can you not see
how a dogpile of downvotes from the local majority opinion might dissuade some individuals from wanting to engage in discussion?
It's clearly psychological. It's not that it quantitatively "matters." I suspect very few people are going around using the downvote button with the intent of statistically decreasing the view count of a particular comment. On these politically charged posts, it's used as a litmus test for whether or not you are part of the 'right side' or 'the other side.'