I generally try not to downvote comments unless someone says something either very stupid or if it's some sort of propaganda. If someone makes a good argument against me, or just disagrees with me, I don't downvote their replies.
With posts, sure, but comments aren't posts or content they are part of a conversation. You don't need to downvote a comment to elevate another. You can simply upvote.Let me put it to you this way, you are having a conversation with someone then a stranger comes along and starts booing at something you said, would you think it's necessary? There is a similar concept in black mirror where people all rate each other. Its not conducive to a welcoming environment when we start downvoting people for things because they are 'dull' and becomes toxic quickly, it's a form of emotional/social violence.