Yeah, this makes sense. Would you feel better about the this: "It's ok to punch fascists"? Using the common definition for all these terms. I know this is slightly different from what OP was posting about.
Yes, I have no problem with that. I think we only need to target people who spread fascist ideology to a wide and/or large audience. The average fascist Joe, has very little reach and it would look bad if it seemed like random nobodies were being targeted.
Edit: When I say the average fascist Joe. I mean the average person who is a fascists. Not anyone named Joe. Sorry for any confusion.