It's an obsolete and relatively obscure word that I think a lot of people don't know - because if most people did know it, no one would ever have gotten into trouble for using it.
It needs to be retired honestly. Like you said, more often than not it's used by smug racists. There are other, better words that mean the same thing like avaricious, miserly, parsimonious, or to keep it simple - cheap.
For those that can't believe it's not a racial slur.
Niggard (14th C) is derived from the Middle English word meaning 'stingy,' nigon, which is probably derived from two other words also meaning 'stingy,' Old Norse hnǫggr and Old English hnēaw.[2] The word niggle, which in modern usage means to give excessive attention to minor details, probably shares an etymology with niggardly.[3]