If you have one copy of a mutated gene, it could be bad. However, one mutated gene is usually recessive or will be co-dominant with the other gene so it isn't that bad. If you have two copies of that gene, it can be a lot worse.
Two close family members are more likely to have the same mutated gene, so the odds of getting that same mutated gene twice are a lot higher.
You're on the right track, and others have explained it. Mutations happen regardless. The important thing is that because of redundancy between two copies, the effects of many mutations can be masked. The mutated alleles circulate throughout the population and only face selection when combined with another loss-of-function allele in the same functional unit (gene).
Those recessive mutations are more likely to be matched with an identical partner allele if you reproduce with close relatives. Even if you don't do that, the odds are higher if you're in a closed community.