Not even inbreeding, strictly speaking, when you go back that far. At 12 generations back you've got lots of ancestors with common ancestors, but they aren't so closely related that conceiving a child between them would be incest. Doesn't have to be siblings; very distant cousins would also reduce the number of total ancestors. They need not even have known their relation to one another.
By second cousins the rate of genetic abnormalities is the same as for distant strangers. You really don't have to go very far back for it to not cause issues.