Indeed they should and are, except when they are in their natural habitat. It's better to not take them out of their habitat in the first place, but it's also better to not release a buch of them in an environment that can't handle them.
If you live in the natural habitat of the north african wildcat, it's a complicated question, because if cats outnumber their wild counterparts and interbreed with them, their descendants risk adopting unnatural behaviors like losing their fear of humans. The higher than normal number of predators could also be a risk for other local species (they may be used to being hunted by wild cats, but not that many...). So, yeah, depending on the cat population density in the area, they should probably stay indoors to.