Yep! You're totally correct. They're indeed highly specialized, it's a chain that starts with a proerythroblast that would look more like a "normal" cell going all the way to the erythrocyte, the familiar red blood cell.
Very frequently in biology we create definitions and groupings that get more and more specific, but without taking away from the original group: for instance, carnivores are a specific group of animals with specialized characteristics... But they're still mammals. And mammals are still animals. And animals are still Eukaryotes. And so on.
Cells do not follow the same type of definitions that we would use for groupings, but the logic still applies - losing a nucleus won't make this cell suddenly not a cell. In fact, the membrane is still highly dynamic, there are still internal and external communication mechanisms, and there's a metabolism - glycolysis.
What exactly they lose depends on the species, human red blood cells indeed discard organelles and the nucleus.
But having organelles and a nucleus are not the formal definition of a cell. To begin with, any prokaryotes won't have a nucleus, and organelles aren't mandatory.