Some are but there are nylon teabags and pyramid teabags made with plastic, and apparently even paper teabags often use glues made of plastic. These days you have to specifically seek out a brand that explicitly says they have no plastic in their bags or just buy loose leaf.
Using loose leaves is so easy and tea shops have so many great fruit combinations with their teas I don't understand why so many people still use the bland tea bags