A horde of trailer-park visigoths can sack the Capitol and Republicans say "what's the big deal?" But let one guy show up in casual clothes and they lose their goddamn minds.
Because he still looks like shit in a suit because he's 6'8" and has odd proportions. So either he looks unprofessional in an ill-fitting, uncomfortable suit or he looks unprofessional in comfortable clothing. Might as well let the guy be comfortable.