There should be a semicolon punctuation for exclamation and question marks.
A difficult part of writing for me is when a single sentence--especially dialogue--contains two tones. It sounds best as a single sentence, but ending with a period, or alternative punctuation, looks wrong. As well as this, using two sentences also looks wrong.
I can't think of a great example right now, but I know I've wanted punctuation that doesn't exist before. I've had moments where it would have been so useful to have a ";!" and a ";?" mark.
Is a new mark necessary? Just taking out the semicolon does exactly what you want.
Hell, the semicolon was pretty much invented because someone thought "What if I want a compound sentence, but can't decide on a conjunction to put between them?"
How often does anybody use THOSE? (In English, C++ doesn't count)
The sentence doesn't have emphasis unless I use italics, which isn't always convenient, or sometimes even possible (writing, software with no formatting).
Also, I use semicolons--and double hyphens, for that matter--quite frequently. I love using them.