What irritates you the most with your own language?
Mine is people who separate words when they write. I'm Norwegian, and we can string together words indefinetly to make a new word. The never ending word may not make any sense, but it is gramatically correct
Still, people write words the wrong way by separating them.
Examples:
"Ananas ringer" means "the pineapple is calling" when written the wrong way. The correct way is "ananasringer" and it means "pineapple rings" (from a tin).
"Prinsesse pult i vinkel" means "a princess fucked at an angle". The correct way to write it is "prinsessepult i vinkel", and it means "an angeled princess desk" (a desk for children, obviously)
"Koke bøker" means "to cook books". The correct way is "kokebøker" and means "cookbooks"
It's not so much a feature of English as it is a recurring bug in the way people use the language...
If you write "of" instead of "have" or "'ve" you need to be taken out back and beaten with a dictionary, preferably until you can apologize to your ancestors in person for the effort they wasted in passing down the English language to you.
Incidentally, when did people start saying "on accident"? It's by accident! Has been for ages! Why this? Why now? I hate it.
With that out of the way... English isn't a language, it's five dialects in a trenchcoat mugging other languages in a dark alley for their loose grammar.
Edit: With regards to OP, "a cookbook" and "to cook the books" are similar phrases in English, too, but have, eh, wildly different meanings. XD
"of" in place of "have" certainly had to come from people mishearing/misunderstanding "ve." There's no other explanation.
The accident one is funny. I had to really think about when I'd use "on", and it's when I say something like: "he did it on accident." Which is wrong when I think about it, but I know I've said this countless times. I can only guess it grew from "an accident" like "it was an accident."
Even though "on"and "by" are the same length, "by" sounds like it takes too much effort to say. How weird.