Neuron activity shows that the brain uses different systems for counting up to four, and for five or more.
An experiment found that the brain uses one set of neural circuits to identify the numbers 1–4; these circuits are very specific to their own numbers. A separate set of circuits respond to the numbers 5–9; these are less precise, and are activated by adjacent numbers.
For this reason, it is easier to determine when there are four things than to determine there are five things.
This has been part of IQ tests for a while: the vast majority of people can hold to 4 items without a problem, some people struggle with that, some people can hold to 5, 6, 7 or rarely more objects without a problem.
It's also why phone numbers in most places started as 5 digit numbers, then as "area code + 7", and now that we use smartphone contact lists to save them, they've gone to full "country code + 9 + extension". There was also an interesting movement opposing the changes: https://www.mentalfloss.com/article/61116/why-did-old-phone-numbers-start-letters
I bet this is also why old-timey phone numbers encoded the first digits with letters. The US had the famous ABC2/DEF3/... system that's still displayed on most keypads, and the former Soviet Union mapped the first letters of the Russian alphabet (skipping З to avoid confusion with 3)...
I don't think that's why, not least because this is a recent discovery. The reason is, presumably, because 26+ keys are difficult to fit on a small but still usable keyboard.