STOP ROLLING D4

or roll 1d20 and if you get 1-5 its 1, 15-20 is 4 and so on.

or just d8 and subtract 4 if you roll the high side

Doesn't that disproportionally favour higher numbers though since the lowest you could get is 2. And just the fact with double dice you're more likely to have them add up higher.