TIL that the first person to receive a speeding ticket was going 8 mph.
According to Guinness World Records, the first person charged with speeding was Walter Arnold from Paddock Wood, Kent. On 28 January 1896, Arnold was seen going 4 times the allowed 2 mph speed limit in his Benz. The constable pursued him on his bicycle, issuing a ticket for £4 7s (roughly £260 in today's money), of which 10 shillings was for speeding.
It would be difficult to keep a horse at 2mph for any length of time. Even for a human that's a pretty damn slow walk.
Horses have 4 speeds; walk, trot, canter, and gallop. A walk is the slowest and will be about 4mph. You can make them go slower for a small amount of time but they'll get frustrated.