Sorry, that seems painful. No need to worry though unless you have trouble moving your fingers or wrist (check if you can clench your fist with good force, and carefully see if you can move and flex your pinky normally). Also look for any part of your forearmarm or hand becoming painfully swollen, or pale and sweaty. It is normal though to have a numb pinky (often affects the ring finger too) for up to several days.
Once I hit my elbow really hard and I had a numb pinky for a couple of weeks, it's uncomfortable, but at least blunt impacts rarely hurt a nerve permanently.
