As others have said, thumb drives are generally designed for lighter loads than running an OS, so yours probably just doesn't have enough of a heat-spreading design to keep it cool.

To help you investigate what's causing the thumb drive to be accessed so much, the iostat -d -x 1 will tell you what its percentage utilisation is (rightmost column). You can then use sudo iotop to see what processes are using the drive.

In the long run though, I'd definitely recommend switching over to a cheap SATA SSD, especially since your mini PC will likely let you install one internally somewhere. In my experience, your system will probably feel snappier too!