Let's be clear, it's really low risk, especially for such a low strength one, but some types of components can get damaged by it, or you could end up dislodging badly soldered or already damaged ones while you move the board around.
In the worst case, if you dislodge them just enough to give you an intermittent problem, good luck debugging it.
My joycons - original ones even! - finally started to drift earlier this year, and I did the same thing replacing them with some hall effect ones. So far I've been really pleased with the results, though initially it was a little weird simply because the sticks themselves felt a tiny bit different than the original ones. But after using them for a bit they felt good.
My Joycons have survived BotW, AC:NH, Three Houses, Metroid Dread and Metroid Prime, which I all played religiously. But they finally started drifting during my early hours of TotK. I just bought the Gulikit ones, switched them in less than half an hour. Great experience, no complaints, just not drifting joy cons.
Those are the same ones I installed in my joycons, too. And looks like the same ones in the photo on this post.
I had the same experience with the usage, AND I played a lot of Smash Bros. on them which would, in theory, destroy them in no time. But they started to drift about two months before ToTK came out, and I couldn't risk playing that all janky.