Recently, I've just given up trying to use cuda for machine learning. Instead, I've been using (relatively) cpu intensive activation functions & architecture to make up the difference. It hasn't worked, but I can at least consistently inch forward.
Related to D: today vscode released an update that made it so you can’t use the remote tools with Ubuntu 18.04 (which is supported with security updates until 2028) 🥴 the only fix is to downgrade
For sure, I’m on the latest LTS! The problem here is that the remote ssh tools don’t work on older servers either, so you can no longer use the same workflow you had yesterday if you’re trying to connect to an 18.04 Ubuntu server
I started working with CUDA at version 3 (so maybe around 2010?) and it was definitely more than rough around the edges at that time. Nah, honestly, it was a nightmare - I discovered bugs and deviations from the documented behavior on a daily basis. That kept up for a few releases, although I'll mention that NVIDIA was/is really motivated to push CUDA for general purpose computing and thus the support was top notch - still was in no way pleasant to work with.
That being said, our previous implementation was using OpenGL and did in fact produce computational results as a byproduct of rendering noise on a lab screen, so there's that.