Guys this is off topic but I'm using Ubuntu server for a jellyfin/coding server and getting headaches trying to do things cli only.. for example connecting openvpn.
My question is how much shame is there using a distro with a gui for a server
You could try a web UI like Webmin. If it's a public-facing system like a VPS, the once you get the VPN working, configure Webmin to only listen on the VPN IP so that it's not exposed publicly.
By the way, I'd recommend Wireguard or Tailscale over OpenVPN. Tailscale uses Wireguard but makes it a lot easier to configure. One of the major advantages is that Wireguard/Tailscale are peer-to-peer models that don't use a central server like OpenVPN does. Wireguard doesn't have servers, only peers.
Basically, if you have two devices on the VPN that want to communicate with each other (eg say you're at a coffee shop and want to connect to your laptop from your phone), they can communicate directly rather than having to go through a server that's potentially located somewhere else.
To be honest the problem is I'm trying to connect to the VPN provider I have with openvpn client. Im with windscribe, I get as far as importing the profile, attempting to start the service, entering the username and password for the VPN but the thing just won't start.
Oh I see... I thought you wanted to set up your own VPN, as that's the usual use case for a VPN (e.g. connecting to a home server while not at home).
Looks like Windscribe have a Wireguard config generator so you could try that. https://windscribe.com/getconfig/wireguard. I don't have an account so I can't try it, but it may just generate a regular Wireguard config you can use with Wireguard directly, without having to use their app.
The problem is now all my port forwards stopped working for accessing services via my external IP so I guess there is some kind of firewall going on via wireguard.
Give tailscale a try! I use zerotier (because I found it first) which is very similar but I think tailscale is slightly easier. I've had trouble getting openvpn working when i was just getting started as well, and actually find zt/tailscale to better suit my needs in general - I just want computer x and computer y to always be able to reach each other.