Help with domain

Lots of questions, let's take it one step at a time. You have a domain, now you can point it to your public IP, so that whenever someone tries to access example.com they ask their DNS server and it replies with 10.172.172.172 (which btw is not a valid public IP). Now that request will hit your router, you need to configure your router to redirect ports 80 and 443 to 192.168.200.101, that way the request to example.com gets to your local machine.

Ok, so now you need your local machine to reply on that port, I recommend using Caddy it's very easy to setup, but NGIX is the more traditional approach. A simple Caddy config would look like:

    
example.com {
    respond "Hello"
}


jellyfin.example.com {
    handle {
        reverse_proxy http://192.168.200.101:1020/
    }
}

  

So after the request reaches Caddy it will see that the person tried to access, example.com and respond with a "Hello".

If instead you had tried jellyfin.example.com the DNS would have sent you to 10.172.172.172, your router would send that to 192.168.200.101, Caddy would then send it to 192.168.200.101:1020, which is Jellyfin so that would get returned.

There are some improvements that can be made, for example if both caddy and Jellyfin are docker you can share a network between them so Jellyfin is only exposed through caddy. Another possibly good idea is to add some authentication service like Authelia or Authentik to harden stuff a little bit. Also as you might have noticed Caddy can forward stuff to other computers, so you can have one machine on your network exposing multiple services on multiple machines.

DNS and domains are just human-friendly IP addresses.

You only have 1 public IP address.
So, to access different services you need to use different ports.
Or run a service on a single port in front of the other services that can understand the connections and forward the connections to the actual services - known as a reverse proxy. In the case of http/https, there are plenty of reverse proxies that can direct requests based on all sorts of parameters, subdomains being one of them.

If you are just starting out, I'd recommend a docker compose stack and Nginx Proxy Manager.
Learning containers & docker makes everything easier.
NPM is a very easy to use reverse proxy with a nice GUI, so you don't have to configure CertBot/ACME or learn the specific config language of Nginx.

If you are unsure of domains and all that, you can try it out for free.
Your computer has a hosts file (/etc/hosts on Linux, I think it's in system32 on windows). This allows you to tell the computer "for the domain example.com use the IP 10.0.0.200" or whatever you want. You need a hosts file entry for each subdomain.
What this means is that you can run up a docker compose stack on your computer and point a bunch of sub domains to 127.0.0.1, use self-signed certs, and play around with nginx proxy manager and docker.
No money spent, no records published, no traffic leaving your computer.
Zero risk.

There are loads of tutorials out there on NPM and docker compose stacks. Probably some close to your specific requirements.

Don't mix your public and private DNS records. Use your public records for public things, and a local DNS forwarder for your local network.

A records only reference IPs and not ports.

SRV can be used to specify where to find ports, but the client needs to support those lookups to properly use it. You can use a reverse proxy or HTTP redirects to point things to different ports.

And is there any problem to setting this up in docker compose?