I have the same Residental Gateway. Using pfSense+ on my end. The BGW320-500 is fiber capable. I assume you're using fiber? If so you cannot hook it into ONT because the RG is the ONT. In my case I get raw fiber into a PON module that hooks into the RG. Best you can do in this case is set the RG to "passthrough mode" via web UI (192.168.1.254).
There is no double nat. Passthrough mode has worked as expected for me. The one issue I have is that the RG will maintain firewall states, so it limits you to the RG hardware for those states. I have a pretty large home network though, tons of devices, IoT, etc, and it has been stable.
Latency seems decent. I have an AT&T fiber 2gb symmetrical connection and a ping to google from my Netgate pfSense machine is around 10-15ms.
What firmware is your BGW320 currently on? There's a method for newish firmwares that should work on the 320, and are confirmed working for the BGW210.
No easily accessible guide for it yet, but for OPNSense and PFSense themselves, there's a simpler bypass available now. It still requires certificates. PFSense has an auth bridge mode that does not require certificates, but requires 3 interfaces and for your modem to still be plugged in.
You will need to connect the ONT ethernet directly to the WAN port for a bypass to work.