Help fixing invalid origin with memos

Running joplin and memos in docker, routed through nginx. Since I don't own a domain I'm just using my public ip with ports and port forwarding. Joplin was throwing the same invalid origin error, but worked after I set APP_BASE_URL: http://<IP>:<port>. I tried setting SITE_URL=http://<IP>:<port2> under environment, which I've read is supposed to fix this exact problem. Same error. The error displays the correct address including port number, so I know that's being passed correctly. I've tried several different variations of the Host, Origin, and Referer header without success. Just for fun I tried directly exposing <port2> on the memos instance and it opened right up in the browser.

PS: Yes, I know I should be using https. I'm lazy. Setting up a cert is on the old todo list.