Honestly, copper is a better conductor than gold, which is why you wouldn't want to make the whole cable out of gold (ignoring price). It works for the connectors because it doesn't oxidize like copper does which means that the connection between the device and cable will be better.
You MIGHT notice a difference between a gold connector and a normal one with an analog signal, but once you have a digital signal like HDMI, the cable will either work or not. You won't get a signal degradation like you would with a RCA cable, it just shuts out.
That being said, I have seen gold connector on the fiber TOSLINK cables too. I'm sure those made the light go SO much faster. /s