“Superhuman” Go AIs still have trouble defending against these simple exploits

By using unorthodox "cyclic" strategies—ones that even a beginning human player could detect and defeat—a crafty human can often exploit gaps in a top-level AI's strategy and fool the algorithm into a loss.

preprint of the actual science article summarized in the ars technica piece:

https://arxiv.org/pdf/2406.12843

Prior work found that superhuman Go AIs like KataGo can be defeated by simple adversarial strategies. In this paper, we study if simple defenses can improve KataGo’s worst-case performance. We test three natural defenses: adversarial training on hand-constructed positions, iterated adversarial training, and changing the network architecture. We find that some of these defenses are able to protect against previously discovered attacks. Unfortunately, we also find that none of these defenses are able to withstand adaptive attacks. In particular, we are able to train new adversaries that reliably defeat our defended agents by causing them to blunder in ways humans would not. Our results suggest that building robust AI systems is challenging even in narrow domains such as Go