"participants who had access to an AI assistant wrote significantly less secure code" and "were also more likely to believe they wrote secure code" - 2023 Stanford University study published at CCS23
I had a student came into office hours asking why their program got a bad grade. I looked and it didn't actually do anything related to the assignment.
Upon further query, they objected saying that the CI pipeline built it just fine.
So ..yeah... You can write a program that builds and runs, but doesn't do the required tasks, which makes it wrong. This was not a concept they'd figured out yet.