Study Finds That 52 Percent of ChatGPT Answers to Programming Questions Are Wrong
Researchers found that 52 percent of answers to programming questions generated by ChatGPT were incorrect.
You're viewing a single thread.
Worth noting this study was done on gpt 3.5, 4 is leagues better than 3.5. I'd be interested to see how this number has changed
4 made up functions that didn't exist last time I asked in a programming question.
This is why I like Bing Chat for this kind of thing, it does a web search in the background and will often be working right from the API documentation.
sure, I'm not saying GPT4 is perfect, just that it's known to be a lot better than 3.5. Kinda why I would be interested to see how much better it actually is.
There is huge gap between 3.5 and 4 especially in coding related questions. GPT3.5 does not have large enough token size to handle harder code related questions.