And you can expect this to possibly get worse. I remember seeing a post about how AI is now having trouble obtaining good quality image data because the internet has become saturated with AI generated art. I imagine text content is even worse. I’ve already seen several news articles lambasting a few lawyers for referring to non-existent cases.

The bottom line is that ChatGPT and other AIs don’t understand what they are writing. They are just stringing words together by association with keywords. It’s writing convincing sounding fiction about the real world. The fact that you can get things like vaguely working code out of it is just a happy accident.

At best, AI it should be seen as an idea generator. It requires that the person be skeptical of whatever is output.