When ChatGPT Goes Wrong - 3 Examples
Dec 12, 2022
ChatGPT has some limitations that can result in silly mistakes and nonsensical outputs.
We have seen tons of impressive examples of how ChatGPT wows and bedazzles with its seemingly superhuman knowledge and writing ability.
But let’s take a look at the other side and see how it can fail just as spectacularly.
The AI does not understand numbers the way we do. It treats numbers like words, and as a result, makes silly mistakes that my 5-year-old could call out.
For example, when asked to compare bed sizes, the AI may say that 108 inches is narrower than 76 inches, which is clearly not true.
I planned to go to Tahoe last weekend but had to cancel because of the impending snow storm and avalanche risk. So I asked the AI to make a joke about it.
Unfortunately, the AI made a spurious connection between snowmen and avalanches, and then it hallucinated the double meaning of blizzard. The result was a joke that made no sense.
When asked to solve the famous Titanic problem with logistic regression, the AI was able to generate a pretty plausible code.
As a follow-up, I asked it to use conformal prediction instead of logistic regression, and it started to hallucinate.
ChatGPT suggested that we import scikit-cp, which stands for Computational Physics, not Conformal Prediction. It looks like it just made the assumption that the package could solve the problem and wrote the code without understanding what it was doing.
In conclusion, ChatGPT is a powerful tool that can generate impressive results, but it is not perfect. It has some limitations that can result in silly mistakes and nonsensical outputs. Some of these limitations include:
- The AI does not understand numbers the way humans do, and can make mistakes when comparing sizes or quantities.
- It can make spurious connections and hallucinations when asked to generate jokes or creative content.
- It may import the wrong packages or use the wrong algorithms when asked to write code, if it does not understand the task properly.
Despite these limitations, ChatGPT is still an impressive tool that can be used for many useful applications. It just needs to be used with care and understanding of its limitations.