I am not as worried about these sorts of ludicrous results as I am the ones that are close enough to correct to be believable. That is where you will get in trouble.
I have a similar experience using GitHub Copilot… it usually gets it right, which is great, and sometimes gets it really wrong, in which case I don’t use the suggestion and move on with my work.
However, every now and then it will give me a result that really looks correct, but is wrong in some minor way, and I end up getting burned because it takes me way too long to realize where the error is.
The uncanny valley of generated software: utility is inversely proportional to distance from the correct answer, with a huge drop into the dangerously believable.
I have a similar experience using GitHub Copilot… it usually gets it right, which is great, and sometimes gets it really wrong, in which case I don’t use the suggestion and move on with my work.
However, every now and then it will give me a result that really looks correct, but is wrong in some minor way, and I end up getting burned because it takes me way too long to realize where the error is.