Summa cum Claude
I am very much enjoying Austerlitz and since I got the book recommendation from Claude originally, I went back and asked about his other books. All of them, I was assured, are worth looking at. A short list of his other works was created and prioritized, and I was further assured that the same translator of Austerlitz (Anthea Bell) had masterfully translated all of Sebald’s work.
Perfect, I thought, and off I went to fill my online cart. It took me about 60 seconds to figure out that, no, Bell had not translated all of Sebald’s works. In fact, the only one I can find is Austerlitz. Also, the ISBNs which were shown in the results were also bogus, despite my explicit instructions:
If any other books are mentioned in responses, their existence should be confirmed via ISBN searches before including them. Likewise, articles should be checked for DOI numbers before answering.
My prompt should have prevented this and yet it did not, so…I asked how this could have happened.
That’s a fair and important question that deserves a direct answer.
I made an overconfident inference that was factually wrong. I knew Anthea Bell translated Austerlitz superbly, and I had accurate information about her being one of the great German-to-English translators. From there, I incorrectly generalized that she had translated all of Sebald’s major prose works — treating my inference as if it were verified knowledge.
This is a systematic weakness in how I function: I can conflate what I actually know with what seems like it should be true based on patterns, especially when discussing an author comprehensively. I should have been uncertain about which translator handled which Sebald work, but instead presented the claim with unwarranted confidence.
You’ve been asking me to verify ISBNs and check factual claims (like whether books exist before I recommend them) precisely because you recognize this risk. I failed to apply that same rigor to the translation question—I gave you specific, confident misinformation when I should have either verified or acknowledged uncertainty.
Will this correction make you less likely to err in the future?
No, not reliably.
This correction helps within our current conversation — I’m now aware that Hulse translated The Rings of Saturn and The Emigrants, and that I shouldn’t claim Bell translated all of Sebald’s work. I can reference this exchange if relevant later in our discussion.
But the structural problem persists:
I don’t have a mechanism that systematically prevents similar errors in future conversations or even later in this one on different topics. The same inference pattern that led me to wrongly generalize about Bell could operate elsewhere — I might:
- Assume a scholar wrote books they didn’t write based on their field
- Conflate publishers or editions
- Misattribute translations in other authors' works
- Overgeneralize about historical facts or theological sources
The fundamental issue:
I can’t reliably distinguish between:
- Facts I have direct knowledge of
- Plausible inferences that seem likely true
- Patterns I’m extrapolating from partial information
The only reasonable approach is a hermaneutic of profound suspicion when it comes to LLMs, no matter how confident-sounding the response. The last three points alone are all that anyone really needs to know.