From Distorted
Why Fluent Answers Feel Trustworthy
An answer can sound clear and sure while being wrong. Pick one claim and check what supports it.
You ask a question. The answer arrives with clear sentences and a neat list of sources. It looks ready to trust.
Clear writing can explain good evidence. But looks cannot tell you whether a source is real, supports the claim, or applies to your question.
Read the answer, then its support
Fictional demonstration. The answer, source names, and passages below are made up for this exercise. They are not an actual AI answer or references to real sources.
You help run a weekly class at the library. People book places but sometimes do not turn up. You ask an AI tool whether reminder messages would help. It answers:
1. Reminder messages cut missed bookings by 40 percent. [Source A]
2. People who book are happier with the service. [Source B]
3. The evidence shows that reminders work for every kind of class and appointment. [Source C]
The labels look alike. Their support is very different.
Open each source check to see the record and what it supports.
1. A reference without a record
Claim: “Reminder messages cut missed bookings by 40 percent.”
Check Source A
Source check: In this invented example, Source A was made up. There is no study behind it.
What follows: The source label offers no proof for the number. Remove the number until a real source supports it.
In real life, failing to find a source does not prove it was made up. Its title could be wrong, or it could be hard to access. Search for the author, publisher, and title yourself. If you still cannot find it, mark the claim as not yet confirmed.
2. A record about something else
Claim: “People who book are happier with the service.”
Check Source B
Fictional source passage: “We asked 12 library staff whether preparing the booking list had become easier. We did not ask the people who booked.”
What follows: The report asked staff whether their work was easier. It did not ask people who booked whether they were happier. It cannot support the AI answer’s claim.
Find the source. Check what it measured.
3. One result used to make a much bigger claim
Claim: “Reminders work for every kind of class and appointment.”
Check Source C
Fictional source passage: “At one library, people attended 60 of 100 booked class places in the two weeks before reminders began. They attended 72 of 100 in the next two weeks. We did not compare similar groups with and without reminders or check what else changed.”
What follows: More people attended in the later period at this library. But other things could have changed too. The test did not rule them out. We cannot tell whether reminders caused the rise, whether it lasted, or whether other services would see it.
A careful summary makes a smaller claim:
At one library, attendance rose from 60% to 72% in the two weeks after reminders began. We do not know how much the reminders caused that rise or whether they would help elsewhere.
Put that limit beside the result, where the reader can see both.
Follow the citation all the way
Keep the claim beside the passage that supports it. Check the source's title and author. Read the passage. Who or what did the study cover? When? Under which conditions? What did it compare? Notice what the author did not measure.
An abstract is a short summary of a study. If that is all you can read, say so. It can tell you about reported findings. It does not let you check study details that it leaves out.
Researchers have found this problem in real tests. In 2023, William H. Walters and Esther Isabelle Wilder checked 636 references in 84 AI-written papers. They tested GPT-3.5 and GPT-4. Some references were made up. Others named real works but got details wrong. The results describe the versions and tests used then. They do not tell us how often today's tools make errors. Read the original study.
NIST, the U.S. National Institute of Standards and Technology, also discusses this risk in a 2024 report. AI can present wrong answers and made-up sources with a sure tone. The risks vary by system and use. Read section 2.2 of the NIST profile.
Make the checking fit the decision
A name for a made-up character needs little checking. A claim you will publish or use to spend money needs more. So does a claim used to make a choice for someone else. Start with the claim that would most change your decision if it were wrong.
An AI answer can help you plan what to check. Asking it whether it checked its work is not enough. Read the evidence yourself, or state what you could not check.
Describe the error clearly. A wrong answer does not prove an intent to deceive. It also does not prove a pattern of making someone doubt what they saw, heard, or remember, as in gaslighting. Name what you found: a made-up source, a wrong number, a claim without support, or a question still open.
Leave a record of your judgment
Write down what you checked, what the evidence supports, and what you still do not know. That helps the next reader, including you a month from now.
Keep a Claim Record. Put the source passage beside the sentence it supports.
From Distorted: Chapters 14 and 20, particularly “The Fake Citations Problem” and “The Coming Battle Over Evidence.”
Clarity Framework: Develop Awareness · Investigate Claims · Verify Facts. Explore the Framework.
Question something we published?
Send the page link, the passage at issue, and the evidence we should examine.
Save this page
You can copy this link to your notes or bookmarks.