From Distorted
Why Fluent Answers Feel Trustworthy
An answer can sound clear and sure while being wrong. Pick one claim and check what supports it.
You ask a question. The answer arrives with clear sentences and a neat list of sources. It looks ready to trust.
Clear writing can explain good evidence. But looks cannot tell you whether a source is real, supports the claim, or applies to your question.
Those questions still need checking.
Read the answer, then its support
Fictional demonstration. The answer, source names, and passages below are made up for this exercise. They are not an actual AI answer or references to real sources.
Imagine asking whether appointment reminders improve a public service. The answer says:
1. Reminders reduce missed appointments by 40 percent. [Source A]
2. They also improve customer satisfaction. [Source B]
3. The evidence shows that they work across all public services. [Source C]
The labels look alike. Their support is very different.
Open each source check to see the record and what it supports.
1. A reference without a record
Claim: “Reminders reduce missed appointments by 40 percent.”
Check Source A
Source check: In this invented example, Source A was made up. There is no study behind it.
What follows: The source label offers no proof for the number. Remove the number until a real source supports it.
In real life, failing to find a source does not prove it was made up. Its title could be wrong, or it could be hard to access. Search for the author, publisher, and title yourself. If you still cannot find it, mark the claim as not yet confirmed.
2. A record about something else
Claim: “They also improve customer satisfaction.”
Check Source B
Fictional source passage: “We asked 52 service employees whether preparing the appointment list had become easier. We did not survey customers.”
What follows: The report asked staff whether a task was easier. It did not ask customers whether they were happier. It concerns the same service, but it does not support this claim.
Find the source. Check what it measured.
3. A finding stretched beyond its reach
Claim: “They work across all public services.”
Check Source C
Fictional source passage: “At one library, 60 of 100 scheduled appointments were attended during the two weeks before reminders began, compared with 72 of 100 during the following two weeks. The periods were not randomized; other influences were not controlled.”
What follows: More people attended in the later period at this library. But other things could have changed too. The test did not rule them out. We cannot tell whether reminders caused the rise, whether it lasted, or whether other services would see it.
A careful summary makes a smaller claim:
At one library, attendance rose from 60 to 72 percent across two consecutive two-week periods when reminders were introduced. This comparison alone does not establish the cause of the increase or whether other services would see similar results.
Put that limit beside the result, where the reader can see both.
Follow the citation all the way
Keep the claim beside the passage that supports it. Check the source's title and author. Read the passage. Who or what did the study cover? When? Under which conditions? What did it compare? Notice what the author did not measure.
An abstract is a short summary of a study. If that is all you can read, say so. It can tell you about reported findings. It does not let you check study details that it leaves out.
Researchers have found this problem in real tests. In 2023, William H. Walters and Esther Isabelle Wilder checked 636 references in 84 AI-written papers. They tested GPT-3.5 and GPT-4. Some references were made up. Others named real works but got details wrong. The results describe the versions and tests used then. They do not tell us how often today's tools make errors. Read the original study.
NIST, the U.S. National Institute of Standards and Technology, also discusses this risk in a 2024 report. AI can present wrong answers and made-up sources with a sure tone. The risks vary by system and use. Read section 2.2 of the NIST profile.
Make the checking fit the decision
A name for a made-up character needs little checking. A claim you will publish or use to spend money needs more. So does a claim used to make a choice for someone else. Start with the claim that would most change your decision if it were wrong.
An AI answer can help you plan what to check. Asking it whether it checked its work is not enough. Read the evidence yourself, or state what you could not check.
Describe the error clearly. A wrong answer does not prove an intent to deceive. It also does not prove a pattern of making someone doubt what they saw, heard, or remember, as in gaslighting. Name what you found: a made-up source, a wrong number, a claim without support, or a question still open.
Leave a record of your judgment
Write down what you checked, what the evidence supports, and what you still do not know. That helps the next reader, including you a month from now.
Keep a Claim Record. Put the source passage beside the sentence it supports.
From Distorted: Chapters 14 and 20, particularly “The Fake Citations Problem” and “The Coming Battle Over Evidence.”
Clarity Framework: Develop Awareness · Investigate Claims · Verify Facts. Explore the Framework.
Save this page
You can copy this link to your notes or bookmarks.