From Distorted
Why Our Own Side Is Harder to Question
Check a claim you like as carefully as one you doubt.
A headline praises a group you support. You share it. A similar headline praises a rival. Now you want to know how the study was done.
That difference is worth a closer look. Perhaps one group has earned more trust. Or perhaps you checked less carefully because you liked the first answer.
Chapter 29 of Distorted asks us to examine that second possibility, especially in “When the Lie Feels Good.” Does a story deserve your trust, or do you want it to be true?
Begin with one claim you can check.
Two claims, the same questions
The organizations, figures, and accounts in this exercise are fictional. The two statements use the same figures on purpose. They help show how to check a claim. They do not compare real groups.
Account A
Our twelve-week reading program transformed outcomes for four in five students. It deserves to become the district's standard approach.
Account B
Our twelve-week reading program transformed outcomes for four in five students. It deserves to become the district's standard approach.
Before learning who said each statement, ask what you need to know. What does “transformed” mean? Four in five of which students? How was progress measured? Why should other schools expect the same result?
Write one sentence about how much you trust these claims. Add one question you would ask both groups. Keep your notes for the next step.
Reveal the imagined speakers
Imagine Account A came from an education group you support. Account B came from a rival group whose approach you often doubt. Both are promoting their own programs. We have no outside test of their results or further evidence about how reliable either group is.
Read your question again. Would you now ask it more gently? Demand stronger proof? Stop asking it? Explain why. This exercise helps you reflect on your judgment. It has not been tested as a way to measure bias, which can mean favoring one side unfairly.
The supplied record
Each fictional group reports the same figures: 80 students signed up. Of those, 50 finished. Forty of those who finished moved up at least one band on the group's own reading test. A band is a level on that test. Later results for the other 30 students are missing.
There is no outside test of the results. There is also no comparison group: similar students whose results could help show what happened without the program.
Forty out of fifty is 80%. That describes those who finished. Forty out of eighty is 50%. That tells us how many of all the students who signed up had a reported gain. Neither figure tells us how the students with missing results did.
The headline's “four in five students” leaves out who finished. “Transformed” suggests a major change. We need to know what moving up one test band means before accepting that word. Using the program across the district raises more questions: Would it suit other students? What would it cost? Would it work elsewhere?
Both reports leave these questions open. That does not prove that the programs work equally well. It does not prove that either one fails. We need more evidence to find out.
Let the source matter for a reason
Who made a claim can matter. Does the source know the subject? Can they see the records? Do they explain their method and correct mistakes? Those are reasons to take a report seriously. Past cases of false reporting are reasons to check more closely.
“They share my values” may explain why you support a group. It does not show that this result is correct.
Ask: What do I know about this source that helps me judge this claim? Look for a record you can check. Being familiar is not the same as being reliable. In the exercise, we learned which group we liked and that both were promoting their work. We learned nothing new about their results.
Name what a correction would cost
Perhaps you told someone to try the program. Perhaps friends volunteer there. Perhaps its purpose matters to you. Correcting the headline could feel like more than fixing a number.
You can notice that feeling without explaining its cause. Write down what you care about, then the claim you need to check. You can still say, “I want children to get good reading support.” You may need to change, “This report proves the program will work across the district.”
Now consider the strongest concerns. Students who finished may differ from those who left. Other help may explain some gains. The test may not measure what schools need to know. These questions deserve answers. Asking them does not mean abandoning the cause. Unanswered questions also do not prove that the program failed.
Make a revision someone can use
A useful correction explains what you got wrong, what the record shows, and what you still do not know:
I shared this as evidence that the program transformed reading for four in five students. Of the 80 students who signed up, 50 finished. Forty of those moved up at least one level on the group's own reading test. Results for the other 30 students are missing. We do not know how much of the gain the program caused or whether other schools would get the same result. I still support the goal. I want those questions answered before recommending a wider rollout.
You do not have to choose a position halfway between two sides. Stronger evidence can support one side more than the other. Explain how you judge a claim. Keep using that standard when the answer is one you dislike.
Take one favored claim further
Choose something you shared, defended, or wanted to be true. Use Compare the Stories to examine another strong explanation. Name evidence that could change your mind.
From Distorted
This guide builds on the book's questions about belonging, speaking up, and checking our own views. See Chapters 8, 9, 19, and 27, and Chapter 29's “When the Lie Feels Good.” The exercise was written for this guide. It does not report a real experiment.
Save this page
You can copy this link to your notes or bookmarks.