Tag: Paul Tough

Falsification: How to Destroy Incorrect Ideas

Sir Karl Popper wrote that the nature of scientific thought is that we could never be sure of anything. The only way to test the validity of any theory was to prove it wrong, a process he labeled falsification. And it turns out we’re quite bad at falsification.

When it comes to testing a theory we don’t instinctively try to find evidence we’re wrong. It’s much easier and more mentally satisfying to find information that proves our intuition. This is known as the confirmation bias.

“The human mind is a lot like the human egg, and the human egg has a shut-off device. When one sperm gets in, it shuts down so the next one can’t get in.”

— Charlie Munger

In Paul Tough’s book, How Children Succeed: Grit, Curiosity, and the Hidden Power of Character, he tells the story of an English psychologist Peter Cathcart Wason, who came up with an “ingenious experiment to demonstrate our natural tendency to confirm rather than disprove our own ideas.”

Subjects were told that they would be given a series of three numbers that followed a certain rule known only to the experimenter. Their assignment was to figure out what the rule was, which they could do by offering the experimenter other strings of three numbers and asking him whether or not these new strings met the rule.

The string of numbers the subjects were given was quite simple:


Try it: What’s your first instinct about the rule governing these numbers? And what’s another string you might test with the experimenter in order to find out if your guess is right? If you’re like most people, your first instinct is that the rule is “ascending even numbers” or “numbers increasing by two.” And so you guess something like:


And the experimenter says, “Yes! That string of numbers also meets the rule.” And your confidence rises. To confirm your brilliance, you test one more possibility, just as due diligence, something like:


“Yes!” says the experimenter. Another surge of dopamine. And you proudly make your guess: “The rule is: even numbers, ascending in twos.” “No!” says the experimenter. It turns out that the rule is “any ascending numbers.” So 8-10-12 does fit the rule, it’s true, but so does 1-2-3. Or 4-23-512. The only way to win the game is to guess strings of numbers that would prove your beloved hypothesis wrong—and that is something each of us is constitutionally driven to avoid.

In the study, only 1 in five people was able to guess the correct rule.

And the reason we’re all so bad at games like this is the tendency toward confirmation bias: It feels much better to find evidence that confirms what you believe to be true than to find evidence that falsifies what you believe to be true. Why go out in search of disappointment?

There is also a video explaining Wason’s work.

Slow Down and Think

Sebastian Garcia made a mistake, but he couldn’t figure it out. At the 2011 National Junior High Chess Championship, he was looking strong and heading towards a victory. Then he made a mistake, squandering his advantage. A few moves later, the collapse was complete. Sebastian shook hands with the boy who had beaten him and walked back to Union B, the conference room down the hall. Union B was the makeshift home for his chess team from Intermediate School 318 in Brooklyn.

Elizabeth Spiegel, the school’s chess teacher, was waiting. It was customary to come back to the room for a postmortem. Sebastian, feeling sorry for himself, slouched into the room, his head held low, and approached Spiegel. “I lost,” he announced.

“Tell me about your game,” Spiegel said. Sebastian flopped into the chair and handed her his notepad, where he’d recorded all the moves for both players in the game.

Sebastian explained that the other guy was simply better. “He had good skills,” he said. “Good strategies.”

And this is the point where many of us would simply say something along the lines of “did you do your best?,” in which case the likely response is “Yes.” Everyone is at least let off the hook. The teacher for ensuring students try their best, the student for having lost to someone better. Spiegel did not take this approach.

“Well, let’s see,” said Spiegel as she started to re-create the game on a chess board. At one point Sebastian fell into a trap. His opponent quickly pounced and took a pawn just four moves into the game. He was already down a piece.

Spiegel looked at him. “How long did you spend on that move,” she asked.

“Two seconds.”

Spiegel’s face tensed. “We did not bring you here so that you could spend two seconds on a move,” she said with an edge in her voice. In the face of this obvious challenge to what he did, Sebastian looked down. “Sebastian!” He looked up. “This is pathetic. If you continue to play like this, I’m going to withdraw you from the tournament, and you can just sit here with your head down for the rest of the weekend. Two seconds is not slow enough.”

Her voice grew more understanding. “Look, if you make a mistake, that’s okay. But you do something without even thinking about it? That’s not okay. I’m very, very, very upset to be seeing such a careless and thoughtless game.”

Then the storm passed. Spiegel resumed moving pieces and examining the game. She pointed out his good moves. “Very clever,” she said when he took the knight. As the game progressed, Spiegel praised his good ideas and asked him to come up with alternatives to others. “You were playing in some ways an excellent game,” she told him, “and then once in a while, you moved superfast and you did something really stupid. If you can stop doing that, you’re going to do very, very well.”

By the thirty-fifth move, Sebastian recovered completely from his early errors. The position on the board favored him. He pushed his queen forward, checking the white king. His opponent countered, drawing a pawn up to block the path. Sebastian moved his queen ahead: check. His opponent moved his king one square out of the queen’s range.

Rather than keep the pressure on, Sebastian went for an easy score: taking a white pawn with his queen. But in this move he had missed the threat. On the other side of the board, his opponent took his bishop and Sebastian’s advantage started to slip away.

“You took the pawn?” Spiegel asked. “Come on. What’s a better move?”

Sebastian didn’t answer.

“What about check?” Spiegel suggested. Sebastian stared at the board evaluating the move.

“Think about it,” Spiegel said. “Remember, when I ask you a question, you don’t have to answer right away. But you do have to be right.”

Suddenly a light went on his head. “I could win the queen,” he said.

Spiegel looked at him and said “Show me.”

Sebastian made the moves, understanding how one more check would have saved his bishop and sealed the game.

“This is the thing,” Spiegel said, moving the pieces back to the point where they were when Sebastian had gone for the easy pawn. “Think back on this moment. When you made this move,”— she captured the white pawn, as Sebastian had done—“ you lost the game. If you had made this move,”— she put the white king in check—“ you would have won the game.” She leaned back in her chair, her gaze fixed on Sebastian. “It’s okay if the loss hurts you a little,” she said. “You should feel bad. You’re a talented player, but you have to slow down and think more. Because now you have”— she checked her watch—“ four hours until the next game, which means that you have four hours to think about the fact that you got beat by this kid.” She tapped the board. “All because of this one time when you could have slowed down, but you didn’t.”

* * *

A version of that story appears in Paul Tough’s incredible book How Children Succeed: Grit, Curiosity, and the Hidden Power of Character. Spiegel’s story is straight out of Hollywood — inner-city teacher and low-income students, against all odds beating the pants off private school students. Often when you look into these stories, you find an asterisk. But in this case, Tough “couldn’t find one.”

Take a look at this tournament, held only a few months before the tournament that Sebastian Garcia was playing in:

Kindergarten Oak Hall School, a private school in Gainesville, Florida
First grade SciCore Academy, a private school in New Jersey
Second grade Dalton School, a private school in New York City
Third grade Hunter College Elementary, an exam school in New York City
Fourth grade Tie between SciCore Academy and Stuart Hall School for Boys, a Catholic school in New Orleans
Fifth grade Regnart Elementary, a public school in Cupertino, California, home of Apple and dozens of software companies
Ninth grade San Benito Veterans Memorial Academy, in southern Texas, a public school whose student body is mostly Hispanic and low income
Tenth grade Horace Mann, a private school in New York City
Eleventh grade Solomon Schechter, a private school in a New York City suburb
Twelfth grade Bronx Science, an exam school in New York City

With the exception of Grade nine, the winning team, “came from a private school, an exam school, a parochial school, or a public school populated by the children of Apple engineers,” Tough writes.

Except, that is, for middle school grades, the space where Elizabeth Spiegel teaches.

Sixth grade IS 318, a low-income public school in Brooklyn
Seventh grade IS 318, a low-income public school in Brooklyn
Eighth grade IS 318, a low-income public school in Brooklyn

These students didn’t win just one grade; they won every grade they entered. “The roster of schools they beat,” Tough writes, “reads like a wealthy parent’s wish list of the most desirable private schools in the country: Trinity, Collegiate, Spence, Dalton, and Horace Mann in New York City, and exclusive private schools in Boston, Miami, and Greenwich, Connecticut.”

The chess program at IS 318 is one of the best in the country because it teaches students how to think.

…they win tournaments because of what Elizabeth Spiegel was sitting in Union B doing that April afternoon: taking eleven-year-old kids, like Sebastian Garcia, who know a little chess but not a lot, and turning them, move by painstaking move, into champions.

“Most of the major academic studies of chess miss much that is essential to the way that chess-player thinks and feels,” Jonathan Rowson wrote in his book The Seven Deadly Chess Sins. “They are guilty of thinking of chess as an almost exclusively cognitive pursuit, where moves are chosen, and positions understood only on the basis of mental patterns and inferences.” In reality, he wrote, if you want to become a great chess player, or even a good one, “your ability to recognize and utilize your emotions is every bit as important as the way you think.”

Most of us probably think that Spiegel was teaching the kids chess. Of course, she often passed along specific chess knowledge: how to weigh the comparative value of moves, etc. “But most of the time,” Tough writes, “it struck me whenever I watched (Elizabeth Spiegel) at work, what she was really doing was far simpler, and also far more complicated: she was teaching her students a new way to think.” More than that, she was teaching students how to think.

Two of the most important executive functions are cognitive flexibility and cognitive self-control. Cognitive flexibility is the ability to see alternative solutions to problems, to think outside the box, to negotiate unfamiliar situations. Cognitive self-control is the ability to inhibit an instinctive or habitual response and substitute a more effective, less obvious one. Both skills are central to the training Spiegel gives to her students. To prevail at chess, she says, you need a heightened ability to see new and different ideas: Which especially creative winning move have you overlooked? And which potentially lethal move of your opponent’s are you blindly ignoring? She also teaches them to resist the temptation to pursue an immediately attractive move, since that type of move (as Sebastian Garcia found out) often leads to trouble down the road. “Teaching chess is really about teaching the habits that go along with thinking,” Spiegel explained to me one morning when I visited her classroom. “Like how to understand your mistakes and how to be more aware of your thought processes.”

Before she was a full-time chess teacher, Spiegel taught an eighth-grade honors, English class. She taught them the same way she taught Sebastian: ruthlessly analyzing everything. She was teaching her English class on how to think.

When students turned in writing assignments, she went through each assignment sentence by sentence with each student, asking, Well, are you sure that’s the best way to say what you want to say? “They looked at me like I was insane,” she told me. “I would write them these long letters about what they’d written. It would take me the whole evening to do six or seven of them.”

Although her teaching style might not have been the right fit for English, this helped her better understand how to teach chess. Rather than follow a set curriculum, she decided to construct her calendar as she went, focusing “entirely on what her students knew and, more important, on what they didn’t know.”

For instance, she would take her students to a weekend tournament and notice that many of them were hanging pieces , meaning they were leaving pieces undefended, which made them easy targets. The following Monday, she would organize the whole class around how not to hang pieces, reconstructing the students’ flawed games on the green felt practice boards hung on hooks at the front of her classroom. Again and again, she would go over her students’ games, both individually and as a class, analyzing exactly where a player had gone wrong, what he could have done differently, what might have happened if he had made the better move, and playing out these counterfactual scenarios for several moves before returning to the moment of error.

Sensible though this process might sound, it’s actually a pretty unusual way to teach chess, or to learn it. “It’s uncomfortable to focus so intensely on what you’re bad at,” Spiegel told me. “So the way people usually study chess is they read a book about chess, which can be fun and often intellectually amusing, but it doesn’t actually translate into skill. If you really want to get better at chess, you have to look at your games and figure out what you’re doing wrong.”

Calibrated Meanness

At the heart of Spiegel’s job was a complex balancing act. She wanted to build up her students’ confidence, to make them believe in their own ability to overcome stronger rivals and master an impossibly complicated game. But the exigencies of her job— and the particularities of her personality— meant that she spent most of her time telling her students how they were messing up. It’s the basic narrative of all postgame chess analysis, in fact: You thought you had a good idea here, but you were wrong.

“I struggle with it all the time,” she told me one day when I visited her class. “Every day. It’s very high on my list of anxieties as a teacher. I feel like I’m very mean to the kids. It kills me sometimes, like I go home and I play through everything I said to every kid and I’m like, ‘What am I doing? I’m damaging the children.’”

After the 2010 girls’ national tournament (which IS 318 won), Spiegel wrote on her blog:

The first day and a half was pretty bad. I was on a complete rampage, going over every game and being a huge bitch all the time: saying things like “THAT IS COMPLETELY UNACCEPTABLE!!!” to 11-year-olds for hanging pieces or not having a reason for a move. I said some amazing things to kids, including “You can count to two, right? Then you should have seen that!!” and “If you are not going to pay more attention, you should quit chess, because you are wasting everyone’s time.” By the end of round three I was starting to feel like an abusive jerk and was about to give up and be fake nice instead. But then in round four everyone took more than an hour and started playing well. And I really believe that’s why we seem to win girls’ nationals sections pretty easily every year: most people won’t tell teenage girls (especially the together, articulate ones) that they are lazy and the quality of their work is unacceptable. And sometimes kids need to hear that, or they have no reason to step up.

Slowing down, examining impulses, and considering alternatives sound reasonable, but it’s “quite rare in contemporary American Schools.”

If you believe that your school’s mission or your job as a teacher is simply to convey information, then it probably doesn’t seem necessary to subject your students to that kind of rigorous self-analysis. But if you’re trying to help them change their character, then conveying information isn’t enough. And while Spiegel didn’t use the word character to describe what she was teaching, there was a remarkable amount of overlap between the strengths David Levin and Dominic Randolph emphasized and the skills that Spiegel tried to inculcate in her students. Every day, in the classroom and at tournaments, I saw Spiegel trying to teach her students grit, curiosity, self-control, and optimism.

How Children Succeed: Grit, Curiosity, and the Hidden Power of Character is an amazing book. Highly recommended.