Author: Farnam Street

The Inner Game: Why Trying Too Hard Can Be Counterproductive

Reading Time: 10 minutes

The standard way of learning is far from being the fastest or most enjoyable. It’s slow, makes us second guess ourselves, and interferes with our natural learning process. Here we explore a better way to learn and enjoy the process.

***

It’s the final moment before an important endeavor—a speech, a performance, a presentation, an interview, a date, or perhaps a sports match. Up until now, you’ve felt good and confident about your abilities. But suddenly, something shifts. You feel a wave of self-doubt. You start questioning how well you prepared. The urge to run away and sabotage the whole thing starts bubbling to the surface.

As hard as you try to overcome your inexplicable insecurity, something tells you that you’ve already lost. And indeed, things don’t go well. You choke up, forget what you were meaning to say, long to just walk out, or make silly mistakes. None of this comes as a surprise—you knew beforehand that something had gone wrong in your mind. You just don’t know why.

Conversely, perhaps you’ve been in a situation where you knew you’d succeeded before you even began. You felt confident and in control. Your mind could focus with ease, impervious to self-doubt or distraction. Obstacles melted away, and abilities you never knew you possessed materialized.

This phenomenon—winning or losing something in your mind before you win or lose it in reality—is what tennis player and coach W. Timothy Gallwey first called “the Inner Game” in his book The Inner Game of Tennis. Gallwey wrote the book in the 1970s when people viewed sport as a purely physical matter. Athletes focused on their muscles, not their mindsets. Today, we know that psychology is in fact of the utmost importance.

Gallwey recognized that physical ability was not the full picture in any sport. In tennis, success is very psychological because there are really two games going on: the Inner Game and the Outer Game. If a player doesn’t pay attention to how they play the Inner Game—against their insecurities, their wandering mind, their self-doubt and uncertainty—they will never be as good as they have the potential to be. The Inner Game is fought against your own self-defeating tendencies, not against your actual opponent. Gallwey writes in the introduction:

Every game is composed of two parts, an outer game, and an inner game. . . . It is the thesis of this book that neither mastery nor satisfaction can be found in the playing of any game without giving some attention to the relatively neglected skills of the inner game. This is the game that takes place in the mind of the player, and it is played against such obstacles as lapses in concentration, nervousness, self-doubt, and self-condemnation. In short, it is played to overcome all habits of mind which inhibit excellence in performance. . . . Victories in the inner game may provide no additions to the trophy case, but they bring valuable rewards which are more permanent and which can contribute significantly to one’s success, off the court as well as on.

Ostensibly, The Inner Game of Tennis is a book about tennis. But dig beneath the surface, and it teems with techniques and insights we can apply to any challenge. The book is really about overcoming the external obstacles we create that prevent us from succeeding. You don’t need to be interested in tennis or even know anything about it to benefit from this book.

One of the most important insights Gallwey shares is that a major thing which leads us to lose the Inner Game is trying too hard and interfering with our own natural learning capabilities. Let’s take a look at how we can win the Inner Game in our own lives by seeing the importance of not forcing things.

The Two Sides of You

Gallwey was not a psychologist. But his experience as both a tennis player and a coach for other players gave him a deep understanding of how human psychology influences playing. The tennis court was his laboratory. As is evident throughout The Inner Game of Tennis, he studied himself, his students, and opponents with care. He experimented and tested out theories until he uncovered the best teaching techniques.

When we’re learning something new, we often internally talk to ourselves. We give ourselves instructions. When Gallwey noticed this in his students, he wondered who was talking to who. From his observations, he drew his key insight: the idea of Self 1 and Self 2.

Self 1 is the conscious self. Self 2 is the subconscious. The two are always in dialogue.

If both selves can communicate in harmony, the game will go well. More often, this isn’t what happens. Self 1 gets judgmental and critical, trying to instruct Self 2 in what to do. The trick is to quiet Self 1 and let Self 2 follow the natural learning process we are all born competent at; this is the process that enables us to learn as small children. This capacity is within us—we just need to avoid impeding it. As Gallwey explains:

Now we are ready for the first major postulate of the Inner Game: within each player the kind of relationship that exists between Self 1 and Self 2 is the prime factor in determining one’s ability to translate his knowledge of technique into effective action. In other words, the key to better tennis—or better anything—lies in improving the relationship between the conscious teller, Self 1, and the natural capabilities of Self 2.

Self 1 tries to instruct Self 2 using words. But Self 2 responds best to images and internalizing the physical experience of carrying out the desired action.

In short, if we let ourselves lose touch with our ability to feel our actions, by relying too heavily on instructions, we can seriously compromise our access to our natural learning processes and our potential to perform.

Stop Trying so Hard

Gallwey writes that “great music and art are said to arise from the quiet depths of the unconscious, and true expressions of love are said to come from a source which lies beneath words and thoughts. So it is with the greatest efforts in sports; they come when the mind is as still as a glass lake.”

What’s the most common piece of advice you’re likely to receive for getting better at something? Try harder. Work harder. Put more effort in. Pay more attention to what you’re doing. Do more.

Yet what do we experience when we are performing at our best? The exact opposite. Everything becomes effortless. We act without thinking or even giving ourselves time to think. We stop judging our actions as good or bad and observe them as they are. Colloquially, we call this being in the zone. In psychology, it’s known as “flow” or a “peak experience.”

Compare this to the typical tennis lesson. As Gallwey describes it, the teacher wants the student to feel that the cost of the lesson was worthwhile. So they give detailed, continuous feedback. Every time they spot the slightest flaw, they highlight it. The result is that the student does indeed feel the lesson fee is justifiable. They’re now aware of dozens of errors they need to fix—so they book more classes.

In his early days as a tennis coach, Gallwey took this approach. Over time, he saw that when he stepped back and gave his students less feedback, not more, they improved faster. Players would correct obvious mistakes without any guidance. On some deeper level, they knew the correct way to play tennis. They just needed to overcome the habits of the mind getting in the way. Whatever impeded them was not a lack of information. Gallwey writes:

I was beginning to learn what all good pros and students of tennis must learn: that images are better than words, showing better than telling, too much instruction worse than none, and that trying too hard often produces negative results.

There are numerous instances outside of sports when we can see how trying too hard can backfire. Consider a manager who feels the need to constantly micromanage their employees and direct every detail of their work, not allowing any autonomy or flexibility. As a result, the employees lose interest in ever taking initiative or directing their own work. Instead of getting the perfect work they want, the manager receives lackluster efforts.

Or consider a parent who wants their child to do well at school, so they control their studying schedule, limit their non-academic activities, and offer enticing rewards for good grades. It may work in the short term, but in the long run, the child doesn’t learn to motivate themselves or develop an intrinsic desire to study. Once their parent is no longer breathing down their neck, they don’t know how to learn.

Positive Thinking Backfires

Not only are we often advised to try harder to improve our skills, we’re also encouraged to think positively. According to Gallwey, when it comes to winning the Inner Game, this is the wrong approach altogether.

To quiet Self 1, we need to stop attaching judgments to our performance, either positive or negative. Thinking of, say, a tennis serve as “good” or “bad” shuts down Self 2’s intuitive sense of what to do. Gallwey noticed that “judgment results in tightness and tightness interferes with the fluidity required for accurate and quick movement. Relaxation produces smooth strokes and results from accepting your strokes as they are, even if erratic.”

In order to let Self 2’s sense of the correct action take over, we need to learn to see our actions as they are. We must focus on what is happening, not what is right or wrong. Once we can see clearly, we can tap into our inbuilt learning process, as Gallwey explains:

But to see things as they are, we must take off our judgmental glasses, whether they’re dark or rose-tinted. This action unlocks a process of natural development, which is as surprising as it is beautiful. . . . The first step is to see your strokes as they are. They must be perceived clearly. This can be done only when personal judgment is absent. As soon as a stroke is seen clearly and accepted as it is, a natural and speedy process of change begins.

It’s hard to let go of judgments when we can’t or won’t trust ourselves. Gallwey noticed early on that negative assessments—telling his students what they had done wrong—didn’t seem to help them. He tried only making positive assessments—telling them what they were doing well. Eventually, Gallwey recognized that attaching any sort of judgment to how his students played tennis was detrimental.

Positive and negative evaluations are two sides of the same coin. To say something is good is to implicitly imply its inverse is bad. When Self 1 hears praise, Self 2 picks up on the underlying criticism.

Clearly, positive and negative evaluations are relative to each other. It is impossible to judge one event as positive without seeing other events as not positive or negative. There is no way to stop just the negative side of the judgmental process.

The trick may be to get out of the binary of good or bad completely by doing more showing and asking questions like “Why did the ball go that way?” or “What are you doing differently now than you did last time?” Sometimes, getting people to articulate how they are doing by observing their own performance removes the judgments and focuses on developmental possibilities. When we have the right image in mind, we move toward it naturally. Value judgments get in the way of that process.

The Inner Game Way of Learning

We’re all constantly learning and picking up new skills. But few of us pay much attention to how we learn and whether we’re doing it in the best possible way. Often, what we think of as “learning” primarily involves berating ourselves for our failures and mistakes, arguing with ourselves, and not using the most effective techniques. In short, we try to brute-force ourselves into adopting a capability. Gallwey describes the standard way of learning as such:

Step 1: Criticize or judge past behavior.

Step 2: Tell yourself to change, instructing with word commands repeatedly.

Step 3: Try hard; make yourself do it right.

Step 4: Critical judgment about results leading to Self 1 vicious cycle.

The standard way of learning is far from being the fastest or most enjoyable. It’s slow, it makes us feel awful about ourselves, and it interferes with our natural learning process. Instead, Gallwey advocates following the Inner Game way of learning.

First, we must observe our existing behavior without attaching any judgment to it. We must see what is, not what we think it should be. Once we are aware of what we are doing, we can move onto the next step: picturing the desired outcome. Gallwey advocates images over outright commands because he believes visualizing actions is the best way to engage Self 2’s natural learning capabilities. The next step is to trust Self 2 and “let it happen!” Once we have the right image in mind, Self 2 can take over—provided we do not interfere by trying too hard to force our actions. The final step is to continue “nonjudgmental, calm observation of the results” in order to repeat the cycle and keep learning. It takes nonjudgmental observation to unlearn bad habits.

Conclusion

Towards the end of the book, Gallwey writes:

Clearly, almost every human activity involves both the outer and inner games. There are always external obstacles between us and our external goals, whether we are seeking wealth, education, reputation, friendship, peace on earth or simply something to eat for dinner. And the inner obstacles are always there; the very mind we use in obtaining our external goals is easily distracted by its tendency to worry, regret, or generally muddle the situation, thereby causing needless difficulties within.

Whatever we’re trying to achieve, it would serve us well to pay more attention to the internal, not just the external. If we can overcome the instinct to get in our own way and be more comfortable trusting in our innate abilities, the results may well be surprising.

Tagged: , , , ,

How To Spot Bad Science

Reading Time: 17 minutes

In a digital world that clamors for clicks, news is sensationalized and “facts” change all the time. Here’s how to discern what is trustworthy and what is hogwash.

***

Unless you’ve studied it, most of us are never taught how to evaluate science or how to parse the good from the bad. Yet it is something that dictates every area of our lives. It is vital for helping us understand how the world works. It might be too much effort and time to appraise research for yourself, however. Often, it can be enough to consult an expert or read a trustworthy source.

But some decisions require us to understand the underlying science. There is no way around it. Many of us hear about scientific developments from news articles and blog posts. Some sources put the work into presenting useful information. Others manipulate or misinterpret results to get more clicks. So we need the thinking tools necessary to know what to listen to and what to ignore. When it comes to important decisions, like knowing what individual action to take to minimize your contribution to climate change or whether to believe the friend who cautions against vaccinating your kids, being able to assess the evidence is vital.

Much of the growing (and concerning) mistrust of scientific authority is based on a misunderstanding of how it works and a lack of awareness of how to evaluate its quality. Science is not some big immovable mass. It is not infallible. It does not pretend to be able to explain everything or to know everything. Furthermore, there is no such thing as “alternative” science. Science does involve mistakes. But we have yet to find a system of inquiry capable of achieving what it does: move us closer and closer to truths that improve our lives and understanding of the universe.

“Rather than love, than money, than fame, give me truth.”

— Henry David Thoreau

There is a difference between bad science and pseudoscience. Bad science is a flawed version of good science, with the potential for improvement. It follows the scientific method, only with errors or biases. Often, it’s produced with the best of intentions, just by researchers who are responding to skewed incentives.

Pseudoscience has no basis in the scientific method. It does not attempt to follow standard procedures for gathering evidence. The claims involved may be impossible to disprove. Pseudoscience focuses on finding evidence to confirm it, disregarding disconfirmation. Practitioners invent narratives to preemptively ignore any actual science contradicting their views. It may adopt the appearance of actual science to look more persuasive.

While the tools and pointers in this post are geared towards identifying bad science, they will also help with easily spotting pseudoscience.

Good science is science that adheres to the scientific method, a systematic method of inquiry involving making a hypothesis based on existing knowledge, gathering evidence to test if it is correct, then either disproving or building support for the hypothesis. It takes many repetitions of applying this method to build reasonable support for a hypothesis.

In order for a hypothesis to count as such, there must be evidence that, if collected, would disprove it.

In this post, we’ll talk you through two examples of bad science to point out some of the common red flags. Then we’ll look at some of the hallmarks of good science you can use to sort the signal from the noise. We’ll focus on the type of research you’re likely to encounter on a regular basis, including medicine and psychology, rather than areas less likely to be relevant to your everyday life.

[Note: we will use the terms “research” and “science” and “researcher” and “scientist” interchangeably here.]

Power Posing

“The saddest aspect of life right now is that science gathers knowledge faster than society gathers wisdom.” ―Isaac Asimov

First, here’s an example of flawed science from psychology: power posing. A 2010 study by Dana Carney, Andy J. Yap, and Amy Cuddy entitled “Power Posing: Brief Nonverbal Displays Effects Neuroendocrine Levels and Risk Tolerance” claimed “open, expansive” poses caused participants to experience elevated testosterone levels, reduced cortisol levels, and greater risk tolerance. These are all excellent things in a high-pressure situation, like a job interview. The abstract concluded that “a person can, via a simple two-minute pose, embody power and instantly become more powerful.” The idea took off. It spawned hundreds of articles, videos, and tweets espousing the benefits of including a two-minute power pose in your day.

Yet at least eleven follow up studies, many led by Joseph Cesario of Michigan State University including “’Power Poses’ Don’t Work, Eleven New Studies Suggest,” failed to replicate the results. None found that power posing has a measurable impact on people’s performance in tasks or on their physiology. While subjects did report a subjective feeling of increased powerfulness, their performance did not differ from subjects who did not strike a power pose.

One of the researchers of the original study, Carney, has since changed her mind about the effect. Carney stated she no longer believe the results of the original study. Unfortunately, this isn’t always how researchers respond when confronted with evidence discrediting their prior work. We all know how uncomfortable changing our minds is.

The notion of power posing is exactly the kind of nugget that spreads fast online. It’s simple, free, promises dramatic benefits with minimal effort, and is intuitive. We all know posture is important. It has a catchy, memorable name. Yet examining the details of the original study reveals a whole parade of red flags. The study had 42 participants. That might be reasonable for preliminary or pilot studies. But is in no way sufficient to “prove” anything. It was not blinded. Feedback from participants was self-reported, which is notorious for being biased and inaccurate.

There is also a clear correlation/causation issue. Powerful, dominant animals tend to use expansive body language that exaggerates their size. Humans often do the same. But that doesn’t mean it’s the pose making them powerful. Being powerful could make them pose that way.

A TED Talk in which Amy Cuddy, the study’s co-author, claimed power posing could “significantly change the way your life unfolds” is one of the most popular to date, with tens of millions of views. The presentation of the science in the talk is also suspect. Cuddy makes strong claims with a single, small study as justification. She portrays power posing as a panacea. Likewise, the original study’s claim that a power pose makes someone “instantly become more powerful” is suspiciously strong.

This is one of the examples of psychological studies related to small tweaks in our behavior that have not stood up to scrutiny. We’re not singling out the power pose study as being unusually flawed or in any way fraudulent. The researchers had clear good intentions and a sincere belief in their work. It’s a strong example of why we should go straight to the source if we want to understand research. Coverage elsewhere is unlikely to even mention methodological details or acknowledge any shortcomings. It would ruin the story. We even covered power posing on Farnam Street in 2016—we’re all susceptible to taking these ‘scientific’ results seriously, without checking on the validity of the underlying science.

It is a good idea to be skeptical of research promising anything too dramatic or extreme with minimal effort, especially without substantial evidence. If it seems too good to be true, it most likely is.

Green Coffee Beans

“An expert is a person who has made all the mistakes that can be made in a very narrow field.” ―Niels Bohr

The world of weight-loss science is one where bad science is rampant. We all know, deep down, that we cannot circumnavigate the need for healthy eating and exercise. Yet the search for a magic bullet, offering results without effort or risks, continues. Let’s take a look at one study that is a masterclass in bad science.

Entitled “Randomized, Double-Blind, Placebo-Controlled, Linear Dose, Crossover Study to Evaluate the Efficacy and Safety of a Green Coffee Bean Extract in Overweight Subjects,” it was published in 2012 in the journal Diabetes, Metabolic Syndrome and Obesity: Targets and Therapy. On the face of it, and to the untrained eye, the study may appear legitimate, but it is rife with serious problems, as Scott Gavura explained in the article “Dr. Oz and Green Coffee Beans – More Weight Loss Pseudoscience” in the publication Science-Based Medicine. The original paper was later retracted by its authors. The Federal Trade Commission (FTC) ordered the supplement manufacturer who funded the study to pay a $3.5 million fine for using it in their marketing materials, describing it as “botched.”

The Food and Drug Administration (FDA) recommends studies relating to weight-loss consist of at least 3,000 participants receiving the active medication and at least 1,500 receiving a placebo, all for a minimum period of 12 months. This study used a mere 16 subjects, with no clear selection criteria or explanation. None of the researchers involved had medical experience or had published related research. They did not disclose the conflict of interest inherent in the funding source. It didn’t cover efforts to avoid any confounding factors. It is vague about whether subjects changed their diet and exercise, showing inconsistencies. The study was not double-blinded, despite claiming to be. It has not been replicated.

The FTC reported that the study’s lead investigator “repeatedly altered the weights and other key measurements of the subjects, changed the length of the trial, and misstated which subjects were taking the placebo or GCA during the trial.” A meta-analysis by Rachel Buchanan and Robert D. Beckett, “Green Coffee for Pharmacological Weight Loss” published in the Journal of Evidence-Based Complementary & Alternative Medicine, failed to find evidence for green coffee beans being safe or effective; all the available studies had serious methodological flaws, and most did not comply with FDA guidelines.

Signs of Good Science

“That which can be asserted without evidence can be dismissed without evidence.” ―Christopher Hitchens

We’ve inverted the problem and considered some of the signs of bad science. Now let’s look at some of the indicators a study is likely to be trustworthy. Unfortunately, there is no single sign a piece of research is good science. None of the signs mentioned here are, alone, in any way conclusive. There are caveats and exceptions to all. These are simply factors to evaluate.

It’s Published by a Reputable Journal

“The discovery of instances which confirm a theory means very little if we have not tried, and failed, to discover refutations.” —Karl Popper

A journal, any journal, publishing a study says little about its quality. Some will publish any research they receive in return for a fee. A few so-called “vanity publishers” claim to have a peer-review process, yet they typically have a short gap between receiving a paper and publishing it. We’re talking days or weeks, not the expected months or years. Many predatory publishers do not even make any attempt to verify quality.

No journal is perfect. Even the most respected journals make mistakes and publish low-quality work sometimes. However, anything that is not published research or based on published research in a journal is not worth consideration. Not as science. A blog post saying green smoothies cured someone’s eczema is not comparable to a published study. The barrier is too low. If someone cared enough about using a hypothesis or “finding” to improve the world and educate others, they would make the effort to get it published. The system may be imperfect, but reputable researchers will generally make the effort to play within it to get their work noticed and respected.

It’s Peer Reviewed

Peer review is a standard process in academic publishing. It’s intended as an objective means of assessing the quality and accuracy of new research. Uninvolved researchers with relevant experience evaluate papers before publication. They consider factors like how well it builds upon pre-existing research or if the results are statistically significant. Peer review should be double-blinded. This means the researcher doesn’t know who is reviewing their work and the reviewer doesn’t know who the researcher is.

Publishers only perform a cursory “desk check” before moving onto peer review. This is to check for major errors, nothing more. They cannot have the expertise necessary to vet the quality of every paper they handle—hence the need for external experts. The number of reviewers and strictness of the process depends on the journal. Reviewers either declare a paper unpublishable or suggest improvements. It is rare for them to suggest publishing without modifications.

Sometimes several rounds of modifications prove necessary. It can take years for a paper to see the light of day, which is no doubt frustrating for the researcher. But it ensures no or fewer mistakes or weak areas.

Pseudoscientific practitioners will often claim they cannot get their work published because peer reviewers suppress anything contradicting prevailing doctrines. Good researchers know having their work challenged and argued against is positive. It makes them stronger. They don’t shy away from it.

Peer review is not a perfect system. Seeing as it involves humans, there is always room for bias and manipulation. In a small field, it may be easy for a reviewer to get past the double-blinding. However, as it stands, peer review seems to be the best available system. In isolation, it’s not a guarantee that research is perfect, but it’s one factor to consider.

The Researchers Have Relevant Experience and Qualifications

One of the red flags in the green coffee bean study was that the researchers involved had no medical background or experience publishing obesity-related research.

While outsiders can sometimes make important advances, researchers should have relevant qualifications and a history of working in that field. It is too difficult to make scientific advancements without the necessary background knowledge and expertise. If someone cares enough about advancing a given field, they will study it. If it’s important, verify their backgrounds.

It’s Part of a Larger Body of Work

“Science, my lad, is made up of mistakes, but they are mistakes which it is useful to make, because they lead little by little to the truth.” ―Jules Verne

We all like to stand behind the maverick. But we should be cautious of doing so when it comes to evaluating the quality of science. On the whole, science does not progress in great leaps. It moves along millimeter by millimeter, gaining evidence in increments. Even if a piece of research is presented as groundbreaking, it has years of work behind it.

Researchers do not work in isolation. Good science is rarely, if ever, the result of one person or even one organization. It comes from a monumental collective effort. So when evaluating research, it is important to see if other studies point to similar results and if it is an established field of work. For this reason, meta-analyses, which analyze the combined results of many studies on the same topic, are often far more useful to the public than individual studies. Scientists are humans and they all make mistakes. Looking at a collective body of work helps smooth out any problems. Individual studies are valuable in that they further the field as a whole, allowing for the creation of meta-studies.

Science is about evidence, not reputation. Sometimes well-respected researchers, for whatever reason, produce bad science. Sometimes outsiders produce amazing science. What matters is the evidence they have to support it. While an established researcher may have an easier time getting support for their work, the overall community accepts work on merit. When we look to examples of unknowns who made extraordinary discoveries out of the blue, they always had extraordinary evidence for it.

Questioning the existing body of research is not inherently bad science or pseudoscience. Doing so without a remarkable amount of evidence is.

It Doesn’t Promise a Panacea or Miraculous Cure

Studies that promise anything a bit too amazing can be suspect. This is more common in media reporting of science or in research used for advertising.

In medicine, a panacea is something that can supposedly solve all, or many, health problems. These claims are rarely substantiated by anything even resembling evidence. The more outlandish the claim, the less likely it is to be true. Occam’s razor teaches us that the simplest explanation with the fewest inherent assumptions is most likely to be true. This is a useful heuristic for evaluating potential magic bullets.

It Avoids or at Least Discloses Potential Conflicts of Interest

A conflict of interest is anything that incentivizes producing a particular result. It distorts the pursuit of truth. A government study into the health risks of recreational drug use will be biased towards finding evidence of negative risks. A study of the benefits of breakfast cereal funded by a cereal company will be biased towards finding plenty of benefits. Researchers do have to get funding from somewhere, so this does not automatically make a study bad science. But research without conflicts of interest is more likely to be good science.

High-quality journals require researchers to disclose any potential conflicts of interest. But not all journals do. Media coverage of research may not mention this (another reason to go straight to the source). And people do sometimes lie. We don’t always know how unconscious biases influence us.

It Doesn’t Claim to Prove Anything Based on a Single Study

In the vast majority of cases, a single study is a starting point, not proof of anything. The results could be random chance, or the result of bias, or even outright fraud. Only once other researchers replicate the results can we consider a study persuasive. The more replications, the more reliable the results are. If attempts at replication fail, this can be a sign the original research was biased or incorrect.

A note on anecdotes: they’re not science. Anecdotes, especially from people close to us or those who have a lot of letters behind their name, have a disproportionate clout. But hearing something from one person, no matter how persuasive, should not be enough to discredit published research.

Science is about evidence, not proof. And evidence can always be discredited.

It Uses a Reasonable, Representative Sample Size

A representative sample represents the wider population, not one segment of it. If it does not, then the results may only be relevant for people in that demographic, not everyone. Bad science will often also use very small sample sizes.

There is no set target for what makes a large enough sample size; it all depends on the nature of the research. In general, the larger, the better. The exception is in studies that may put subjects at risk, which use the smallest possible sample to achieve usable results.

In areas like nutrition and medicine, it’s also important for a study to last a long time. A study looking at the impact of a supplement on blood pressure over a week is far less useful than one over a decade. Long-term data smooths out fluctuations and offers a more comprehensive picture.

The Results Are Statistically Significant

Statistical significance refers to the likelihood, measured in a percentage, that the results of a study were not due to pure random chance. The threshold for statistical significance varies between fields. Check if the confidence interval is in the accepted range. If it’s not, it’s not worth paying attention to.

It Is Well Presented and Formatted

“When my information changes, I alter my conclusions. What do you do, sir?” ―John Maynard Keynes

As basic as it sounds, we can expect good science to be well presented and carefully formatted, without prominent typos or sloppy graphics.

It’s not that bad presentation makes something bad science. It’s more the case that researchers producing good science have an incentive to make it look good. As Michael J. I. Brown of Monash University explains in How to Quickly Spot Dodgy Science, this is far more than a matter of aesthetics. The way a paper looks can be a useful heuristic for assessing its quality. Researchers who are dedicated to producing good science can spend years on a study, fretting over its results and investing in gaining support from the scientific community. This means they are less likely to present work looking bad. Brown gives an example of looking at an astrophysics paper and seeing blurry graphs and misplaced image captions—then finding more serious methodological issues upon closer examination. In addition to other factors, sloppy formatting can sometimes be a red flag. At the minimum, a thorough peer-review process should eliminate glaring errors.

It Uses Control Groups and Double-Blinding

A control group serves as a point of comparison in a study. The control group should be people as similar as possible to the experimental group, except they’re not subject to whatever is being tested. The control group may also receive a placebo to see how the outcome compares.

Blinding refers to the practice of obscuring which group participants are in. For a single-blind experiment, the participants do not know if they are in the control or the experimental group. In a double-blind experiment, neither the participants nor the researchers know. This is the gold standard and is essential for trustworthy results in many types of research. If people know which group they are in, the results are not trustworthy. If researchers know, they may (unintentionally or not) nudge participants towards the outcomes they want or expect. So a double-blind study with a control group is far more likely to be good science than one without.

It Doesn’t Confuse Correlation and Causation

In the simplest terms, two things are correlated if they happen at the same time. Causation is when one thing causes another thing to happen. For example, one large-scale study entitled “Are Non-Smokers Smarter than Smokers?” found that people who smoke tobacco tend to have lower IQs than those who don’t. Does this mean smoking lowers your IQ? It might, but there is also a strong link between socio-economic status and smoking. People of low income are, on average, likely to have lower IQ than those with higher incomes due to factors like worse nutrition, less access to education, and sleep deprivation. A study by the Centers for Disease Control and Prevention entitled “Cigarette Smoking and Tobacco Use Among People of Low Socioeconomic Status,” people of low socio-economic status are also more likely to smoke and to do so from a young age. There might be a correlation between smoking and IQ, but that doesn’t mean causation.

Disentangling correlation and causation can be difficult, but good science will take this into account and may detail potential confounding factors of efforts made to avoid them.

Conclusion

“The scientist is not a person who gives the right answers, he’s one who asks the right questions.” ―Claude Lévi-Strauss

The points raised in this article are all aimed at the linchpin of the scientific method—we cannot necessarily prove anything; we must consider the most likely outcome given the information we have. Bad science is generated by those who are willfully ignorant or are so focused on trying to “prove” their hypotheses that they fudge results and cherry-pick to shape their data to their biases. The problem with this approach is that it transforms what could be empirical and scientific into something subjective and ideological.

When we look to disprove what we know, we are able to approach the world with a more flexible way of thinking. If we are unable to defend what we know with reproducible evidence, we may need to reconsider our ideas and adjust our worldviews accordingly. Only then can we properly learn and begin to make forward steps. Through this lens, bad science and pseudoscience are simply the intellectual equivalent of treading water, or even sinking.

Article Summary

  • Most of us are never taught how to evaluate science or how to parse the good from the bad. Yet it is something that dictates every area of our lives.
  • Bad science is a flawed version of good science, with the potential for improvement. It follows the scientific method, only with errors or biases.
  • Pseudoscience has no basis in the scientific method. It does not attempt to follow standard procedures for gathering evidence. The claims involved may be impossible to disprove.
  • Good science is science that adheres to the scientific method, a systematic method of inquiry involving making a hypothesis based on existing knowledge, gathering evidence to test if it is correct, then either disproving or building support for the hypothesis.
  • Science is about evidence, not proof. And evidence can always be discredited.
  • In science, if it seems too good to be true, it most likely is.

Signs of good science include:

  • It’s Published by a Reputable Journal
  • It’s Peer Reviewed
  • The Researchers Have Relevant Experience and Qualifications
  • It’s Part of a Larger Body of Work
  • It Doesn’t Promise a Panacea or Miraculous Cure
  • It Avoids or at Least Discloses Potential Conflicts of Interest
  • It Doesn’t Claim to Prove Anything Based on a Single Study
  • It Uses a Reasonable, Representative Sample Size
  • The Results Are Statistically Significant
  • It Is Well Presented and Formatted
  • It Uses Control Groups and Double-Blinding
  • It Doesn’t Confuse Correlation and Causation
Tagged: , , , , , , ,

Is Vulnerability a Choice?

Reading Time: 6 minutes

Being vulnerable is not a choice. It’s a reality of living. What we do with that vulnerability can either open doors to deeper connection, or throw up walls that stifle growth and fulfillment.

***

Vulnerability: the quality or state of being exposed to the possibility of being attacked or harmed, either physically or emotionally.

Given the potential consequences, why would anyone ever choose to be vulnerable? Who wants to risk an emotional or physical attack?

At the basic biological level, it seems to make very little sense to be vulnerable. When we are, we can more easily get hurt. We can get physically maimed or killed by a predator. Emotional attacks can make us afraid of rejection. Since the vast majority of us don’t want to die and instead pass on our genes, avoiding vulnerability seems to make perfect sense. Be tough in order to increase your chances of a long life. Don’t give anyone the opportunity to hurt you.

However, humans usually want to do more than just survive. We focus on the quality of our lives as well. Yes, we want our lives to be long. But we also want them to be good.

Part of a good life is having good relationships. We are social creatures and live longer, healthier lives when we have people around us that we trust and love. We want to be around people who can make us laugh and help us through life’s inevitable hard times. Our lives are less stressful when we have people with whom we can relax and be authentic. Without genuine vulnerability, it’s impossible to build the types of relationships that can provide comfort and increase resilience. The risks of vulnerability may be high, but the rewards of positive, strong relationships are even higher.

The reality is, we are vulnerable in some way at all times. We are vulnerable to viruses and accidents, misunderstandings and the pain caused by our fears and anxieties. Vulnerability is a part of life for all of us. Having close relationships where we can be vulnerable is actually a way to reduce our overall weakness. As Dr. Sue Johnson said on The Knowledge Project, “We need connection with others like we need oxygen. We’re way too vulnerable without it.”

The only choice we really have when it comes to vulnerability is the choice to acknowledge it or not. There is no doubt it can be hard to be vulnerable, especially if we didn’t have positive experiences with it as children. But social connections sustain us, and meaningful social connections are hard to build and maintain without mutual vulnerability.

Some people constantly pretend they have no vulnerabilities. Those people are frustrating to be around. Why? Because everyone is vulnerable in some way, so we know that those who say they aren’t are lying. No one likes to spend time around people who can’t be honest. Furthermore, people who refuse to acknowledge their vulnerabilities (at least to themselves) don’t make great friends or partners because we can’t learn much from them to help us process our own vulnerabilities. Even if it’s hard to pinpoint, we sense something is missing in our interactions with them. They don’t trust us enough to risk hurt.

Someone who goes on about how everything in their life is okay can’t offer much insight into how to deal with things that are most definitely not okay. And someone who thinks they are infallible tends to blame others when things don’t work out. They can’t admit to being wrong, which is another drawback to having them as a friend.

In her TED talk on the subject, author Brené Brown says, “The more afraid we are, the more vulnerable we are, the more afraid we are.” We develop these lists of all the things we won’t do and all the ways in which we won’t engage with people in order to protect ourselves. Our vulnerabilities get registered as something that could be exploited to hurt us. So we put up big buffers of denial and anger because it seems that if we admit we are afraid of something, our whole lives are going to come crashing down as people rush in to take advantage of our weaknesses. Except that isn’t true.

When we allow ourselves to be vulnerable (most often to those we are closest to, but also occasionally to others when the situation would benefit from us putting ourselves out there), we can create amazing reciprocal interactions that empower all parties.

When we are able to say the following: “I don’t know,” “I made a mistake,” “I’m sorry for causing you pain,” “I’m scared,” “I cried last night,” or “I’m struggling with this,” we actually free up energy because we no longer have to put effort into maintaining our buffers and our illusions. When we open up and admit to our vulnerabilities, we give people the opportunity to safely admit to theirs as well. We might hear back: “I make mistakes all the time,” “I’m scared as well,” “I cry too,” or “I also struggle with that.” And in that shared space, we can let go of some of the fear and make room for a deeper connection. When we are vulnerable with someone who doesn’t judge us for it, we can grow stronger. We can become less affected by situations that normally cause us stress.

Most importantly, we strengthen our connection with the people we are sharing with.

Although someone may react by ridiculing you when you admit to a fear, a far more common reaction is respect for your bravery and a sigh of relief over a shared circumstance. Someone doesn’t have to share your particular fear to feel a connection. We’re all afraid of something, and by being honest about your fears, you have signaled that others can share their fears with you in return.

We have written before about the social media prism and how it distorts reality, leading most of us to believe we are the only ones whose lives suck sometimes. The endless posts about career successes and fabulous vacations are really a large-scale representation of the fear of vulnerability. Complex, varied lives become little more than a glittering highlight reel. We never get to see the outtakes.

But coming clean about the downs increases the value of sharing the ups. At the very least, it’s more relatable. We learn more through failure than we do through success. And since we can’t try everything, learning from others’ failures is exceptionally valuable. To just hear the story of the person who made it big and sold their company is not useful. To hear about their multiple failures, their trials, their stops and starts and all the times they doubted themselves—now that’s an insight worth sharing.

Being vulnerable starts with being honest with yourself. How can you get better if you can’t admit that you could be better? How are you going to be a better partner or friend if you can’t admit that sometimes you aren’t a great one? How will you learn from your mistakes if you don’t acknowledge making any?

When we share that vulnerability and find people we can be open with, we form valuable connections. After all, to really trust someone, we need to know if they are going to be there when we are vulnerable. As Dr. Sue Johnson explained on The Knowledge Project, “When you can be vulnerable for a moment, and that person tunes in and cares about your vulnerability, that’s the person to go with.” In this way, vulnerability can also serve as a litmus test for your close relationships. If you can’t be vulnerable with someone, why bother? What can you really get from a relationship in which you can never relax and be yourself?

When we have people with whom we can be vulnerable, we actually reduce our exposure to potential harm and improve the quality of our life. By putting ourselves out there and risking hurt, we often find that we create more meaningful interactions with the people in our lives. When we have people we can trust with our deepest vulnerabilities, we increase our ability to be resilient in the face of chance and change.

Tagged: , , ,

The Top 5 Episodes of The Knowledge Project 2019

Reading Time: 2 minutes

Through conversations, we are able to learn from others, reflect on ourselves, and better navigate a conscious life.

The goal of our podcast, The Knowledge Project, is to help you think, reflect, and better understand the complexities and interconnections in the world in which we live.

If done right, listeners should walk away from episodes with a deeper understanding and a renewed sense of curiosity. Of course, not all of the conversations or guests will appeal to everyone.  And that is the point. We consciously want to explore the thinking, ideas, and methods of thoughtful people to deepen our understanding, challenge our ideas, and gain a broader perspective.

Of the 23 interviews that we published in 2019, these are the top five (as measured by downloads in the first 30 days):

  • #62 Cracking the Code of Love with Dr. Sue Johnson — Dr. Sue Johnson is a researcher, clinical psychologist, and the developer of EFT or Emotionally Focused Therapy. In this interview, we discuss how to create, protect, and nourish fulfilling sexual and emotional relationships.
  • #68 Putting Your Intuition on Ice with Daniel Kahneman — In this fascinating episode of the Knowledge Project Podcast, Psychologist and Nobel laureate Daniel Kahneman reveals the actions we can take to overcome the biases that cripple our decision-making, damper our thinking, and limit our effectiveness. Listen and Learn from the master.
  • #57 Decoding Difficult Conversations with Sheila Heen — Two-time NY Times best-selling author, consultant, and lecturer at Harvard Law School, Sheila Heen makes the tough talks easier by breaking down the three layers that make up every difficult conversation.
  • #67 Keeping the Flywheel in Motion with Jim Collins — An earnest student and powerful teacher, mega best-selling author Jim Collins goes under the hood and shows what all enduring companies have in common. We talk luck, leadership, and business longevity.
  • #60 Leading Above the Line with Jim Dethmer — Jim Dethmer, founder of The Conscious Leadership Group, shares practical advice about becoming more self-aware, ditching the victim mindset, and connecting more fully with the people in our lives.

One episode that just missed the cut but warrants your attention is #71 Cultivating Desire with relationship expert Esther Perel.

Thanks for listening, Because of people like you sharing our show with friends, family, and colleagues, we crossed 12 million downloads this year and were selected as “Best of” Apple Podcasts in 2019.

Subscribe on Apple Podcasts | YouTube | Spotify | Overcast | Google Podcasts

Want more? Last year was a great year for the podcast as well. Here are the top 5 episodes from 2018.

Tagged:

The Best of Farnam Street 2019

Reading Time: 3 minutes

We read for the same reasons we have conversations — to enrich our lives.

Reading helps us to think, feel, and reflect — not only upon ourselves and others but upon our ideas, and our relationship with the world. Reading deepens our understanding and helps us live consciously.

Of the 31 articles we published on FS this year, here are the top ten as measured by a combination of page views, responses, and feeling.

How Not to Be Stupid — Stupidity is overlooking or dismissing conspicuously crucial information. Here are seven situational factors that compromise your cognitive ability and result in increased odds of stupidity.

The Danger of Comparing Yourself to Others — When you stop comparing yourself to others and turn your focus inward, you start being better at what really matters: being you.

Yes, It’s All Your Fault: Active vs. Passive Mindsets — The hard truth is that most things in your life – good and bad – are your fault. The sooner you realize that, the better things will be. Here’s how to cultivate an active mindset and take control of your life.

Getting Ahead By Being Inefficient — Inefficient does not mean ineffective, and it is certainly not the same as lazy. You get things done – just not in the most effective way possible. You’re a bit sloppy, and use more energy. But don’t feel bad about it. There is real value in not being the best.

How to Do Great Things — If luck is the cause of a person’s success, why are so many so lucky time and time again? Learn how to create your own luck by being intelligently prepared.

The Anatomy of a Great Decision — Making better decisions is one of the best skills we can develop. Good decisions save time, money, and stress. Here, we break down what makes a good decision and what we can do to improve our decision-making processes.

The Importance of Working With “A” Players — Building a team is more complicated than collecting talent. I once tried to solve a problem by putting a bunch of PhDs in a room. While comments like that sounded good and got me a lot of projects above my level, they were rarely effective at delivering actual results.

Compounding Knowledge — The filing cabinet of knowledge stored in Warren Buffett’s brain has helped make him the most successful investor of our time. But it takes much more than simply reading a lot. In this article, learn how to create your own “snowball effect” to compound what you know into opportunity.

An Investment Approach That Works — There are as many investment strategies as there are investment opportunities. Some are good; many are terrible. Here’s the one that I lean on the most when I’m looking for low risk and above average returns.

Resonance: How to Open Doors For Other People — Opening doors for other people is a critical concept to understand in life. Read this article to learn more about how to show people that you care.

More interesting things, you might have missed

Thank you

As we touched on in the annual letter, it’s been a wonderful year at FS. We are looking forward to a wider variety of content on the blog in 2020 with a mix of deep dives and pieces exploring new subjects.

Thank you for an amazing 2019 and we look forward to learning new things with you in 2020.

Still curious? You can find the top five podcast episodes in 2019 here. Our Best of Farnam Street archive can be found here.

Tagged: , , , , , ,

The Gift of Hope

Reading Time: 2 minutes

It can be daunting, wondering what to give people, especially at this time of year. What gift properly communicates the feelings you have for someone? One idea is to give yourself. Another is to give the gift of hope.

In Becoming Wise, Krista Tippett writes about hope, and how transformative it can be.

“In a century of staggering open questions, hope becomes a calling for those of us who can hold it, for the sake of the world. Hope is distinct, in my mind, from optimism or idealism. It has nothing to do with wishing. It references reality at every turn and reveres truth. It lives open eyed and wholehearted with the darkness that is woven ineluctably into the light of life and sometimes seems to overcome it. Hope, like every virtue, is a choice that becomes a practice that becomes spiritual muscle memory. It’s a renewable resource for moving through life as it is, not as we wish it to be.”

Hope makes us resilient and courageous. We can share our hope with others, engaging in a reciprocal exchange that makes us all stronger. Tippett writes:

“We want to be called to our best selves. We long to figure out what that would look like. And we are figuring out that we need each other to do so.”

and

“Hope is an orientation, and insistence on wresting wisdom and joy from the endlessly fickle fabric of space and time.”

So how might we give hope? Tippett shares this insight:

“There are millions of people at any given moment, young and old, giving themselves over to service, risking hope, and all the while ennobling us all. To take such goodness in and let it matter-to let it define our take on reality as much as headlines of violence-is a choice we can make to live by the light in the darkness, to be brave and free.”

And thus hope, as explained by Tippett, is a resource we can grow, give, and share together.

“It is a relief to claim our love of each other and take that on as an adventure, a calling. It is a pleasure to wonder at the mystery we are and find delight in the vastness of reality that is embedded in our beings. It is a privilege to hold something robust and resilient called hope, which has the power to shift the world on its axis.”

Tagged: , , ,

Explore

Browse

123›»