Tag: Science

The Half-life of Facts

Facts change all the time. Smoking has gone from doctor recommended to deadly. We used to think the Earth was the center of the universe and that Pluto was a planet. For decades we were convinced that the brontosaurus was a real dinosaur.

Knowledge, like milk, has an expiry date. That’s the key message behind Samuel Arbesman’s excellent new book The Half-life of Facts: Why Everything We Know Has an Expiration Date.

We’re bombarded with studies that seemingly prove this or that. Caffeine is good for you one day and bad for you the next. What we think we know and understand about the world is constantly changing. Nothing is immune. While big ideas are overturned infrequently, little ideas churn regularly.

As scientific knowledge grows, we end up rethinking old knowledge. Abresman calls this “a churning of knowledge.” But understanding that facts change (and how they change) helps us cope in a world of constant uncertainty. We can never be too sure of what we know.

In introducing this idea, Abresam writes:

Knowledge is like radioactivity. If you look at a single atom of uranium, whether it’s going to decay — breaking down and unleashing its energy — is highly unpredictable. It might decay in the next second, or you might have to sit and stare at it for thousands, or perhaps even millions, of years before it breaks apart.

But when you take a chunk of uranium, itself made up of trillions upon trillions of atoms, suddenly the unpredictable becomes predictable. We know how uranium atoms work in the aggregate. As a group of atoms, uranium is highly regular. When we combine particles together, a rule of probability known as the law of large numbers takes over, and even the behavior of a tiny piece of uranium becomes understandable. If we are patient enough, half of a chunk of uranium will break down in 704 million years, like clock-work. This number — 704 million years — is a measurable amount of time, and it is known as the half-life of uranium.

It turns out that facts, when viewed as a large body of knowledge, are just as predictable. Facts, in the aggregate, have half-lives: We can measure the amount of time for half of a subject’s knowledge to be overturned. There is science that explores the rates at which new facts are created, new technologies developed, and even how facts spread. How knowledge changes can be understood scientifically.

This is a powerful idea. We don’t have to be at sea in a world of changing knowledge. Instead, we can understand how facts grow and change in the aggregate, just like radioactive materials. This book is a guide to the startling notion that our knowledge — even what each of us has in our head — changes in understandable and systematic ways.

Why does this happen? Why does knowledge churn? In Zen and the Art of Motocycle Maintenance, Robert Pirsig writes:

If all hypotheses cannot be tested, then the result of any experiment are inconclusive and the entire scientific method falls short of its goal of establishing proven knowledge.

About this Einstein had said, “Evolution has shown that at any given moment out of all conceivable constructions a single one has always proved itself absolutely superior to the rest,” and let it go at that.

… But there it was, the whole history of science, a clear story of continuously new and changing explanations of old facts. The time spans of permanence seemed completely random, he could see no order in them. Some scientific truths seemed to last for centuries, others for less than a year. Scientific truth was not dogma, good for eternity, but a temporal quantitative entity that could be studied like anything else.

A few pages later, Pirsig continues:

The purpose of scientific method is to select a single truth from among many hypothetical truths. That, more than anything else, is what science is all about. But historically science has done exactly the opposite. Through multiplication upon multiplication of facts, information, theories and hypotheses, it is science itself that is leading mankind from single absolute truths to multiple, indeterminate, relative ones.

With that, lets dig into how this looks. Arbesman offers a example:

A few years ago a team of scientists at a hospital in Paris decided to actually measure this (churning of knowledge). They decided to look at fields that they specialized in: cirrhosis and hepatitis, two areas that focus on liver diseases. They took nearly five hundred articles in these fields from more than fifty years and gave them to a battery of experts to examine.

Each expert was charged with saying whether the paper was factual, out-of-date, or disproved, according to more recent findings. Through doing this they were able to create a simple chart (see below) that showed the amount of factual content that had persisted over the previous decades. They found something striking: a clear decay in the number of papers that were still valid.

Furthermore, they got a clear measurement of the half-life of facts in these fields by looking at where the curve crosses 50 percent on this chart: 45 years. Essentially, information is like radioactive material: Medical knowledge about cirrhosis or hepatitis takes about forty-five years for half of it to be disproven or become out-of-date.

half-life of facts, decay in the truth of knowledge

Old knowledge, however, isn’t a waste. It’s not like we have to start from scratch. “Rather,” writes Arbesman, “the accumulation of knowledge can then lead us to a fuller and more accurate picture of the world around us.”

Isaac Asimov, in a wonderful essay, uses the Earth’s curvature to help explain this:

When people thought the earth was flat, they were wrong. When people thought the earth was spherical, they were wrong. But if you think that that thinking the earth is spherical is just as wrong as thinking the earth is flat, then your view is wronger than both of them put together.

When our knowledge in a field is immature, discoveries come easily and often explain the main ideas. “But there are uncountably more discoveries, although far rarer, in the tail of this distribution of discovery. As we delve deeper, whether it’s intro discovering the diversity of life in the oceans or the shape of the earth, we begin to truly understand the world around us.”

So what we’re really dealing with the long tail of discovery. Our search for what’s way out at the end of that tail, while it might not be as important or as Earth-shattering as the blockbuster discoveries, can be just as exciting and surprising. Each new little piece can teach us something about what we thought was possible in the world and help us to asymptotically approach a more complete understanding of our surroundings.

In an interview with the Economist, Arbesman was asked which scientific fields decay the slowest-and fastest-and what causes that difference.

Well it depends, because these rates tend to change over time. For example, when medicine transitioned from an art to a science, its half-life was much more rapid than it is now. That said, medicine still has a very short half-life; in fact it is one of the areas where knowledge changes the fastest. One of the slowest is mathematics, because when you prove something in mathematics it is pretty much a settled matter unless someone finds an error in one of your proofs.

One thing we have seen is that the social sciences have a much faster rate of decay than the physical sciences, because in the social sciences there is a lot more “noise” at the experimental level. For instance, in physics, if you want to understand the arc of a parabola, you shoot a cannon 100 times and see where the cannonballs land. And when you do that, you are likely to find a really nice cluster around a single location. But if you are making measurements that have to do with people, things are a lot messier, because people respond to a lot of different things, and that means the effect sizes are going to be smaller.

Arbesman concludes his economist interview:

I want to show people how knowledge changes. But at the same time I want to say, now that you know how knowledge changes, you have to be on guard, so you are not shocked when your children (are) coming home to tell you that dinosaurs have feathers. You have to look things up more often and recognise that most of the stuff you learned when you were younger is not at the cutting edge. We are coming a lot closer to a true understanding of the world; we know a lot more about the universe than we did even just a few decades ago. It is not the case that just because knowledge is constantly being overturned we do not know anything. But too often, we fail to acknowledge change.

Some fields are starting to recognise this. Medicine, for example, has got really good at encouraging its practitioners to stay current. A lot of medical students are taught that everything they learn is going to be obsolete soon after they graduate. There is even a website called “up to date” that constantly updates medical textbooks. In that sense we could all stand to learn from medicine; we constantly have to make an effort to explore the world anew—even if that means just looking at Wikipedia more often. And I am not just talking about dinosaurs and outer space. You see this same phenomenon with knowledge about nutrition or childcare—the stuff that has to do with how we live our lives.

Even when we find new information that contradicts what we thought we knew, we’re likely to be slow to change our minds. “A prevailing theory or paradigm is not overthrown by the accumulation of contrary evidence,” writes Richard Zeckhauser, “but rather by a new paradigm that, for whatever reasons, begins to be accepted by scientists.”

In this view, scientific scholars are subject to status quo persistence. Far from being objective decoders of the empirical evidence, scientists have decided preferences about the scientific beliefs they hold. From a psychological perspective, this preference for beliefs can be seen as a reaction to the tensions caused by cognitive dissonance.

A lot of scientific advancement happens only when the old guard dies off. Many years ago Max Planck offered this insight: “A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it.”

While we have the best intentions and our minds change slowly, a lot of what we think we know is actually just a temporary knowledge to be updated in the future by more complete knowledge. I think this is why Nassim Taleb argues that we should read Seneca and not worry about someone like Jonah Lehrer bringing us sexy narratives of the latest discoveries. It turns out most of these discoveries are based on very little data and, while they may add to our cumulative knowledge, they are not likely to be around in 10 years.

The Half-life of Facts is a good read that help puts what we think we understand about the world into perspective.

Follow your curiosity and read my interview with the author. Knowing that knowledge has a half-life isn’t enough, we can use this to help us determine what to read.

Freeman Dyson on The Difference Between Science and Philsophy

Freeman Dyson, writing in The NY Review of Books:

On quantum physics

The essence of quantum physics is unpredictability. At every instant, the objects in our physical environment—the atoms in our lungs and the light in our eyes—are making unpredictable choices, deciding what to do next. According to Everett and Deutsch, the multiverse contains a universe for every combination of choices. There are so many universes that every possible sequence of choices occurs in at least one of them. Each universe is constantly splitting into many alternative universes, and the alternatives are recombining when they arrive at the same final state by different routes. The multiverse is a huge network of possible histories diverging and reconverging as time goes on. The “quantum weirdness” that we observe in the behavior of atoms, the “spooky action at a distance” that Einstein famously disliked, is the result of universes recombining in unexpected ways.

On the difference between science and philosophy

Science is about facts that can be tested and mysteries that can be explored, and I see no way of testing hypotheses of the multiverse. Philosophy is about ideas that can be imagined and stories that can be told.

On philosophy

When and why did philosophy lose its bite? How did it become a toothless relic of past glories? These are the ugly questions that Jim Holt’s book compels us to ask. Philosophers became insignificant when philosophy became a separate academic discipline, distinct from science and history and literature and religion. The great philosophers of the past covered all these disciplines. Until the nineteenth century, science was called natural philosophy and officially recognized as a branch of philosophy. The word “scientist” was invented by William Whewell, a nineteenth-century Cambridge philosopher who became master of Trinity College and put his name on the building where Wittgenstein and I were living in 1946. Whewell introduced the word in the year 1833. He was waging a deliberate campaign to establish science as a professional discipline distinct from philosophy.

Whewell’s campaign succeeded. As a result, science grew to a dominant position in public life, and philosophy shrank. Philosophy shrank even further when it became detached from religion and from literature. The great philosophers of the past wrote literary masterpieces such as the Book of Job and the Confessions of Saint Augustine. The latest masterpieces written by a philosopher were probably Friedrich Nietzsche’s Thus Spoke Zarathustra in 1885 and Beyond Good and Evil in 1886. Modern departments of philosophy have no place for the mystical.

Richard Feynman on Why Questions

In this beautiful video, Richard Feynman on the nature of why questions and how they help us understand the world.

Interviewer: If you get hold of two magnets, and you push them, you can feel this pushing between them. Turn them around the other way, and they slam together. Now, what is it, the feeling between those two magnets?

Feynman: What do you mean, “What’s the feeling between the two magnets?”

There’s something there, isn’t there? The sensation is that there’s something there when you push these two magnets together.

Listen to my question. What is the meaning when you say that there’s a feeling? Of course, you feel it. Now, what do you want to know?

What I want to know is what’s going on between these two bits of metal?

They repel each other.

What does that mean, or why are they doing that, or how are they doing that? I think that’s a perfectly reasonable question.

Of course, it’s an excellent question. But the problem, you see, when you ask why something happens, how does a person answer why something happens? For example, Aunt Minnie is in the hospital. Why? Because she went out, slipped on the ice, and broke her hip. That satisfies people. It satisfies, but it wouldn’t satisfy someone who came from another planet and knew nothing about why when you break your hip do you go to the hospital. How do you get to the hospital when the hip is broken? Well, because her husband, seeing that her hip was broken, called the hospital up and sent somebody to get her. All that is understood by people. And when you explain a why, you have to be in some framework that you allow something to be true. Otherwise, you’re perpetually asking why. Why did the husband call up the hospital? Because the husband is interested in his wife’s welfare. Not always, some husbands aren’t interested in their wives’ welfare when they’re drunk, and they’re angry.

“But the problem, you see, when you ask why something happens, how does a person answer why something happens?”

— Richard Feynman

And you begin to get a very interesting understanding of the world and all its complications. If you try to follow anything up, you go deeper and deeper in various directions. For example, if you go, “Why did she slip on the ice?” Well, ice is slippery. Everybody knows that, no problem. But you ask why is ice slippery? That’s kinda curious. Ice is extremely slippery. It’s very interesting. You say, how does it work? You could either say, “I’m satisfied that you’ve answered me. Ice is slippery; that explains it,” or you could go on and say, “Why is ice slippery?” and then you’re involved with something, because there aren’t many things as slippery as ice. It’s not very hard to get greasy stuff, but that’s sort of wet and slimy. But a solid that’s so slippery? Because it is, in the case of ice, when you stand on it (they say) momentarily the pressure melts the ice a little bit so you get a sort of instantaneous water surface on which you’re slipping. Why on ice and not on other things? Because water expands when it freezes, so the pressure tries to undo the expansion and melts it. It’s capable of melting, but other substances get cracked when they’re freezing, and when you push them they’re satisfied to be solid.

Why does water expand when it freezes and other substances don’t? I’m not answering your question, but I’m telling you how difficult the why question is. You have to know what it is that you’re permitted to understand and allow to be understood and known, and what it is you’re not. You’ll notice, in this example, that the more I ask why, the deeper a thing is, the more interesting it gets. We could even go further and say, “Why did she fall down when she slipped?” It has to do with gravity, involves all the planets and everything else. Nevermind! It goes on and on. And when you’re asked, for example, why two magnets repel, there are many different levels. It depends on whether you’re a student of physics or an ordinary person who doesn’t know anything. If you’re somebody who doesn’t know anything at all about it, all I can say is the magnetic force makes them repel, and that you’re feeling that force.

You say, “That’s very strange, because I don’t feel a kind of force like that in other circumstances.” When you turn them the other way, they attract. There’s a very analogous force, electrical force, which is the same kind of a question, that’s also very weird. But you’re not at all disturbed by the fact that when you put your hand on a chair, it pushes you back. But we found out by looking at it that that’s the same force, as a matter of fact (an electrical force, not magnetic exactly, in that case). But it’s the same electric repulsions that are involved in keeping your finger away from the chair because it’s electrical forces in minor and microscopic details. There are other forces involved, connected to electrical forces. It turns out that the magnetic and electrical force with which I wish to explain this repulsion in the first place is what ultimately is the deeper thing that we have to start with to explain many other things that everybody would just accept. You know you can’t put your hand through the chair; that’s taken for granted. But that you can’t put your hand through the chair, when looked at more closely, why, involves the same repulsive forces that appear in magnets. The situation you then have to explain is why, in magnets, it goes over a bigger distance than ordinarily. There it has to do with the fact that in iron all the electrons are spinning in the same direction, they all get lined up, and they magnify the effect of the force ’til it’s large enough, at a distance, that you can feel it. But it’s a force which is present all the time and very common and is a basic force of almost – I mean, I could go a little further back if I went more technical – but on an early level I’ve just got to tell you that’s going to be one of the things you’ll just have to take as an element of the world: the existence of magnetic repulsion, or electrical attraction, magnetic attraction.

I can’t explain that attraction in terms of anything else that’s familiar to you. For example, if we said the magnets attract like rubber bands, I would be cheating you. Because they’re not connected by rubber bands. I’d soon be in trouble. And secondly, if you were curious enough, you’d ask me why rubber bands tend to pull back together again, and I would end up explaining that in terms of electrical forces, which are the very things that I’m trying to use the rubber bands to explain. So I have cheated very badly, you see. So I am not going to be able to give you an answer to why magnets attract each other except to tell you that they do. And to tell you that that’s one of the elements in the world – there are electrical forces, magnetic forces, gravitational forces, and others, and those are some of the parts. If you were a student, I could go further. I could tell you that the magnetic forces are related to the electrical forces very intimately, that the relationship between the gravity forces and electrical forces remains unknown, and so on. But I really can’t do a good job, any job, of explaining magnetic force in terms of something else you’re more familiar with, because I don’t understand it in terms of anything else that you’re more familiar with.

Physicist Richard Feynman on Beauty of a Flower

Richard Feynman talking about the beauty of the natural world.

I have a friend who’s an artist and has sometimes taken a view which I don’t agree with very well. He’ll hold up a flower and say “look how beautiful it is,” and I’ll agree. Then he says “I as an artist can see how beautiful this is but you as a scientist take this all apart and it becomes a dull thing,” and I think that he’s kind of nutty. First of all, the beauty that he sees is available to other people and to me too, I believe. Although I may not be quite as refined aesthetically as he is … I can appreciate the beauty of a flower. At the same time, I see much more about the flower than he sees. I could imagine the cells in there, the complicated actions inside, which also have a beauty. I mean it’s not just beauty at this dimension, at one centimeter; there’s also beauty at smaller dimensions, the inner structure, also the processes. The fact that the colors in the flower evolved in order to attract insects to pollinate it is interesting; it means that insects can see the color. It adds a question: does this aesthetic sense also exist in the lower forms? Why is it aesthetic? All kinds of interesting questions which the science knowledge only adds to the excitement, the mystery and the awe of a flower. It only adds. I don’t understand how it subtracts.

Richard Feynman Teaches you the Scientific Method

The scientific method refers to a process of thought based on integrating previous knowledge, observing, measuring, and logical reasoning.

“If it disagrees with experiment, it’s wrong. In that simple statement is the key to science.”

— Richard Feynman

In this short video taken from his lectures, Physicist Richard Feynman offers perhaps one of the greatest definitions of science and the scientific method that I’ve ever heard. And he does it in about a minute.

Now I’m going to discuss how we would look for a new law. In general, we look for a new law by the following process. First, we guess it (audience laughter), no, don’t laugh, that’s the truth. Then we compute the consequences of the guess, to see what, if this is right, if this law we guess is right, to see what it would imply and then we compare the computation results to nature or we say compare to experiment or experience, compare it directly with observations to see if it works.

If it disagrees with experiment, it’s wrong. In that simple statement is the key to science. It doesn’t make any difference how beautiful your guess is, it doesn’t matter how smart you are who made the guess, or what his name is … If it disagrees with experiment, it’s wrong. That’s all there is to it.

For more color watch the longer version below, which offers the next 9 minutes of the lecture. In this clip Feynman explains that guessing is not unscientific: “It is not unscientific to take a guess, although many people who are not in science believe that it is.”

The Scientific Method is part of the Farnam Street Latticework of Mental Models.