Nassim Taleb: The Big Errors of Big Data


I am not saying here that there is no information in big data.
There is plenty of information.

The problem — the central issue — is that the needle
comes in an increasingly larger haystack.


Nassim Taleb offers another way to look at big data.

We’re more fooled by noise than ever before, and it’s because of a nasty phenomenon called “big data.” With big data, researchers have brought cherry-picking to an industrial level.

Modernity provides too many variables, but too little data per variable. So the spurious relationships grow much, much faster than real information.

In other words: Big data may mean more information, but it also means more false information.

To me, this relates to reading the news.

We’re consumed—bombarded even—by all of this incoming information that’s constructed in a way to capture and maintain our attention. Somewhat counter-intuitively, this distraction offers negative, not positive utility. Not only does it give us easily accessible information that’s full of noise from people that are not deeply fluent in the subject they are talking about but we rarely consider the opportunity cost of this time or the false confidence it gives us (which causes us to take undue risks).

It would be much better to focus our limited attention in two places.

The first, our niche. That is our narrow specialization or our circle of competence. This is after all how we’ll make a living.

The second, on how the world works. These are the time-tested ideas that repeat throughout history and don’t change as time passes.

These are the mental models that you can use to not only better understand how the world works and why people behave as they do, but also to make better decisions.

They combine a narrow specialization with a general view of how the world works. If you further add how to think, now you’re really getting somewhere.