The Hidden Danger of Big Data
With big data, we can multiply our options and filter out things we don’t want to see. But there is much to be said for making discoveries through pure serendipity: contingency and randomness often furnish the transformational or counterintuitive ideas that propel humanity forward.
CAMBRIDGE – In game theory, the “price of anarchy” describes how individuals acting in their own self-interest within a larger system tend to reduce that larger system’s efficiency. It is a ubiquitous phenomenon, one that almost all of us confront, in some form, on a regular basis.
For example, if you are a city planner in charge of traffic management, there are two ways you can address traffic flows in your city. Generally, a centralized, top-down approach – one that comprehends the entire system, identifies choke points, and makes changes to eliminate them – will be more efficient than simply letting individual drivers make their own choices on the road, with the assumption that these choices, in aggregate, will lead to an acceptable outcome. The first approach reduces the cost of anarchy and makes better use of all available information.
The world today is awash in data. In 2015, mankind produced as much information as was created in all previous years of human civilization. Every time we send a message, make a call, or complete a transaction, we leave digital traces. We are quickly approaching what Italian writer Italo Calvino presciently called the “memory of the world”: a full digital copy of our physical universe.