Elif Ayiter/Flickr

Living Big Data

Used carefully and accurately, big data creates unprecedented scope to understand society and improve how we live and work. But big data requires big experiments, because what works in theory may not translate well in the real world, where complex human interactions cannot always be captured, even by the most sophisticated models.

CAMBRIDGE – Big data is made from the digital trail that we leave behind when we use credit cards, mobile phones, or the Web. Used carefully and accurately, these data give us unprecedented scope to understand our society, and improve the way we live and work. But what works in theory may not translate well in the real world, where complex human interactions cannot always be captured, even by the most sophisticated models. Big data requires us to experiment on a big scale.

My own laboratory, for example, is building a Web site which, based on Google maps, uses society’s digital trail to map poverty, infant mortality, crime rates, changes in GDP, and other social indicators, neighborhood by neighborhood – all of which will be updated daily. This new capability allows viewers to see, for example, where government initiatives are working or failing.

But, while such impressive visualization tools can dramatically enhance transparency and public knowledge, they are surprisingly limited when applied to solving society’s problems. One reason is that such rich streams of data encourage spurious correlations.

To continue reading, please log in or enter your email address.

To read this article from our archive, please log in or register now. After entering your email, you'll have access to two free articles from our archive every month. For unlimited access to Project Syndicate, subscribe now.

required

By proceeding, you agree to our Terms of Service and Privacy Policy, which describes the personal data we collect and how we use it.

Log in

http://prosyn.org/vuqJzwv;

Cookies and Privacy

We use cookies to improve your experience on our website. To find out more, read our updated cookie policy and privacy policy.