Used carefully and accurately, big data creates unprecedented scope to understand society and improve how we live and work. But big data requires big experiments, because what works in theory may not translate well in the real world, where complex human interactions cannot always be captured, even by the most sophisticated models.
CAMBRIDGE – Big data is made from the digital trail that we leave behind when we use credit cards, mobile phones, or the Web. Used carefully and accurately, these data give us unprecedented scope to understand our society, and improve the way we live and work. But what works in theory may not translate well in the real world, where complex human interactions cannot always be captured, even by the most sophisticated models. Big data requires us to experiment on a big scale.
My own laboratory, for example, is building a Web site which, based on Google maps, uses society’s digital trail to map poverty, infant mortality, crime rates, changes in GDP, and other social indicators, neighborhood by neighborhood – all of which will be updated daily. This new capability allows viewers to see, for example, where government initiatives are working or failing.
But, while such impressive visualization tools can dramatically enhance transparency and public knowledge, they are surprisingly limited when applied to solving society’s problems. One reason is that such rich streams of data encourage spurious correlations.
CAMBRIDGE – Big data is made from the digital trail that we leave behind when we use credit cards, mobile phones, or the Web. Used carefully and accurately, these data give us unprecedented scope to understand our society, and improve the way we live and work. But what works in theory may not translate well in the real world, where complex human interactions cannot always be captured, even by the most sophisticated models. Big data requires us to experiment on a big scale.
My own laboratory, for example, is building a Web site which, based on Google maps, uses society’s digital trail to map poverty, infant mortality, crime rates, changes in GDP, and other social indicators, neighborhood by neighborhood – all of which will be updated daily. This new capability allows viewers to see, for example, where government initiatives are working or failing.
But, while such impressive visualization tools can dramatically enhance transparency and public knowledge, they are surprisingly limited when applied to solving society’s problems. One reason is that such rich streams of data encourage spurious correlations.