Exceptions Become the Rule

With the advent of "big data," we now can look at massive amounts of information and make reasonably accurate predictions about many things, like traffic flows or a new drug's effectiveness. But now we can also do “small data,” enabling us to treat many things like individuals.

NEW YORK – Long ago, I worked as an analyst on Wall Street. The first company that I analyzed was Federal Express, which at the time had not yet shipped its first package.

The idea behind FedEx was simple and compelling: The cost of complexity was higher than the cost of air transport, so the company would ship all its packages overnight to Memphis, Tennessee. By radically simplifying the myriad combinations of start and end points – the only routes were to and from Memphis – all of the packages could be delivered the next day reliably.

All of that has changed, thanks to developments in information technology. With today’s powerful computers, we can look at massive amounts of information and simulate complex situations, making pretty good predictions about many things: What will traffic flows look like on Tuesday at 5:00 in the afternoon if we put a detour at this highway intersection? What percentage of people on this drug will get better – and, more interestingly, which particular individuals will respond positively, and which ones are likely to be harmed?

To continue reading, please log in or enter your email address.

To access our archive, please log in or register now and read two articles from our archive every month for free. For unlimited access to our archive, as well as to the unrivaled analysis of PS On Point, subscribe now.

required

By proceeding, you agree to our Terms of Service and Privacy Policy, which describes the personal data we collect and how we use it.

Log in

http://prosyn.org/LqeIlc4;

Cookies and Privacy

We use cookies to improve your experience on our website. To find out more, read our updated cookie policy and privacy policy.