According to IBM, the computers with which we have surrounded ourselves are now generating 2.5 quintillion bytes of data a day around the world. That's about half a CD's worth of data per person per day. “Big data” is the topic of countless breathless conference presentations and consultants' reports. What, then, might it contribute to economics?
Not everyone means the same thing when they talk about “big data” but here are a few common threads. First, the dataset is far too big for a human to comprehend without a lot of help from some sort of visualisation software. The time-honoured trick of plotting a scatter graph to see what patterns or anomalies it suggests is no use here. Second, the data is often available at short notice, at least to some people. Your mobile phone company knows where your phone is right now. Third, the data may be heavily interconnected - in principle Google could have your email, your Android phone location, knowledge of who is your friend on the Google Plus social network, and your online search history. Fourth, the data is messy: videos that you store on your phone are “big data” but a far cry from neat database categories - date of birth, employment status, gender, income.
This hints at problems for economists. We have been rather spoiled: in the 1930s and 1940s pioneers such as Simon Kuznets and Richard Stone built tidy, intellectually coherent systems of national accounts. Literally billions of individual transactions are summarised as “UK GDP in 2012”; billions of price movements are represented by a single index of inflation. The data come in nice “rectangular” form - inflation for x countries over y years, for instance.