Prof. Sir David F. Hendry, speaks about Mining Big Data by Statistical Methods in his latest article in the European Financial Review. Snippet below and the full article here.
Big Data offer potential benefits, but statistical modelling confronts problems of false positives, mistaking correlations for causes, sampling biases, and selecting by inappropriate methods. Below, David Hendry explains that resolution requires commencing from general models, using high quality data, selecting at tight significance levels, testing for well-specified invariant relationships, using a powerful algorithm like Autometrics to handle large numbers of possible models.