Using machine learning and big data to analyse the business cycle

Prepared by Dominik Hirschbühl, Luca Onorante and Lorena Saiz

Published as part of the ECB Economic Bulletin, Issue 5/2021.

1 Introduction

Policymakers take decisions in real time based on incomplete information about current economic conditions. Central banks and economic analysts largely rely on official statistics together with soft data and surveys, to assess the state of the economy. Although a wide range of high-quality conventional data is available, the datasets are released with lags ranging from a few days or weeks to several months after the reference period. For these reasons, central banks have been looking at ways to exploit timelier data and employ more sophisticated methods to enhance accuracy when forecasting metrics that are relevant for policymaking.

Over recent years, policy institutions have started to explore new sources of data and alternative statistical methods for the real-time assessment of economic activity. Since the financial crisis, they have stepped up their efforts to systematically use micro and survey data to better gauge changes in aggregate consumption, investment and the labour market. In parallel, technological advances have allowed users to start examining unconventional sources such as text data and images from newspaper articles, social media and the internet together with numerical data from payments. Also now available are alternative statistical methods such as regression trees, neural networks and support-vector machines that may help the potential insights that can be gained from these data sources to be fully exploited.

The coronavirus (COVID-19) pandemic has accelerated this trend. The crisis associated with the pandemic has shown that “big data” can provide timely signals on the state of the economy and help to track economic activity alongside more traditional data. Big data are commonly characterised as having three Vs: high volume, high velocity and high variety.^[1] High volume refers to the massive amounts of data generated as a result of the proliferation of devices, services and human interaction. High velocity refers to the fast speed at which the data are created and processed. High variety relates to the wide range and complexity of data types and sources.^[2] Big data are appealing because they are available at high frequency; however, they are often relatively unstructured and are, by definition, large in size. This in turn poses various challenges for traditional econometric models. Some of these can be addressed by machine learning (ML) algorithms, which also have the advantage of potentially capturing complex non-linear relationships. Even though there is no single definition of machine learning, the basic idea behind it is that computers (machines) can learn from past data, identify general patterns – often characterised by non-linear relationships – and make predictions using algorithms capturing those patterns. Machine learning is therefore a subset of artificial intelligence, and most of its methods are largely based on concepts from statistics and statistical learning theory.^[3]

This article reviews how policy institutions – international organisations and central banks – use big data and/or machine learning methods to analyse the business cycle. Specifically, these new data sources and tools are used to improve nowcasting and short-term forecasting of real GDP. They are also employed to gain useful insights for assessing cyclical developments and building narratives. A number of illustrative examples are provided.

The article is organised as follows. Section 2 reviews the main sources of big data that central banks and other policy institutions have been exploring for business cycle analysis over recent years. It provides an overview of existing literature and also includes two examples of how big data have been used to monitor economic activity and labour market developments during the pandemic. Section 3 discusses the main advantages of ML methods in dealing with big data and analysing the business cycle. This section includes two examples using newspaper articles to build measures of economic sentiment and economic policy uncertainty. Section 4 presents the main conclusions and discusses opportunities and challenges faced by central banks when using machine learning and big data.

2 How do big data help to gauge the current state of the economy?

Policy institutions have recently started to incorporate structured and unstructured big data in their economic analysis. Big data can be structured – such as those collected in large financial datasets that can be matched to firm-level financial statements – or unstructured. Unstructured data range from large and near-real-time data gleaned from the internet (e.g. internet search volumes, data from social networks such as Twitter and Facebook, newspaper articles) to large-volume data obtained from non-official sources (e.g. trading platforms and payment systems or GPS-based technologies).

Structured data, such as those from financial and payment transactions, can provide critical real-time information for assessing aggregate consumption and economic activity. As the use of credit and debit cards to purchase goods and services has increased, the underlying financial transaction data have provided useful information to track consumption and economic activity. At the same time, payments data are available promptly and subject to few revisions since they are financial records. Central banks had already started to regard these data as a valuable source of information before the pandemic emerged. Analysis based on data for the Netherlands, Norway. Portugal and Spain, among others, finds that retail payment systems data (i.e. credit and debit card payments at the point of sale and ATM withdrawals) helped retail sales, private consumption (especially of non-durables) and even real GDP to be forecast in the previous expansionary phase.^[4] For Italy, some gains in forecast accuracy have been reported when information from highly aggregated but large value payments (i.e. TARGET2) has been included in GDP nowcasting models.^[5]

Turning to unstructured big data, the use of text data from newspapers to understand and forecast the business cycle has increased significantly in the recent years. In business cycle analysis, text data from newspapers and social media have been used to construct proxy measures for unobservable variables such as “sentiment” or “uncertainty” which are likely to be associated with macroeconomic fluctuations. These proxies can be obtained at relatively low cost (in contrast to expensive survey-based measures) and on a timely basis (e.g. daily) by means of automated natural language processing methods. For instance, news-based sentiment indicators can serve as early warning indicators of financial crises.^[6] Newspaper-based sentiment and economic policy uncertainty indexes for Italy and Spain have proved helpful in monitoring economic activity in real time and nowcasting GDP. ^[7] Similarly, in Belgium daily average economic media news sentiment is found to be useful for nowcasting survey-based consumer confidence.^[8] At the ECB, newspaper-based daily sentiment indicators have been estimated for the four largest euro area countries and the euro area as a whole. These indicators demonstrate a high correlation with survey-based sentiment indicators and real GDP; they are also found to be useful for nowcasting GDP, particularly at the beginning of the quarter when other more traditional indicators (e.g. surveys) referring to the current quarter have not been released yet (see Box 3 in Section 3). In addition, economic policy uncertainty indexes have been estimated for the same set of countries. The ML methods employed also allow uncertainty to be decomposed into sub-components that point towards the main sources (see Box 4 in Section 3).

Similarly, the use of internet searches has also started to feature in short-term forecasting models. Several Eurosystem studies show that internet searches can provide information about future consumption decisions. Recent examples include analysis linking Google search data to euro area car sales, the use of Google search data to enhance German GDP nowcasting model and the analysis exploiting synthetic indicators based on Google searches for forecasting private consumption in Spain. For the euro area as a whole, Google data provide useful information for GDP nowcasting when macroeconomic information is lacking (i.e. in the first four weeks of the quarter), but as soon as official data relating to the current quarter become available, their relative nowcasting power diminishes.^[9]

Internet-based data can also help when assessing the tightness of the labour and housing markets. Analysis for the US labour market shows that including Google-based job-search indicators improves the accuracy of unemployment forecasts, particularly over the medium-term horizon (i.e. three to 12 months ahead).^[10] In the euro area, a measure of labour market tightness based on the number of clicks on job postings has recently been built for the Irish economy.^[11] For the housing market, analysis for Italy found that metrics based on web-scraped data from an online portal for real estate services can be a leading indicator of housing prices.^[12] During the pandemic, Google searches on topics related to job retention schemes and layoffs provided early insight into the strong impact of the pandemic and related policy measures. Moreover, online data on job posting and hiring in the euro area have complemented official statistics (see Box 1).

The experience gained with structured and unstructured data prior to the pandemic made it easier to deploy models quickly to facilitate the real-time assessment of the economic situation during the pandemic. In particular, these data have been used to assess the degree of slack in the labour market and to measure the decline in economic activity, seen from both the supply and the demand side. During this period of sudden economic disruption, high-frequency alternative data such as electricity consumption, card payments, job postings, air quality and mobility statistics have been crucial for gaining a timely picture of the economic impact of the pandemic and the associated containment measures, weeks before hard and survey data were released. Payment data have been key to understanding the developments in private consumption, one of the demand components most severely affected by the crisis.^[18] Consumption of key inputs such electricity, gas and fuel was used as a proxy for production in some sectors. A timely understanding of developments in the services sector, with a special focus on small businesses in certain service activities such as tourism which have borne the brunt of the crisis, was also very important. High-frequency information available for these sectors related mostly to sales (e.g. sales in tax returns, card payments), online bookings and Google searches. Other indicators such as freight movements, numbers of flights and air quality were informative as rough proxies for economic activity.

One effective way of summarising information from a set of high-frequency indicators is to use economic activity trackers. Box 2 provides an example of a weekly economic activity tracker for the euro area devised by the ECB. Similarly, the European Commission’s Joint Research Centre and Directorate-General for Economic and Financial Affairs have been tracking the COVID-19 crisis by combining traditional macroeconomic indicators with a high number of non-conventional, real-time and extremely heterogeneous indicators for the four largest economies in the euro area.^[19] They have developed a toolbox with a suite of diverse models, including linear and non-linear models and several ML methods, to exploit the large number of indicators in the dataset for nowcasting GDP. The GDP forecasts are produced by first estimating the whole set (thousands) of models and then applying automatic model selection to average out the forecasts and produce the final forecast.

3 What makes machine learning algorithms useful tools for analysing big data?

While big data can help improve the forecasts of GDP and other macroeconomic aggregates, their full potential can be exploited by employing ML algorithms. Section 2 shows that in many cases, the improvement in forecasting performance relates to specific situations, such as when traditional monthly indicators for the reference quarter are not yet available. This section focuses on the modelling framework, arguing that ML methods help to reap the benefits of using big data. The main goal of ML techniques is to find patterns in data or to predict a target variable. Although ML algorithms estimate and validate predictive models in a subset of data (training sample), the ultimate aim is to obtain the best forecasting performance using a different subset of data (test sample). The distinction between machine learning and traditional methods is not clear-cut since some traditional methods (e.g. linear regression, principal components) are also quite popular in the ML literature. However, the literature on machine learning has developed a host of new and sophisticated models that promise to strongly enrich the toolbox of applied economists. Moreover, it also seems fair to say that, so far, machine learning has been mostly focused on prediction, while more traditional econometric and statistical analysis is also interested in uncovering the causal relationships between economic variables.^[30] This is changing fast, as more and more researchers in the ML field address the issue of inference and causality, although this frontier research is not yet widely applied in the policy context.^[31] The aim of this section is to discuss how machine learning can usefully complement traditional econometric methods, in particular to leverage the opportunities for analysing the business cycle offered by big data. It also reviews several contributions to forecasting/nowcasting GDP (see Box 3) and provides examples of how ML algorithms can provide interesting insights for policy, such as pointing towards the sources of economic policy uncertainty (see Box 4).

The size of the newly available databases in itself often constitutes an obstacle to the use of traditional econometrics. Techniques have been adopted to reduce the dimensionality of the data, including traditional methods such as factor models and principal component analysis, but more often going into newer versions of machine learning. While a description of specific methods is beyond the scope of this article, it is important to note that ML methods have several desirable features for summarising the data, allowing precise reduction of high-dimensional data into a number of manageable indicators.

The first key advantage of ML methods is their ability to extract and select the relevant information from large volumes of, unstructured data. When dealing with big data, the presence of a large amount of mostly irrelevant information engenders the problem of data selection. This issue is magnified by the presence of large, unstructured datasets.^[32] In some simple cases, the forecaster can pick variables manually; this is normally possible when forecasting very specific quantities. The seminal work of Choi and Varian with Google Trends, for instance, focuses on car sales, unemployment claims, travel destination planning and consumer confidence.^[33] Where macroeconomic aggregates are involved, the choosing of relevant variables quickly becomes intractable. ML methods offer very useful tools for selecting the most informative variables and exploiting their information potential. Several techniques derived from the model-averaging literature have also proved popular and successful in improving forecasting accuracy. In these methods, a large number of econometric models are first estimated, their forecasting performance is then evaluated, and the final forecast is obtained by averaging the forecasts of the best models, thus retaining those models and explanatory variables that provide useful information. Similarly, what are known as ensemble methods such as random forests and bagging combine different “views” of the data given by competing models, adding flexibility and robustness to the predictions.

The second key advantage of ML methods is their ability to capture quite general forms of non-linearities. This is a general advantage of ML methods, regardless of the volume of data concerned; however, the issue is that, by their very nature, big data may be particularly prone to non-linearities. For instance, the data stemming from social networks present a good way to understand these inherent non-linearities. In this case, a specific topic can generate cascade or snowball effects within the network which cannot be channelled in linear regression models. Other examples include Google Trends and Google search categories, which are compiled using ML algorithms that determine the category to which an internet search belongs.^[34] Text data are also obtained by applying highly non-linear ML algorithms to news items, for example. More generally, non-linearities and interactions between variables are common in macroeconomics owing to the presence of financial frictions and uncertainty. Several works have found that ML methods can be useful for macroeconomic forecasting, since they better capture non-linearities (e.g. Coulombe et al.). These methods can, for instance, capture the non-linear relationship between financial conditions and economic activity, among others, and hence more accurately predict activity and recessions in particular (see Box 3). Also, ML methods can outperform standard methods (e.g. credit scoring models, logistic regression) when predicting consumer and corporate defaults, since they capture non-linear relationships between the incidence of default and the characteristics of the individuals.^[35]

The COVID-19 pandemic is an important source of non-linearities. During the pandemic, many macroeconomic variables have recorded extreme values that are far from the range of past values. Econometric methods such as linear time series analysis seek to find average patterns in past data. If current data are very different, linearly extrapolating from past patterns may lead to biased results. Central banks, the European Commission and other institutions have adapted their nowcasting frameworks to capture non-standard data and non-linearities.^[36]

Finally, ML techniques are the main tool used to capture a wide set of phenomena that would otherwise remain unquantified. The most prominent example in recent years is the dramatic surge of text data analysis. Today, broad corpuses of text are analysed and converted into numbers that forecasters can use. For instance, a wide range of timely, yet noisy confidence indicators based on text currently complement the traditional surveys, which are available with considerable lags and where agents do not necessarily “vote with their behaviour”, as well as market-based indicators, where expectations and other factors such as risk aversion compound in the data. A first generation of work built on word counts has been followed by more sophisticated approaches.^[37] Second-generation techniques based on unsupervised learning are also used in public institutions, and in particular in central banks, to assess the effect of their communication. Finally, following Baker et al., concepts such as economic policy uncertainty which were previously difficult to quantify are now currently assessed on the basis of their economic consequences and used in forecasting.^[38] See Box 3 and Box 4 for examples.

4 Conclusions, challenges and opportunities

This article has described how big data and ML methods can complement standard analysis of the business cycle. A case in point is the coronavirus pandemic, which represents an extraordinary shock. This crisis has propelled the dissemination and refinement of ML techniques and big data at an unprecedented speed. In particular, it has shown that alternative sources of data can provide more timely signals on the state of the economy and help to track economic activity. Furthermore, it is an important showcase for non-linearities in the economy, which has required existing models to be adapted or new approaches to be developed. In this respect, ML methods can deal with non-linearities more easily than traditional methods. Besides new opportunities, these new data sources and methods also pose some challenges.

Big data allow a wider range of timely indicators to be used for forecasting (e.g. text-based or internet-based indicators), although in some cases this can entail replicability and accountability issues. Text-based sentiment indicators are particularly useful, for instance, given that they can be produced automatically at higher frequency and at lower cost than survey-based indicators. While the construction of conventional economic data, such as industrial production, follows harmonised procedures to ensure high quality, continuity and comparability over time and countries, alternative data are neither collected primarily for economic analysis, nor sourced and validated by independent statistical offices. Therefore, their application in decision-making processes exposes central banks to various risks, given that the replicability of results and accountability could be impaired. Since alternative data are collected for other purposes (e.g. credit card transactions) or come as the by-product of another service (e.g. news articles from the digitisation of newspapers), the data are often very noisy and require careful treatment. Moreover, the existence of significant data accessibility issues and limitations to data sharing could impair the replicability of the results in some cases. All these risks require careful consideration when investing scarce resources in software development and legal issues, as well as customising IT infrastructure.^[49]

Although useful as complements, at the moment these tools cannot be considered as substitutes for standard data and methods due to issues of interpretability and statistical inference. ML methods can help overcome the shortcomings of big data and exploit their full potential. When combined with big data, ML methods are capable of outperforming traditional statistical methods and providing an accurate picture of economic developments. Despite the good forecasting performance, the complexity of the methods often makes it difficult to interpret revisions to the forecasts and most importantly to communicate them. However, rapid advances are being made on enhancing the interpretability of ML techniques (most recently based on Shapley values).^[50] In addition, ML techniques are not originally designed to identify causal relationships, which is of critical importance to policymakers. Enhancing the ability of ML methods to capture causality is currently the biggest challenge; this has the potential to make ML techniques promising complements and viable alternatives to established methods.^[51]