Search Results
Working Paper
Total Recall? Evaluating the Macroeconomic Knowledge of Large Language Models
We evaluate the ability of large language models (LLMs) to estimate historical macroeconomic variables and data release dates. We find that LLMs have precise knowledge of some recent statistics, but performance degrades as we go farther back in history. We highlight two particularly important kinds of recall errors: mixing together first print data with subsequent revisions (i.e., smoothing across vintages) and mixing data for past and future reference periods (i.e., smoothing within vintages). We also find that LLMs can often recall individual data release dates accurately, but aggregating ...
Working Paper
Manufacturing Sentiment: Forecasting Industrial Production with Text Analysis
This paper examines the link between industrial production and the sentiment expressed in natural language survey responses from U.S. manufacturing firms. We compare several natural language processing (NLP) techniques for classifying sentiment, ranging from dictionary-based methods to modern deep learning methods. Using a manually labeled sample as ground truth, we find that deep learning models partially trained on a human-labeled sample of our data outperform other methods for classifying the sentiment of survey responses. Further, we capitalize on the panel nature of the data to train ...
Working Paper
Improving the Accuracy of Economic Measurement with Multiple Data Sources: The Case of Payroll Employment Data
This paper combines information from two sources of U.S. private payroll employment to increase the accuracy of real-time measurement of the labor market. The sources are the Current Employment Statistics (CES) from BLS and microdata from the payroll processing firm ADP. We briefly describe the ADP-derived data series, compare it to the BLS data, and describe an exercise that benchmarks the data series to an employment census. The CES and the ADP employment data are each derived from roughly equal-sized samples. We argue that combining CES and ADP data series reduces the measurement error ...
Journal Article
Understanding the evolution of trade deficits: trade elasticities of industrialized countries
In this article, the authors present updated trade elasticities?measures of how much imports and exports change in response to income and price changes?for the U.S. and six other industrialized countries, collectively known as the Group of Seven. They find that the imports and exports of these countries are slightly more responsive to changes in a country?s total income over a period that ends in 2006, compared with a period that ends in 1994.
Discussion Paper
Measuring AI Uptake in the Workplace
Artificial Intelligence (AI) may be poised to raise productivity across various domains, including writing (Noy and Zhang 2023), programming (Peng et al. 2023), and research and development (Toner-Rodgers 2024; Korinek 2023). However, understanding the extent to which AI—and generative AI in particular—has been adopted as part of the production process remains an open question. This note reviews the extant surveys on AI adoption at both the employee and firm levels. Surveys of firms show a wide spread of adoption rates, ranging from 5 percent to about 40 percent. Surveys of workers show ...
Working Paper
LLM on a Budget: Active Knowledge Distillation for Efficient Classification of Large Text Corpora
Large Language Models (LLMs) are highly accurate in classification tasks, however, substantial computational and financial costs hinder their large-scale deployment in dynamic environments. Knowledge Distillation (KD) where a LLM ""teacher"" trains a smaller and more efficient ""student"" model, offers a promising solution to this problem. However, the distillation process itself often remains costly for large datasets, since it requires the teacher to label a vast number of samples while incurring significant token consumption. To alleviate this challenge, in this work we explore the ...
Working Paper
Business Exit During the COVID-19 Pandemic: Non-Traditional Measures in Historical Context
Lags in official data releases have forced economists and policymakers to leverage "alternative" or "non-traditional" data to measure business exit resulting from the COVID- 19 pandemic. We first review official data on business exit in recent decades to place the alternative measures of exit within historical context. For the U.S., business exit is countercyclical and fairly common, with about 7.5 percent of firms exiting annually in recent years. Both the high level and the cyclicality of exit are driven by very small firms and establishments. We then explore a range of alternative measures ...