Search Results
Working Paper
Total Recall? Evaluating the Macroeconomic Knowledge of Large Language Models
We evaluate the ability of large language models (LLMs) to estimate historical macroeconomic variables and data release dates. We find that LLMs have precise knowledge of some recent statistics, but performance degrades as we go farther back in history. We highlight two particularly important kinds of recall errors: mixing together first print data with subsequent revisions (i.e., smoothing across vintages) and mixing data for past and future reference periods (i.e., smoothing within vintages). We also find that LLMs can often recall individual data release dates accurately, but aggregating ...
Working Paper
Manufacturing Sentiment: Forecasting Industrial Production with Text Analysis
This paper examines the link between industrial production and the sentiment expressed in natural language survey responses from U.S. manufacturing firms. We compare several natural language processing (NLP) techniques for classifying sentiment, ranging from dictionary-based methods to modern deep learning methods. Using a manually labeled sample as ground truth, we find that deep learning models partially trained on a human-labeled sample of our data outperform other methods for classifying the sentiment of survey responses. Further, we capitalize on the panel nature of the data to train ...
Working Paper
Improving the Accuracy of Economic Measurement with Multiple Data Sources: The Case of Payroll Employment Data
This paper combines information from two sources of U.S. private payroll employment to increase the accuracy of real-time measurement of the labor market. The sources are the Current Employment Statistics (CES) from BLS and microdata from the payroll processing firm ADP. We briefly describe the ADP-derived data series, compare it to the BLS data, and describe an exercise that benchmarks the data series to an employment census. The CES and the ADP employment data are each derived from roughly equal-sized samples. We argue that combining CES and ADP data series reduces the measurement error ...
Journal Article
Understanding the evolution of trade deficits: trade elasticities of industrialized countries
In this article, the authors present updated trade elasticities?measures of how much imports and exports change in response to income and price changes?for the U.S. and six other industrialized countries, collectively known as the Group of Seven. They find that the imports and exports of these countries are slightly more responsive to changes in a country?s total income over a period that ends in 2006, compared with a period that ends in 1994.
Discussion Paper
Measuring AI Uptake in the Workplace
Artificial Intelligence (AI) may be poised to raise productivity across various domains, including writing (Noy and Zhang 2023), programming (Peng et al. 2023), and research and development (Toner-Rodgers 2024; Korinek 2023). However, understanding the extent to which AI—and generative AI in particular—has been adopted as part of the production process remains an open question. This note reviews the extant surveys on AI adoption at both the employee and firm levels. Surveys of firms show a wide spread of adoption rates, ranging from 5 percent to about 40 percent. Surveys of workers show ...
Working Paper
AI and Coder Employment: Compiling the Evidence
We evaluate whether LLMs have had any discernible impact on the aggregate labor market so far. We focus on occupations that are computer programming-intensive, motivated by data showing that coding is one of the most LLM-exposed tasks. Linking O*NET to CPS we find that aggregate employment of coders has decelerated sharply since the introduction of ChatGPT. Using a novel control variable for industry-level shocks we show that the deceleration is not attributable to the exposure of coders to slowing industries, suggesting instead that coders experienced an occupation-specific shock around ...
Working Paper
LLM on a Budget: Active Knowledge Distillation for Efficient Classification of Large Text Corpora
Large Language Models (LLMs) are highly accurate in classification tasks, however, substantial computational and financial costs hinder their large-scale deployment in dynamic environments. Knowledge Distillation (KD) where a LLM ""teacher"" trains a smaller and more efficient ""student"" model, offers a promising solution to this problem. However, the distillation process itself often remains costly for large datasets, since it requires the teacher to label a vast number of samples while incurring significant token consumption. To alleviate this challenge, in this work we explore the ...