Search Results
Working Paper
Total Recall? Evaluating the Macroeconomic Knowledge of Large Language Models
We evaluate the ability of large language models (LLMs) to estimate historical macroeconomic variables and data release dates. We find that LLMs have precise knowledge of some recent statistics, but performance degrades as we go farther back in history. We highlight two particularly important kinds of recall errors: mixing together first print data with subsequent revisions (i.e., smoothing across vintages) and mixing data for past and future reference periods (i.e., smoothing within vintages). We also find that LLMs can often recall individual data release dates accurately, but aggregating ...
Working Paper
Manufacturing Sentiment: Forecasting Industrial Production with Text Analysis
This paper examines the link between industrial production and the sentiment expressed in natural language survey responses from U.S. manufacturing firms. We compare several natural language processing (NLP) techniques for classifying sentiment, ranging from dictionary-based methods to modern deep learning methods. Using a manually labeled sample as ground truth, we find that deep learning models partially trained on a human-labeled sample of our data outperform other methods for classifying the sentiment of survey responses. Further, we capitalize on the panel nature of the data to train ...
Working Paper
Generative AI at the Crossroads: Light Bulb, Dynamo, or Microscope?
With the advent of generative AI (genAI), the potential scope of artificial intelligence has increased dramatically, but the future effect of genAI on productivity remains uncertain. The effect of the technology on the innovation process is a crucial open question. Some inventions, such as the light bulb, temporarily raise productivity growth as adoption spreads, but the effect fades when the market is saturated; that is, the level of output per hour is permanently higher but the growth rate is not. In contrast, two types of technologies stand out as having longer-lived effects on ...
Discussion Paper
Measuring AI Uptake in the Workplace
Artificial Intelligence (AI) may be poised to raise productivity across various domains, including writing (Noy and Zhang 2023), programming (Peng et al. 2023), and research and development (Toner-Rodgers 2024; Korinek 2023). However, understanding the extent to which AI—and generative AI in particular—has been adopted as part of the production process remains an open question. This note reviews the extant surveys on AI adoption at both the employee and firm levels. Surveys of firms show a wide spread of adoption rates, ranging from 5 percent to about 40 percent. Surveys of workers show ...
Working Paper
AI and Coder Employment: Compiling the Evidence
We evaluate whether LLMs have had any discernible impact on the aggregate labor market so far. We focus on occupations that are computer programming-intensive, motivated by data showing that coding is one of the most LLM-exposed tasks. Linking O*NET to CPS we find that aggregate employment of coders has decelerated sharply since the introduction of ChatGPT. Using a novel control variable for industry-level shocks we show that the deceleration is not attributable to the exposure of coders to slowing industries, suggesting instead that coders experienced an occupation-specific shock around ...