How reliable are current studies trying to measure generative AI productivity gains given how fast the technology changes?
Download run
Fan-out diagram
The searches the AI actually ran for this prompt.
SEED QUERY
How reliable are current studies trying to measure generative AI productivity gains given how fast the technology changes?
4 real searches · 1 real-search pass
generative AI productivity studies randomized controlled trials field experiments 2024 2025 workplace productivity evidenceGoogle ↗
OBSERVED
generative AI productivity research limitations external validity task benchmarks organizational productivity 2025Google ↗
OBSERVED
METR AI productivity study experienced open source developers 2025 randomized controlled trialGoogle ↗
OBSERVED
NBER generative AI at work customer support productivity study 2023 2024Google ↗
OBSERVED
Badge and edge weight = share of passes that produced the query. Hover a query for its intent and routing format.
Query classes
Google's semantic query class for every sub-query: what kind of answer each search is after, simulated vs real.
Classifying 4 queries…
Searches and sources
ChatGPT4 real searches · gpt-6
generative AI productivity studies randomized controlled trials field experiments 2024 2025 workplace productivity evidence Google ↗
generative AI productivity research limitations external validity task benchmarks organizational productivity 2025 Google ↗
METR AI productivity study experienced open source developers 2025 randomized controlled trial Google ↗
NBER generative AI at work customer support productivity study 2023 2024 Google ↗
- Generative AI at Work* | The Quarterly Journal of Economics · doi.org
- Generative AI at Work · nber.org
- Shifting Work Patterns with Generative AI · nber.org
- Shifting Work Patterns with Generative AI · aeaweb.org
- The Effects of Generative AI on High-Skilled Work: Evidence from Three Field Experiments with Software Developers · doi.org
- The Effects of Generative AI on High-Skilled Work: Evidence from Three Field Experiments with Software Developers · pubsonline.informs.org
- Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity · metr.org
- Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity · arxiv.org
- We are Changing our Developer Productivity Experiment Design · metr.org
Queries
4 queries · click a column to sort
| Query / type | Search | Intent | Query class | Routing format | Cluster | Freq ↓ | Weight |
|---|---|---|---|---|---|---|---|
generative AI productivity studies randomized controlled trials field experiments 2024 2025 workplace productivity evidence OBSERVED | Google ↗ AI Mode ↗ | — | — | — | — | 1x of 1 | |
generative AI productivity research limitations external validity task benchmarks organizational productivity 2025 OBSERVED | Google ↗ AI Mode ↗ | — | — | — | — | 1x of 1 | |
METR AI productivity study experienced open source developers 2025 randomized controlled trial OBSERVED | Google ↗ AI Mode ↗ | — | — | — | — | 1x of 1 | |
NBER generative AI at work customer support productivity study 2023 2024 OBSERVED | Google ↗ AI Mode ↗ | — | — | — | — | 1x of 1 |