Notes
On papers, ideas, and things I'm thinking about.
Filter:
All
AI R&D
AI progress
benchmarks
economics
evals
forecasting
functional forms
growth models
software optimization
Evidence on AI R&D Progress from NanoGPT
How much are AI agents accelerating AI R&D, and how is that changing over time? The NanoGPT speedrun leaderboard is one source of evidence: a public, cumulative record of pretraining optimizations contributed by humans and, recently, by agents.
On the Subtleties of Software Optimization Evals
Small metric choices in software optimization benchmarks can create brittle aggregates and misleading capability conclusions. A look at how harmonic mean aggregation and correctness penalties interact in optimization evals.
Building Blocks to AI Timelines
Notes and intuitions on economic growth models, functional forms, and how they connect to forecasting AI progress. Part of a series on economics basics for AI forecasting.