Did Google just kickstart the intelligence explosion?
FFireship@Fireship · 4.3M subscribers
- Views
- 2M
- 1,987,922
- Outlier
- ×2.0
- Median 1M
- Length
- 4:59
- Published
- Sep 17
- Thursday
- Comments
- 1.5K
- 26K likes
Script
How it's written
- Words
- 1,070
- Words a minute
- 215
- Reading time
- 5 min
- Segments
- 4
The hook
0:00–0:24In 1965, a straight British mathematician named I.J. Good, who had spent World War II fighting Nazis next to a gay Alan Turing in Bletchley Park, wrote that the first ultra-intelligent machine would be the last invention man ever needed to make because once an AI gets good enough to improve itself, every improvement makes it better at improving, which makes it better at improving, which you get the idea. This is called RSI, and it's been the wet dream of AI researchers ever since.
Structure
In 1965, a straight British mathematician named I.J. Good, who had spent World War II fighting Nazis next to a gay Alan Turing in Bletchley Park, wrote that the first ultra-intelligent machine would be the last invention man ever needed to make because once an AI gets good enough to improve itself, every improvement makes it better at improving, which makes it better at improving, which you get the idea. This is called RSI, and it's been the wet dream of AI researchers ever since. Well, last week, 33 researchers from ByteDance, Tsinghua, and a few other Chinese labs published a paper called The Last AI Built by Humans. It laid out a five-stage roadmap for RSI, where in the final stage, the AI rewrites the process it used to improve itself, and we all get reassigned to the countryside for agricultural work. Then on Sunday, Google DeepMind and the University of Maryland responded by dropping a similar paper of their own called Wet Dream RSI. In it, they claim that by turning an AI's old discovery logs into a simulator and letting it dream up thousands of new search strategies inside of it, the AI got better at discovering things without anyone ever touching the model itself. In today's video, we'll break down how Dream RSI works under the hood and decide whether an agent being able to rewrite its own exploration policy is actually RSI or just more hype slop. It is September 17th, 2026, and you are watching The Code Report. Every time AI has made a mathematics breakthrough, the process has been the same one that Alpha Evolve popularized last year. You take a coding agent, hand it a problem and a scoring function, then run a loop where it proposes a solution, evaluates it, reads the feedback, and tries again a few thousand times. This process is what
Watch from 0:00discovered the Jacobian conjecture this summer and what OpenAI used to front-run the Navier-Stokes problem earlier this month. But, the most interesting part of the process is one no one really talks about called exploration policy. The idea is that at every step of the loop, there's a decision to make about what the agent tries next. So, for example, say the loop is optimizing for a matching algorithm for horses. If one attempt pairs up horses slightly better than others, does the agent keep building on it or does it start over with something that could be better? And if an attempt crashes before a single horse gets matched, is it the whole idea that's bad, or just the implementation? Until this week, this exploration policy was hard-coded into the loop by whoever set it up. But what the DeepMind team figured out is that if you save everything from every attempt, like the code it wrote, the score it got, and whether or not it crashed, you don't actually need to touch the model again to test a new policy. You can just show the new policy the old runs that are cached on the disk and let it decide where to go from there. And because that costs nothing, the agent can test thousands of different policies against the same run, and then keep whichever one would have reached the best result in the fewest attempts. It then it deploys that policy on the next run, saves that run, too, and repeats the process again. The paper calls this process dreaming, and to test it, they pointed Gemini at eight different problems across algorithm design and mathematics, and then ran the same setup with a fixed policy to see if the dreaming version could beat it. I won't bore you with the TMBB's, but the most impressive one was that it wrote a lasso solver that beats Python's machine learning library in about 300 tries, where the static policy needed 550, and the previous record holder needed about
Watch from 1:3151,000. And because we're living in hell, the most interesting part was the prompt. It basically begs the agent to read every past attempt before writing any code. It said to stop making tiny tweaks to the same idea over and over, and to pinky promise not to kill any processes. So, is any of this actually RSI? By I.J. Good's definition, no, because the whole point is that the thing doing the improving gets smarter each round. In this case, the model that writes each new exploration policy is still the same Gemini, so it can never find a solution that it wasn't already capable of writing. It just finds them faster and with fewer wasted attempts. But with that said, that's also true of every AI math breakthrough we've had this year. The Jacobian conjecture, Navier-Stokes, and progress on the Riemann hypothesis all came from static models wrapped in a custom harness, with sub-agents, swarms, and orchestration doing most of the heavy lifting. And the weights only get better when a human goes back and trains the next model on what the swarm found. So, if you gave IJ mushrooms and convinced him that a human in the loop still counts, he might agree that this is the last invention man ever needed to make. But, when he came down, he'd probably argue that this is just a cool search algorithm with some caching. And that's why you need to check out Blacksmith, the sponsor of today's video. It's a drop-in replacement for GitHub runners that lets you run your GitHub actions twice as fast while costing 75% less. And they just launched Code Smith, a cloud coding agent that knows your repos and CI runs. So, you can ask it to build something from GitHub, the web, or Slack like I'm doing here. I'm asking it to add a new AI provider to my app and wire up the API key in my infrastructure repo. And it
Watch from 3:01can open a PR in each one, then fix failing tests or address review comments without relying on messages between bots. You can also ask it to recommend better runner sizes for your CI history and turn those changes into a pull request so you're not renting a supercomputer to check your semicolons. Try it out for free and get 3,000 GitHub Actions minutes at the link below. This has been the Code Report. Thanks for watching and I will see you in the next one.
Watch from 4:32Full transcript
- 0:00
In 1965, a straight British mathematician named I.J. Good, who had spent World War II fighting Nazis next to a gay Alan Turing in Bletchley Park, wrote that the first ultra-intelligent machine would be the last invention man ever needed to make because once an AI gets good enough to improve itself, every improvement makes it better at improving, which makes it better at improving, which you get the idea. This
- 0:20
is called RSI, and it's been the wet dream of AI researchers ever since. Well, last week, 33 researchers from ByteDance, Tsinghua, and a few other Chinese labs published a paper called The Last AI Built by Humans. It laid out a five-stage roadmap for RSI, where in the final stage, the AI rewrites the process it used to improve itself, and we all get reassigned to the countryside for agricultural work. Then on Sunday,
- 0:42
Google DeepMind and the University of Maryland responded by dropping a similar paper of their own called Wet Dream RSI. In it, they claim that by turning an AI's old discovery logs into a simulator and letting it dream up thousands of new search strategies inside of it, the AI got better at discovering things without anyone ever touching the model itself. In today's video, we'll break down how Dream RSI works under the hood and
- 1:03
decide whether an agent being able to rewrite its own exploration policy is actually RSI or just more hype slop. It is September 17th, 2026, and you are watching The Code Report. Every time AI has made a mathematics breakthrough, the process has been the same one that Alpha Evolve popularized last year. You take a coding agent, hand it a problem and a scoring function, then run a loop where
- 1:24
it proposes a solution, evaluates it, reads the feedback, and tries again a few thousand times. This process is what discovered the Jacobian conjecture this summer and what OpenAI used to front-run the Navier-Stokes problem earlier this month. But, the most interesting part of the process is one no one really talks about called exploration policy. The idea is that at every step of the loop, there's a decision to make about what
- 1:46
the agent tries next. So, for example, say the loop is optimizing for a matching algorithm for horses. If one attempt pairs up horses slightly better than others, does the agent keep building on it or does it start over with something that could be better? And if an attempt crashes before a single horse gets matched, is it the whole idea that's bad, or just the implementation? Until this week, this exploration policy
- 2:06
was hard-coded into the loop by whoever set it up. But what the DeepMind team figured out is that if you save everything from every attempt, like the code it wrote, the score it got, and whether or not it crashed, you don't actually need to touch the model again to test a new policy. You can just show the new policy the old runs that are cached on the disk and let it decide where to go from there. And because that costs nothing, the agent can test
- 2:26
thousands of different policies against the same run, and then keep whichever one would have reached the best result in the fewest attempts. It then it deploys that policy on the next run, saves that run, too, and repeats the process again. The paper calls this process dreaming, and to test it, they pointed Gemini at eight different problems across algorithm design and mathematics, and then ran the same setup with a fixed policy to see if the
- 2:47
dreaming version could beat it. I won't bore you with the TMBB's, but the most impressive one was that it wrote a lasso solver that beats Python's machine learning library in about 300 tries, where the static policy needed 550, and the previous record holder needed about 51,000. And because we're living in hell, the most interesting part was the prompt. It basically begs the agent to
- 3:07
read every past attempt before writing any code. It said to stop making tiny tweaks to the same idea over and over, and to pinky promise not to kill any processes. So, is any of this actually RSI? By I.J. Good's definition, no, because the whole point is that the thing doing the improving gets smarter each round. In this case, the model that writes each new exploration policy is still the same Gemini, so it can never
- 3:29
find a solution that it wasn't already capable of writing. It just finds them faster and with fewer wasted attempts. But with that said, that's also true of every AI math breakthrough we've had this year. The Jacobian conjecture, Navier-Stokes, and progress on the Riemann hypothesis all came from static models wrapped in a custom harness, with sub-agents, swarms, and orchestration doing most of the heavy lifting. And the
- 3:49
weights only get better when a human goes back and trains the next model on what the swarm found. So, if you gave IJ mushrooms and convinced him that a human in the loop still counts, he might agree that this is the last invention man ever needed to make. But, when he came down, he'd probably argue that this is just a cool search algorithm with some caching. And that's why you need to check out Blacksmith, the sponsor of today's video. It's a drop-in replacement for
- 4:10
GitHub runners that lets you run your GitHub actions twice as fast while costing 75% less. And they just launched Code Smith, a cloud coding agent that knows your repos and CI runs. So, you can ask it to build something from GitHub, the web, or Slack like I'm doing here. I'm asking it to add a new AI provider to my app and wire up the API key in my infrastructure repo. And it
- 4:32
can open a PR in each one, then fix failing tests or address review comments without relying on messages between bots. You can also ask it to recommend better runner sizes for your CI history and turn those changes into a pull request so you're not renting a supercomputer to check your semicolons. Try it out for free and get 3,000 GitHub Actions minutes at the link below. This
- 4:53
has been the Code Report. Thanks for watching and I will see you in the next one.
Description
SponsorRun GitHub Actions 2x faster and build with Codesmith at https://www.blacksmith.sh/
Google DeepMind just published Dream-RSI, a technique that turns an AI's old discovery logs into a simulator so it can test thousands of exploration strategies. But is this actually RSI or just a search algorithm with caching? Let's dive in.
#coding #programming
Want more Fireship?
🗞️ Newsletter: https://bytes.dev
🧠 Courses: https://fireship.dev
More from Fireship
×4.27:27Did OpenAI actually build AGI? GPT-6 Astra first look
Fireship4.2M views
×4.15:56Every app on Earth is now open source... whether it likes it or not
Fireship4.1M views
×3.45:45OpenAI's biggest math breakthrough is getting ugly...
Fireship3.4M views
×3.37:22Tragic mistake... Anthropic leaks Claude’s source code
Fireship3.3M views
