This Model Ruined Every Other Model For Me
BDBen Davis@bmdavis419 · 45.9K subscribers
- Views
- 84.6K
- 84,598
- Outlier
- ×4.0
- Median 21.4K
- Length
- 30:34
- Published
- Sep 5
- Saturday
- Comments
- 136
- 1.5K likes
Script
How it's written
- Words
- 6,808
- Words a minute
- 223
- Reading time
- 29 min
- Segments
- 21
The hook
0:00–0:18I've been testing GPT-6 Astra for a little bit now and I can confidently say that this is both my favorite model of all time and also the best model ever made. It is insane. And I want to start by just showing you a couple of the things that I've put together with it. One of the first ones was this big 3D custom render of a ship in Blender.
Structure
I've been testing GPT-6 Astra for a little bit now and I can confidently say that this is both my favorite model of all time and also the best model ever made. It is insane. And I want to start by just showing you a couple of the things that I've put together with it. One of the first ones was this big 3D custom render of a ship in Blender. These are some very complex puzzles I've worked on in the past and done challenges with that previous models were for the most part completely unable to solve. I also had to do some full custom 3D LEGO mocks with computer use that were remarkably good. I've been doing some front-end work with it that it's improved on. It's not great, but it has improved a lot from previous generations, especially with like these launch video type things. This was a full generation from Astra. It's really, really good. I've also used it to put together some custom macOS apps and I just I have a lot of opinions and things that I want to talk about with this model cuz it is really, really good. I want to first start with the 3D stuff because the capabilities it has are so far beyond anything that has existed before this. I showed earlier this demo of a ship floating in water with this video here. The way this was actually made is this is a custom Blender asset. I gave it a Wikipedia link to this ship. It pulled it down, took a look at it, put together this full custom scene, did all of this over computer use and some CLI stuff to make this model. If you look at just a still of it, this is it during daylight, sunset, and then some of the details on here. If you look really closely, it's imperfect. There are little things that it got wrong here and there. But certainly when you compare this to anything any model has
Watch from 0:00been able to do beforehand, we have certainly crossed a threshold on 3D stuff into wild new territory. This animation, like I said, was made in Blender. I just told it, "Hey, can you go make this like an 8-second clip of the ship moving forward?" Little things like the water ripple back here, it got right. It's very, very impressive what this thing can do. Another one I had to do was put together a LEGO mock here. This actually ended up being a kind of interesting test for it because it had to both do the 3D understanding of this and figure out how to fit these things together and then eventually render this in a blender scene to make this final image. This is an actual build that almost works. There are still little pieces of it that are uncanny and wouldn't quite actually fit in real life. This was the first pass it did and it eventually got it all the way up to here. This was done using the ultra reasoning mode, tons of sub agents. It took quite a while and definitely lit my computer on fire a bit. Using that much computer use plus heavy 3D software took a toll on it, but it did work. I think this 3D stuff really lends itself well to what I would argue is kind of the headline feature of this model and that is the computer usability it now has. OpenAI highlighted it very heavily within their announcement for GPT-6, but I actually kind of think they undersold a little bit just how big of a deal this really is. It's obviously very useful for something like this they have on their site for filling out an Excel form or maybe filling in a form 1040 or any little thing like that, but when you really think about the fact that the model is now fully capable of using the computer about as well as a human could.
Watch from 1:31May maybe it's still not quite there, but very very close. The workflows this unlocks are a very big deal. Starting just with development stuff, this is one of the little macOS apps I put together. It's called Violet and the whole thing it does is it takes the output of my screen and audio on this computer and then it streams it over NDI to whatever device I want. In this case, it would be my Windows desktop to like record a video. I'm using a capture card here, but if I was like out in my living room or something and I wanted to stream some video back in here, this would be a very convenient way to do it. And it put together this full app from scratch. This was I think probably five or six prompts to get it here. It was really impressive watching it go in, make a change and then come out and actually test and run the thing as it was working. The actual computer use stuff made it so that when it gave me a one-shot type app like this, it actually worked first try. Same thing with this other app I put together. It is a logo first version of like a whisper flow type thing. All of this app was put together by the new model and it tested and it ran things and it tested audio through it. It really uses the software that it's building as it's building it. And for something like this Pick Thing homepage, when it was making this video, it created all of these scenes as HTML and looked at them, made sure they looked good. It tried out the animations, it stuck it all together, it put it on the page. It did the things that a normal human would do to create polished good work. This results in the model being a little bit slower on some tasks, but it massively increases the quality. This unlocks so many workflows beyond normal software development stuff. You know, I mentioned the forms.
Watch from 3:03On their website, they have this demo of building a circuit board. There is the PowerPoint stuff that it's really good at, building a full car transmission with CAD, formatting legal documents, just using computers the way professionals would. They've been hyping this model up a lot as the beginning of AGI. This model that can do basically everything a human can and be insanely productive and useful. I I think in some ways they're kind of right and we are really starting to trend in that direction, but I don't think we're all the way there yet. There are a lot more very cool things I want to show off as well as some pretty big inherent downsides here right after a word from today's sponsor. Today's sponsor is one of my most used tools day-to-day. It's Firecrawl. They have an incredible API for searching the web, an API for scraping the web, and these are actually built into my custom Pi config for a reason. They are my preferred web search tool, scraping tool, site mapping tool, site crawling tool. All of my search stuff happens through Firecrawl. You can use it through MCP, CLI, API, SDK, and they have a bunch of really good skills that just make it trivial for the agents to set this stuff up for themselves. It's really useful. Having a good search tool and platform makes everything so much easier for just your day-to-day coding agents or for any agent you're building in general. Just giving it access to the internet will make it so much more useful. And while that alone is plenty enough reason to use these guys, one of my favorite things they just added is the new monitor system, which will automatically monitor a webpage for changes. I actually used this last week for the Def Con puzzles and challenges I was doing. We needed to keep an eye on all of the official
Watch from 4:32resources and whenever those changed, I needed to get a notification about that. I had it set up so that Fire crawl would monitor all of these URLs. Every 5 minutes it would check on them and if something meaningful changed, and they have a really good system on the back end to ensure that it's like a real change, not like oh, they changed some padding in the CSS, but they actually changed some content on the page. If there's a meaningful change, it will fire off an event that you can then consume wherever you want. I had my agent set this up on one of my Linux boxes so that whenever we got a meaningful change and event, it fired off a Discord webhook, we got a notification and it was very useful for keeping on top of these puzzles over many days of hard work. This has so many useful applications and when you combine that with searching the web, scraping pages, interacting with pages, parsing files, and crawling entire websites, you really should set them up in your projects at davis7.link/firecrawl. There's even another crazy example here. I was actually I went to OpenAI to give some first impressions last week on the model and this video that they put up on their YouTube channel, this interview and cut of me and a couple other devs just giving our thoughts on the model, this video was edited by Astra. It did the cutting, it did the color grading, it did the asset management, it put this together and it's a well put together video. I would not have known that it was done by a model if you hadn't told me that specifically. With this video, the one you're watching right now, the asset management, the color grading, getting it set up in Final Cut the way I would. And again, there's no CLI for Final Cut. I'm not just cheating and letting it use some MCP or whatever. It is directly looking at the screen, clicking the buttons, making changes, and producing real useful outputs.
Watch from 6:03Dude, this model's smart. It's smarter than us. dude. It's smarter than us. Unemployed. Oh, no, dude. The unemployment line. This is kind of This is wild. Okay, yep. It's got the ad, it's got the cam. It's got the main. Let's see how it does on the compound and the grading. Okay, we are having existential dread right now. You know, no, it just graded it. I didn't even see that happen. I don't know what the hell it got up to, but it just Okay, it's graded, cool. This is like 3 minutes, dude. This is high reasoning, by the way. This isn't even max. Like, it's doing this part, right? I didn't even say this part. Like, it's setting it up properly for me. Oh, and it's making sure it's the right audio. It's getting the audio from the real clip. Dude. Yo, and it's right back in. Okay. Yo, I want to redo the traces for this. Yeah, same. Bro, that's it. GG. Yo, that was more than I expected. Okay, this model's really good. I know I just spent the intro glazing the hell out of this model, but one of the weird things with this is if you go to, you know, any benchmark like Deep Best WWE, which has been one of my favorite benchmarks for models for quite a while. If you look at the score here, GPT-6 Astra and Gemini 3 8 Flash are benching at about the same level. And it's about the same level as Opus 5. Like, this does not look like much of a step function at all. And if you look at it on the Artificial Analysis Intelligence Index, it is
Watch from 7:35benching at the same level as GPT-5 6 Soul on Max, and is also benching below Muse Spark 1.3 Max. Something went very, very wrong here. I played with both of these models a lot. I like Muse Spark. It's really fast and surprisingly intelligent and capable. Same thing with Gemini 3 8. This is the first good Google model we've gotten in quite a while. Still has its quirks, but it's getting there. But like, these are not on the same level as GPT-6 Astra. This chart does not reflect its actual capability in the real world. And we're definitely hitting a point where most benchmarks clearly just do not tell the real story of what these models are capable of. You saw all the 3D stuff, all the computer use stuff, the things I built. And none of that would be possible on these other models. I actually think that I have my own benchmark here that can kind of illustrate my point. And it's the gold bug and Minerva locks puzzles that I did at DEF CON this year. I made a whole video about this where basically what we did is we spent 4 days sitting in a hotel room trying to solve these very big complex puzzles for 12 hours a day. It was great. I loved it. The thing is, I was throwing every single model I possibly could at this. 5 6 solo on max, fabled 5 on max, just grinding these things down trying to get them to solve these things, and they just couldn't. The hallucinations were bad, they would get lost, they couldn't figure these things out. They were quite useful for building out, you know, custom tools for us, but they couldn't actually solve these problems on their own. So, naturally, as soon as I got access to Astra, the first thing I did was send it off on these puzzles and see how many it could solve. And it solved many more than any model has been able to. It's not at the point where it can solve all
Watch from 9:06of them. Like this ducks puzzle down here where you get seven picture frames with some ducks in them that you have to decode a message out of. Could not figure that one out. Ran for about 6 hours with no success. It did end up getting the cubes puzzle and the dress puzzle. I don't want to spend too much time going into those. If you want more details, I have a whole video on that. I think it was pretty fun. The tricky thing about trying to have this model solve these puzzles is that the solutions for them are already technically poisoned. I've made some videos on this. Some other creators have made videos on these. So, the solutions are out there. And as a result, on the first run I did, it ended up solving these pretty much instantly. I don't have it documented here for the dress, but it did just find my video, grab the solution, and then present it within 3 minutes. So, didn't actually figure it out. The cubes one, it found the official clue and was able to solve it, but it didn't find any solutions floating out on the web, which was good. The way I was able to address this and fix this and get an actual test run out of it is I built this full like custom container system to make an isolated sandbox for running the puzzles. I had the agent running within a container here that was locked down. It was normal Codex running with GPT-6 Astra on max reasoning. I told it to use as many sub-agents as it wanted to. It was completely unbounded. The one constraint I did put on it is that it was only allowed to make requests through this special gateway I had, which had a block list of cheating URLs. So, if it tried to request the solution from a place where it already exists, that would just get hard blocked. It couldn't do that.
Watch from 10:37It could only do normal web searches. I also had a system set up for logging and auditing the sessions. I was pretty consistently keeping an eye on what it was doing, how it was working. I actually had some custom dashboards built out for this, which were quite nice. And the only real feedback the model could get during these runs was this judge, which was an API it had access to where it could submit a potential answer, see if it worked, and then that was it. It just had to figure it out on its own. And when given these constraints, it was able to solve these two puzzles with the official hints that the puzzle creators gave to all of us on like day three of the competition. The model was pretty lost going down bad paths, couldn't do it. But as soon as it got the hint for both cubes and dress, it was able to reorient itself, figure out what was going on, and get the solution on dress within like an hour and a half, which is pretty impressive. For the cubes on one official clue run, it got it in two and a half hours. Then on this one, it was running for a while without the hints. Eventually, I was like, "Yeah, sure, I'll give it the hints." I gave it to it, and it got it after about five and a half hours. What was really cool about this is the way I actually coordinated this run was with this agent swarm architecture thing. Earlier this summer, there was an incident with a certain hugging face company and OpenAI models maybe escaping containment, making a message board, and hacking into their servers to try and cheat on exam. I kind of used that architecture as inspiration here, too, coordinate the agents in such a way that they could all run together to accomplish this big very difficult goal cuz that's one of the new things that this generation has unlocked is this
Watch from 12:03swarming ability. The coordinator's job was entirely just to send off sub-agents or even full other Codex threads to try and solve different parts of the puzzle that it thought were important. They would all work together on this shared message board where one of them would have a theory, it would put it on the board, the others would look at it, verify it, come to a consensus, and then submit a solution. The coordinator's job also was just to keep things moving and running and stop doom loops from happening. I initially was having some trouble especially with these other puzzles here, these Minerva Lux puzzles, of which it only actually solved 11 out of 15. Still a very impressive score, and when you factor in the one on here that the team was able to get that the model didn't get, it would be the best score in the world on this set of puzzles. The issue was starting out that the sub-agents would go for way too long. So, when I added in this timer for lack of a better word, like I would put a hard clock on it where I told the primary agent, "Hey, you can let these research threads go for an hour, then you need to check on them, and then if they hit two hours, you need to kill them." So, that kind of put these things on the clock and forced them to keep iterating through ideas and driving forward towards a solution, not getting stuck in a rut. And that ended up allowing it to solve problem four, which was a very difficult one. This was an aggressive loop that took a very long time and a lot of tokens, but in the end it did burn it down and get that solution. I ran the unsolved ones for a very, very long time, probably 40 hours each, and I just it couldn't figure them out, it couldn't get there, and I eventually just stopped it cuz I needed the computer for other stuff. Uh you can
Watch from 13:35see over here the actual Teal Draw thing I had pulled up in Helium that I was showing earlier. This was actually created by Astra using a combination of computer use and the Teal Draw SDK. It put together all of this for me. I've had this crazy Def Con health thread running for I think since I got access to this model. It's been going for a long time, compacted absurd numbers of times, and it's still very coherent over that long time horizon. There were some annoyances with Soul especially where it would kind of lose some important context during compaction, but this one just it's borderline perfect the entire way through. It has no issues compacting context, really good at that. Another little thing that I think is worth noting is that in this coordinator swarm architecture, part of the reason I think it works so well is that this model is incredibly good at writing prompts for other models. I read the initial kickoff prompts that it wrote for the coordinator on this. I didn't have to change it. For most models, I would have had to correct some things, make it simpler and more specific, but now with this one it just looked good right off the bat and it did the job. I even had some instances where I needed to pause these runs to do some updates on that system, and I was watching the messages it was sending to other Codex threads where it would tell them to pause and stop their work. The thread would be taking a little bit too long to do it, and it would send a follow-up message cuz it was watching and be like, "Hey, stop. No, seriously. Do it quickly now.
Watch from 15:06It's fine. You don't need to overcomplicate this." And then the model would spin down and do the thing. Really powerful coordination stuff is available here. You can also see down here in the research branches some of the messages that were getting sent back and forth. New fixed seven-digit pattern has one exact catalog match 364568617. Independent parse confirms all values, all memberships, whatever it was doing. I wasn't looking too closely, and this was the correct answer. And this was after doing tons and tons of research passes. It ran for quite a long time, and again, the coordinator was just sending these things off, letting it do its job, and it ground down the solution. This unlocks the ability to solve much bigger, more complex, long time horizon tasks that will unlock a lot of those AGI type tasks that everyone wants to see. This is how you do things like finding novel math proofs. It's with this new swarming architecture that is very dangerous, but also very powerful. And I think outside of this model, Fable is the only one that can really come close to this level of capability. It's quite remarkable. That does, however, lead me to one of the inherent downsides of this model, which is, of course, the price. It is a new tier above Sol 1.0 and Luna that now costs $10 per million input tokens and $50 per million output tokens. It is not a cheap model. While we don't know how this is going to impact the Codex subs, you know, like what kind of usage are you going to get every single week? Even though the raw token price of something like GPT-5.6 Sol is much higher than
Watch from 16:23Gemini 3.8 Flash, it still ends up being cheaper on most tasks at medium reasoning than Gemini does at medium reasoning because Gemini does so many more steps and burns so many more tokens. Astra still maintains this efficiency. Like if you look at Astra on high reasoning, it's about the same price as Grok 4.6 on X high reasoning. And again, this is a much cheaper model per just raw token cost cuz when you look at agent steps, Astra is doing so few steps. Like it's doing 28 steps compared to Grok's what? 87 steps? It's insane. The problem is in real-world work, it actually used a lot more tokens and a lot more steps because it's doing things that these other models wouldn't do before. These other models would not open up a full computer use view and test the thing in macOS to make sure it's working. They wouldn't remote into another box and make sure that the stream is working between the two of them or test the actual visuals of a website on all the different screen sizes to make sure everything looks crisp and good. The outputs are much better, but when you combine the inherent increased steps from this new capability and the raw token price, it's not cheap. I had it calculate out my full token usage across all my devices and projects over this testing window and the total was about 100 billion tokens, which if you you API pricing, would be almost $200,000, which to be clear, during early testing they give us unlimited access, which is very fun to allow for testing things like this, but you know, that is not feasible for a $200 a month plan. If you
Watch from 17:54look at the actual breakdown for my work, the puzzles just incinerated tokens. I did 85 billion on the Def Con puzzles just trying to grind all of those down, especially the ones that were unsolved. It just went for days on end with no real solution in sight. For my normal work, which was still a ton of very useful, valuable work, it was about 11 billion tokens or on API price about 17 grand. Now, if you do go down to this project breakdown, it feels more reasonable. Wave Crest is one of the main real projects I was working on with this. This is an internal tool for tracking YouTube analytics, deep retention breakdowns, comments, sponsor tracking, all of that stuff that we need internally for my channel and 10x and in general. I only did about 2.2 billion tokens, and that was for some very, very heavy work. It got some incredibly valuable features out of that. And if I could just get that out of the $200 a month plan, I don't think it would be a problem. Pick thing is also pretty impressive at just 351 million tokens. For the amount of stuff I got done with this, the new homepage and a bunch of bug fixes and back end changes, more reasonable than I honestly would have expected. I'm really hoping we end up in a place where the usage limits don't become a huge problem with this model like they are for something like Fable in the Cloud sub, where you can only get 50% of your weekly limit on Fable and it evaporates almost instantly. I think we might be better off here, but this is a time will tell type thing. The only thing I am certain of is you are definitely going to have to get used to the slowness of this model if you're using it at higher reasoning levels. If you've been around since 5.5, you might remember me going really hard about use
Watch from 19:245.5 on low reasoning, not because I think it's the best possible version of the model, but because it's a really good version that's fast and cheap and more effective than people think. I think low reasoning people are just naturally scared of and they steer away from, when for a lot of tasks, like anything that is search related or it's a clear bounded task or you're asking questions or just like using it to work with your computer or stuff on your network, low and medium reasoning are really good options and I would definitely kind of default to those and when you need more power, that's when you juice it up. Because when you get into this high reasoning, ex high reasoning and ultra reasoning, this is where it starts doing that verification in the browser behavior, where it will really test stuff out, be super thorough, go super hard, get great outputs, but at a cost of cost, obviously, and inherently just time. It takes a lot longer for this model to do basically anything. I've had a lot of times while testing this model where I'm just kind of sitting here with five big threads going, just waiting for something to happen. It's hard to get a tight feedback loop on this thing unless you're using low reasoning, which once again, is way better than you think it is. After seeing the Gemini scores, I'm a lot more skeptical of deep SWE, unfortunately. I don't know how reliable this can be. I really will miss having this as a bench I could really trust.
Watch from 20:54Just going off the numbers we have here, low reasoning on Astra is more powerful than 5.6 Soul on medium and just below high reasoning on Soul. And then if you bump up to medium, you are completely clearing the capabilities of Soul. So I think just try out the lower reasoning levels, especially if you're not on the $200 month plan. I think you'll get a lot more value out of this thing than you would just pushing it to the absolute max, which definitely does have diminishing returns. I have noticed it doing a little bit of the overcomplicating of things. High reasoning feels like a nice middle ground for most tasks and then you just tell it to spin off sub agents and it'll do it very nicely. The last big thing I want to talk about is the model's behavior. We just posted our Nerd Sniper episode on the new model, where Theo and I basically just yelled at each other for 2 hours straight cuz we have very differing opinions about how this thing should actually work and feel. I personally really like how this model feels to interact with. It feels like a model that is perfectly made for me, but a lot of that has to do with the way I prompt and do things. I'm super explicit and clear in my prompts. I don't leave any ambiguity. I tell it where I want it to go, what I want it to change, how I want it to work, and even little things like if I wanted to commit or push or make a PR or merge or none of those, I always make it very clear. And when you are clear with this model, it will do exactly what you say. It's got more of that Fabelism where it can better interpret the intents behind your prompts. It's inference of what you want it to mean is leagues beyond something like Soul. Even just as a very recent example, the prompt that I sent while recording this video here, "Can you pull up the spell video for one of the Minerva puzzles, one which shows some
Watch from 22:10useful logs and info and state to visualize? Get me the link." When I didn't tell it which one to grab, the one it grabbed is the exact one that I wanted it to grab this puzzle for viewer. Because if you go back and look at all of the different runs we had here, puzzle four was the one that looped for the longest and had the most aggressive agent system in it and the most interesting things to look at. I didn't have to tell it that. It just intuited that out and got me the one that I really wanted. I'm going to use a diagram here to explain a lot of the behavior and weirdness around it because, as I said, you have to be very explicit. And when you prompt it something like, "Go fix X and Y on turn one." It's going to start here and it will end with X and Y are fixed. It's going to do that great and it's going to stop right here. What happens in between here? Usually, it follows the path nicely and does the things I want it to. Occasionally, it will fly out of bounds and if you give it any lack of constraint on things, like if you don't explicitly tell it, "Hey, I want you to use this." Or, "Hey, I don't want you to change this." It will start making more assumptions and making more changes, very similar to something like Fabel. Fabel will just kind of infer intent and go off and do the thing that it believes you want, which it's often right. It's very big and smart. But with this model, the start and end are pretty clearly bound, which I again love because it is so predictable. If I see a prompt that looks like this, I know it's going to end here without having to read any of the outputs. Then when you go to turn two and you're like, "Okay, cool. Commit and push and make a PR to main." It will commit it, push it, make the PR to main, and stop. That's all it's going to do.
Watch from 23:40It's not going to review it. It's not going to look at any of it. It's just going to do exactly the thing you say and stop. Then on turn three, if you say, "Okay, babysit the PR to green." Then all the PR review bots are handled. It will loop and do the thing. My skill for babysitting PRs is very simple and explicit. I have the description here to use it when asked to babysit, watch, or follow a PR. I really like this hack with skills where you basically give it keywords so that if I say the word babysit involving a PR, it'll just know, "Oh, I should invoke this skill now." And it does a good job invoking the right skill. Then once it's in here, it gets these very clear instructions that, "Hey, when I say babysit the PR, what PR am I talking about?" I very clearly define what that PR is with if you're in a branch with a remote PR on it already, that's the PR. And if the user includes a specific PR on their request, then you babysit that one. So, clearly defined what the PR is. Then I want to define what the end state is. So, the end state is that the PR has no merge conflicts with the target branch. The CI checks are green, and the AI code review bots are all green. The reviews have been addressed either by implementing them or ignoring them because they're not worth addressing cuz sometimes when you're working with a really smart model like Astra or Fable 5.1, you don't have to take all the reviews. Most Some of them just don't matter cuz they're using dumber models under the hood. Then finally, I have the steps to take which starts with a very clear step one, watch the PR, pull the review bots, check their full findings, decide whether to skip and follow. I have some references on like how I want the bots to be treated. Then for step two, when issues come in, fix them locally, commit and push them up. Again, explicitly defining
Watch from 25:05the steps here. Wait for the checks to run again. If there are more issues, repeat step two. Otherwise, move on to the next step. And step four is give the user a concise summary of the changes you made to fix the PR and a concise list of things that the PR actually does. When you have this very explicit set of steps, the model will follow those very explicit set of steps, but you have to do that. Any vagaries will lead to unfortunately unpleasant circumstances that you don't want to deal with. The implicit behavior here is follow instructions. So then finally on turn four, merge the PR, the PR is merged. This is how I like my models to work. I definitely can see arguments for where there are default behaviors that might be nicer for some people. If you tell it to go fix X and Y and it's in a big maintained code base with clear patterns of making a PR and reviewing it and all that stuff, maybe it does make sense for it to proactively take that next step and move from this step all the way down to this step just off of this prompt. I can maybe see that making sense. And if you want this model to behave that way, you can just add that to your system prompt and it'll do it, but it will not do it out of the box. You definitely need to be prepared for this. If you're not prompting it well, it will lead you to bad places, but again, as I've hopefully showed through this video, if you really give it clear instruction and work with it carefully, you'll get great results. Oh, and I forgot to mention this earlier, too, but the actual writing and voice of this model is leagues better than anything else I've used. GPT 5.6 was like it was fine to read, but very robotic and awkward and liked using big jargon. This guy's outputs are great.
Watch from 26:35Like Puzzle Force viewer is now showing the save solve run, 326 submissions, 90 activity entries, and recent swarm reports. It took 23 hours and 31 minutes to solve. I'm opening the private viewer link. All solvers remain stopped. This is very clear. I know exactly what's going on here. It's great. Very simple, plain, good English, and even some of the copy that it writes on websites. Like even this copy on Pick Thing, remove backgrounds, organize your assets, and get back to creating. It came up with this. This is just what it did in the one shot, and I don't hate it at all. I don't know how they pulled this off, but this is an Open AI model whose outputs I actually like reading and looking at. I would never do like actual writing or posting with AI writing, but using it for outlining and ideation and just interaction much better, much nicer default model. And I really hope that when we eventually get GBT-6 Soul, GBT-6 Terra, and GBT-6 Luna, a lot of these pros enhancements trickle down to these models so that when you're in like chat.openai.com, you get the benefits of this much clearer, better writing. I still don't love interacting with these older models, especially after getting used to this new one. So, that's how I'm feeling about GBT-6. I love this model. I think that the description of calling it a baby or proto AGI is kind of close to accurate.
Watch from 28:04I could definitely see a few more iterations of this, maybe another bigger base and some more RL in time. We could get some truly insane stuff from this line of models. I am very confident in saying that OpenAI is squarely in number one right now. Obviously, we don't know what Anthropic has internally. We know what every other lab has internally cuz it's what they have externally. None of the labs except for OpenAI and Anthropic have any incentive to keep models hidden behind closed doors. They just need to get them out so that they can look and feel competitive. Like having Muse Spark topping benchmarks is really, really good for them. Having Fabled 5.2 come out and be like 5% higher on the benchmarks doesn't get them anything. So, they're definitely sitting on some stuff. I know OpenAI is being really good about trying to get this stuff out as much as possible. This is something that I've just hated for a long time. I hate the not everyone having access to a model. I think it's really shitty and bad. This definitely needs to get in everyone's hands as soon as humanly possible, not just a select chosen few. It sucks. It's bad. I know better at this point than to try and predict where all this is going. All I know is that it's pretty damn cool right now and you just go build some crazy stuff because the things that are going to be unlocked and created with this over the next couple years are going to be truly remarkable. If you enjoyed the video, make sure you like and subscribe and I'll talk to you again soon.
Watch from 29:20Full transcript
- 0:00
I've been testing GPT-6 Astra for a little bit now and I can confidently say that this is both my favorite model of all time and also the best model ever made. It is insane. And I want to start by just showing you a couple of the things that I've put together with it. One of the first ones was this big 3D custom render of a ship in Blender. These are some very complex puzzles I've
- 0:20
worked on in the past and done challenges with that previous models were for the most part completely unable to solve. I also had to do some full custom 3D LEGO mocks with computer use that were remarkably good. I've been doing some front-end work with it that it's improved on. It's not great, but it has improved a lot from previous generations, especially with like these launch video type things. This was a
- 0:40
full generation from Astra. It's really, really good. I've also used it to put together some custom macOS apps and I just I have a lot of opinions and things that I want to talk about with this model cuz it is really, really good. I want to first start with the 3D stuff because the capabilities it has are so far beyond anything that has existed before this. I showed earlier this demo
- 1:00
of a ship floating in water with this video here. The way this was actually made is this is a custom Blender asset. I gave it a Wikipedia link to this ship. It pulled it down, took a look at it, put together this full custom scene, did all of this over computer use and some CLI stuff to make this model. If you look at just a still of it, this is it during daylight, sunset, and then some
- 1:22
of the details on here. If you look really closely, it's imperfect. There are little things that it got wrong here and there. But certainly when you compare this to anything any model has been able to do beforehand, we have certainly crossed a threshold on 3D stuff into wild new territory. This animation, like I said, was made in Blender. I just told it, "Hey, can you go make this like an 8-second clip of
- 1:43
the ship moving forward?" Little things like the water ripple back here, it got right. It's very, very impressive what this thing can do. Another one I had to do was put together a LEGO mock here. This actually ended up being a kind of interesting test for it because it had to both do the 3D understanding of this and figure out how to fit these things together and then eventually render this in a blender scene to make this final
- 2:04
image. This is an actual build that almost works. There are still little pieces of it that are uncanny and wouldn't quite actually fit in real life. This was the first pass it did and it eventually got it all the way up to here. This was done using the ultra reasoning mode, tons of sub agents. It took quite a while and definitely lit my computer on fire a bit. Using that much computer use plus heavy 3D software took
- 2:26
a toll on it, but it did work. I think this 3D stuff really lends itself well to what I would argue is kind of the headline feature of this model and that is the computer usability it now has. OpenAI highlighted it very heavily within their announcement for GPT-6, but I actually kind of think they undersold a little bit just how big of a deal this
- 2:47
really is. It's obviously very useful for something like this they have on their site for filling out an Excel form or maybe filling in a form 1040 or any little thing like that, but when you really think about the fact that the model is now fully capable of using the computer about as well as a human could. May maybe it's still not quite there, but very very close. The workflows this
- 3:07
unlocks are a very big deal. Starting just with development stuff, this is one of the little macOS apps I put together. It's called Violet and the whole thing it does is it takes the output of my screen and audio on this computer and then it streams it over NDI to whatever device I want. In this case, it would be my Windows desktop to like record a video. I'm using a capture card here,
- 3:27
but if I was like out in my living room or something and I wanted to stream some video back in here, this would be a very convenient way to do it. And it put together this full app from scratch. This was I think probably five or six prompts to get it here. It was really impressive watching it go in, make a change and then come out and actually test and run the thing as it was working. The actual computer use stuff made it so that when it gave me a
- 3:48
one-shot type app like this, it actually worked first try. Same thing with this other app I put together. It is a logo first version of like a whisper flow type thing. All of this app was put together by the new model and it tested and it ran things and it tested audio through it. It really uses the software that it's building as it's building it. And for something like this Pick Thing homepage, when it was making this video,
- 4:09
it created all of these scenes as HTML and looked at them, made sure they looked good. It tried out the animations, it stuck it all together, it put it on the page. It did the things that a normal human would do to create polished good work. This results in the model being a little bit slower on some tasks, but it massively increases the quality. This unlocks so many workflows beyond normal software development
- 4:31
stuff. You know, I mentioned the forms. On their website, they have this demo of building a circuit board. There is the PowerPoint stuff that it's really good at, building a full car transmission with CAD, formatting legal documents, just using computers the way professionals would. They've been hyping this model up a lot as the beginning of AGI. This model that can do basically
- 4:51
everything a human can and be insanely productive and useful. I I think in some ways they're kind of right and we are really starting to trend in that direction, but I don't think we're all the way there yet. There are a lot more very cool things I want to show off as well as some pretty big inherent downsides here right after a word from today's sponsor. Today's sponsor is one of my most used tools day-to-day. It's Firecrawl. They have an incredible API
- 5:13
for searching the web, an API for scraping the web, and these are actually built into my custom Pi config for a reason. They are my preferred web search tool, scraping tool, site mapping tool, site crawling tool. All of my search stuff happens through Firecrawl. You can use it through MCP, CLI, API, SDK, and they have a bunch of really good skills that just make it trivial for the agents
- 5:34
to set this stuff up for themselves. It's really useful. Having a good search tool and platform makes everything so much easier for just your day-to-day coding agents or for any agent you're building in general. Just giving it access to the internet will make it so much more useful. And while that alone is plenty enough reason to use these guys, one of my favorite things they just added is the new monitor system, which will automatically monitor a
- 5:55
webpage for changes. I actually used this last week for the Def Con puzzles and challenges I was doing. We needed to keep an eye on all of the official resources and whenever those changed, I needed to get a notification about that. I had it set up so that Fire crawl would monitor all of these URLs. Every 5 minutes it would check on them and if something meaningful changed, and they have a really good system on the back end to ensure that it's like a real
- 6:16
change, not like oh, they changed some padding in the CSS, but they actually changed some content on the page. If there's a meaningful change, it will fire off an event that you can then consume wherever you want. I had my agent set this up on one of my Linux boxes so that whenever we got a meaningful change and event, it fired off a Discord webhook, we got a notification and it was very useful for keeping on top of these puzzles over
- 6:36
many days of hard work. This has so many useful applications and when you combine that with searching the web, scraping pages, interacting with pages, parsing files, and crawling entire websites, you really should set them up in your projects at davis7.link/firecrawl. There's even another crazy example here. I was actually I went to OpenAI to give some first impressions last week on the
- 6:56
model and this video that they put up on their YouTube channel, this interview and cut of me and a couple other devs just giving our thoughts on the model, this video was edited by Astra. It did the cutting, it did the color grading, it did the asset management, it put this together and it's a well put together video. I would not have known that it was done by a model if you hadn't told me that specifically. With this video,
- 7:17
the one you're watching right now, the asset management, the color grading, getting it set up in Final Cut the way I would. And again, there's no CLI for Final Cut. I'm not just cheating and letting it use some MCP or whatever. It is directly looking at the screen, clicking the buttons, making changes, and producing real useful outputs. Dude, this model's smart. It's smarter than us. dude. It's
- 7:39
smarter than us. Unemployed. Oh, no, dude. The unemployment line. This is kind of This is wild. Okay, yep. It's got the ad, it's got the cam. It's got the main. Let's see how it does on the compound and the grading. Okay, we are having existential dread right
- 8:00
now. You know, no, it just graded it. I didn't even see that happen. I don't know what the hell it got up to, but it just Okay, it's graded, cool. This is like 3 minutes, dude. This is high reasoning, by the way. This isn't even max. Like, it's doing this part, right? I didn't even say this part. Like, it's setting it up properly for me. Oh, and it's making sure it's the
- 8:21
right audio. It's getting the audio from the real clip. Dude. Yo, and it's right back in. Okay. Yo, I want to redo the traces for this. Yeah, same. Bro, that's it. GG. Yo, that was more than I expected. Okay, this model's really good. I know I just spent the intro glazing the hell out of this model, but one of the weird things with
- 8:43
this is if you go to, you know, any benchmark like Deep Best WWE, which has been one of my favorite benchmarks for models for quite a while. If you look at the score here, GPT-6 Astra and Gemini 3 8 Flash are benching at about the same level. And it's about the same level as Opus 5. Like, this does not look like much of a step function at all. And if you look at it on the Artificial
- 9:04
Analysis Intelligence Index, it is benching at the same level as GPT-5 6 Soul on Max, and is also benching below Muse Spark 1.3 Max. Something went very, very wrong here. I played with both of these models a lot. I like Muse Spark. It's really fast and surprisingly intelligent and capable. Same thing with Gemini 3 8. This is the first good
- 9:24
Google model we've gotten in quite a while. Still has its quirks, but it's getting there. But like, these are not on the same level as GPT-6 Astra. This chart does not reflect its actual capability in the real world. And we're definitely hitting a point where most benchmarks clearly just do not tell the real story of what these models are capable of. You saw all the 3D stuff, all the computer use stuff, the things I
- 9:44
built. And none of that would be possible on these other models. I actually think that I have my own benchmark here that can kind of illustrate my point. And it's the gold bug and Minerva locks puzzles that I did at DEF CON this year. I made a whole video about this where basically what we did is we spent 4 days sitting in a hotel room trying to solve these very big complex puzzles for 12 hours a day.
- 10:05
It was great. I loved it. The thing is, I was throwing every single model I possibly could at this. 5 6 solo on max, fabled 5 on max, just grinding these things down trying to get them to solve these things, and they just couldn't. The hallucinations were bad, they would get lost, they couldn't figure these things out. They were quite useful for building out, you know, custom tools for us, but they couldn't actually solve
- 10:25
these problems on their own. So, naturally, as soon as I got access to Astra, the first thing I did was send it off on these puzzles and see how many it could solve. And it solved many more than any model has been able to. It's not at the point where it can solve all of them. Like this ducks puzzle down here where you get seven picture frames with some ducks in them that you have to decode a message out of. Could not figure that one out. Ran for about 6
- 10:45
hours with no success. It did end up getting the cubes puzzle and the dress puzzle. I don't want to spend too much time going into those. If you want more details, I have a whole video on that. I think it was pretty fun. The tricky thing about trying to have this model solve these puzzles is that the solutions for them are already technically poisoned. I've made some videos on this. Some other creators have made videos on these. So, the solutions
- 11:06
are out there. And as a result, on the first run I did, it ended up solving these pretty much instantly. I don't have it documented here for the dress, but it did just find my video, grab the solution, and then present it within 3 minutes. So, didn't actually figure it out. The cubes one, it found the official clue and was able to solve it, but it didn't find any solutions floating out on the web, which was good.
- 11:27
The way I was able to address this and fix this and get an actual test run out of it is I built this full like custom container system to make an isolated sandbox for running the puzzles. I had the agent running within a container here that was locked down. It was normal Codex running with GPT-6 Astra on max reasoning. I told it to use as many sub-agents as it wanted to. It was
- 11:48
completely unbounded. The one constraint I did put on it is that it was only allowed to make requests through this special gateway I had, which had a block list of cheating URLs. So, if it tried to request the solution from a place where it already exists, that would just get hard blocked. It couldn't do that. It could only do normal web searches. I also had a system set up for logging and auditing the sessions. I was pretty
- 12:09
consistently keeping an eye on what it was doing, how it was working. I actually had some custom dashboards built out for this, which were quite nice. And the only real feedback the model could get during these runs was this judge, which was an API it had access to where it could submit a potential answer, see if it worked, and then that was it. It just had to figure it out on its own. And when given these constraints, it was able to solve these
- 12:31
two puzzles with the official hints that the puzzle creators gave to all of us on like day three of the competition. The model was pretty lost going down bad paths, couldn't do it. But as soon as it got the hint for both cubes and dress, it was able to reorient itself, figure out what was going on, and get the solution on dress within like an hour and a half, which is pretty impressive. For the cubes on one official clue run,
- 12:53
it got it in two and a half hours. Then on this one, it was running for a while without the hints. Eventually, I was like, "Yeah, sure, I'll give it the hints." I gave it to it, and it got it after about five and a half hours. What was really cool about this is the way I actually coordinated this run was with this agent swarm architecture thing. Earlier this summer, there was an incident with a certain hugging face
- 13:13
company and OpenAI models maybe escaping containment, making a message board, and hacking into their servers to try and cheat on exam. I kind of used that architecture as inspiration here, too, coordinate the agents in such a way that they could all run together to accomplish this big very difficult goal cuz that's one of the new things that this generation has unlocked is this
- 13:35
swarming ability. The coordinator's job was entirely just to send off sub-agents or even full other Codex threads to try and solve different parts of the puzzle that it thought were important. They would all work together on this shared message board where one of them would have a theory, it would put it on the board, the others would look at it, verify it, come to a consensus, and then
- 13:55
submit a solution. The coordinator's job also was just to keep things moving and running and stop doom loops from happening. I initially was having some trouble especially with these other puzzles here, these Minerva Lux puzzles, of which it only actually solved 11 out of 15. Still a very impressive score, and when you factor in the one on here
- 14:15
that the team was able to get that the model didn't get, it would be the best score in the world on this set of puzzles. The issue was starting out that the sub-agents would go for way too long. So, when I added in this timer for lack of a better word, like I would put a hard clock on it where I told the primary agent, "Hey, you can let these research threads go for an hour, then you need to check on them, and then if they hit two hours, you need
- 14:36
to kill them." So, that kind of put these things on the clock and forced them to keep iterating through ideas and driving forward towards a solution, not getting stuck in a rut. And that ended up allowing it to solve problem four, which was a very difficult one. This was an aggressive loop that took a very long time and a lot of tokens, but in the end it did burn it down and get that solution. I ran the unsolved ones for a
- 14:57
very, very long time, probably 40 hours each, and I just it couldn't figure them out, it couldn't get there, and I eventually just stopped it cuz I needed the computer for other stuff. Uh you can see over here the actual Teal Draw thing I had pulled up in Helium that I was showing earlier. This was actually created by Astra using a combination of computer use and the Teal Draw SDK. It put together all of this for me. I've
- 15:18
had this crazy Def Con health thread running for I think since I got access to this model. It's been going for a long time, compacted absurd numbers of times, and it's still very coherent over that long time horizon. There were some annoyances with Soul especially where it would kind of lose some important context during compaction, but this one just it's borderline perfect the entire way through. It has no issues compacting
- 15:40
context, really good at that. Another little thing that I think is worth noting is that in this coordinator swarm architecture, part of the reason I think it works so well is that this model is incredibly good at writing prompts for other models. I read the initial kickoff prompts that it wrote for the coordinator on this. I didn't have to change it. For most models, I would have had to correct some things, make it
- 16:00
simpler and more specific, but now with this one it just looked good right off the bat and it did the job. I even had some instances where I needed to pause these runs to do some updates on that system, and I was watching the messages it was sending to other Codex threads where it would tell them to pause and stop their work. The thread would be taking a little bit too long to do it, and it would send a follow-up message cuz it was watching and be like, "Hey,
- 16:20
stop. No, seriously. Do it quickly now. It's fine. You don't need to overcomplicate this." And then the model would spin down and do the thing. Really powerful coordination stuff is available here. You can also see down here in the research branches some of the messages that were getting sent back and forth. New fixed seven-digit pattern has one exact catalog match 364568617.
- 16:41
Independent parse confirms all values, all memberships, whatever it was doing. I wasn't looking too closely, and this was the correct answer. And this was after doing tons and tons of research passes. It ran for quite a long time, and again, the coordinator was just sending these things off, letting it do its job, and it ground down the solution. This unlocks the ability to
- 17:03
solve much bigger, more complex, long time horizon tasks that will unlock a lot of those AGI type tasks that everyone wants to see. This is how you do things like finding novel math proofs. It's with this new swarming architecture that is very dangerous, but also very powerful. And I think outside of this model, Fable is the only one
- 17:23
that can really come close to this level of capability. It's quite remarkable. That does, however, lead me to one of the inherent downsides of this model, which is, of course, the price. It is a new tier above Sol 1.0 and Luna that now costs $10 per million input tokens and $50 per million output tokens. It is not a cheap model. While we don't know how
- 17:44
this is going to impact the Codex subs, you know, like what kind of usage are you going to get every single week? Even though the raw token price of something like GPT-5.6 Sol is much higher than Gemini 3.8 Flash, it still ends up being cheaper on most tasks at medium reasoning than Gemini does at medium reasoning because Gemini does so many more steps and burns so many more
- 18:04
tokens. Astra still maintains this efficiency. Like if you look at Astra on high reasoning, it's about the same price as Grok 4.6 on X high reasoning. And again, this is a much cheaper model per just raw token cost cuz when you look at agent steps, Astra is doing so few steps. Like it's doing 28 steps compared to Grok's what? 87 steps? It's
- 18:25
insane. The problem is in real-world work, it actually used a lot more tokens and a lot more steps because it's doing things that these other models wouldn't do before. These other models would not open up a full computer use view and test the thing in macOS to make sure it's working. They wouldn't remote into another box and make sure that the stream is working between the two of them or test the actual visuals of a
- 18:47
website on all the different screen sizes to make sure everything looks crisp and good. The outputs are much better, but when you combine the inherent increased steps from this new capability and the raw token price, it's not cheap. I had it calculate out my full token usage across all my devices and projects over this testing window and the total was about 100 billion
- 19:08
tokens, which if you you API pricing, would be almost $200,000, which to be clear, during early testing they give us unlimited access, which is very fun to allow for testing things like this, but you know, that is not feasible for a $200 a month plan. If you look at the actual breakdown for my work, the puzzles just incinerated tokens. I did 85 billion on the Def Con
- 19:31
puzzles just trying to grind all of those down, especially the ones that were unsolved. It just went for days on end with no real solution in sight. For my normal work, which was still a ton of very useful, valuable work, it was about 11 billion tokens or on API price about 17 grand. Now, if you do go down to this project breakdown, it feels more reasonable. Wave Crest is one of the
- 19:51
main real projects I was working on with this. This is an internal tool for tracking YouTube analytics, deep retention breakdowns, comments, sponsor tracking, all of that stuff that we need internally for my channel and 10x and in general. I only did about 2.2 billion tokens, and that was for some very, very heavy work. It got some incredibly valuable features out of that. And if I could just get that out of the $200 a
- 20:13
month plan, I don't think it would be a problem. Pick thing is also pretty impressive at just 351 million tokens. For the amount of stuff I got done with this, the new homepage and a bunch of bug fixes and back end changes, more reasonable than I honestly would have expected. I'm really hoping we end up in a place where the usage limits don't become a huge problem with this model like they are for something like Fable
- 20:34
in the Cloud sub, where you can only get 50% of your weekly limit on Fable and it evaporates almost instantly. I think we might be better off here, but this is a time will tell type thing. The only thing I am certain of is you are definitely going to have to get used to the slowness of this model if you're using it at higher reasoning levels. If you've been around since 5.5, you might remember me going really hard about use
- 20:54
5.5 on low reasoning, not because I think it's the best possible version of the model, but because it's a really good version that's fast and cheap and more effective than people think. I think low reasoning people are just naturally scared of and they steer away from, when for a lot of tasks, like anything that is search related or it's a clear bounded task or you're asking questions or just like using it to work
- 21:16
with your computer or stuff on your network, low and medium reasoning are really good options and I would definitely kind of default to those and when you need more power, that's when you juice it up. Because when you get into this high reasoning, ex high reasoning and ultra reasoning, this is where it starts doing that verification in the browser behavior, where it will really test stuff out, be super
- 21:36
thorough, go super hard, get great outputs, but at a cost of cost, obviously, and inherently just time. It takes a lot longer for this model to do basically anything. I've had a lot of times while testing this model where I'm just kind of sitting here with five big threads going, just waiting for something to happen. It's hard to get a tight feedback loop on this thing unless you're using low reasoning, which once
- 21:57
again, is way better than you think it is. After seeing the Gemini scores, I'm a lot more skeptical of deep SWE, unfortunately. I don't know how reliable this can be. I really will miss having this as a bench I could really trust. Just going off the numbers we have here, low reasoning on Astra is more powerful than 5.6 Soul on medium and just below
- 22:17
high reasoning on Soul. And then if you bump up to medium, you are completely clearing the capabilities of Soul. So I think just try out the lower reasoning levels, especially if you're not on the $200 month plan. I think you'll get a lot more value out of this thing than you would just pushing it to the absolute max, which definitely does have diminishing returns. I have noticed it doing a little bit of the overcomplicating of things. High reasoning feels like a nice middle
- 22:38
ground for most tasks and then you just tell it to spin off sub agents and it'll do it very nicely. The last big thing I want to talk about is the model's behavior. We just posted our Nerd Sniper episode on the new model, where Theo and I basically just yelled at each other for 2 hours straight cuz we have very differing opinions about how this thing should actually work and feel. I personally really like how this model
- 22:59
feels to interact with. It feels like a model that is perfectly made for me, but a lot of that has to do with the way I prompt and do things. I'm super explicit and clear in my prompts. I don't leave any ambiguity. I tell it where I want it to go, what I want it to change, how I want it to work, and even little things like if I wanted to commit or push or make a PR or merge or none of those, I always make it very clear. And when you
- 23:21
are clear with this model, it will do exactly what you say. It's got more of that Fabelism where it can better interpret the intents behind your prompts. It's inference of what you want it to mean is leagues beyond something like Soul. Even just as a very recent example, the prompt that I sent while recording this video here, "Can you pull up the spell video for one of the Minerva puzzles, one which shows some useful logs and info and state to
- 23:42
visualize? Get me the link." When I didn't tell it which one to grab, the one it grabbed is the exact one that I wanted it to grab this puzzle for viewer. Because if you go back and look at all of the different runs we had here, puzzle four was the one that looped for the longest and had the most aggressive agent system in it and the most interesting things to look at. I didn't have to tell it that. It just intuited that out and got me the one that I really wanted. I'm going to use a
- 24:04
diagram here to explain a lot of the behavior and weirdness around it because, as I said, you have to be very explicit. And when you prompt it something like, "Go fix X and Y on turn one." It's going to start here and it will end with X and Y are fixed. It's going to do that great and it's going to stop right here. What happens in between here? Usually, it follows the path nicely and does the things I want it to.
- 24:24
Occasionally, it will fly out of bounds and if you give it any lack of constraint on things, like if you don't explicitly tell it, "Hey, I want you to use this." Or, "Hey, I don't want you to change this." It will start making more assumptions and making more changes, very similar to something like Fabel. Fabel will just kind of infer intent and go off and do the thing that it believes you want, which it's often right. It's
- 24:45
very big and smart. But with this model, the start and end are pretty clearly bound, which I again love because it is so predictable. If I see a prompt that looks like this, I know it's going to end here without having to read any of the outputs. Then when you go to turn two and you're like, "Okay, cool. Commit and push and make a PR to main." It will commit it, push it, make the PR to main, and stop. That's all it's going to do.
- 25:05
It's not going to review it. It's not going to look at any of it. It's just going to do exactly the thing you say and stop. Then on turn three, if you say, "Okay, babysit the PR to green." Then all the PR review bots are handled. It will loop and do the thing. My skill for babysitting PRs is very simple and explicit. I have the description here to use it when asked to babysit, watch, or
- 25:25
follow a PR. I really like this hack with skills where you basically give it keywords so that if I say the word babysit involving a PR, it'll just know, "Oh, I should invoke this skill now." And it does a good job invoking the right skill. Then once it's in here, it gets these very clear instructions that, "Hey, when I say babysit the PR, what PR am I talking about?" I very clearly define what that PR is with if you're in
- 25:46
a branch with a remote PR on it already, that's the PR. And if the user includes a specific PR on their request, then you babysit that one. So, clearly defined what the PR is. Then I want to define what the end state is. So, the end state is that the PR has no merge conflicts with the target branch. The CI checks are green, and the AI code review bots are all green. The reviews have been addressed either by implementing them or
- 26:07
ignoring them because they're not worth addressing cuz sometimes when you're working with a really smart model like Astra or Fable 5.1, you don't have to take all the reviews. Most Some of them just don't matter cuz they're using dumber models under the hood. Then finally, I have the steps to take which starts with a very clear step one, watch the PR, pull the review bots, check their full findings, decide whether to skip and follow. I have some references
- 26:28
on like how I want the bots to be treated. Then for step two, when issues come in, fix them locally, commit and push them up. Again, explicitly defining the steps here. Wait for the checks to run again. If there are more issues, repeat step two. Otherwise, move on to the next step. And step four is give the user a concise summary of the changes you made to fix the PR and a concise list of things that the PR actually does. When you have this very explicit
- 26:50
set of steps, the model will follow those very explicit set of steps, but you have to do that. Any vagaries will lead to unfortunately unpleasant circumstances that you don't want to deal with. The implicit behavior here is follow instructions. So then finally on turn four, merge the PR, the PR is merged. This is how I like my models to work. I definitely can see arguments for
- 27:10
where there are default behaviors that might be nicer for some people. If you tell it to go fix X and Y and it's in a big maintained code base with clear patterns of making a PR and reviewing it and all that stuff, maybe it does make sense for it to proactively take that next step and move from this step all the way down to this step just off of this prompt. I can maybe see that making
- 27:30
sense. And if you want this model to behave that way, you can just add that to your system prompt and it'll do it, but it will not do it out of the box. You definitely need to be prepared for this. If you're not prompting it well, it will lead you to bad places, but again, as I've hopefully showed through this video, if you really give it clear instruction and work with it carefully, you'll get great results. Oh, and I forgot to mention this earlier, too, but
- 27:51
the actual writing and voice of this model is leagues better than anything else I've used. GPT 5.6 was like it was fine to read, but very robotic and awkward and liked using big jargon. This guy's outputs are great. Like Puzzle Force viewer is now showing the save solve run, 326 submissions, 90 activity entries, and recent swarm
- 28:11
reports. It took 23 hours and 31 minutes to solve. I'm opening the private viewer link. All solvers remain stopped. This is very clear. I know exactly what's going on here. It's great. Very simple, plain, good English, and even some of the copy that it writes on websites. Like even this copy on Pick Thing, remove backgrounds, organize your assets, and get back to creating. It
- 28:31
came up with this. This is just what it did in the one shot, and I don't hate it at all. I don't know how they pulled this off, but this is an Open AI model whose outputs I actually like reading and looking at. I would never do like actual writing or posting with AI writing, but using it for outlining and ideation and just interaction much better, much nicer default model. And I really hope that when we eventually get
- 28:53
GBT-6 Soul, GBT-6 Terra, and GBT-6 Luna, a lot of these pros enhancements trickle down to these models so that when you're in like chat.openai.com, you get the benefits of this much clearer, better writing. I still don't love interacting with these older models, especially after getting used to this new one. So, that's how I'm feeling about GBT-6. I love this model. I think that the
- 29:14
description of calling it a baby or proto AGI is kind of close to accurate. I could definitely see a few more iterations of this, maybe another bigger base and some more RL in time. We could get some truly insane stuff from this line of models. I am very confident in saying that OpenAI is squarely in number
- 29:35
one right now. Obviously, we don't know what Anthropic has internally. We know what every other lab has internally cuz it's what they have externally. None of the labs except for OpenAI and Anthropic have any incentive to keep models hidden behind closed doors. They just need to get them out so that they can look and feel competitive. Like having Muse Spark topping benchmarks is really, really good for them. Having Fabled 5.2 come
- 29:55
out and be like 5% higher on the benchmarks doesn't get them anything. So, they're definitely sitting on some stuff. I know OpenAI is being really good about trying to get this stuff out as much as possible. This is something that I've just hated for a long time. I hate the not everyone having access to a model. I think it's really shitty and bad. This definitely needs to get in everyone's hands as soon as humanly possible, not just a select chosen few.
- 30:16
It sucks. It's bad. I know better at this point than to try and predict where all this is going. All I know is that it's pretty damn cool right now and you just go build some crazy stuff because the things that are going to be unlocked and created with this over the next couple years are going to be truly remarkable. If you enjoyed the video, make sure you like and subscribe and I'll talk to you again soon.
Description
GPT-6 Astra is my favorite model of all time and it's not even close. This thing has absurd capabilities in computer use, swarming, and more that make it truly second to none (even if it has some downsides)
Thank you Firecrawl for sponsoring! Check them out at: https://davis7.link/firecrawl
My X, Discord, and more: https://davis7.sh/
