OpenAI and Anthropic Finally Have Competition
BDBen Davis@bmdavis419 · 45.9K subscribers
- Views
- 13.9K
- 13,918
- Outlier
- ×0.7
- Median 21.4K
- Length
- 26:05
- Published
- Aug 20
- Thursday
- Comments
- 46
- 443 likes
Script
How it's written
- Words
- 5,826
- Words a minute
- 223
- Reading time
- 25 min
- Segments
- 18
The hook
0:00–0:25You know, I've often lamented it really sucks that only Anthropic and OpenAI felt competitive for the longest time. It really just didn't seem like any of the other labs could even get close to competing. But that has finally changed because SpaceX AAI, at least for right now, with Gro 46 coming out, cursor getting really good, the new Grockbot stuff they're doing, the origin code hosting, all of these things together, have made it so that I finally have
Structure
You know, I've often lamented it really sucks that only Anthropic and OpenAI felt competitive for the longest time. It really just didn't seem like any of the other labs could even get close to competing. But that has finally changed because SpaceX AAI, at least for right now, with Gro 46 coming out, cursor getting really good, the new Grockbot stuff they're doing, the origin code hosting, all of these things together, have made it so that I finally have something other than codeex or cloud code that I want to use every single day. I also want to really talk about the current competitive landscape. If you want to see that, make sure you stick around to the end. Now, let's start with Grock Force 6. I overall really like this model. I've enjoyed working with it and has a lot of very good characteristics that I haven't seen from Grock models before, but it's kind of a double-edged sword model where this feels like an interim type step where Grock 47 is probably going to be much much better. It improves on Grock 45 in a lot of ways, but it also regresses in some other ones. If you look at it on a benchmark, like the main benchmark I always use, deep SWE, it benches way better than Gro 45 did. It's down at 54 versus Gro 46 is all the way up at 67. Really improved here. But you can already instantly see there is some weirdness with this curve because the medium reasoning seems to be benching the best out of all the different levels. Like it actually dips on score with high reasoning. X high is about the same but not quite as good. So there's some weirdness with that. It's also a pretty big regression in efficiency of the model. So looking at cost, it is
Watch from 0:00just slightly more expensive than Grock 45 at 550 a task versus $242 a task. When you look at output tokens, it regresses even more up to 71k for X high versus 36k for high from before cuz I think that's the max the last one went up to. But when you get into agent steps, this is where it really starts to get painful. 61 steps on high versus 70 steps on medium. There is definitely a um I don't want to call this benchmaxing, but this is more just kind of the thing a lot of models used to run into. Claude models still often run into. The GVT models have really gotten better about this. They've been getting more and more efficient over time. But when you're looking at the price and speed of models, it is really important to not just look at the raw here's the input and output token prices. If you just look at that, something like Grock 45 might seem way more cost effective, but then when you look at it on the cost graph, the medium reasoning version is the same price per task as the high reasoning version of GPT56 salt. And the reason for that is even though the tokens are way more expensive since it has to do less work and it's more capable of just getting to the right answer faster, that ultimately means that over time you end up having the bigger, more expensive model actually be the cheaper, sometimes faster model. The efficiency is 100% the biggest regression as compared to Grock 45. I really hope that it's able to do the thing that GBT models have been doing where they get further and further up and to the right on this chart, just more efficient over time. But at least for now, I will take the trade-off since it is a huge improvement. Like it is noticeable in day-to-day work. The quality of output you get from Grock 46
Watch from 1:30is much higher than that of Grock 45. It runs for much longer periods of time, much more effectively. It's able to handle sub agents much better. It has a better understanding of its environments and can solve problems better. Still doesn't quite have that crisp feel that something like GBT 56 or Fable has, but it can get there and it's just a slightly higher effort version of those two models. These are the three big closed source models that I find myself using day-to-day. Fable is the easiest one to use because you just don't have to put that much effort in to get the actual output you want. Then the second easiest to use is GPT56 where if you don't direct it well enough, it will just end up vomiting out obscene amounts of overcomplex Python looking code. There's a real quality problem there that can be solved, but it just requires more effort, more tuning of skills, more tuning of the prompts, really putting effort into reading and working on the codebase. Fable just naturally figures that stuff out. 56 doesn't. And Grock is even worse about this. You really have to make sure that you are directing it properly. Otherwise, it's just not going to get you what you're looking for, especially on long horizon tasks, which is what a lot of agent work is these days. One of the things that everyone loved about Gro 45 was just how insanely fast it felt. You could run something and really be in the loop and basically instantly get an answer. It's really not the case anymore. The tokens per second is still decent. 49 to 54 TPS on Open Router compared to 56 souls like 41 to 43 TPS. The point is, it doesn't feel instant. And most of that is just because of the efficiency losses they had here. Like if you look at the Gro 45
Watch from 3:02charts here, it's 52 to 45 tokens per second on Gro 45. So like the TPS didn't really change, but the feel day-to-day has gotten noticeably slower since again it has to do more steps to get a better result, but still it's more steps, more time, more cost. You get the idea. Lately, I've been building so much more, making so many more PRs to the point where CI is actually becoming a real bottleneck for me. GitHub just isn't cutting it. Which is why today's sponsor, Blacksmith, makes so much sense. It is a oneline change to your GitHub actions that results in you getting 2x faster hardware, 4x faster cache downloads, and 40x faster Docker builds. It is insane. There are humongous companies like Mercury, Subase, Ashb, Clerk, and many more using Blacksmith for a reason. Actually, including T3 Code. All of T3 Codes builds go through Blacksmith because they are just so cheap, so fast, so good. I I don't really know what else to say. If you're working on any even remotely serious project, you're going to have a ton of CI. And if you have CI, you're just burning money by not using Blacksmith. And outside of GitHub actions, the new Codesmith background coding agent is really good. You can set it up in Slack or you can also just tag it in from GitHub or the web UI. And once you have it set up, it can help with planning, implementing, optimizing code, also autofixing PRs, which is incredibly useful. the amount of like autonomous loops I have running for PRs at this point where I will have CI run, some random thing fails, it's really useful to just have an agent instantly be spun off without me having to do anything, make the required fix, and just update the PR accordingly so that by the time I go look at it, everything
Watch from 4:34is already exactly where I want it to be. Blacksmith will improve literally everything about your GitHub actions and CI experience at dav7.link/blacksmith. Regardless, doing the full cursor setup and using this thing a bunch for some real work over the past few days has felt quite nice. I am really liking the full experience they've got here. It took quite a while. Cursor was in a really rough state like four or five months ago. The first release of their glass version, which is this desktop app, just was not there. It had horrible, horrible performance. There were a bunch of weird bugs. It just felt very halfbaked. But I think this is a lot of just kind of the way that company and SpaceX in general works now is they will ship something that is good, but then they will just iterate rapidly over and over again to where 3 months later it's in a really good state and it is now in a very good state. When you combine it up with their like cloud agent environments, I've been very happy working with this. They're very clearly trying to build a full ecosystem with this. Like uh the origin cloud hosting, which is their GitHub competitor, just dropped in beta. I think it was yesterday. The first impressions are mostly good, but this is clearly a beta. It is not there yet. I have it synced up and working in my organization and repos. Uh like the two that I'm mostly working on right now, Wave Crest and A Drive. If I go into this guy, it takes a moment to load the like first thing. But once I have this loaded, if I go into apps and then web and then the package JSON, like the navigation here is so fast. It feels so good having gotten used to GitHub's garbage speeds and UI and everything. It's a very promising
Watch from 6:05product that I want to see them iterate a ton on. There's even little things like when you go into the pull request tab, it will show link GitHub to cursor, but I have linked it half a dozen times. I remove the integration, readded it. You can see it just keeps like refreshing over and over. There's just something broken on the back end. This is not what I would migrate all my repos to yet. But since it is currently set up to mirror from GitHub, I'm totally fine with using this as kind of just a better UI for viewing my repos that maybe over time we can get a uh full better experience that I can fully migrate off GitHub because the reliability there has been atrocious lately as I'm sure you've probably run into. I want to talk about Grockbot as well. It is not a super heavy dev product the same way Cursor is. Like when you use it, you can feel this is definitely intended to be used by a much wider range of people. I still don't think that this is quite everybody friendly. Like I could not see my mom using this quite yet, but it is getting much closer to something that could potentially be adopted by the masses. And I really like the new organization system that they put together here. I have it on my machine. I also have it on my phone. I wish I could show mine, but unfortunately since the way you use this thing is mostly for non-code work. So I have to manage six different emails, two different Slacks, bunch of Google Drive things, bunch of Notion stuff. I have many admin tasks that have to be done.
Watch from 7:37The way I was doing those before is mostly through Hermes agent, but this is kind of just a more polished version of that. We had the first iteration of these things which were the coding agents. Then they took the coding agents and built this crazy more proactive assistanty type harness with OpenClaw. That was a huge innovation there that then spawned Hermes, which is in a lot of ways a more polished version of Open Claw is kind of the way I think about it. And now we have this Grockbot which is a closed source full application much less customizable than something like Hermes but also right out of the box a much better experience than Hermes because just going into Discord to manage all your threads is functional but it's not nearly as nice as having a bunch of different agents cuz the whole thing isn't set up to be a bunch of different threads where you make a new chat, send off a prompt, do the thing. It is a bunch of persistent agents like you could have an inbox manager. I have something very similar to this where it has a system prompt in there which defines what it should be doing. It can also define tasks within that. So recurring crons that go off every single day or every week or whatever. It also has the full cursor cloud sandbox environment thing attached to it. So if you have some like browser type task, you can just log into that on that agent's computer, have that all set up, and whenever it needs to do that, it can just use the computer. It's very similar to the Cursor Cloud stuff cuz I think they actually developed this in parallel with the Cursor desktop app. This is very much a Cursor product even though it has Grock on it. This came from Cursor. I remember when the Cursor acquisition was announced at that big $60 billion price tag, which at the time most people thought would just kind of
Watch from 8:54be for the data and the training, which a lot of it was. Groforce 6 would not have been nearly as good as it is without Cursor's data and expertise. The harness is good, but everything else was kind of fine. But now that all this stuff is really coming together, it feels like they are building a full ecosystem. And this was one of the best acquisitions we have seen in a long time. Curser and Grock and SpaceX, all of these are in a actually quite remarkably good state. I think of as far as the quote product offerings of any of the labs go right now between Grockbot, Cursor's desktop app, the Grock build CLI, the cursor cloud agents, now origin code hosting, all of these things come together to make a cohesive ecosystem that no one else can really compete with at the moment. The pricing on all this is definitely not light. In order to use Grockbot at all, you need to at least be on the $200 a month Cursor Ultra plan. I think the actual best value out of all of these is the $300 a month Gro Heavy. This is what I switch my stuff over to. Reason being, you get the X premium Twitter premium thing since I use that for work. I need that. I was paying for it anyway. Might as well roll it into this. It also has a ton of Gro usage, a bunch of like extra usage for uh different models within Cursor. You get the full Cursor Ultra sub from the Gro Heavy. It just gives you a bunch of stuff allin-one that feels like a pretty decent value. I actually think if you are looking at all of these $200 a month subs and you're trying to figure out which one makes the most sense, if you want to have more diversity in which models you can use, while you will get less usage, the best bang for your buck will always be either the Codeex $200 a
Watch from 10:25month plan or the Claw $200 a month plan. If you want everything, this is the one to probably go with right now. You just have to know that like you will be locked into this ecosystem in kind of in a way surpassed claude with the whole ecosystem thing cuz we always complained about claude code being a very closed ecosystem and them wanting to do the Apple thing where you can only use your claude code sub within cla code itself. They banned open code. They banned all third party harnesses. You can kind of use the SDK for something like T3 code but even that is like a little on the line and someday they will probably do something about that. The only one that feels truly open right now is actually the codec sub. You can use the codec sub literally wherever you want. I use it within a bunch of Hermes agents. I use it in pi. I use it in codecs. I use it even in custom projects. Like they do not care. You can just use that inference wherever you would like to with the cursor or gro usage, whatever you want to call it. This is locked into their services. You can use it in cursor. You can use it in Grockbot. I do think that they have integrations for stuff like Hermes and OpenClaw. I think for the actual Grock usage, they're pretty generous about where you can use that thing. I know that they were really proactive and great about getting the Grock build stuff implemented into T3 code. They drove a lot of that. They've been a very good partner on that kind of thing. So, they are trying to spread out the Grock stuff a lot more, but for like cursor usage, so other models and things like that, you can only use that within cursor itself. The natural question then is like, okay, am I going to move all of my stuff over from codecs and cloud code into cursor and just build entirely into that? Because the one thing I really
Watch from 11:58didn't mention earlier is that in order to build deeply into the whole like cursor cloud setup and that kind of thing, it requires a lot of effort. You have to set your repos and projects up in such a way that they fit into this ecosystem, especially now that we have origin. You got to set up the repo, then you got to set up the cloud environment. You have to manage these things properly, have your projects put together in such a way that it fits nicely into that paradigm. Once it is set up and working, it feels phenomenal. But it is not nearly as frictionless as something like Codex is. Codex, I just like open up a random directory on my computer and tell it to go do something and it goes off and does it. I'm really leaning hard into cloud agency type workflows right now. I think that is the best way to work overall. And there's kind of two competing ways of doing this. The only real options here right now are cursor cloud agents and the Devon cloud agents. Those are the only two that really crossed my bar into I would use this every day. I've heard very good things about AMP's orb system, but I haven't tried it enough personally to have anything intelligent to say about it. So, for now, I'll just talk about this. And once I have a project set up and working in the cloud agents, like I get the environment variables in there, I get the setup scripts ready, I have the project structured in such a way that it makes sense to run in a remote environment, I have the preview environments, the agent MD things, like all of that is ready to go. It does feel quite nice going into my cursor desktop app. I open up a new thread. I'm in the A drive project. I have it set to be cloud. Anything I put in here to this prompt will look like it's running on my
Watch from 13:28machine normally, except it's actually running in the cloud. So instead of doing work trees the way we have been for a long time, the work tree is now just a brand new cloud sandbox spun up for this exact workflow. It is a really good way to work, especially when you're parallelizing things and running things for a very long period of time. I like this a lot. And if you are not going into this other thing I'm about to talk about, I would recommend trying this more heavily and seeing how it feels because getting work off your main laptop, it makes life so much better. I really have gotten to the point where my laptop is mostly just an interface for using my servers that are spread across the normal cloud and then my home server, which is where I am currently running my big T3 code instance. This is the thing that drives most of my stuff these days. I do most of my work here. And the way I have this set up is in my home server setup. I have a framework desktop that is on 24/7. I have tail scale set up on it and I have T3 code running 247 so that I can just go to this URL in the browser, kick off some work, have it run. I also have some agent MD things in there so that whenever it's spins up a dev server or something for a project, it knows that it needs to spin it up so that it can work over the local network. And since I'm tail scaled in 24/7 between my devices, it just sends me a preview URL. I go to it. I can test the thing out. It works really nicely. It's like just having a full big private server that you can use 24/7 as much as you want.
Watch from 14:58And managing these Linux boxes has gone from kind of a pain in the ass to completely trivial with things like Codeex. I do all of my system management between devices just with the Codex CLI. It's great. The problem with the setup too is just the fact that like now you have to buy a whole extra computer to have running at home. You have to deal with the remote management stuff. You really need to set up something like a KVM so that you can remote in and do system updates and recover crashes. You now have to be on tail scale 24/7. You are dealing with the inherent limitations of like it is your own computer and with Linux you can run pretty much infinite agents at this point. The efficiency of that operating system just blows my mind every time. Especially being used to Mac OS where five or six codeex threads and you crash your computer. It's stupid and really annoying. But that's a Mac OS problem, not a Codex problem. Codex runs wonderfully on the Linux machines. again, you have to go buy another computer and hardware prices are really high right now. If your home network isn't super robust, it also makes it a little bit worse. This is a much lower friction version of that new workflow that will just make your life so much easier. I do recommend trying to figure out which version of this works best for you and get that set up and move your work over to that. It really has sped up my development and made me want to use these things more. This probably should be its own video, so I will keep this section fairly brief. The main thing I wanted to talk about here with the battle between the labs is it kind of feels like there is a real dividing line being created right now where at least from the outside kind of feels like anthropic versus the world right now. A lot of this is kind of vibes based but if you just look at the behavior of
Watch from 16:20these companies and the way they act and talk very much all of these companies are focused heavily on getting really good models into as many people's hands as humanly possible. I mean, for meta, Zuck just put out this big article, the future is for everyone. Very long. Actually worth reading in my opinion. Regardless of what you think of Zuck, whether or not this is a purely like his back is against the wall, so he's taking this strong open weight is good and everyone should have access to super intelligence position because he's scared of anthropic. Maybe that's the case. Maybe he just genuinely believes this. Regardless, I like a lot of what he says here, and this is kind of how I feel about this stuff right now, where I want to have the best possible models, and I want everyone to have the best possible models and be able to do really cool stuff with these things. Enthropic does not seem very concerned about anything right now other than their biggest best model scaling as large and powerful as humanly possible. When you look at the Opus 5 release, that was clearly just a distilled version of Fable that was smaller and rldled into hell to just not be very good. Sonnet 5 was a disaster. That thing was pretty useless. Haiku, we haven't had a new Haiku model for almost a year at this point. I doubt we will ever get one again. They just do not seem to really care about anything other than Mythos and whatever is going to come after Mythos. Be it their like model 2 thing that came from I think it was like an investor call or something last week.
Watch from 17:50That big new model that doesn't seem to be getting released to the public. I'm sure we will get a Fable 5.1 at some point, but like they're not that concerned with lowering prices, making things more efficient, getting it in more people's hands. They just want to build God, but all these other companies really seem like they're just trying to get these things to be the most useful possible AIs for everybody. There's two ways of thinking about AI. If you actually understand what's going on right now, there is like the AGI view of the world where AI is going to keep getting better and better, but it is going to always have some fundamental limitation of like you still need a human driving it in order to get anything useful out of it. I experienced this myself last week when I went to Defcon. I had it try and solve a bunch of these really difficult problems that weren't easily verified. And since there was no verification, the models were able to just hallucinate a bunch of garbage mid-run and as a result couldn't actually solve any of them. They solved three out of the 13 puzzles that we were able to solve by orchestrating and running and using them. But we needed to know what we were doing. We needed to drive. And if you've seen any AI code, it can get really far and it can get you a correct solution. But getting the right solution still requires a lot of human intervention and work. Maybe we'll get to a point where none of that really matters and we'll have that other view of the world which is kind of the ASI view like the super intelligence view where AI gets to the point that it is so much better than everybody at everything and nothing else really matters. The model is so good and so big and there's so much compute behind it that it will just naturally figure these things out.
Watch from 19:10Fable is kind of a glimpse of that. It is the easiest to work with model because it has great discernment. It writes great code. It is a very wise model compared to any of these other ones. These other ones require you to be more in the loop, but you still can get really good stuff out of them. Even something like Deep Seek V4 Flash, which I've been running on my DJX Sparks in my home server locally, is actually getting some really good stuff. It just requires more effort from me. I have to be more specific with what I want. I have to give it more direction. I have to check its work more. It's not quite as much of a like fire and forget as the anthropic models. And with this kind of split in the way they're doing things, I think everyone is just trying to kind of stop Anthropic from scaling out and ahead of everybody. OpenAI is the one kind of exception to this rule. They do live kind of in the middle. I think they are the only other lab that has a real chance of scaling up this crazy ASI thing that is like the god machine. The new Astra stuff from everything we've seen publicly on like it solving a bunch of these math problems and the really crazy stuff it can do. Clearly, this is a very powerful model and they actually just announced today that they are pausing the RL job on their next big crazy model because they are concerned about the safety risks. So, they're definitely scaling in there and they're definitely seeing some crazy stuff in there. But, I know for a fact they are working really hard to get their models in everybody's hand. Like, Astra will be publicly released. Sam Holman confirmed that publicly today. They really want everyone to have this stuff. I know that for a fact. This might be an overly cynical view, but I don't think Anthropic cares that much. I think the consumer stuff is more of a means to an end. They really are just concerned
Watch from 20:41about building the best possible AI that is the big god machine. The safety stuff is really important there. It has huge implications. And like honestly, I do kind of get it from their perspective. If you do fundamentally believe that these things can just scale up to the point where they can do everything on Earth better than a human, it does suddenly make sense why none of this other stuff matters. And if that thesis is true, and frankly I don't know where I fall on this line from a more visceral standpoint, I really do fall on the side of like these things are tools. They require a human to direct them. I have a really hard time imagining them being able to make good long horizon decisions without hallucinating to hell. But also, if you look at the amount of progress we've gotten in the last like two or 3 years, like I don't remember when GPT 3.5 came out, but it wasn't that long ago, all things considered. And now we have Mythos. And Mythos is insane. But still, it's not quite there yet. And while I academically understand and can be convinced depending on the day of the week that just scaling will get us there. Bigger, smarter models and like a lot of the questions of like, okay, well, how are they going to get the discernment? How are they going to be able to figure these things out? How are they going to touch the real world? And a lot of the answers of those questions you'll get is just, well, you make the model smarter and bigger. Yeah, like honestly, I can see that working. Like I I don't know what the implications of at or above human genius level intelligence that is completely unbounded by time will be. Like maybe capabilities will keep being super spiky, but even if the capabilities are spiky, if you throw enough compute and enough agents at it, over time they'll just kind of naturally
Watch from 22:13whittle these problems down and create the thing that you're looking for and just naturally improve over time. I don't know how this is going to work. It's going to be very strange. I would love to live in this world. I'm concerned we there's a very good chance we'll live in this world, but time will tell at least for right now where we stand. The ranking of the labs is very clearly OpenAI and Anthropic are in a league of their own number one. Then SpaceX is pretty clearly the number two spot. The Chinese labs are down below them depending on the day of the week. They could fluctuate a bit. I think Kim K3 is overall the most powerful one we have there. But Deepseek V4 Flash is an awesome model. V4 Pro looks pretty good, but I don't have a huge amount of experience with it. GLM 5.3 once the weights come out for that one. I'm very excited to see what that can do. Meta looks promising. They seem to be going in the right direction and I really like how fast their new model is and it does actually feel different which is dope. Not there yet. Has potential. Honestly, the way I felt about XAI 4 months ago was basically how I feel about Meta now. They have a lot of potential. SpaceX is realizing that potential right now. Meta could get there. Google um they uh how do I put they they exist um like you know the the Je Gemini 3.7 Flash came out. I'm sure we're all using a lot of that right now. Um yeah, we're just going to we're going to put that way down there. Yeah.
Watch from 23:43Yeah. This feels good. I I think I'll just leave it here. I don't know. Like am I going to keep beating the Google horse? Like it's just sad at this point. Like what am I supposed to say? It's Google. It's going to be a weird couple years here. I I at the end of these videos I always like I wish I had something more concrete to say. I just don't on where all this stuff is going to go. All I know is that right here right now the things that I'm using are a lot of cursor cuz I really like that whole ecosystem. Grockbot feels great. Codex desktop app is still excellent. Claude Code CLI Fable is still a really powerful model when I just want highquality code generated without having to think too hard about it. T3 code in my whole remote setup also feels really good. Open weight models are also getting to the point where like I'm really enjoying using them. I've been testing out Deep Seek V4 Flash with the Deep Seek harness all running locally and it is dope. It is so cool to see like an Opus 46 level model running on my network at over 100 TBS. There's a lot of cool stuff happening right now. If you enjoyed this video, make sure you like and subscribe.
Watch from 25:07Full transcript
- 0:00
You know, I've often lamented it really sucks that only Anthropic and OpenAI felt competitive for the longest time. It really just didn't seem like any of the other labs could even get close to competing. But that has finally changed because SpaceX AAI, at least for right now, with Gro 46 coming out, cursor getting really good, the new Grockbot stuff they're doing, the origin code
- 0:21
hosting, all of these things together, have made it so that I finally have something other than codeex or cloud code that I want to use every single day. I also want to really talk about the current competitive landscape. If you want to see that, make sure you stick around to the end. Now, let's start with Grock Force 6. I overall really like this model. I've enjoyed working with it and has a lot of very good characteristics that I haven't seen
- 0:43
from Grock models before, but it's kind of a double-edged sword model where this feels like an interim type step where Grock 47 is probably going to be much much better. It improves on Grock 45 in a lot of ways, but it also regresses in some other ones. If you look at it on a benchmark, like the main benchmark I always use, deep SWE, it benches way better than Gro 45 did. It's down at 54
- 1:05
versus Gro 46 is all the way up at 67. Really improved here. But you can already instantly see there is some weirdness with this curve because the medium reasoning seems to be benching the best out of all the different levels. Like it actually dips on score with high reasoning. X high is about the same but not quite as good. So there's some weirdness with that. It's also a
- 1:25
pretty big regression in efficiency of the model. So looking at cost, it is just slightly more expensive than Grock 45 at 550 a task versus $242 a task. When you look at output tokens, it regresses even more up to 71k for X high versus 36k for high from before cuz I think that's the max the last one went
- 1:46
up to. But when you get into agent steps, this is where it really starts to get painful. 61 steps on high versus 70 steps on medium. There is definitely a um I don't want to call this benchmaxing, but this is more just kind of the thing a lot of models used to run into. Claude models still often run into. The GVT models have really gotten better about this. They've been getting more and more efficient over time. But
- 2:08
when you're looking at the price and speed of models, it is really important to not just look at the raw here's the input and output token prices. If you just look at that, something like Grock 45 might seem way more cost effective, but then when you look at it on the cost graph, the medium reasoning version is the same price per task as the high reasoning version of GPT56 salt. And the reason for that is even though the
- 2:28
tokens are way more expensive since it has to do less work and it's more capable of just getting to the right answer faster, that ultimately means that over time you end up having the bigger, more expensive model actually be the cheaper, sometimes faster model. The efficiency is 100% the biggest regression as compared to Grock 45. I really hope that it's able to do the thing that GBT models have been doing
- 2:49
where they get further and further up and to the right on this chart, just more efficient over time. But at least for now, I will take the trade-off since it is a huge improvement. Like it is noticeable in day-to-day work. The quality of output you get from Grock 46 is much higher than that of Grock 45. It runs for much longer periods of time, much more effectively. It's able to handle sub agents much better. It has a
- 3:09
better understanding of its environments and can solve problems better. Still doesn't quite have that crisp feel that something like GBT 56 or Fable has, but it can get there and it's just a slightly higher effort version of those two models. These are the three big closed source models that I find myself using day-to-day. Fable is the easiest one to use because you just don't have to put that much effort in to get the
- 3:31
actual output you want. Then the second easiest to use is GPT56 where if you don't direct it well enough, it will just end up vomiting out obscene amounts of overcomplex Python looking code. There's a real quality problem there that can be solved, but it just requires more effort, more tuning of skills, more tuning of the prompts, really putting effort into reading and working on the
- 3:52
codebase. Fable just naturally figures that stuff out. 56 doesn't. And Grock is even worse about this. You really have to make sure that you are directing it properly. Otherwise, it's just not going to get you what you're looking for, especially on long horizon tasks, which is what a lot of agent work is these days. One of the things that everyone loved about Gro 45 was just how insanely fast it felt. You could run something
- 4:13
and really be in the loop and basically instantly get an answer. It's really not the case anymore. The tokens per second is still decent. 49 to 54 TPS on Open Router compared to 56 souls like 41 to 43 TPS. The point is, it doesn't feel instant. And most of that is just because of the efficiency losses they had here. Like if you look at the Gro 45
- 4:34
charts here, it's 52 to 45 tokens per second on Gro 45. So like the TPS didn't really change, but the feel day-to-day has gotten noticeably slower since again it has to do more steps to get a better result, but still it's more steps, more time, more cost. You get the idea. Lately, I've been building so much more, making so many more PRs to the point
- 4:54
where CI is actually becoming a real bottleneck for me. GitHub just isn't cutting it. Which is why today's sponsor, Blacksmith, makes so much sense. It is a oneline change to your GitHub actions that results in you getting 2x faster hardware, 4x faster cache downloads, and 40x faster Docker builds. It is insane. There are humongous companies like Mercury, Subase, Ashb, Clerk, and many more using
- 5:15
Blacksmith for a reason. Actually, including T3 Code. All of T3 Codes builds go through Blacksmith because they are just so cheap, so fast, so good. I I don't really know what else to say. If you're working on any even remotely serious project, you're going to have a ton of CI. And if you have CI, you're just burning money by not using Blacksmith. And outside of GitHub actions, the new Codesmith background
- 5:37
coding agent is really good. You can set it up in Slack or you can also just tag it in from GitHub or the web UI. And once you have it set up, it can help with planning, implementing, optimizing code, also autofixing PRs, which is incredibly useful. the amount of like autonomous loops I have running for PRs at this point where I will have CI run, some random thing fails, it's really useful to just have an agent instantly
- 5:58
be spun off without me having to do anything, make the required fix, and just update the PR accordingly so that by the time I go look at it, everything is already exactly where I want it to be. Blacksmith will improve literally everything about your GitHub actions and CI experience at dav7.link/blacksmith. Regardless, doing the full cursor setup and using this thing a bunch for some real work over the past few days has
- 6:19
felt quite nice. I am really liking the full experience they've got here. It took quite a while. Cursor was in a really rough state like four or five months ago. The first release of their glass version, which is this desktop app, just was not there. It had horrible, horrible performance. There were a bunch of weird bugs. It just felt very halfbaked. But I think this is a lot of just kind of the way that company
- 6:40
and SpaceX in general works now is they will ship something that is good, but then they will just iterate rapidly over and over again to where 3 months later it's in a really good state and it is now in a very good state. When you combine it up with their like cloud agent environments, I've been very happy working with this. They're very clearly trying to build a full ecosystem with this. Like uh the origin cloud hosting,
- 7:01
which is their GitHub competitor, just dropped in beta. I think it was yesterday. The first impressions are mostly good, but this is clearly a beta. It is not there yet. I have it synced up and working in my organization and repos. Uh like the two that I'm mostly working on right now, Wave Crest and A Drive. If I go into this guy, it takes a moment to load the like first thing. But
- 7:21
once I have this loaded, if I go into apps and then web and then the package JSON, like the navigation here is so fast. It feels so good having gotten used to GitHub's garbage speeds and UI and everything. It's a very promising product that I want to see them iterate a ton on. There's even little things like when you go into the pull request
- 7:42
tab, it will show link GitHub to cursor, but I have linked it half a dozen times. I remove the integration, readded it. You can see it just keeps like refreshing over and over. There's just something broken on the back end. This is not what I would migrate all my repos to yet. But since it is currently set up to mirror from GitHub, I'm totally fine with using this as kind of just a better
- 8:02
UI for viewing my repos that maybe over time we can get a uh full better experience that I can fully migrate off GitHub because the reliability there has been atrocious lately as I'm sure you've probably run into. I want to talk about Grockbot as well. It is not a super heavy dev product the same way Cursor is. Like when you use it, you can feel this is definitely intended to be used by a much wider range of people. I still
- 8:23
don't think that this is quite everybody friendly. Like I could not see my mom using this quite yet, but it is getting much closer to something that could potentially be adopted by the masses. And I really like the new organization system that they put together here. I have it on my machine. I also have it on my phone. I wish I could show mine, but unfortunately since the way you use this
- 8:43
thing is mostly for non-code work. So I have to manage six different emails, two different Slacks, bunch of Google Drive things, bunch of Notion stuff. I have many admin tasks that have to be done. The way I was doing those before is mostly through Hermes agent, but this is kind of just a more polished version of that. We had the first iteration of these things which were the coding agents. Then they took the coding agents
- 9:05
and built this crazy more proactive assistanty type harness with OpenClaw. That was a huge innovation there that then spawned Hermes, which is in a lot of ways a more polished version of Open Claw is kind of the way I think about it. And now we have this Grockbot which is a closed source full application much less customizable than something like Hermes but also right out of the box a
- 9:26
much better experience than Hermes because just going into Discord to manage all your threads is functional but it's not nearly as nice as having a bunch of different agents cuz the whole thing isn't set up to be a bunch of different threads where you make a new chat, send off a prompt, do the thing. It is a bunch of persistent agents like you could have an inbox manager. I have something very similar to this where it
- 9:46
has a system prompt in there which defines what it should be doing. It can also define tasks within that. So recurring crons that go off every single day or every week or whatever. It also has the full cursor cloud sandbox environment thing attached to it. So if you have some like browser type task, you can just log into that on that agent's computer, have that all set up, and whenever it needs to do that, it can
- 10:07
just use the computer. It's very similar to the Cursor Cloud stuff cuz I think they actually developed this in parallel with the Cursor desktop app. This is very much a Cursor product even though it has Grock on it. This came from Cursor. I remember when the Cursor acquisition was announced at that big $60 billion price tag, which at the time most people thought would just kind of be for the data and the training, which
- 10:27
a lot of it was. Groforce 6 would not have been nearly as good as it is without Cursor's data and expertise. The harness is good, but everything else was kind of fine. But now that all this stuff is really coming together, it feels like they are building a full ecosystem. And this was one of the best acquisitions we have seen in a long time. Curser and Grock and SpaceX, all of these are in a actually quite
- 10:49
remarkably good state. I think of as far as the quote product offerings of any of the labs go right now between Grockbot, Cursor's desktop app, the Grock build CLI, the cursor cloud agents, now origin code hosting, all of these things come together to make a cohesive ecosystem that no one else can really compete with at the moment. The pricing on all this
- 11:09
is definitely not light. In order to use Grockbot at all, you need to at least be on the $200 a month Cursor Ultra plan. I think the actual best value out of all of these is the $300 a month Gro Heavy. This is what I switch my stuff over to. Reason being, you get the X premium Twitter premium thing since I use that for work. I need that. I was paying for it anyway. Might as well roll it into this. It also has a ton of Gro usage, a
- 11:31
bunch of like extra usage for uh different models within Cursor. You get the full Cursor Ultra sub from the Gro Heavy. It just gives you a bunch of stuff allin-one that feels like a pretty decent value. I actually think if you are looking at all of these $200 a month subs and you're trying to figure out which one makes the most sense, if you want to have more diversity in which models you can use, while you will get
- 11:53
less usage, the best bang for your buck will always be either the Codeex $200 a month plan or the Claw $200 a month plan. If you want everything, this is the one to probably go with right now. You just have to know that like you will be locked into this ecosystem in kind of in a way surpassed claude with the whole ecosystem thing cuz we always complained
- 12:13
about claude code being a very closed ecosystem and them wanting to do the Apple thing where you can only use your claude code sub within cla code itself. They banned open code. They banned all third party harnesses. You can kind of use the SDK for something like T3 code but even that is like a little on the line and someday they will probably do something about that. The only one that feels truly open right now is actually
- 12:34
the codec sub. You can use the codec sub literally wherever you want. I use it within a bunch of Hermes agents. I use it in pi. I use it in codecs. I use it even in custom projects. Like they do not care. You can just use that inference wherever you would like to with the cursor or gro usage, whatever you want to call it. This is locked into their services. You can use it in cursor. You can use it in Grockbot. I do
- 12:55
think that they have integrations for stuff like Hermes and OpenClaw. I think for the actual Grock usage, they're pretty generous about where you can use that thing. I know that they were really proactive and great about getting the Grock build stuff implemented into T3 code. They drove a lot of that. They've been a very good partner on that kind of thing. So, they are trying to spread out the Grock stuff a lot more, but for like cursor usage, so other models and things
- 13:16
like that, you can only use that within cursor itself. The natural question then is like, okay, am I going to move all of my stuff over from codecs and cloud code into cursor and just build entirely into that? Because the one thing I really didn't mention earlier is that in order to build deeply into the whole like cursor cloud setup and that kind of thing, it requires a lot of effort. You
- 13:36
have to set your repos and projects up in such a way that they fit into this ecosystem, especially now that we have origin. You got to set up the repo, then you got to set up the cloud environment. You have to manage these things properly, have your projects put together in such a way that it fits nicely into that paradigm. Once it is set up and working, it feels phenomenal. But it is not nearly as frictionless as something like Codex is. Codex, I just
- 13:58
like open up a random directory on my computer and tell it to go do something and it goes off and does it. I'm really leaning hard into cloud agency type workflows right now. I think that is the best way to work overall. And there's kind of two competing ways of doing this. The only real options here right now are cursor cloud agents and the Devon cloud agents. Those are the only
- 14:19
two that really crossed my bar into I would use this every day. I've heard very good things about AMP's orb system, but I haven't tried it enough personally to have anything intelligent to say about it. So, for now, I'll just talk about this. And once I have a project set up and working in the cloud agents, like I get the environment variables in there, I get the setup scripts ready, I have the project structured in such a way that it makes sense to run in a
- 14:41
remote environment, I have the preview environments, the agent MD things, like all of that is ready to go. It does feel quite nice going into my cursor desktop app. I open up a new thread. I'm in the A drive project. I have it set to be cloud. Anything I put in here to this prompt will look like it's running on my machine normally, except it's actually running in the cloud. So instead of
- 15:02
doing work trees the way we have been for a long time, the work tree is now just a brand new cloud sandbox spun up for this exact workflow. It is a really good way to work, especially when you're parallelizing things and running things for a very long period of time. I like this a lot. And if you are not going into this other thing I'm about to talk about, I would recommend trying this more heavily and seeing how it feels
- 15:22
because getting work off your main laptop, it makes life so much better. I really have gotten to the point where my laptop is mostly just an interface for using my servers that are spread across the normal cloud and then my home server, which is where I am currently running my big T3 code instance. This is the thing that drives most of my stuff these days. I do most of my work here.
- 15:42
And the way I have this set up is in my home server setup. I have a framework desktop that is on 24/7. I have tail scale set up on it and I have T3 code running 247 so that I can just go to this URL in the browser, kick off some work, have it run. I also have some agent MD things in there so that whenever it's spins up a dev server or something for a project, it knows that it needs to spin it up so that it can
- 16:05
work over the local network. And since I'm tail scaled in 24/7 between my devices, it just sends me a preview URL. I go to it. I can test the thing out. It works really nicely. It's like just having a full big private server that you can use 24/7 as much as you want. And managing these Linux boxes has gone from kind of a pain in the ass to completely trivial with things like
- 16:25
Codeex. I do all of my system management between devices just with the Codex CLI. It's great. The problem with the setup too is just the fact that like now you have to buy a whole extra computer to have running at home. You have to deal with the remote management stuff. You really need to set up something like a KVM so that you can remote in and do system updates and recover crashes. You now have to be on tail scale 24/7. You
- 16:46
are dealing with the inherent limitations of like it is your own computer and with Linux you can run pretty much infinite agents at this point. The efficiency of that operating system just blows my mind every time. Especially being used to Mac OS where five or six codeex threads and you crash your computer. It's stupid and really annoying. But that's a Mac OS problem, not a Codex problem. Codex runs wonderfully on the Linux machines.
- 17:08
again, you have to go buy another computer and hardware prices are really high right now. If your home network isn't super robust, it also makes it a little bit worse. This is a much lower friction version of that new workflow that will just make your life so much easier. I do recommend trying to figure out which version of this works best for you and get that set up and move your work over to that. It really has sped up
- 17:28
my development and made me want to use these things more. This probably should be its own video, so I will keep this section fairly brief. The main thing I wanted to talk about here with the battle between the labs is it kind of feels like there is a real dividing line being created right now where at least from the outside kind of feels like anthropic versus the world right now. A lot of this is kind of vibes based but
- 17:49
if you just look at the behavior of these companies and the way they act and talk very much all of these companies are focused heavily on getting really good models into as many people's hands as humanly possible. I mean, for meta, Zuck just put out this big article, the future is for everyone. Very long. Actually worth reading in my opinion. Regardless of what you think of Zuck, whether or not this is a purely like his
- 18:11
back is against the wall, so he's taking this strong open weight is good and everyone should have access to super intelligence position because he's scared of anthropic. Maybe that's the case. Maybe he just genuinely believes this. Regardless, I like a lot of what he says here, and this is kind of how I feel about this stuff right now, where I want to have the best possible models, and I want everyone to have the best
- 18:31
possible models and be able to do really cool stuff with these things. Enthropic does not seem very concerned about anything right now other than their biggest best model scaling as large and powerful as humanly possible. When you look at the Opus 5 release, that was clearly just a distilled version of Fable that was smaller and rldled into hell to just not be very good. Sonnet 5
- 18:52
was a disaster. That thing was pretty useless. Haiku, we haven't had a new Haiku model for almost a year at this point. I doubt we will ever get one again. They just do not seem to really care about anything other than Mythos and whatever is going to come after Mythos. Be it their like model 2 thing that came from I think it was like an investor call or something last week. That big new model that doesn't seem to
- 19:12
be getting released to the public. I'm sure we will get a Fable 5.1 at some point, but like they're not that concerned with lowering prices, making things more efficient, getting it in more people's hands. They just want to build God, but all these other companies really seem like they're just trying to get these things to be the most useful possible AIs for everybody. There's two ways of thinking about AI. If you
- 19:33
actually understand what's going on right now, there is like the AGI view of the world where AI is going to keep getting better and better, but it is going to always have some fundamental limitation of like you still need a human driving it in order to get anything useful out of it. I experienced this myself last week when I went to Defcon. I had it try and solve a bunch of these really difficult problems that
- 19:53
weren't easily verified. And since there was no verification, the models were able to just hallucinate a bunch of garbage mid-run and as a result couldn't actually solve any of them. They solved three out of the 13 puzzles that we were able to solve by orchestrating and running and using them. But we needed to know what we were doing. We needed to drive. And if you've seen any AI code, it can get really far and it can get you
- 20:14
a correct solution. But getting the right solution still requires a lot of human intervention and work. Maybe we'll get to a point where none of that really matters and we'll have that other view of the world which is kind of the ASI view like the super intelligence view where AI gets to the point that it is so much better than everybody at everything and nothing else really matters. The
- 20:35
model is so good and so big and there's so much compute behind it that it will just naturally figure these things out. Fable is kind of a glimpse of that. It is the easiest to work with model because it has great discernment. It writes great code. It is a very wise model compared to any of these other ones. These other ones require you to be more in the loop, but you still can get really good stuff out of them. Even
- 20:55
something like Deep Seek V4 Flash, which I've been running on my DJX Sparks in my home server locally, is actually getting some really good stuff. It just requires more effort from me. I have to be more specific with what I want. I have to give it more direction. I have to check its work more. It's not quite as much of a like fire and forget as the anthropic models. And with this kind of split in
- 21:15
the way they're doing things, I think everyone is just trying to kind of stop Anthropic from scaling out and ahead of everybody. OpenAI is the one kind of exception to this rule. They do live kind of in the middle. I think they are the only other lab that has a real chance of scaling up this crazy ASI thing that is like the god machine. The new Astra stuff from everything we've seen publicly on like it solving a bunch
- 21:37
of these math problems and the really crazy stuff it can do. Clearly, this is a very powerful model and they actually just announced today that they are pausing the RL job on their next big crazy model because they are concerned about the safety risks. So, they're definitely scaling in there and they're definitely seeing some crazy stuff in there. But, I know for a fact they are working really hard to get their models in everybody's hand. Like, Astra will be
- 21:58
publicly released. Sam Holman confirmed that publicly today. They really want everyone to have this stuff. I know that for a fact. This might be an overly cynical view, but I don't think Anthropic cares that much. I think the consumer stuff is more of a means to an end. They really are just concerned about building the best possible AI that is the big god machine. The safety stuff is really important there. It has huge
- 22:20
implications. And like honestly, I do kind of get it from their perspective. If you do fundamentally believe that these things can just scale up to the point where they can do everything on Earth better than a human, it does suddenly make sense why none of this other stuff matters. And if that thesis is true, and frankly I don't know where I fall on this line from a more visceral standpoint, I really do fall on the side
- 22:41
of like these things are tools. They require a human to direct them. I have a really hard time imagining them being able to make good long horizon decisions without hallucinating to hell. But also, if you look at the amount of progress we've gotten in the last like two or 3 years, like I don't remember when GPT 3.5 came out, but it wasn't that long ago, all things considered. And now we have Mythos. And Mythos is insane. But
- 23:03
still, it's not quite there yet. And while I academically understand and can be convinced depending on the day of the week that just scaling will get us there. Bigger, smarter models and like a lot of the questions of like, okay, well, how are they going to get the discernment? How are they going to be able to figure these things out? How are they going to touch the real world? And a lot of the answers of those questions you'll get is just, well, you make the
- 23:24
model smarter and bigger. Yeah, like honestly, I can see that working. Like I I don't know what the implications of at or above human genius level intelligence that is completely unbounded by time will be. Like maybe capabilities will keep being super spiky, but even if the capabilities are spiky, if you throw enough compute and enough agents at it, over time they'll just kind of naturally whittle these problems down and create
- 23:46
the thing that you're looking for and just naturally improve over time. I don't know how this is going to work. It's going to be very strange. I would love to live in this world. I'm concerned we there's a very good chance we'll live in this world, but time will tell at least for right now where we stand. The ranking of the labs is very clearly OpenAI and Anthropic are in a league of their own number one. Then SpaceX is pretty clearly the number two
- 24:07
spot. The Chinese labs are down below them depending on the day of the week. They could fluctuate a bit. I think Kim K3 is overall the most powerful one we have there. But Deepseek V4 Flash is an awesome model. V4 Pro looks pretty good, but I don't have a huge amount of experience with it. GLM 5.3 once the weights come out for that one. I'm very excited to see what that can do. Meta looks promising. They seem to be going
- 24:29
in the right direction and I really like how fast their new model is and it does actually feel different which is dope. Not there yet. Has potential. Honestly, the way I felt about XAI 4 months ago was basically how I feel about Meta now. They have a lot of potential. SpaceX is realizing that potential right now. Meta could get there. Google
- 24:50
um they uh how do I put they they exist um like you know the the Je Gemini 3.7 Flash came out. I'm sure we're all using a lot of that right now. Um yeah, we're just going to we're going to put that way down there. Yeah. Yeah. This feels good. I I think I'll just leave it here. I don't know. Like
- 25:11
am I going to keep beating the Google horse? Like it's just sad at this point. Like what am I supposed to say? It's Google. It's going to be a weird couple years here. I I at the end of these videos I always like I wish I had something more concrete to say. I just don't on where all this stuff is going to go. All I know is that right here right now the things that I'm using are a lot of cursor cuz I really like that whole ecosystem. Grockbot feels great.
- 25:31
Codex desktop app is still excellent. Claude Code CLI Fable is still a really powerful model when I just want highquality code generated without having to think too hard about it. T3 code in my whole remote setup also feels really good. Open weight models are also getting to the point where like I'm really enjoying using them. I've been testing out Deep Seek V4 Flash with the Deep Seek harness all running locally
- 25:52
and it is dope. It is so cool to see like an Opus 46 level model running on my network at over 100 TBS. There's a lot of cool stuff happening right now. If you enjoyed this video, make sure you like and subscribe.
Description
SpaceX AI finally feels like a real competitor to OpenAI and Anthropic. Grok 4.6 is far from perfect, but Cursor, GrokBot, and Origin are becoming an ecosystem I actually want to use every day.
Thank you Blacksmith for sponsoring! Check them out at: https://davis7.link/blacksmith
My X, Discord, and more: https://davis7.sh/
