Are These Actually Useful?
BDBen Davis@bmdavis419 · 45.9K subscribers
- Views
- 37.6K
- 37,617
- Outlier
- ×1.8
- Median 21.4K
- Length
- 20:32
- Published
- Oct 7
- Wednesday
- Comments
- 96
- 615 likes
Script
How it's written
- Words
- 4,528
- Words a minute
- 221
- Reading time
- 20 min
- Segments
- 14
The hook
0:00–0:25These personal assistant agent things seem to be this big new thing that all of the labs have kind of jumped on. OpenAI just dropped their new DOTs. Grock did Grockbot earlier this summer. Even Meta has been doing Muse, which seems to be doing really well. It's this new weird paradigm of working with an agent because we we're all mostly using coding agents, but now these are like coding agents and they're very similar under the hood, but they're not quite
Structure
These personal assistant agent things seem to be this big new thing that all of the labs have kind of jumped on. OpenAI just dropped their new DOTs. Grock did Grockbot earlier this summer. Even Meta has been doing Muse, which seems to be doing really well. It's this new weird paradigm of working with an agent because we we're all mostly using coding agents, but now these are like coding agents and they're very similar under the hood, but they're not quite coding agents cuz they have some extra stuff attached to them to make them more useful for daily productivity and also hopefully appeal to a much wider audience. And the thing that made me want to make this video was OpenAI's dot release. It has some promise to it, but I actually have a lot of problems with DOT. I am not using it heavily personally. The one that I have built most of my stuff into and I use daily is Grockbot. Muse is definitely not for me, but we'll talk about that later. But I think that the things that they screwed up here are very much worth talking about. I want to start by just breaking down the difference between these coding agents and personal assistants. I also want to talk about how they connect to different tools, how they work, where they run, what all of that actually means, the ones that I like that I don't like, and just try and kind of explain how all this fits together because it's weird. It is getting very complicated. It has been for a while. Long gone are the days where we were just using something like cloud code or codeex or pi or open code or whatever to do all of our work and it would just run on one machine and that machine would be the thing we interact with. It would do some tool calls to interact with our file system, maybe write some code and that
Watch from 0:00was it. That's what a coding agent was. At the end of the day, these are best described as a while loop. The model generates tokens which then have tool calls inside of them which are called with code inside of the actual harness and then that will keep going until it wants to stop. That simple core makes cloud code work, codecs work, the desktop apps for all these things work. We've done a ton of crazy stuff with these and now there are these new personal assistants which are at their core very similar. It's the same idea. Something like dots from OpenAI is very much a codeex instance. It feels like I it's clearly running a codeex thread or something like it under the hood. It's just an agent loop. These assistants have a lot of extra stuff stacked on top of them. Like for example, they might have a memory system so they can write to markdown files to save some information about you that can be then referenced in the future to better answer other questions. Or maybe they'll have a cron job so that every day at 5:00 p.m. it will fire off a little thing that will cause them to start running. Then they will do something during that time. Maybe it's summarizing your email for the day. Maybe it's uh triaging some like all the PRs that happen over the course of the day. Whatever it ends up being, it's just some ability for them to wake. They also have lots of built-in native connectors to things like Gmail or Outlook or Calendar or Notion or whatever you need to do more knowledge productivity work.
Watch from 1:30I think a lot of the positioning for these things is the coding agents are still for coding. like I'm not going to do my coding work through Grockbot, but Grockbot is very useful for orchestrating and managing things almost like at a PM level for more dev speak way of putting it. Uh they work really well for that and they also tend to have and I think all of these do they have their own computer quote unquote which is really just a VM running in the cloud that is their little box that they can play around in. They can write files to it. They can use a browser in it so that if you say for example log into Amazon on one of your bots, it can then go in computer use and order stuff for you directly anytime from your phone just in the background. It is really cool. I do really like these things. I don't want to downplay what they do, but they are just stacking new features on top of the core of coding agents. I have a bigger diagram here to try and explain how all of this stuff fits together. It's actually more chaotic and complicated than you might think. I want to try to explain that and then also like which one of these bots is actually worth using right after today's sponsor. Today's sponsor is the ultimate AI code reviewer. It's GPile. They have such a powerful system for reviewing PRs that I would not want to have any real project without it. But I think the easiest way to show you why is to show you this comments that it left on a recent PR that I just merged. And you'll notice here it says that the new languages cannot be saved. Bunch of details on what went wrong. But the really interesting piece here are the artifacts. If I open up this artifact, you'll notice that it's showing me a
Watch from 2:50bunch of like command outputs. And then it's showing me this big long script of something and then another set of outputs. And what this is is these are artifacts generated from Reptiles's T-Rex product, which is their system for actually running and testing your code live in a sandbox on every PR, which means that they're able to do things like this where they call these endpoints, make sure they work. Found out that they would 400 and ended up leaving this very useful comment with a bunch of evidence that my agents were able to just find, grab, fix, put it back up, and it just worked. It's such a powerful thing to have in your coding agent loop at this point where all of my PRs go through multiple rounds of reviews from Greile before I ever even see them. Companies like Nvidia, Work OS, and 22,000 others are relying on Greile right now, and you should be too at davis 7.link/gretile. I know I've got a lot of stuff up on screen here. There's really only three things that are actually here. The first of which is the core agent itself. That is the loop that runs the tokens and that would be the coding agents andor the personal assistants. Then the next section you have is the context and execution layer. So this would be where memories, computer use files, uh executing terminal commands or running code, all of that random stuff that lives within like another environment or whatever.
Watch from 4:21You can put that wherever you want. Where this stuff runs is another whole question. But that's like another like I don't know what the right word for it is. But another like piece of the larger system that is an agent quote unquote. And then of course there's the connections and these are like the external things that your agents can interact with in order to make them useful. This has been a problem for a long time that has mostly gotten easier to solveish with the advent of you know Grockbot dots muse. They really worked hard to make the connection layer on these things much easier. So it's much easier to sign into notion or Gmail or Google calendar whatever to actually get useful stuff done. I have an extra section in the middle here for gateways because this is something that I personally really like. Having all these connections is great, but you want to have these connections for your personal assistants, but you can also have these in your coding agents. I know a lot of people who for their personal agent workflow, quote unquote, it's just connecting Gmail into normal codecs and asking it to do stuff in Gmail cuz it can just fine with zero issue. There are two gateways right now that I think really kind of fit this bill. The first of which is Upsy. It's a new thing from Upstash that they're working on right now for basically giving your agent a computer that has all of that uh context layer stuff on it as well as the external connections. So, you hook up all of these different services and then you just connect this one MCP into whatever agent you want and it can just now go off and do those things easily.
Watch from 5:37You manage the permissions, keep some memories and skills and whatever in there. It's a really nice system for that. Executor is another one that does this really well. I have really liked what Ree is doing with all of this. He's close to getting V2 out. Once V2 is out, this will be a very powerful way of doing these things, but it's again a similar concept, one singular gateway to connect all these different tools to whatever agent you want. And having that one core gateway piece that exists that you can then fan out to all your different agents because I find myself switching a bunch between, you know, Cloud Code, Codeex, Grock, Build, Grockbot, Dots, Pi. I I use all of these. So, one MCP endpoint that I just have to add into whatever harness and suddenly everything works. It's great. It's very, very helpful. And I think that leads to the next natural question of like, okay, where do all of these things actually run? Because computers have gotten much more distributed and abstract for me, especially since agents have gotten good. To try and keep this somewhat simple, I broke it up into three like distinct things that need computers for the whole agent to work. The first of which is, okay, where do the tokens come from? Where are these actually going to run? Generally speaking, the 90% of the time this will be from some hosted external API like OpenAI's API. They are running a bunch of GPUs and computers to serve you those tokens. You could also do this like I have a bunch of DJX sparks in my closet.
Watch from 6:59I I can run the tokens layer there. I can get all those tokens from there. It doesn't really matter where you get them from. They just have to run somewhere so that you can access them. Then there's the agent. And the agent really just needs some light isolated instance to run in. It doesn't even have to be isolated. Like you could run this in a normal like NodeJS backend in on Verscell or something. All it's doing is requesting tokens from the token gateway processing those in a while loop and then doing the last step here which is calling the actual tools. And this is the one that I think gets really interesting is the agent is calling the tools but the tools can effectively run wherever. It's a function you're calling in code in the agent and that function could be calling maybe a bash command on that machine or maybe it could be calling a MCP on a different machine and it's just doing an HTTP request on that actual agent that then triggers something on a remote box to run it there. So say I am working in Grockbot and I want to check my calendar for what's coming up tomorrow. The model will fire off a tool call that will then go to executor over HTTP or something else. executor will then call code to pull from the calendar in that external API, send it back down here, send it back down to Grockbot. It gets that info. Great. So, that ran that way, but Grockbot is running on cursor servers or I guess SpaceX AI servers now. Then executor is running on my own self-hosted instance and then this is running on Google's computers and we're all just like connecting back and forth and talking to each other there. It's this crazy weird thing that you kind of have to wrap your brain around, but once you do, you realize that all of these
Watch from 8:18things can be connected in these weird random ways and you can make very powerful systems with this. This is definitely getting deep into the technical details on all this stuff. I I do want to go back and just now talk about the actual agents themselves because for a lot of people, they're not going to be thinking about all this stuff. Most of this is being abstracted away by these new agent products. So, I think starting with the new dots one from OpenAI, I don't love what they've done here with this. I don't think it makes all that much sense. There's a weird thing that I found OpenAI tends to do and they're really doing a lot now where they are trying to serve both professional developers and power users and also the normal person who doesn't really use this stuff for work and just kind of wants a fun semi-useful little AI bot to help them out. Most people don't actually care or need a deep complex personal assistance system like what I just showed you. This right here is the actual UI for Dot. You'll notice that it looks nothing like that crazy complicated thing I just talked about. This is just basically iMessage for lack of better word where you can send a message into this agent and the agent will go do something. It's very abstracted and that is by design. All of these do kind of do that where they don't want you to be looking at the tool calls or the big complicated things happening under the hood. They just want you to ask for it to do something and then it will just magically go off and do the thing. So you get this seamless assistant experience. Personally, I don't like this at all. For the type of work I do and the variety of things I do, having one singular hell thread, quote unquote, for everything is just not viable at all. I have lots of
Watch from 9:48different things that I need to bucket where I need to bucket, you know, agency work, I need to bucket sales work, I need to bucket dev work, I need to bucket research or personal or finance or whatever. All of these things exist separately. Controlling all of them through one singular iMessage stream that very easily goes off topic and is not the most coherent thing in the world and is also a huge problem here is it is insanely slow and awkward. It's very hard to see what's actually happening. When I was trying to connect up my hosted instance of executor, it just was taking forever. I would send it a message and it would just sit there. It would like read my message, do the red thing like you get in iMessage and then just like kind of do nothing for 10 minutes because it was working in the background doing normal coding agent things, but it doesn't give you any progress updates. It just sends you a text quote unquote at the very end with some very highle information that's hard to actually understand what's going on under the hood. I know that 99% of people are not going to understand all of this nonsense and how it fits together, nor should they, but I need something. I I need more feedback to actually see what's going on when I'm working with this thing. It is so abstracted away that I I just don't love it. I I mentioned earlier that this kind of feels like a codec thread and it definitely is where sometimes it will shell stuff off to your codeex instance. It it could be the codec stuff on your computer. You can see I have my computer here and I also have the dots computer here because the dot gets its own cloud hosted computer. So, if it does some work, maybe it will run that on the cloud computer or on my computer. It's kind of weird how all of this fits
Watch from 11:20together. It's It's the right idea for sure. Like, I do like that they have this a lot, but the weirdness and lack of trust I have for the coordination happening here just isn't great. It also just kind of suffers from the thing that a lot of Open AAI apps have been suffering from recently, which is just the fact that it feels like they are adding too much too fast and things are borderline regressing sometimes. Like the performance on this isn't great. The mobile app is kind of awkward. There's a lot of weird things in there that just don't feel as good as I think they probably should. I hope that they will patch this up and fix it in the future. They usually do. But I honestly, for lack of a better word, I I think that the thing that's so weird about this is it is too abstracted and simple for me to get my own serious complex work done with it. But it also doesn't quite seem to be as simple and easy and userfriendly nor me friendly as something like Muse is. It's just kind of in the middle and I just don't quite know who this is for to be honest with you. I have not been a huge fan of it personally. Maybe it'll get there. I'm not entirely sure. But at least for right now, DOT is not what I am building my stuff into and through. The one that I'm pretty much entirely built into at this point is Grockbot. I love Grockbot. The only problem I have with it is that like showing and demoing it is really hard because I have too much just like personal stuff and business stuff that I can't really show on video. So, I can't really open the Grockbot app to show everything I'm doing. But, they actually have a really good page for this to see how it works where you have a bot for each little thing. Like for me personally, I have a personal bot, a finance bot, an agency bot, a my company
Watch from 12:50bot, a YouTube bot, a GitHub bot. All of these things off doing their own things. They all have their own routines. So things that they do at certain times of the day like pull certain briefs based on what happened or at the beginning of the day I get this summary of everything that I need to do for the day from the to-do list from the calendar that sort of stuff. It's an assistant and DOT can do this kind of thing. I just find that the singlethreaded nature makes it a lot more complex and annoying to deal with. And also the hard reality is that SpaceX AAI has the best engineering of any of the labs right now. their mobile app, the desktop app for bots, the websites they have for all this stuff is just bestin-class by far. I've been super impressed with it. It's fast, robust, stable. I really, really like it. And to go back here, Grockbot is built using the same general systems and sandboxes as the cursor cloud agents. So each bot gets its own dedicated computer. So for example, mine has an Amazon instance on it that is logged in and whenever I need to order like cat food or something, I'm just like, "Hey, go get the cat food." and it's like, "Hey, can I order this? Here's the thing I'm going to do." You answer the permission prompt, give it a yes, and then it goes off and does the thing. Really, really useful. But this like computer layer right here is living on Grockbot server. So, it's on 24/7. Same thing with files, memories, routines, skills, all that stuff live within the Grockbot computer elsewhere.
Watch from 14:21But you could also have Grockbot on the desktop, connect it to the desktop, and then run some code or whatever on that machine specifically. And you can network these machines together. So, if I'm on my phone, but my laptop's turned on, I can then be like, "Hey, can you go do this on my laptop? Spin up an iOS simulator." And it'll go do that. The bots are also capable of like talking to each other. So, they have this like semi-warming behavior to them where I have this like centralized orchestrator bot that is like the main place I interface with. And then if I need to like, oh, I need to order this thing or I need to go check on this email or whatever, it will send a message to the other bot, which acts basically like a sub agent to grab all that context, send it back up to the original one, and then deliver that to me. And since they all have their own little tasks and roles or whatever, their system prompts stay cleaner, their memories stay cleaner, their routines stay cleaner, and it's much easier to parallelize into different tasks. When you're not stuck in just one singular hell thread that, from what I understand has a maximum of five sub agents on it, it doesn't feel very good. The biggest advantage that DOT has right now is the fact that it has Astra under the hood. Astra is running this and Astra is a much, much better model than Gro 4.7. I made a video on that model. It's not good. They screwed up the pricing. The performance isn't good. It is not Frontier. But they did RL it directly on Grockbot and it seems like that paid off. Grockbot is in a great state like using itwise. The model feels great inside of it specifically. It definitely kind of seems like they sacrificed that model on the alter of Grockbot to make Grockbot better, but you know, we'll see what the
Watch from 15:40future releases look like. I'm ambivalent. I think the point I'm really trying to make here with all of that nonsense I just spouted is that serious employed people are using Grockbot for professional work. It's not the thing that I would expect like my mom or little sister to use for like fun oneoff day-to-day things. The one for personal stuff is Muse. And Muse, frankly, if you're this far into the video after I went through all of that random technical nonsense, Muse is not for you. It is not. I have not used it. I have no intentions of using it. I don't like Instagram. I don't like Facebook. I don't I don't want anything to do with Meta Products to be honest with you. I do like their model. It's pretty good, but I'm not using Muse. Hell no. But, you know, when you just like look at the web page for it, you have this little example right here of, you know, fill out this permission slip for a field trip and then this little reply here with the emoji reaction. Like, this is it's meant to feel like a conversation in iMessage and be much more natural for someone who is not thinking about this. They just want something that is useful for little one-off things. Maybe tuning their Instagram, maybe sending some messages, maybe handling basic stuff like this. This is the simple version of these agents for consumer. And I think it's going to slaughter in consumer. The fact that when you open Instagram, you get this big banner ad for Muse telling you to set it up and shows which ones of your friends are already using Muse and all this stuff. Like it's this is the normal people personal agent. I think that's awesome. Like genuinely, I think this is a very good thing to have and I'm glad this exists. It's just not for me personally. So, I know this video was
Watch from 17:12kind of a chaotic mess because the reality is these agents are a bit of a chaotic mess right now. My genuine personal recommendation if you're this far into the video is go try out like a Grockbot type thing. It's very useful for dev stuff or just general management tasks that you probably don't want to deal with. They did a really good job on it, but also at the same time, you don't necessarily need it. The big things you're getting out of these personal assistants is the always on nature, a better mobile experience, the ability to have them have like built-in routines and their own computer and stuff like that. It's a lot of convenience stuff. I could build my own personal Grockbot or whatever, but I don't want to. Self-hosting all this stuff and managing it is kind of a pain in the ass to be honest with you. The coding agents are for coding, but also for just like simple one-off stuff. Like I do really like using codecs for, you know, a quick thing on notion or a quick thing in Gmail, whatever. That's great. But once you go beyond that, it's much nicer to just have a dedicated place for that. And I think that's where these personal assistants lie. This space is definitely going to grow. I think that this is a pretty decent shape of things that I know OpenAI is going to be investing very heavily in. They are very bullish on DOTS. I do think that they'll be able to get there. It's just going to take some time and some iteration. It's not quite there yet. I think this is one of those videos and topics where I really don't have a good gut feel of how familiar most people are with all this stuff. So, this time more than most, let let me know in the comments what stuff does and doesn't make sense to you on these things and what you're curious about and would want to see other videos on because I I want to try and better cover how to stick all of these weird new agent systems into all the different places of like work and life because it's insanely useful and valuable. It's just kind of hard to explain how all this stuff works right now. The labs are struggling with it, too, to be fair.
Watch from 18:42Full transcript
- 0:00
These personal assistant agent things seem to be this big new thing that all of the labs have kind of jumped on. OpenAI just dropped their new DOTs. Grock did Grockbot earlier this summer. Even Meta has been doing Muse, which seems to be doing really well. It's this new weird paradigm of working with an agent because we we're all mostly using
- 0:20
coding agents, but now these are like coding agents and they're very similar under the hood, but they're not quite coding agents cuz they have some extra stuff attached to them to make them more useful for daily productivity and also hopefully appeal to a much wider audience. And the thing that made me want to make this video was OpenAI's dot release. It has some promise to it, but I actually have a lot of problems with
- 0:42
DOT. I am not using it heavily personally. The one that I have built most of my stuff into and I use daily is Grockbot. Muse is definitely not for me, but we'll talk about that later. But I think that the things that they screwed up here are very much worth talking about. I want to start by just breaking down the difference between these coding agents and personal assistants. I also want to talk about how they connect to different tools, how they work, where
- 1:02
they run, what all of that actually means, the ones that I like that I don't like, and just try and kind of explain how all this fits together because it's weird. It is getting very complicated. It has been for a while. Long gone are the days where we were just using something like cloud code or codeex or pi or open code or whatever to do all of our work and it would just run on one
- 1:24
machine and that machine would be the thing we interact with. It would do some tool calls to interact with our file system, maybe write some code and that was it. That's what a coding agent was. At the end of the day, these are best described as a while loop. The model generates tokens which then have tool calls inside of them which are called with code inside of the actual harness and then that will keep going until it
- 1:44
wants to stop. That simple core makes cloud code work, codecs work, the desktop apps for all these things work. We've done a ton of crazy stuff with these and now there are these new personal assistants which are at their core very similar. It's the same idea. Something like dots from OpenAI is very much a codeex instance. It feels like I it's clearly running a codeex thread or
- 2:06
something like it under the hood. It's just an agent loop. These assistants have a lot of extra stuff stacked on top of them. Like for example, they might have a memory system so they can write to markdown files to save some information about you that can be then referenced in the future to better answer other questions. Or maybe they'll have a cron job so that every day at 5:00 p.m. it will fire off a little thing that will cause them to start
- 2:27
running. Then they will do something during that time. Maybe it's summarizing your email for the day. Maybe it's uh triaging some like all the PRs that happen over the course of the day. Whatever it ends up being, it's just some ability for them to wake. They also have lots of built-in native connectors to things like Gmail or Outlook or Calendar or Notion or whatever you need
- 2:47
to do more knowledge productivity work. I think a lot of the positioning for these things is the coding agents are still for coding. like I'm not going to do my coding work through Grockbot, but Grockbot is very useful for orchestrating and managing things almost like at a PM level for more dev speak way of putting it. Uh they work really well for that and they also tend to have
- 3:07
and I think all of these do they have their own computer quote unquote which is really just a VM running in the cloud that is their little box that they can play around in. They can write files to it. They can use a browser in it so that if you say for example log into Amazon on one of your bots, it can then go in computer use and order stuff for you directly anytime from your phone just in
- 3:28
the background. It is really cool. I do really like these things. I don't want to downplay what they do, but they are just stacking new features on top of the core of coding agents. I have a bigger diagram here to try and explain how all of this stuff fits together. It's actually more chaotic and complicated than you might think. I want to try to explain that and then also like which one of these bots is actually worth
- 3:50
using right after today's sponsor. Today's sponsor is the ultimate AI code reviewer. It's GPile. They have such a powerful system for reviewing PRs that I would not want to have any real project without it. But I think the easiest way to show you why is to show you this comments that it left on a recent PR that I just merged. And you'll notice
- 4:10
here it says that the new languages cannot be saved. Bunch of details on what went wrong. But the really interesting piece here are the artifacts. If I open up this artifact, you'll notice that it's showing me a bunch of like command outputs. And then it's showing me this big long script of something and then another set of outputs. And what this is is these are artifacts generated from Reptiles's
- 4:32
T-Rex product, which is their system for actually running and testing your code live in a sandbox on every PR, which means that they're able to do things like this where they call these endpoints, make sure they work. Found out that they would 400 and ended up leaving this very useful comment with a bunch of evidence that my agents were able to just find, grab, fix, put it back up, and it just worked. It's such a
- 4:53
powerful thing to have in your coding agent loop at this point where all of my PRs go through multiple rounds of reviews from Greile before I ever even see them. Companies like Nvidia, Work OS, and 22,000 others are relying on Greile right now, and you should be too at davis 7.link/gretile. I know I've got a lot of stuff up on screen here. There's really only three things that
- 5:14
are actually here. The first of which is the core agent itself. That is the loop that runs the tokens and that would be the coding agents andor the personal assistants. Then the next section you have is the context and execution layer. So this would be where memories, computer use files, uh executing terminal commands or running code, all of that random stuff that lives within
- 5:34
like another environment or whatever. You can put that wherever you want. Where this stuff runs is another whole question. But that's like another like I don't know what the right word for it is. But another like piece of the larger system that is an agent quote unquote. And then of course there's the connections and these are like the external things that your agents can interact with in order to make them
- 5:55
useful. This has been a problem for a long time that has mostly gotten easier to solveish with the advent of you know Grockbot dots muse. They really worked hard to make the connection layer on these things much easier. So it's much easier to sign into notion or Gmail or Google calendar whatever to actually get useful stuff done. I have an extra section in the middle here for gateways
- 6:15
because this is something that I personally really like. Having all these connections is great, but you want to have these connections for your personal assistants, but you can also have these in your coding agents. I know a lot of people who for their personal agent workflow, quote unquote, it's just connecting Gmail into normal codecs and asking it to do stuff in Gmail cuz it can just fine with zero issue. There are
- 6:35
two gateways right now that I think really kind of fit this bill. The first of which is Upsy. It's a new thing from Upstash that they're working on right now for basically giving your agent a computer that has all of that uh context layer stuff on it as well as the external connections. So, you hook up all of these different services and then you just connect this one MCP into whatever agent you want and it can just
- 6:56
now go off and do those things easily. You manage the permissions, keep some memories and skills and whatever in there. It's a really nice system for that. Executor is another one that does this really well. I have really liked what Ree is doing with all of this. He's close to getting V2 out. Once V2 is out, this will be a very powerful way of doing these things, but it's again a similar concept, one singular gateway to
- 7:18
connect all these different tools to whatever agent you want. And having that one core gateway piece that exists that you can then fan out to all your different agents because I find myself switching a bunch between, you know, Cloud Code, Codeex, Grock, Build, Grockbot, Dots, Pi. I I use all of these. So, one MCP endpoint that I just have to add into whatever harness and
- 7:38
suddenly everything works. It's great. It's very, very helpful. And I think that leads to the next natural question of like, okay, where do all of these things actually run? Because computers have gotten much more distributed and abstract for me, especially since agents have gotten good. To try and keep this somewhat simple, I broke it up into three like distinct things that need computers for the whole agent to work.
- 8:00
The first of which is, okay, where do the tokens come from? Where are these actually going to run? Generally speaking, the 90% of the time this will be from some hosted external API like OpenAI's API. They are running a bunch of GPUs and computers to serve you those tokens. You could also do this like I have a bunch of DJX sparks in my closet. I I can run the tokens layer there. I
- 8:21
can get all those tokens from there. It doesn't really matter where you get them from. They just have to run somewhere so that you can access them. Then there's the agent. And the agent really just needs some light isolated instance to run in. It doesn't even have to be isolated. Like you could run this in a normal like NodeJS backend in on Verscell or something. All it's doing is requesting tokens from the token gateway processing those in a while loop and
- 8:42
then doing the last step here which is calling the actual tools. And this is the one that I think gets really interesting is the agent is calling the tools but the tools can effectively run wherever. It's a function you're calling in code in the agent and that function could be calling maybe a bash command on that machine or maybe it could be calling a MCP on a different machine and
- 9:03
it's just doing an HTTP request on that actual agent that then triggers something on a remote box to run it there. So say I am working in Grockbot and I want to check my calendar for what's coming up tomorrow. The model will fire off a tool call that will then go to executor over HTTP or something else. executor will then call code to pull from the calendar in that external
- 9:23
API, send it back down here, send it back down to Grockbot. It gets that info. Great. So, that ran that way, but Grockbot is running on cursor servers or I guess SpaceX AI servers now. Then executor is running on my own self-hosted instance and then this is running on Google's computers and we're all just like connecting back and forth and talking to each other there. It's this crazy weird thing that you kind of
- 9:45
have to wrap your brain around, but once you do, you realize that all of these things can be connected in these weird random ways and you can make very powerful systems with this. This is definitely getting deep into the technical details on all this stuff. I I do want to go back and just now talk about the actual agents themselves because for a lot of people, they're not going to be thinking about all this stuff. Most of this is being abstracted away by these new agent products. So, I
- 10:07
think starting with the new dots one from OpenAI, I don't love what they've done here with this. I don't think it makes all that much sense. There's a weird thing that I found OpenAI tends to do and they're really doing a lot now where they are trying to serve both professional developers and power users and also the normal person who doesn't
- 10:27
really use this stuff for work and just kind of wants a fun semi-useful little AI bot to help them out. Most people don't actually care or need a deep complex personal assistance system like what I just showed you. This right here is the actual UI for Dot. You'll notice that it looks nothing like that crazy complicated thing I just talked about. This is just basically iMessage for lack
- 10:48
of better word where you can send a message into this agent and the agent will go do something. It's very abstracted and that is by design. All of these do kind of do that where they don't want you to be looking at the tool calls or the big complicated things happening under the hood. They just want you to ask for it to do something and then it will just magically go off and do the thing. So you get this seamless assistant experience. Personally, I
- 11:10
don't like this at all. For the type of work I do and the variety of things I do, having one singular hell thread, quote unquote, for everything is just not viable at all. I have lots of different things that I need to bucket where I need to bucket, you know, agency work, I need to bucket sales work, I need to bucket dev work, I need to bucket research or personal or finance or whatever. All of these things exist
- 11:31
separately. Controlling all of them through one singular iMessage stream that very easily goes off topic and is not the most coherent thing in the world and is also a huge problem here is it is insanely slow and awkward. It's very hard to see what's actually happening. When I was trying to connect up my hosted instance of executor, it just was taking forever. I would send it a
- 11:51
message and it would just sit there. It would like read my message, do the red thing like you get in iMessage and then just like kind of do nothing for 10 minutes because it was working in the background doing normal coding agent things, but it doesn't give you any progress updates. It just sends you a text quote unquote at the very end with some very highle information that's hard to actually understand what's going on under the hood. I know that 99% of
- 12:13
people are not going to understand all of this nonsense and how it fits together, nor should they, but I need something. I I need more feedback to actually see what's going on when I'm working with this thing. It is so abstracted away that I I just don't love it. I I mentioned earlier that this kind of feels like a codec thread and it definitely is where sometimes it will shell stuff off to your codeex instance.
- 12:35
It it could be the codec stuff on your computer. You can see I have my computer here and I also have the dots computer here because the dot gets its own cloud hosted computer. So, if it does some work, maybe it will run that on the cloud computer or on my computer. It's kind of weird how all of this fits together. It's It's the right idea for sure. Like, I do like that they have this a lot, but the weirdness and lack
- 12:56
of trust I have for the coordination happening here just isn't great. It also just kind of suffers from the thing that a lot of Open AAI apps have been suffering from recently, which is just the fact that it feels like they are adding too much too fast and things are borderline regressing sometimes. Like the performance on this isn't great. The mobile app is kind of awkward. There's a lot of weird things in there that just
- 13:17
don't feel as good as I think they probably should. I hope that they will patch this up and fix it in the future. They usually do. But I honestly, for lack of a better word, I I think that the thing that's so weird about this is it is too abstracted and simple for me to get my own serious complex work done with it. But it also doesn't quite seem to be as simple and easy and
- 13:39
userfriendly nor me friendly as something like Muse is. It's just kind of in the middle and I just don't quite know who this is for to be honest with you. I have not been a huge fan of it personally. Maybe it'll get there. I'm not entirely sure. But at least for right now, DOT is not what I am building my stuff into and through. The one that I'm pretty much entirely built into at this point is Grockbot. I love Grockbot.
- 13:59
The only problem I have with it is that like showing and demoing it is really hard because I have too much just like personal stuff and business stuff that I can't really show on video. So, I can't really open the Grockbot app to show everything I'm doing. But, they actually have a really good page for this to see how it works where you have a bot for each little thing. Like for me personally, I have a personal bot, a finance bot, an agency bot, a my company
- 14:21
bot, a YouTube bot, a GitHub bot. All of these things off doing their own things. They all have their own routines. So things that they do at certain times of the day like pull certain briefs based on what happened or at the beginning of the day I get this summary of everything that I need to do for the day from the to-do list from the calendar that sort of stuff. It's an assistant and DOT can do this kind of thing. I just find that
- 14:42
the singlethreaded nature makes it a lot more complex and annoying to deal with. And also the hard reality is that SpaceX AAI has the best engineering of any of the labs right now. their mobile app, the desktop app for bots, the websites they have for all this stuff is just bestin-class by far. I've been super impressed with it. It's fast, robust, stable. I really, really like it. And to
- 15:03
go back here, Grockbot is built using the same general systems and sandboxes as the cursor cloud agents. So each bot gets its own dedicated computer. So for example, mine has an Amazon instance on it that is logged in and whenever I need to order like cat food or something, I'm just like, "Hey, go get the cat food." and it's like, "Hey, can I order this? Here's the thing I'm going to do." You
- 15:23
answer the permission prompt, give it a yes, and then it goes off and does the thing. Really, really useful. But this like computer layer right here is living on Grockbot server. So, it's on 24/7. Same thing with files, memories, routines, skills, all that stuff live within the Grockbot computer elsewhere. But you could also have Grockbot on the desktop, connect it to the desktop, and
- 15:44
then run some code or whatever on that machine specifically. And you can network these machines together. So, if I'm on my phone, but my laptop's turned on, I can then be like, "Hey, can you go do this on my laptop? Spin up an iOS simulator." And it'll go do that. The bots are also capable of like talking to each other. So, they have this like semi-warming behavior to them where I have this like centralized orchestrator
- 16:05
bot that is like the main place I interface with. And then if I need to like, oh, I need to order this thing or I need to go check on this email or whatever, it will send a message to the other bot, which acts basically like a sub agent to grab all that context, send it back up to the original one, and then deliver that to me. And since they all have their own little tasks and roles or whatever, their system prompts stay
- 16:25
cleaner, their memories stay cleaner, their routines stay cleaner, and it's much easier to parallelize into different tasks. When you're not stuck in just one singular hell thread that, from what I understand has a maximum of five sub agents on it, it doesn't feel very good. The biggest advantage that DOT has right now is the fact that it has Astra under the hood. Astra is running this and Astra is a much, much
- 16:46
better model than Gro 4.7. I made a video on that model. It's not good. They screwed up the pricing. The performance isn't good. It is not Frontier. But they did RL it directly on Grockbot and it seems like that paid off. Grockbot is in a great state like using itwise. The model feels great inside of it specifically. It definitely kind of seems like they sacrificed that model on
- 17:08
the alter of Grockbot to make Grockbot better, but you know, we'll see what the future releases look like. I'm ambivalent. I think the point I'm really trying to make here with all of that nonsense I just spouted is that serious employed people are using Grockbot for professional work. It's not the thing that I would expect like my mom or little sister to use for like fun oneoff day-to-day things. The one for personal
- 17:30
stuff is Muse. And Muse, frankly, if you're this far into the video after I went through all of that random technical nonsense, Muse is not for you. It is not. I have not used it. I have no intentions of using it. I don't like Instagram. I don't like Facebook. I don't I don't want anything to do with Meta Products to be honest with you. I do like their model. It's pretty good, but I'm not using Muse. Hell no. But, you know, when you just like look at the
- 17:51
web page for it, you have this little example right here of, you know, fill out this permission slip for a field trip and then this little reply here with the emoji reaction. Like, this is it's meant to feel like a conversation in iMessage and be much more natural for someone who is not thinking about this. They just want something that is useful for little one-off things. Maybe tuning
- 18:12
their Instagram, maybe sending some messages, maybe handling basic stuff like this. This is the simple version of these agents for consumer. And I think it's going to slaughter in consumer. The fact that when you open Instagram, you get this big banner ad for Muse telling you to set it up and shows which ones of your friends are already using Muse and all this stuff. Like it's this is the
- 18:33
normal people personal agent. I think that's awesome. Like genuinely, I think this is a very good thing to have and I'm glad this exists. It's just not for me personally. So, I know this video was kind of a chaotic mess because the reality is these agents are a bit of a chaotic mess right now. My genuine personal recommendation if you're this far into the video is go try out like a Grockbot type thing. It's very useful
- 18:54
for dev stuff or just general management tasks that you probably don't want to deal with. They did a really good job on it, but also at the same time, you don't necessarily need it. The big things you're getting out of these personal assistants is the always on nature, a better mobile experience, the ability to have them have like built-in routines and their own computer and stuff like
- 19:14
that. It's a lot of convenience stuff. I could build my own personal Grockbot or whatever, but I don't want to. Self-hosting all this stuff and managing it is kind of a pain in the ass to be honest with you. The coding agents are for coding, but also for just like simple one-off stuff. Like I do really like using codecs for, you know, a quick thing on notion or a quick thing in Gmail, whatever. That's great. But once
- 19:35
you go beyond that, it's much nicer to just have a dedicated place for that. And I think that's where these personal assistants lie. This space is definitely going to grow. I think that this is a pretty decent shape of things that I know OpenAI is going to be investing very heavily in. They are very bullish on DOTS. I do think that they'll be able to get there. It's just going to take some time and some iteration. It's not
- 19:56
quite there yet. I think this is one of those videos and topics where I really don't have a good gut feel of how familiar most people are with all this stuff. So, this time more than most, let let me know in the comments what stuff does and doesn't make sense to you on these things and what you're curious about and would want to see other videos on because I I want to try and better cover how to stick all of these weird
- 20:18
new agent systems into all the different places of like work and life because it's insanely useful and valuable. It's just kind of hard to explain how all this stuff works right now. The labs are struggling with it, too, to be fair.
Description
I wanted to make a vid on OpenAI's new Dots thing, Grok Bot, Muse, and what these things actually are.
Thank you Greptile for sponsoring! Check them out at: https://davis7.link/greptile
Dot crashout - https://x.com/davis7/status/210575332...
My X, Discord, and more: https://davis7.sh/
