You can rewrite typescript in rust for $420k
TTTheo - t3.gg@t3dotgg · 572K subscribers
- Views
- 174K
- 173,892
- Outlier
- ×1.0
- Median 167K
- Length
- 47:40
- Published
- Oct 10
- Saturday
- Comments
- 531
- 2.8K likes
Script
How it's written
- Words
- 9,879
- Words a minute
- 207
- Reading time
- 42 min
- Segments
- 33
The hook
0:00–0:25Announcing TSC RS aka Typescript Rust. My complete rewrite of the TypeScript compiler, Type Checker, and LSP all in Rust. This is a project I've had agents working on for 5 months now. I burned $400,000 of codeex tokens and got nowhere. I burned $20,000 of Opus 5.5 and I got there in 2 weeks. An important note, I have not read a single line of
Structure
Announcing TSC RS aka Typescript Rust. My complete rewrite of the TypeScript compiler, Type Checker, and LSP all in Rust. This is a project I've had agents working on for 5 months now. I burned $400,000 of codeex tokens and got nowhere. I burned $20,000 of Opus 5.5 and I got there in 2 weeks. An important note, I have not read a single line of code. If you've been around for a bit, you probably know I've been using this TypeScript Rust rewrite as my favorite little playground for trying out new models and seeing where capabilities are going. I did that because I was looking for things that just seemed impossible for the models to do. The reason I first started the TypeScript Rust project was because I had a crazy amount of inference with the early access testing for GPT56 soul and it was crushing everything I threw at it. So, I decided to reach for something a bit bolder. Not a bit bolder, like just impossible. It didn't seem viable to just rewrite all of TypeScript in Rust. It seemed like a meme and it was the furthest I got using OpenAI models was roughly 84% compatibility. And while like impressive that it could get that far, didn't really seem like it was going to get much further because at that point the last 16% is way harder. And as you get closer and closer, the edges that are left get worse and worse. Which is why it feels extra insane that not only is TS Rust now out and available for people to use with a real package on npm, more importantly, it's way faster when compared to TypeScript v6, which was the last version written in TypeScript and JavaScript itself. It is 12.5 times
Watch from 0:00faster. When compared to TSGO, which is TypeScript v7, the version that the official folks at Microsoft built, it's around two times faster. We're not the only ones who did this. Bun also dropped bun check which is a rewrite of the typescript checker in Rust that is also way faster and I'll be real is even faster than what we built with PS Rust. But there are catches to all of this from how I built it to how I verified it to the secret reasons I actually wanted this to exist which believe it or not aren't just RAM usage and how long it takes to run. I have one very interesting secret reason I wanted to build this and I actually have a demo of it working now. This little vibe coding demo here is a lot cooler than it might seem initially. I want to talk about all of this from how much it cost to how it worked to why I care so much about this project and most importantly what does this mean for engineering going forward. But after burning $420,000 of tokens, I hope you can forgive me for a real quick break for today's sponsor. How the heck did we get here? AI models are capable of rewriting pretty much any piece of software from scratch in whatever language you want. But trying to find the right three rows in your database using an agent is nearly impossible. Once the data isn't perfectly structured and doesn't have the exact keywords the model's looking for, it's not going to find it. This is why good vector search and retrieval systems are so important. And today's sponsor, Quadrant, has it absolutely figured out. I'm going to quickly mention two things to make the haters go away. It's fully open source and also written in Rust. You can use the cloud if you want or you can self-host. Both are incredible. My team's been using the cloud version a bunch and the things
Watch from 1:31we've been building are genuinely super useful. For example, let's look for videos where I talk about my agent MD. When I do basic keyword search, I get 10 results that aren't really good fits for what I was looking for. But on Quadrant, I'm getting ranked results based on how well my question actually matches the content. And obviously, the video on the right here is a much better fit than the one on the left. I can even get more vague and search for something like anti-claude. And what's funny here is the video that was recommended on the left is the same as before, because this is basic keyword search, like what your agents do by default. on the right with the vector search, we get videos that are very much me being anti-anthropic. I didn't plan this one ahead. I did not think this gap was going to be so big. That's hilarious. The more vague the query, the more valuable it is. For example, I can have it go through all of my issues and PRs on GitHub for T3 Code and ask something vague like T3 Code is making my whole Windows PC lag even when it's sitting idle. I fire this, I get no results from the keyword search, and I get eight relevant results from Quadrant. This makes dduping and finding similar issues so much easier. We're even using it now to index all the faces I use in my thumbnails. I can search for shrugging with both hands out and immediately get results that are good matches for that request. Or I can search something like angry and get all the thumbnails where I looked kind of mad. There's a reason companies like Canva, Discord, and more have been moving to Quadrant. Give your agents search that feels magical at swive.link/quadrant.
Watch from 3:02I'm gonna do everything in my power to not make this video two to four hours long, which is tough because I have so much to say about this, but I think we should do a brief history lesson to explain what this is and why it exists. Typescript is a language kind of. The point of Typescript is to build a system on top of JavaScript so that we can have more confidence about the behaviors of our JavaScript apps. The main thing that we're talking about today is TSC, the TypeScript compiler. The point of the TypeScript compiler is of course to turn the TypeScript that is in your editor into JavaScript that your browser can run because your browser doesn't know how to deal with TypeScript. If you have a function like function user name string console.log user.name. Wow, we're writing code. Such advanced stuff. Been a while, right guys? Wait, there's even a code button there. Look at that. If you took this function and ran it in Chrome or Node or any other JavaScript runtime, it will immediately fail an error because the syntax for the type definition here where I describe user as a parameter that is of type object with a name key that's a string. This is not valid JavaScript syntax. You can't just do this there. Also, the function needs a name. So, log user. You get the idea, though. This would not be valid JavaScript because of this guy here.
Watch from 4:21Turning this into valid JavaScript is actually quite easy. You can go through and trim out all of the TypeScript isms in the JavaScript code. And once you've done that, you can just run it and it works. So the first important thing to understand here, turning TypeScript files into JS files is easy, borderline trivial, and there have been lots of cool tools doing this for a while now. In particular, one blew up a few years ago and kind of changed how we think about webdev. ESB build. ES build was built by Evan W who was the CTO of Figma because he wanted to make it faster to get updates when you change files in your editor to see them in your UI and running all of your code through JavaScript just to remove some things from it so you can put it in the browser was not efficient. So ESB build was built to do a lot of those steps, not just the turning typescript to JavaScript, but also a lot of the other steps like bundling in the CSS, creating the other assets needed, turning it all into one file the browser could actually receive instead of all of these files scattered around your codebase. All the things you need to bundle the app to get it ready for your browser written in Go so that it could be as efficient as possible reasonably without having to like reinvent the universe and manage all your memory yourself. and ESB build was incredible, but it was missing something. We're just talking about turning TypeScript into JavaScript.
Watch from 5:41That's an easy part. The actual hard part is checking the TypeScript, making sure the types are valid and being used and honored correctly throughout your application. I'm going to be so real with y'all. Doing the translation, turning TS into JS was so easy that even just doing it in JS wasn't that big of a deal. Checking the TypeScript to make sure it's correct, way harder, and also way slower. This ended up being where most of the time was spent. As a result of this, the maintainers of lots of big code bases split their transpolation and typeecking compilation paths separately. I remember how bad the hot reloading was initially at Twitch. I would run the dev command, it would take 2 minutes to start up, and then I would go and edit a file, save, and it would take 30 seconds to a minute to appear in the browser. I was so hyped that we got it under 5 seconds by making it so the path that puts it in the browser is separate from the path that is type checking the code to make sure it's correct in its type definitions. Something like ES build let you go even further for those hot reloads from saving to seeing it in the browser, but it doesn't give you enough info to know something like did the types check correctly. And as you've probably seen if you've watched my old videos, the TypeScript type system is incredibly complex and flexible. It's a touring complete system. Here's a fun throwback. One of my friends actually ported all of Doom into Typescript's types, not Typescript, the language.
Watch from 6:58Typescript, the type system. Insane project. Highly recommend watching that video if you haven't seen it before. The point I'm trying to make here is that TypeScript's type system is complex. And it's complex because it can represent all the things you do in JS. Therefore, rewriting the checker in a non-JavaScript language would inherently suck because of all the weird things that JavaScript does inherently, like the way you can change what variables types are as you execute your code. The way that objects inherent traits and the way all of the prototyping systems work and all the the weirdness of JavaScript that TypeScript faithfully encodes is a lot easier to encode when the language encoding it is the same language as the one you're encoding. A sillier way to explain this is that it's much easier to explain how English works in English than it would be to explain how English works in Spanish. Just imagine that complexity, but like times a 100. That's what makes this so hard is all the weirdness of JS has to be encoded faithfully in whatever thing you choose instead. But that was not enough to stop Microsoft, which is why Microsoft decided to rewrite TSC. They wanted it to be native and super fast to make these compilation and type check times way better, which is why they picked Go. You might be confused. In fact, I'll switch over to chat and see how people feel about their choice for Go. What language do you guys think they should have chosen? Jai, C++, C. C's a funny one cuz the creator of Typescript's also the creator of C#. I have the hot take that Go is the right language for this.
Watch from 8:17I know everybody else has opinions, but let me explain why. I recently got dunked on for these tweets, and I think they're relevant here more than ever. I posted in February of 2024, "Go is probably the right language for most JavaScript build and dev tooling." I followed up with the following. Tweeting this now for reference in 3 years when we realize we up going allin on Rust. This take aged poorly, but not for the right reasons. I need to explain the Go one first because this almost directly contradicts with my other takes on Go. For example, here where I'm making fun of Melky, saying, "I suddenly understand why you think Go is a good language. I'm notoriously a Go hater. I just never liked the language much. I I get where its benefits are. I just makes you feel like a monkey more than an engineer from being real. So why did Go make sense here? Go's biggest strengths compared to something like Rust are its flexibility because it's not trying anywhere near as hard to be like super type safe and memory safe. Its type system is a meme. Its concurrency stuff built in is awesome. Like Go's concurrency model has its issues. like channels are stupid, but the like core parts in Go routines, there's some really cool stuff in there. And concurrency is one of the big things that the team wanted with the TypeScript rewrite so that they could take more advantage of your cores. It syntax was surprisingly similar to JS which made it easier to look at two functions from one and the other and understand what they're doing. But I would argue the biggest piece garbage collection.
Watch from 9:47Garbage collection is one of the flaws in Go because it means that your memory isn't being managed by you and at some point it has to be cleaned up which means that the performance characteristics of your app are not reliable. Usually it will respond to a request pretty fast but sometimes if that request comes in when you happen to be out of memory and need to do garbage collection then you'll have these random latency spikes that suck. The benefit is that you get crazy concurrency and you have to think about memory constantly at the cost of garbage collection causing your latency to spike. So if you have a system that you care a lot about and like you want to make sure every request is resolved in under 25 milliseconds, Go might not be the right language because sometimes it will have to garbage collect and it will hold off on executing code until that garbage collection pass is done. To be clear here, JavaScript and Typescript, those are garbage collected as well. Go is an interesting language because it's both low-level and garbage collected. So why would you ever want to use this for a compiler then? Isn't that going to be super slow? No. If you have a 100,000 requests that all come in at 25 milliseconds and then you have to do GC and the thousand first request comes in at 300 milliseconds instead. That seems horrible because that one request is now 10 plus times slower than all the other requests were. But if one out of a thousand requests is 10 times slower, the total time is like less than 1% slower, it's nothing. The impact of garbage collection on the scale of a long process with lots of things in it is much less than when you're timing each individual request. So if you're doing work that you want to have
Watch from 11:13resolved in milliseconds, a garbage collecting language will have flaws you should work around. But if your average request is going to take more than a 100 milliseconds, like I don't know, multiple seconds because you're compiling a giant app, I don't care if you garbage collect one, two, three, 10, 100 times during that request because the garbage collection is going to make up such a small percentage of the runtime. It barely matters. It washes out. So if I was building a web service I wanted to have respond as quickly as possible, Go is not the right language. But if I'm trying to take something written in JavaScript that's really slow that takes minutes to run and shave the time down and get more concurrency, Go is one of the best languages to do this with. But it couldn't have been too hard to do the Rust thing, right? Cuz you had an agent do it in 2 weeks. Now I would like to introduce TS Rust. Writing all of Typescript from JS to Rust would not have been viable. I know this to be the case because of how much the other models I tried like 56 Soul and Astra struggled with this port. I think a lot of that was because going from JavaScript to Rust is not trivial with things as complex as TypeScript itself. And this is where things start to get interesting with my TS Rust port. If we hop into the crates, a lot of things were done, many of which are uh I think questionable is a fair word for it. One of those things is this go std. This is the standard library from Go ported to Rust because the way Opus chose to do this wasn't to rewrite things from Go to Rust. It was to go through this Goport line by line and faithfully port
Watch from 12:44it over to the Rust language. And the result of this was actually quite a bit slower than the Go version because when you have to reimplement things like Go routines, which is in here somewhere, I forgot which it's in. I think it's in the runtime RS. Yeah, runtime RS. This is Go routines ported to Rust. This happened because Go was a much better starting point for a Rust port than JavaScript is. The Microsoft team faithfully by hand went through every line of code in Typescript, ported it to Go, and then started refining it to make it better and faster in Go. My agents did the same thing from the Go version to Rust, and they succeeded. I didn't even know that my agents had done this until somebody else pointed it out on Twitter. And as soon as I saw that, I was like, "Oh, that's hilarious. That explains a lot." Because again, when TSGO RS was ready for its first run when Opus finished, it was two to three times slower than the Go version. So, I told it in a/goal, your objective is to make it at least two times faster in every way. And then I let it run for 2 weeks. More importantly, I let it run on my six cla accounts. All of the work done in TS Rust with Opus 55, which effectively gave up on the old version and restarted, was roughly $24,000 of inference if I had paid full API prices. But, as you know, I didn't because I have subscriptions. So, I also tried pricing it out using all my logs to see how much of my utilization went to this. The equivalent in your sub usage for all the work I did here was between 925% and 983% of one account's
Watch from 14:14weekly limit, which means about two and a half months of usage on a cloud code sub. When you consider that we got a free reset, this ends up being about $500ish give or take of subscription usage. And at that price, this is actually a good value to have built. I know it sounds insane, but $500 to have TypeScript in Rust, which is more memory efficient and faster, 2x faster than the Go version, is genuinely super useful. And I am to me, this is money well spent. And I know that seems insane because the 20,000 or the $420,000 headline number sounds crazy, but with the subsidization from a subscription and the fact that only the 20k of Claude mattered, this being 10 weeklys of usage is just hilariously good value in my opinion. So, if you were thinking, man, the Microsoft TypeScript team wasted so much money and time porting to Go, they could have just waited and let AI do it in Rust. No, I don't think the AI would have done this viably from JavaScript and TypeScript to Rust, but it was able to do it with Go relatively trivially. And the result is a pretty much 100% drop in replacement for TSGO and TSC7. I know that for a very silly reason, the issues, there were five, there are now four. I audited these issues. Well, my agent and I audited these issues. And of the six issues that have been opened total so far, one of them was specific to TS Rust. The other five issues were problems that exist in TSC7, the specific version that we were targeting.
Watch from 15:46So, these aren't things that are wrong with the rewrite. These are bugs in Typescript itself currently that were faithfully ported to my Rust version. And I think that is hilarious that not only did I port the intentional things, I ported the bugs, too, because this truly is a fully compatible drop-in replacement. I suppose now would be a good time to talk about the resulting performance. Since I started this before bun was even a thought from Jared, I was initially running all my numbers against Typescript 7. And for realworld projects like T3 code, we saw a 2x improvement from TS7 which is TSGO and as much as a 20x improvement on various things for TSC 6 which was the last TypeScript JavaScript based TSC. I started with the T3 code checks because I did something fun here. One of the things effect TS does and if you're not familiar effect is a popular library in Typescript to make building reliable concurrent systems more viable and composable as well. effect uses and abuses TypeScript so heavily that they've actually built their own fork of TypeScript effectively. It's a patch you apply to your TypeScript config and your TypeScript like package in your repo to get more diagnostics and features from effect into the type checking. I built that directly into TS Rust because we rely on this a lot for things like T3 code. So for our use case in T3 code, not only are we way faster than TS7 with effects TSGO plugin, we're actually way faster, like fourish times faster than bun as well, not because bun check is slower. Bun check is super fast, but because we have to do the effect check on top of it after. And as soon as
Watch from 17:11you're doing both, it just ends up way slower end to end. But this is just checking in T3 code. Let's check other projects. some real simple things like I don't know the entirety of TRPC server or centuries codebase or excaladraw and funny enough I was checking effect as well I actually think bun couldn't pass on the effect codebase if I recall bun also raised incorrect errors on sentry and tRPC so the bun checker while incredibly impressive and faster in a lot of cases does not have faithful enough checking for me to trust it in these types of big projects I think bun check fits well with like the other bun tools tools in that like if you start a new project with bun and you use it for everything, bun is a great solution. I actually honestly would recommend bun over ts rust for almost everything simply because I have higher faith it'll be maintained. But tsc my project ts rust is a way more faithful recreation and as a result it does not get weird errors in these projects. It works faithfully in projects like TRPC in projects like T3 code in projects like the effect repo itself. And the improvements we see here for most of this stuff is 13x plus for going from TSC6 to TSC RS. And it's usually a 2x improvement from the Go version. But ready for the spicy take that's going to make all of this way more confusing. I didn't do this for performance. I did not make TypeScript Rust because I wanted something faster than TypeScript Go. So why did I do this? There is the silly reason and then the potentially groundbreaking one. And I'm going to start with the silly one. Silly reason is because LLMs couldn't do it. Models
Watch from 18:42were not able to make these types of ports. It had become a meme. I got in flame wars with people for claiming they would be able to port TypeScript to like C++ and things in a month, which just obviously you couldn't do. And I got in a flame war with somebody, ended up getting blocked by them. It was a show. This is like back like the old 2020s, like early days of Typescript posting on Twitter before AI took over all of it. And I was getting in like heated fights about these things because it was obvious you couldn't just rewrite TypeScript in another language, which is why it was always my like reach goal, the impossible thing that like you could never actually have the unobtanium to an extent in the web dev world. And as such, it was a fun thing to throw practically unlimited tokens at. I had never tried an anthropic model on this task because they would burn my usage too fast. But with Opus 5.5, usage was burning so slowly and I had so much I said, "Fuck it. Let's see what happens." and it was able to do it way more effectively than I ever could have guessed. I did it because I didn't think models could. And now that they can, I'm running out of things to throw the LMS at. Like I don't know what tasks an LLM would ever be able to do that are impossible for them now, but could be possible in the future. It's like hard for me to comprehend. Like I am out of engineering ideas that are like too much for the model. This was my too much for the model and now it's done it. But what's the real reason? What reason did I have to want a Rustbased version of Typescript when we already had a super fast go one? I got almost everything I wanted out of TSGO. Why did I want something else? There are small things like I wanted the effect stuff built in.
Watch from 20:13I wanted to be a little easier to extend and play around with. But the main reason was this goes back to the garbage collection bit we talked about before. Garbage collection is an incredible way to take a burden off of engineers to allow them to ship software faster that is still really fast. But since garbage collection is a complex process that requires you to analyze memory and do a lot of memory management stuff, taking a Go app and bundling it into web assembly is not trivial because you can't just bundle the code the developer wrote. You also have to bundle all of the garbage collection. The the end result here is that it's pretty difficult to write a Go app and then bundle it as W was less than 10 megs and it's still going to have lots of weird performance issues because of that garbage collection aspect of it. This is why tools like ESB build struggled to get WSOM working for so long. This was also one of the things we lost with TypeScript Go. Previously, since Typescript was all just JS in the end, you could throw it in the browser, you could throw it in a V8 isolate, you could run it in Node and it just worked. With TSGO, we kind of lost that. In fact, we lost a lot of the JavaScript APIs we use to interface with TypeScript compiler in the past. Why do I care that much though? We're all running our agents on our computers or someone else's computer, right? What if we're not in the future? Intelligence is expensive. People like Lauren aka Potato at Cursor and SpaceX are spending allegedly 132 mill. This number is I'll explain why in that other video. Subscribe if you haven't, by the way. That red button helps us a ton and you'll get to be
Watch from 21:43around for this rant which will be fun. Ignoring the fact that those numbers aren't real, the rate at which prices go down is real. Flashbang warning for artificial analysis. I need to port this particular bench over and I haven't yet. This is the cost efficiency over time chart from artificial analysis. This takes a given score that models could get. So like DeepSync R1 getting a score above a 10 on their index in January of 2025 and it tracks how much the price for models of that level of intelligence goes up or down over time. Where things get really crazy is all the way over here on the right where we see models that are super smart like Opus 5, which broke into the 50 point range for the first time costing $5.86 per task when they ran it. And now we have models like 61 Soul that get the same score for 32. So it's already decreased by 10x in cost for that level of intelligence in just like four months. At a given level of intelligence, cost goes down absurdly fast. Like 10x a year is a conservative estimate here. If you take the 30 point range, which cost $2.50 in February of this year and got down to 3 by summer of this year, that is 8x per month decrease, averaging out to a,000x a year annualized. The rate at which prices for intelligence goes down is insane. But intelligence isn't the only thing we're paying for. When you're building apps with AI, you also have to pay to host it. You have to pay to store the data in the database. You have to pay for
Watch from 23:13whatever like systems and providers you're doing. You have to pay Apple to distribute the app. You have to pay for all these other things. But there's one more thing you have to pay for that people don't think about a lot. You got to pay for the compute that your agent runs on. If you're one of those $5 VPS bros, now's your time to shine because we actually need to talk. One of the cool things cursor does right now is the cursor cloud. When you spin up a cloud thread in cursor, it will run on a full VM with 32 gigs of RAM. I think it might be 16. I'm pretty sure it's 32 in Cursor's cloud with a real guey, real processor, real everything it needs in Linux so that it can spin up your app and test it with a real computer. Every thread gets its own computer. You spin up four threads, that's 128 gigs of RAM reserved to you for that bit. And right now, Cursor doesn't charge for that. So the problem now is that your inference is way more expensive than the VMs to the point where the VMs are almost a rounding error, which is why giving you those free VMs is not really a big cost. But remember what I just said, the cost of inference is going down and it's going down rapidly. What about the cost of compute? What about the cost of VMs? See where I'm going here? The cost of renting a 16 or 32 gig box is going up steadily thanks to AI sweeping all the demand for RAM. The result is that VMs and compute that you want to run your agents on, not the compute running the models, the compute running the code that the model's editing. That compute has been slowly going up in price over time, but the inference compute is plummeting in price. So, one of these
Watch from 24:44things goes up by 25 to 30% every year. the other goes down by 10 to 100x every year. We're going to quickly get to a point where inference is cheaper than the VM. With a model like GLM53 flash, running it for an hour straight is going to cost you a few dollars. The VM it's running on is likely more expensive than the model in the inference you're doing in that window. And as models of that price get smarter and as models at that tier get cheaper, this will be hit quickly. at which point compute will be a constraint for when we're building with agents. Right now we're being given the VMs for free, but that is not going to last. Even Cloud Code isn't doing this anymore. Cloud code now gives every subscriber a credit that they can use towards the cloud compute that they use if they use Claude's cloud product. There's a lot of layers to why it works this way. The compute and the memory are expensive, so they have to like worry about that. The reason that it has to be reserved this way is because of how our agents work. Our agents expect to run on a real computer with a real terminal with real bash so that it can run commands. It can run rip grape. It can search through the codebase and find things. It expects to have native tools for editing, for checking how much CPU is being used, for searching for files, for moving files, for running git commands, for all these things. Our agents expect and are trained on real Linux boxes or Mac or whatever else.
Watch from 26:14They're trained for real systems that they can do things on. And surely you can sandbox them some amount, but all of those additional layers cost additional resources. Sandboxing harder is not going to make the compute cheaper. If anything, it's going to make the comput and the memory more expensive. Which means we need a better abstraction layer. Ideally, in a future where agents are cheap and writing code is incredibly cheap because the models and inferred have gotten so cheap, the thing that they write them on and to test them on should not be this expensive. Thankfully, this is a thing we have explored for a very, very long time in the industry. And this is a war you guys have heard from here. And we get to revisit an old war that we have not talked about in a while. We get to talk about big orange. Cloudflare. What is the word that y'all associate most with Cloudflare? Spam me chat. When you think Cloudflare, what do you think? Cost, cheap, free, cheap. There's like cloud and CDN. Most of them here are free. cheap cost related. A lot of what people associate with Cloudflare is cheapness. The reason for that is how Cloudflare's platform works. When I write code for a backend and I host it on Cloudflare, I'm not wrapping it in a Docker image that they deploy on a server. In fact, when I put code on Cloudflare, it's not deployed. It's not running. It runs when a request comes in. The request triggers Cloudflare to spin up a worker. A worker is an isolated JS process inside of V8,
Watch from 27:35the JavaScript engine powering Chrome and a lot of other things. And that isolate isn't a kernel. It's not Linux. It's not an OS. It is a isolated execution layer where your browser or whatever else can execute JavaScript. This is really useful in browsers because you can have different tabs doing different things and not have access to each other's stuff. or you can have one tab with four things running in it that are different that can communicate with each other but not interfere with each other. V8's isolation layer is incredibly powerful. It's not the most secure thing, but it's pretty secure. And what Cloudflare has done is taken those isolates, made them as secure as possible so that when a user goes to a website you built and deployed on Cloudflare, it can grab that code from storage, throw it on this isolate that's just sitting there waiting, execute it, then kill it. So on a platform like Verscell or Heroku or Hetsnner or any other traditional host, if you and I both host an app on that platform, we both have our own Linux instance effectively that is running with our code in it. On Cloudflare, your code and my code are executing in the same instance of Linux, just split higher up in the V8 layer instead of with VMs and Docker. Hopefully this all makes sense. If it doesn't, ask your agent. I'm not going to keep ranting about this. I've talked about this in a lot of my videos before. There are problems with this model, though. If we're running inside of a JavaScript isolate, we lose a lot of things like we can't read files from the file system because the file system isn't in the isolate and that would allow us to escape and screw with other people's stuff. We can't run native code because native code escapes the isolate and
Watch from 29:06needs to run in the actual like system which we can't do here which means we can't run a lot of other languages in Cloudflare workers. Means we can't do traditional persistent servers in workers. There's a lot of these things and Cloudflare has built lots of clever solutions to these problems, but the code that we're talking about, the JavaScript is running in this virtual layer above V8 instead of running at the VM layer. This makes Cloudflare an incredibly cost-effective and memory usage effective platform. And that's why it's so cheap. It's actually cheap to run. They're not just pricing things cheaper because they want to win. They're doing it because their platform allows them to and still have margin. In this sense, Verscell is not overpriced because they want to make more money than Cloudflare. They are charging probably similar margins, but it's higher cost to them, too. But since Cloudflare is just running in this JavaScript layer, you can't really run an agent in that because you can't write and read files. You can't run a dev server. You certainly can't run TypeScript Go to turn your TypeScript into JavaScript and get feedback on what's wrong to your agent. Oh, huh. Weird. It looks like I did exactly that. This is the demo I built in Astra, and it has some dumb decisions that are wasting a bunch of memory. So, I'm going to switch to this version that Opus made because it's a better representation of what I have in mind here. Before I can justify the chaos I'm about to show you, I need to show one last reference.
Watch from 30:37There's a library called Just Bash. Just Bash was created by Malta at Verscell and has become a pretty legit library. The point of just bash is to provide agents a full fake bash environment in memory in JavaScript so that you can import this bash from just bash and then execute most traditional bash commands with a fake file system, fake processes, fake everything, allowing for an agent to operate as though it is working within bash, even if it isn't. But in order for this to work, you also need to be able to compile and type check and do all the other things that you would do in a normal file system if you wanted to use this to build. If you set this up as a way for your agent to explore fake files to find content, awesome. But you're not going to build a real app with this. You need too many other things, including TypeScript Rust. Well, TypeScript Go or TSC traditional. But now with TypeScript Rust, we can do really stupid things like here where I had GBD61 Soul build a full app that is entirely functioning all in a V8 isolate. This one is running in node on a server technically, but this could just be run on your browser. There are some limitations, but not as many as you'd think. And this all just ran and worked and did its thing. And the entirety of utilization for this, everything that was used here, from the agent to the compute it ran on to the compute that the app itself is running on to the type checking and compilation and everything else just ran on my isolate and used under 260 megs of RAM.
Watch from 31:54and it peaked at like 4% CPU. Do you understand how cool this is? This will allow for you to have an agent build an app where both the agent and the compute it runs on, not the like compute for the inference, but the compute that it's running the code on and building on all together for pennies. And doing this with Go would suck because TSGO wastes a ton of resources simply because porting that to JS is not viable. WASM, but you get the idea. Web Assembly is not a good fit for Go. It is an incredible fit for Rust. And just to emphasize my point here, when I had Astra build this, it chose to use ESB for the bundler. And just sitting here idle, it's using 2x the RAM of the other version because the inclusion of a go-based package like ES build inherently just eats memory and resources and the data that has to be loaded in. Like we have to load way more in because of how much bigger the bundle is. I see people confused about how this would work on Cloudflare. Do you know what Cloudflare, Chrome, and Node all have in common? They're all using V8 isolates. The way these things work is that they are all using the same JavaScript engine. And when you load a Chrome extension or you load a new file or you have some process in Wom, those are all different isolate workers is the term for it. And they can communicate with each other if they're allowed, but they cannot interfere with each other.
Watch from 33:22What I built here isn't impressive because I rebuilt Bolt but used TS Rust. What I built here is impressive because I could turn my file system on read only and it still works. Nothing here left RAM. Nothing here left V8. I'm not doing anything on the system. This all exists within V8. The whole dev loop exists in V8. There isn't anything comparable. I see people mentioning like random like database companies and stuff that let you execute code of them. No, no, you're you're not getting it. And if you guys aren't getting it, you're my stream audience. I am not excited to see how others deal with this. Let's talk about how computers work really quick. At the bottom you have hardware, CPU, RAM, whatever else. Above that you have drivers. Drivers are provided by the thing above the kernel. The drivers are the method in which the kernel interfaces with hardware. Above this we have the operating system. Above this we have applications. I would argue actually we have file system kind of. Then you have applications. So let's figure out how something like Heroku Hetner or any normal cloud works. If we wanted to deploy a Docker image on AWS, we're not deploying it here because these things have to exist on AWS's hardware so that they can put more systems above. So what they'll do is in the application layer they'll put something like Docker and you know what you do after that you copy paste everything and do this.
Watch from 34:45This is how hosting on normal VPS providers works on top of the file system and operating system. You have an application, usually something like Docker, and it's not technically running a new kernel instance always. Usually, it's doing a shim kernel. So, I could claim like for this layer here that this is all like the like Docker hypervisor I don't fully understand it, but like docker shim to kernel, which makes the kernel stuff more efficient for sure. So this shim's there to allow you to skip steps when it makes sense to. And then you're building an OS on top, which is why on something like AWS, you can have one box that AWS is providing that has one person running Ubuntu, one person running Windows Server, and one person running, I don't know, Debian or something. That's all happening in the application layer of Docker on top of the operating system. Which means every one of these things takes RAM. This is why every time you spin up a new Docker container on your computer, it's using a bunch of resources. It takes resources to run operating systems in your operating system. Let's compare this to how JavaScript works. So, we have our whole stack here, but let's say the application is Chrome. Inside of Chrome, you have V8, which is the JavaScript runtime. And in V8, you have all of these isolates. And all of these isolates can execute different JavaScript files and functions at the same time. This is way more efficient because now every file or application if it fits in one of these isolates, it doesn't need to shim out to the kernel,
Watch from 36:11run an operating system, and steal a bunch of resources. Like, like, let's be real here. What takes more resources on your computer? A new Docker instance of Postgress or a new tab in Chrome? Tabs in Chrome, as slow as y'all might think Chroma is, obviously uses way less resources. Absurdly, comically, obviously. The reason why is the insane billions of dollars that Google has pissed into V8. They wanted to make this engine as efficient as possible so that the web could be a compelling platform so they could finally stop making shitty Windows apps. Google spent a billion dollars so that Chrome would be fast enough so they could stop making Windows versions of Google Earth and That was a lot of money and effort put in to V8. And this has benefited the whole ecosystem in ways that are hard to comprehend, including all of Cloudflare. Cloudflare's whole platform is built on the idea that virtualizing at the operating system level by putting an OS at the app layer is stupid and VH is a much better abstraction for putting applications that customers are hosting on top of. The catch is that these are just JavaScript. They don't have full system capabilities. You can't call the kernel because there's no kernel you have access to. You can't write files because there's no file system. It is just JavaScript. Anything that a Chrome tab can't do, a V8 isolate can't do with the exception of cores. I don't even talk about cores now, but remember all the problems here are you don't have a file system. You don't have native code access. You can't do the things you need to build. Isolates are a good place to put the code once it's built. They are not a good place to build. On Versel,
Watch from 37:42for example, Verscell sandboxes are the same compute and primitives as hosting on Verscell is. on Cloudflare. Cloudflare sandboxes suck because Cloudflare does not know how to make things work well with Docker because that's not their focus. Their focus is this isolate layer. So, Cloudflare sandboxes run on entirely different compute with entirely different platforms than Cloudflare's traditional like worker hosting. So, the thing I did here, the thing I enabled inspired by Just Bash because the thing that makes Just Bash special is it lies to agents living in this isolate layer. So they think they have access to the file system and to these other things. And that is what we have built here. If I was to flip a switch on the file system for the server running this demo to be read only. So I literally could not edit files on the computer. I could only do things in RAM in my V8 isolated node. This will still run exactly the same. That is what makes this demo cool. It would be fully possible to run this entire demo in a Cloudflare worker. So instead of needing half a gig of RAM and a bunch of reserved CPU, I can throw it in here and it costs nothing. Another thing to think about here is how CPU allocation works. If you have a VM that has two CPUs and I have a VPM that has two CPUs and both are idling, we're still wasting four vCPUs. There are some things that make it easier to like reallocate and have virtual like cores not be guaranteed the same way. But the way it's working on Cloudflare is with the built-in event loop where when your code isn't doing anything, it doesn't cost anything other than a bit of RAM.
Watch from 39:13So when you're waiting for the next token from an inference provider, that's not costing you any compute cuz the CPU is doing nothing. So that CPU is being used by others. When suddenly data comes in, it will execute the next task because your app, my app, and another person's app are all sharing the same event loop. We're all sharing the same scheduler effectively. So it becomes way more efficient for us to run things here. or even a long running task that takes hours probably spends most of its time idle. Now that only costs money for when a token comes in, not for the time between tokens. If inference continues to get cheaper, this type of building will become essential because this will enable us to build a gentic developer experiences that are cheap on all sides where running the agent is cheap and giving the agent compute to execute on is also cheap. I see some people in chat catching on here asking is this related to lakebed? I won't say more yet. I need to give you guys something to be excited about because I just wanted to focus on this. The fact that TS Rust is one of the final pieces I need to take a full endtoend TypeScript developer experience and do it in an isolate instead of doing it in a VM. For those who are curious and want to see the very complex prompts I use to build TS Rust, here you go. I told it to work in code sandbox ts on this computer. Read the agents MD and the type check or accountability uh files first. Those files included audit notes from the 16-hour run and asked for some things. Make the cargo job count configurable. Add script state and split yada yada. And then update the agency so
Watch from 40:41we can do more parallel work. To which I said, do it. Great work. I trust your judgment fully. Keep going and doing what you think is best to continue improving compatibility and performance. You can probably move to main soon as well. I did a follow-up at some point saying get whatever the best current state of the work is and set it as the main branch ASAP. I don't care how. I trust your judgment. Then later I did a slashef effort cuz I noticed I was on the wrong effort level. Actually, I I was on ultra code and it kept bumping me down to medium when it crashed because it was running out of memory and crashing systemd constantly, but once I like added some things at the OS level to fix this, I had to rerun/ef effort to put on ultra code again on XI and then the / goal of complete the port, get as close as possible to 100% compatibility with TSGO, and then also improve performance. Do the efforts in parallel. Multiple workflows and sub aents should be running at all times to maximize parallelism. Orchestrate how you need to keep them working independently all on this machine. And I noticed it was running out of compute all the time. So I took other machines I had in my fleet, configured them for SSH access, and told it you have SSH access to other machines now like Cup 2 and Alvin, two of my uh VPSs. I noticed it kept stopping because it like got full compat and then like would make one performance room and be like, look, I fixed performance. So it needed a better end state. So I told it with a new goal, our target is minimum 2x faster than the go build. Continue iterating until we get there. And then it did for two weeks and now we are here. So, as you see, I wrote some very complex prompts for all of this. Super super high IQ things I was doing here.
Watch from 42:12I'm a mega prompter. Real talk though, the things I did here were not around the prompts. It was around the environment and setup to allow it to do this stuff. So I like I had other threads going where I was auditing constantly figuring out where the blockers were, what's taking a while, what failure modes are happening again and again, and then updating the environment to allow agents to operate more effectively in parallel without stepping on each other. And after a bit, we built systems that allowed that to work and ended up shipping much faster once we did. But my energy was honestly similar to how I work as a manager where I wasn't spending my time in the trenches writing code. I was spending my time making sure that my team had everything they needed to be as effective as possible in finding where the slowdown was and helping them get around it so that the slowdown would stop happening. Ended up being really fun. And I know I was making a joke about being a mega prompter, but uh I have been politely requested to mention mega dev a bit more. I am finally doing a paid course for the first time ever. Feels weird and I want to be very clear about this. Any information that will help you get a job is not going in here. It's just going on my channel for free. The point of this is for employed developers that want to maximize how much they use and abuse AI to ship real software in the real world. I'm working with three other incredible instructors, people who have helped me learn how the web and technologies work for years now.
Watch from 43:38I'll do the sidebar so you can see Angie, who I love to death, Kent, who taught me half of what I learned about React back in the day, and John Linquist, the creator of Egghhead. If you have to pay for this with your own money, I wouldn't recommend it. But if you can get your boss to sign off on using internal like expenses for educational stuff for it, it'll be fun. And I'm going to do a live workshop with it as well, which should be very fun. So that's where I'll get to talk more about all of these chaotic strategies for maximizing what your agents are capable of, which should be fun stuff. What a journey this one was. I my AI psychosis was already pretty bad, but the fact that all these truly crazy ideas I've been thinking about for years now are actually doable. Like I've always wanted to have a full stack developer experience that runs purely within V8 back when Bolt was still stack blitz and they first put out their full like JS runtime thing inside of WASM. So you could have a full VS codeike experience with apps bundling in the browser. I was blown away. I wanted to go further with that with Cloudflare and I went as far as trying to reverse engineer Stack Blitz and like decode the JS binary and their WASM to figure out how they were doing it because they're there was so much cool stuff there. That's how I became friends with Eric back in the day and that stuff was so cool. I've always been a nerd about how the code we write actually runs and executes and how these systems compose and operate. It's such fun stuff and now I get to try out my crazy ideas. Previously, the vast majority of this was just like imaginary. Like I was playing these games in my head, and if any of those ideas went far enough, I could maybe
Watch from 44:56talk about it with a smarter person and maybe inspire them to build something kind of like it in 5 years in the future. Now, when I have one of these insane ideas, I tell Opus and then I see if it works or not. Even now, I wanted to see if this will actually run on Cloudflare. So, I just told Opus, "Go do it." And it probably will. Absolutely insane. I on one hand, the engineering we know and love is dead. And very, very much so. these types of YOLO projects should not be viable at all. Like you shouldn't even be able to get it to pass the tests, much less actually work as a drop in replacement for TypeScript. But yeah, if you still think AI dev isn't the future and isn't obviously where all of software dev is going, you deserve the lack of employment you'll have in 3 months because this is insane. It's actually hard to fathom how capable these tools have gotten. I think this is everything I had to say on this one. I was up till 6:00 in the morning prepping all of these demos and I'm so proud of what Opus and I were able to cook. Not because I think everyone should use this, but because I can find out if these ideas work or not in ways that are just hard to fathom. I actually feel like we're peering into the future with some of this stuff. And it's so cool to see. I cannot imagine where we will be next year. And I hope you guys are excited to go on this journey with me because it's only going to get more chaotic from here. And with that, until next time, peace nerds.
Watch from 46:27Full transcript
- 0:00
Announcing TSC RS aka Typescript Rust. My complete rewrite of the TypeScript compiler, Type Checker, and LSP all in Rust. This is a project I've had agents working on for 5 months now. I burned $400,000 of codeex tokens and got nowhere. I burned $20,000 of Opus 5.5 and I got there in 2 weeks. An important
- 0:21
note, I have not read a single line of code. If you've been around for a bit, you probably know I've been using this TypeScript Rust rewrite as my favorite little playground for trying out new models and seeing where capabilities are going. I did that because I was looking for things that just seemed impossible for the models to do. The reason I first started the TypeScript Rust project was because I had a crazy amount of
- 0:42
inference with the early access testing for GPT56 soul and it was crushing everything I threw at it. So, I decided to reach for something a bit bolder. Not a bit bolder, like just impossible. It didn't seem viable to just rewrite all of TypeScript in Rust. It seemed like a meme and it was the furthest I got using OpenAI models was roughly 84% compatibility. And while like impressive
- 1:04
that it could get that far, didn't really seem like it was going to get much further because at that point the last 16% is way harder. And as you get closer and closer, the edges that are left get worse and worse. Which is why it feels extra insane that not only is TS Rust now out and available for people to use with a real package on npm, more importantly, it's way faster when
- 1:25
compared to TypeScript v6, which was the last version written in TypeScript and JavaScript itself. It is 12.5 times faster. When compared to TSGO, which is TypeScript v7, the version that the official folks at Microsoft built, it's around two times faster. We're not the only ones who did this. Bun also dropped bun check which is a rewrite of the typescript checker in Rust that is also
- 1:47
way faster and I'll be real is even faster than what we built with PS Rust. But there are catches to all of this from how I built it to how I verified it to the secret reasons I actually wanted this to exist which believe it or not aren't just RAM usage and how long it takes to run. I have one very interesting secret reason I wanted to build this and I actually have a demo of
- 2:09
it working now. This little vibe coding demo here is a lot cooler than it might seem initially. I want to talk about all of this from how much it cost to how it worked to why I care so much about this project and most importantly what does this mean for engineering going forward. But after burning $420,000 of tokens, I hope you can forgive me for a real quick break for today's sponsor.
- 2:29
How the heck did we get here? AI models are capable of rewriting pretty much any piece of software from scratch in whatever language you want. But trying to find the right three rows in your database using an agent is nearly impossible. Once the data isn't perfectly structured and doesn't have the exact keywords the model's looking for, it's not going to find it. This is why good vector search and retrieval systems are so important. And today's
- 2:49
sponsor, Quadrant, has it absolutely figured out. I'm going to quickly mention two things to make the haters go away. It's fully open source and also written in Rust. You can use the cloud if you want or you can self-host. Both are incredible. My team's been using the cloud version a bunch and the things we've been building are genuinely super useful. For example, let's look for videos where I talk about my agent MD. When I do basic keyword search, I get 10 results that aren't really good fits for
- 3:12
what I was looking for. But on Quadrant, I'm getting ranked results based on how well my question actually matches the content. And obviously, the video on the right here is a much better fit than the one on the left. I can even get more vague and search for something like anti-claude. And what's funny here is the video that was recommended on the left is the same as before, because this is basic keyword search, like what your
- 3:32
agents do by default. on the right with the vector search, we get videos that are very much me being anti-anthropic. I didn't plan this one ahead. I did not think this gap was going to be so big. That's hilarious. The more vague the query, the more valuable it is. For example, I can have it go through all of my issues and PRs on GitHub for T3 Code and ask something vague like T3 Code is making my whole Windows PC lag even when
- 3:52
it's sitting idle. I fire this, I get no results from the keyword search, and I get eight relevant results from Quadrant. This makes dduping and finding similar issues so much easier. We're even using it now to index all the faces I use in my thumbnails. I can search for shrugging with both hands out and immediately get results that are good matches for that request. Or I can search something like angry and get all
- 4:13
the thumbnails where I looked kind of mad. There's a reason companies like Canva, Discord, and more have been moving to Quadrant. Give your agents search that feels magical at swive.link/quadrant. I'm gonna do everything in my power to not make this video two to four hours long, which is tough because I have so much to say about this, but I think we should do a brief history lesson to
- 4:33
explain what this is and why it exists. Typescript is a language kind of. The point of Typescript is to build a system on top of JavaScript so that we can have more confidence about the behaviors of our JavaScript apps. The main thing that we're talking about today is TSC, the TypeScript compiler. The point of the TypeScript compiler is of course to turn the TypeScript that is in your editor
- 4:54
into JavaScript that your browser can run because your browser doesn't know how to deal with TypeScript. If you have a function like function user name string console.log user.name. Wow, we're writing code. Such advanced stuff. Been a while, right guys? Wait, there's even a code button there. Look at that. If you took this function and ran it in Chrome or Node or
- 5:16
any other JavaScript runtime, it will immediately fail an error because the syntax for the type definition here where I describe user as a parameter that is of type object with a name key that's a string. This is not valid JavaScript syntax. You can't just do this there. Also, the function needs a name. So, log user. You get the idea,
- 5:36
though. This would not be valid JavaScript because of this guy here. Turning this into valid JavaScript is actually quite easy. You can go through and trim out all of the TypeScript isms in the JavaScript code. And once you've done that, you can just run it and it works. So the first important thing to understand here, turning TypeScript
- 5:56
files into JS files is easy, borderline trivial, and there have been lots of cool tools doing this for a while now. In particular, one blew up a few years ago and kind of changed how we think about webdev. ESB build. ES build was built by Evan W who was the CTO of Figma because he wanted to make it faster to get updates when you change files in
- 6:17
your editor to see them in your UI and running all of your code through JavaScript just to remove some things from it so you can put it in the browser was not efficient. So ESB build was built to do a lot of those steps, not just the turning typescript to JavaScript, but also a lot of the other steps like bundling in the CSS, creating the other assets needed, turning it all into one file the browser could actually
- 6:38
receive instead of all of these files scattered around your codebase. All the things you need to bundle the app to get it ready for your browser written in Go so that it could be as efficient as possible reasonably without having to like reinvent the universe and manage all your memory yourself. and ESB build was incredible, but it was missing something. We're just talking about turning TypeScript into JavaScript.
- 6:58
That's an easy part. The actual hard part is checking the TypeScript, making sure the types are valid and being used and honored correctly throughout your application. I'm going to be so real with y'all. Doing the translation, turning TS into JS was so easy that even just doing it in JS wasn't that big of a deal. Checking the TypeScript to make sure it's correct, way harder, and also
- 7:20
way slower. This ended up being where most of the time was spent. As a result of this, the maintainers of lots of big code bases split their transpolation and typeecking compilation paths separately. I remember how bad the hot reloading was initially at Twitch. I would run the dev command, it would take 2 minutes to start up, and then I would go and edit a file, save, and it would take 30 seconds to a minute to appear in the browser. I
- 7:41
was so hyped that we got it under 5 seconds by making it so the path that puts it in the browser is separate from the path that is type checking the code to make sure it's correct in its type definitions. Something like ES build let you go even further for those hot reloads from saving to seeing it in the browser, but it doesn't give you enough info to know something like did the types check correctly. And as you've
- 8:03
probably seen if you've watched my old videos, the TypeScript type system is incredibly complex and flexible. It's a touring complete system. Here's a fun throwback. One of my friends actually ported all of Doom into Typescript's types, not Typescript, the language. Typescript, the type system. Insane project. Highly recommend watching that video if you haven't seen it before. The point I'm trying to make here is that
- 8:23
TypeScript's type system is complex. And it's complex because it can represent all the things you do in JS. Therefore, rewriting the checker in a non-JavaScript language would inherently suck because of all the weird things that JavaScript does inherently, like the way you can change what variables types are as you execute your code. The
- 8:43
way that objects inherent traits and the way all of the prototyping systems work and all the the weirdness of JavaScript that TypeScript faithfully encodes is a lot easier to encode when the language encoding it is the same language as the one you're encoding. A sillier way to explain this is that it's much easier to explain how English works in English than it would be to explain how English
- 9:04
works in Spanish. Just imagine that complexity, but like times a 100. That's what makes this so hard is all the weirdness of JS has to be encoded faithfully in whatever thing you choose instead. But that was not enough to stop Microsoft, which is why Microsoft decided to rewrite TSC. They wanted it to be native and super fast to make these compilation and type check times
- 9:25
way better, which is why they picked Go. You might be confused. In fact, I'll switch over to chat and see how people feel about their choice for Go. What language do you guys think they should have chosen? Jai, C++, C. C's a funny one cuz the creator of Typescript's also the creator of C#. I have the hot take that Go is the right language for this.
- 9:47
I know everybody else has opinions, but let me explain why. I recently got dunked on for these tweets, and I think they're relevant here more than ever. I posted in February of 2024, "Go is probably the right language for most JavaScript build and dev tooling." I followed up with the following. Tweeting this now for reference in 3 years when we realize we up going allin on
- 10:08
Rust. This take aged poorly, but not for the right reasons. I need to explain the Go one first because this almost directly contradicts with my other takes on Go. For example, here where I'm making fun of Melky, saying, "I suddenly understand why you think Go is a good language. I'm notoriously a Go hater. I just never liked the language much. I I
- 10:29
get where its benefits are. I just makes you feel like a monkey more than an engineer from being real. So why did Go make sense here? Go's biggest strengths compared to something like Rust are its flexibility because it's not trying anywhere near as hard to be like super type safe and memory safe. Its type system is a meme. Its concurrency stuff built in is awesome.
- 10:49
Like Go's concurrency model has its issues. like channels are stupid, but the like core parts in Go routines, there's some really cool stuff in there. And concurrency is one of the big things that the team wanted with the TypeScript rewrite so that they could take more advantage of your cores. It syntax was surprisingly similar to JS which made it easier to look at two functions from one and the other and understand what they're doing. But I would argue the
- 11:10
biggest piece garbage collection. Garbage collection is one of the flaws in Go because it means that your memory isn't being managed by you and at some point it has to be cleaned up which means that the performance characteristics of your app are not reliable. Usually it will respond to a request pretty fast but sometimes if that request comes in when you happen to be out of memory and need to do garbage
- 11:31
collection then you'll have these random latency spikes that suck. The benefit is that you get crazy concurrency and you have to think about memory constantly at the cost of garbage collection causing your latency to spike. So if you have a system that you care a lot about and like you want to make sure every request is resolved in under 25 milliseconds, Go might not be the right language because
- 11:51
sometimes it will have to garbage collect and it will hold off on executing code until that garbage collection pass is done. To be clear here, JavaScript and Typescript, those are garbage collected as well. Go is an interesting language because it's both low-level and garbage collected. So why would you ever want to use this for a compiler then? Isn't that going to be super slow? No. If you have a 100,000
- 12:12
requests that all come in at 25 milliseconds and then you have to do GC and the thousand first request comes in at 300 milliseconds instead. That seems horrible because that one request is now 10 plus times slower than all the other requests were. But if one out of a thousand requests is 10 times slower, the total time is like less than 1%
- 12:32
slower, it's nothing. The impact of garbage collection on the scale of a long process with lots of things in it is much less than when you're timing each individual request. So if you're doing work that you want to have resolved in milliseconds, a garbage collecting language will have flaws you should work around. But if your average request is going to take more than a 100 milliseconds, like I don't know,
- 12:54
multiple seconds because you're compiling a giant app, I don't care if you garbage collect one, two, three, 10, 100 times during that request because the garbage collection is going to make up such a small percentage of the runtime. It barely matters. It washes out. So if I was building a web service I wanted to have respond as quickly as possible, Go is not the right language. But if I'm trying to take something written in JavaScript that's really slow that takes minutes to run and shave the
- 13:15
time down and get more concurrency, Go is one of the best languages to do this with. But it couldn't have been too hard to do the Rust thing, right? Cuz you had an agent do it in 2 weeks. Now I would like to introduce TS Rust. Writing all of Typescript from JS to Rust would not have been viable. I know this to be the case because of how much the other models I tried like 56 Soul and Astra
- 13:36
struggled with this port. I think a lot of that was because going from JavaScript to Rust is not trivial with things as complex as TypeScript itself. And this is where things start to get interesting with my TS Rust port. If we hop into the crates, a lot of things were done, many of which are uh I think questionable is a fair word for it. One of those things is this go std. This is
- 13:58
the standard library from Go ported to Rust because the way Opus chose to do this wasn't to rewrite things from Go to Rust. It was to go through this Goport line by line and faithfully port it over to the Rust language. And the result of this was actually quite a bit
- 14:19
slower than the Go version because when you have to reimplement things like Go routines, which is in here somewhere, I forgot which it's in. I think it's in the runtime RS. Yeah, runtime RS. This is Go routines ported to Rust. This happened because Go was a much better starting point for a Rust port than JavaScript is. The Microsoft team
- 14:41
faithfully by hand went through every line of code in Typescript, ported it to Go, and then started refining it to make it better and faster in Go. My agents did the same thing from the Go version to Rust, and they succeeded. I didn't even know that my agents had done this until somebody else pointed it out on Twitter. And as soon as I saw that, I was like, "Oh, that's hilarious. That explains a lot." Because again, when
- 15:01
TSGO RS was ready for its first run when Opus finished, it was two to three times slower than the Go version. So, I told it in a/goal, your objective is to make it at least two times faster in every way. And then I let it run for 2 weeks. More importantly, I let it run on my six cla accounts. All of the work done in TS Rust with Opus 55, which effectively
- 15:22
gave up on the old version and restarted, was roughly $24,000 of inference if I had paid full API prices. But, as you know, I didn't because I have subscriptions. So, I also tried pricing it out using all my logs to see how much of my utilization went to this. The equivalent in your sub usage for all the work I did here was between 925% and 983% of one account's
- 15:46
weekly limit, which means about two and a half months of usage on a cloud code sub. When you consider that we got a free reset, this ends up being about $500ish give or take of subscription usage. And at that price, this is actually a good value to have built. I know it sounds insane, but $500 to have TypeScript in Rust, which is more memory
- 16:06
efficient and faster, 2x faster than the Go version, is genuinely super useful. And I am to me, this is money well spent. And I know that seems insane because the 20,000 or the $420,000 headline number sounds crazy, but with the subsidization from a subscription and the fact that only the 20k of Claude mattered, this being 10 weeklys of usage is just hilariously good value in my
- 16:26
opinion. So, if you were thinking, man, the Microsoft TypeScript team wasted so much money and time porting to Go, they could have just waited and let AI do it in Rust. No, I don't think the AI would have done this viably from JavaScript and TypeScript to Rust, but it was able to do it with Go relatively trivially. And the result is a pretty much 100% drop in replacement for TSGO and TSC7. I
- 16:50
know that for a very silly reason, the issues, there were five, there are now four. I audited these issues. Well, my agent and I audited these issues. And of the six issues that have been opened total so far, one of them was specific to TS Rust. The other five issues were problems that exist in TSC7, the specific version that we were targeting.
- 17:11
So, these aren't things that are wrong with the rewrite. These are bugs in Typescript itself currently that were faithfully ported to my Rust version. And I think that is hilarious that not only did I port the intentional things, I ported the bugs, too, because this truly is a fully compatible drop-in replacement. I suppose now would be a good time to talk about the resulting performance. Since I started this before
- 17:32
bun was even a thought from Jared, I was initially running all my numbers against Typescript 7. And for realworld projects like T3 code, we saw a 2x improvement from TS7 which is TSGO and as much as a 20x improvement on various things for TSC 6 which was the last TypeScript JavaScript based TSC. I started with the T3 code checks because I did something fun here. One of the things effect TS
- 17:53
does and if you're not familiar effect is a popular library in Typescript to make building reliable concurrent systems more viable and composable as well. effect uses and abuses TypeScript so heavily that they've actually built their own fork of TypeScript effectively. It's a patch you apply to your TypeScript config and your TypeScript like package in your repo to
- 18:14
get more diagnostics and features from effect into the type checking. I built that directly into TS Rust because we rely on this a lot for things like T3 code. So for our use case in T3 code, not only are we way faster than TS7 with effects TSGO plugin, we're actually way faster, like fourish times faster than
- 18:34
bun as well, not because bun check is slower. Bun check is super fast, but because we have to do the effect check on top of it after. And as soon as you're doing both, it just ends up way slower end to end. But this is just checking in T3 code. Let's check other projects. some real simple things like I don't know the entirety of TRPC server or centuries codebase or excaladraw and
- 18:55
funny enough I was checking effect as well I actually think bun couldn't pass on the effect codebase if I recall bun also raised incorrect errors on sentry and tRPC so the bun checker while incredibly impressive and faster in a lot of cases does not have faithful enough checking for me to trust it in these types of big projects I think bun check fits well with like the other bun
- 19:16
tools tools in that like if you start a new project with bun and you use it for everything, bun is a great solution. I actually honestly would recommend bun over ts rust for almost everything simply because I have higher faith it'll be maintained. But tsc my project ts rust is a way more faithful recreation and as a result it does not get weird errors in these projects. It works faithfully in projects like TRPC in
- 19:38
projects like T3 code in projects like the effect repo itself. And the improvements we see here for most of this stuff is 13x plus for going from TSC6 to TSC RS. And it's usually a 2x improvement from the Go version. But ready for the spicy take that's going to make all of this way more confusing. I didn't do this for performance. I did
- 19:59
not make TypeScript Rust because I wanted something faster than TypeScript Go. So why did I do this? There is the silly reason and then the potentially groundbreaking one. And I'm going to start with the silly one. Silly reason is because LLMs couldn't do it. Models were not able to make these types of ports. It had become a meme. I got in flame wars with people for claiming they would be able to port TypeScript to like
- 20:19
C++ and things in a month, which just obviously you couldn't do. And I got in a flame war with somebody, ended up getting blocked by them. It was a show. This is like back like the old 2020s, like early days of Typescript posting on Twitter before AI took over all of it. And I was getting in like heated fights about these things because it was obvious you couldn't just rewrite TypeScript in another language,
- 20:40
which is why it was always my like reach goal, the impossible thing that like you could never actually have the unobtanium to an extent in the web dev world. And as such, it was a fun thing to throw practically unlimited tokens at. I had never tried an anthropic model on this task because they would burn my usage too fast. But with Opus 5.5, usage was burning so slowly and I had so much I
- 21:01
said, "Fuck it. Let's see what happens." and it was able to do it way more effectively than I ever could have guessed. I did it because I didn't think models could. And now that they can, I'm running out of things to throw the LMS at. Like I don't know what tasks an LLM would ever be able to do that are impossible for them now, but could be possible in the future. It's like hard for me to comprehend. Like I am out of
- 21:22
engineering ideas that are like too much for the model. This was my too much for the model and now it's done it. But what's the real reason? What reason did I have to want a Rustbased version of Typescript when we already had a super fast go one? I got almost everything I wanted out of TSGO. Why did I want something else? There are small things like I wanted the effect stuff built in.
- 21:43
I wanted to be a little easier to extend and play around with. But the main reason was this goes back to the garbage collection bit we talked about before. Garbage collection is an incredible way to take a burden off of engineers to allow them to ship software faster that is still really fast. But since garbage collection is a complex process that
- 22:05
requires you to analyze memory and do a lot of memory management stuff, taking a Go app and bundling it into web assembly is not trivial because you can't just bundle the code the developer wrote. You also have to bundle all of the garbage collection. The the end result here is that it's pretty difficult to write a Go app and then bundle it as W was less
- 22:25
than 10 megs and it's still going to have lots of weird performance issues because of that garbage collection aspect of it. This is why tools like ESB build struggled to get WSOM working for so long. This was also one of the things we lost with TypeScript Go. Previously, since Typescript was all just JS in the end, you could throw it in the browser, you could throw it in a V8 isolate, you could run it in Node and it just worked.
- 22:46
With TSGO, we kind of lost that. In fact, we lost a lot of the JavaScript APIs we use to interface with TypeScript compiler in the past. Why do I care that much though? We're all running our agents on our computers or someone else's computer, right? What if we're not in the future? Intelligence is expensive. People like Lauren aka Potato at Cursor and SpaceX are spending allegedly 132 mill. This number is
- 23:08
I'll explain why in that other video. Subscribe if you haven't, by the way. That red button helps us a ton and you'll get to be around for this rant which will be fun. Ignoring the fact that those numbers aren't real, the rate at which prices go down is real. Flashbang warning for artificial analysis. I need to port this particular bench over and I haven't yet. This is the cost efficiency over time
- 23:30
chart from artificial analysis. This takes a given score that models could get. So like DeepSync R1 getting a score above a 10 on their index in January of 2025 and it tracks how much the price for models of that level of intelligence goes up or down over time. Where things get really crazy is all the way over here on the right where we see models that are super smart like Opus 5, which
- 23:52
broke into the 50 point range for the first time costing $5.86 per task when they ran it. And now we have models like 61 Soul that get the same score for 32. So it's already decreased by 10x in cost for that level of intelligence in just like four months. At a given level of intelligence, cost goes down absurdly
- 24:13
fast. Like 10x a year is a conservative estimate here. If you take the 30 point range, which cost $2.50 in February of this year and got down to 3 by summer of this year, that is 8x per month decrease, averaging out to a,000x a year annualized. The rate at which prices for intelligence goes down is insane. But
- 24:35
intelligence isn't the only thing we're paying for. When you're building apps with AI, you also have to pay to host it. You have to pay to store the data in the database. You have to pay for whatever like systems and providers you're doing. You have to pay Apple to distribute the app. You have to pay for all these other things. But there's one more thing you have to pay for that people don't think about a lot. You got to pay for the compute that your agent runs on. If you're one of those $5 VPS
- 24:56
bros, now's your time to shine because we actually need to talk. One of the cool things cursor does right now is the cursor cloud. When you spin up a cloud thread in cursor, it will run on a full VM with 32 gigs of RAM. I think it might be 16. I'm pretty sure it's 32 in Cursor's cloud with a real guey, real processor, real everything it needs in Linux so that it can spin up your app
- 25:17
and test it with a real computer. Every thread gets its own computer. You spin up four threads, that's 128 gigs of RAM reserved to you for that bit. And right now, Cursor doesn't charge for that. So the problem now is that your inference is way more expensive than the VMs to the point where the VMs are almost a rounding error, which is why giving you those free VMs is not really a big cost.
- 25:39
But remember what I just said, the cost of inference is going down and it's going down rapidly. What about the cost of compute? What about the cost of VMs? See where I'm going here? The cost of renting a 16 or 32 gig box is going up steadily thanks to AI sweeping all the demand for RAM. The result is that VMs
- 26:00
and compute that you want to run your agents on, not the compute running the models, the compute running the code that the model's editing. That compute has been slowly going up in price over time, but the inference compute is plummeting in price. So, one of these things goes up by 25 to 30% every year. the other goes down by 10 to 100x every year. We're going to quickly get to a
- 26:22
point where inference is cheaper than the VM. With a model like GLM53 flash, running it for an hour straight is going to cost you a few dollars. The VM it's running on is likely more expensive than the model in the inference you're doing in that window. And as models of that price get smarter and as models at that tier get cheaper, this will be hit
- 26:44
quickly. at which point compute will be a constraint for when we're building with agents. Right now we're being given the VMs for free, but that is not going to last. Even Cloud Code isn't doing this anymore. Cloud code now gives every subscriber a credit that they can use towards the cloud compute that they use if they use Claude's cloud product. There's a lot of layers to why it works this way. The compute and the memory are
- 27:05
expensive, so they have to like worry about that. The reason that it has to be reserved this way is because of how our agents work. Our agents expect to run on a real computer with a real terminal with real bash so that it can run commands. It can run rip grape. It can search through the codebase and find things. It expects to have native tools for editing, for checking how much CPU
- 27:25
is being used, for searching for files, for moving files, for running git commands, for all these things. Our agents expect and are trained on real Linux boxes or Mac or whatever else. They're trained for real systems that they can do things on. And surely you can sandbox them some amount, but all of those additional layers cost additional resources. Sandboxing harder is not
- 27:48
going to make the compute cheaper. If anything, it's going to make the comput and the memory more expensive. Which means we need a better abstraction layer. Ideally, in a future where agents are cheap and writing code is incredibly cheap because the models and inferred have gotten so cheap, the thing that they write them on and to test them on should not be this expensive. Thankfully, this is a thing we have explored for a very, very long time in
- 28:10
the industry. And this is a war you guys have heard from here. And we get to revisit an old war that we have not talked about in a while. We get to talk about big orange. Cloudflare. What is the word that y'all associate most with Cloudflare? Spam me chat. When you think Cloudflare, what do you think? Cost, cheap, free, cheap. There's like cloud
- 28:31
and CDN. Most of them here are free. cheap cost related. A lot of what people associate with Cloudflare is cheapness. The reason for that is how Cloudflare's platform works. When I write code for a backend and I host it on Cloudflare, I'm not wrapping it in a Docker image that they deploy on a server. In fact, when I
- 28:53
put code on Cloudflare, it's not deployed. It's not running. It runs when a request comes in. The request triggers Cloudflare to spin up a worker. A worker is an isolated JS process inside of V8, the JavaScript engine powering Chrome and a lot of other things. And that isolate isn't a kernel. It's not Linux. It's not an OS. It is a isolated
- 29:14
execution layer where your browser or whatever else can execute JavaScript. This is really useful in browsers because you can have different tabs doing different things and not have access to each other's stuff. or you can have one tab with four things running in it that are different that can communicate with each other but not interfere with each other. V8's isolation layer is incredibly powerful.
- 29:34
It's not the most secure thing, but it's pretty secure. And what Cloudflare has done is taken those isolates, made them as secure as possible so that when a user goes to a website you built and deployed on Cloudflare, it can grab that code from storage, throw it on this isolate that's just sitting there waiting, execute it, then kill it. So on a platform like Verscell or Heroku or
- 29:55
Hetsnner or any other traditional host, if you and I both host an app on that platform, we both have our own Linux instance effectively that is running with our code in it. On Cloudflare, your code and my code are executing in the same instance of Linux, just split higher up in the V8 layer instead of with VMs and Docker. Hopefully this all makes sense. If it doesn't, ask your
- 30:16
agent. I'm not going to keep ranting about this. I've talked about this in a lot of my videos before. There are problems with this model, though. If we're running inside of a JavaScript isolate, we lose a lot of things like we can't read files from the file system because the file system isn't in the isolate and that would allow us to escape and screw with other people's stuff. We can't run native code because native code escapes the isolate and
- 30:37
needs to run in the actual like system which we can't do here which means we can't run a lot of other languages in Cloudflare workers. Means we can't do traditional persistent servers in workers. There's a lot of these things and Cloudflare has built lots of clever solutions to these problems, but the code that we're talking about, the JavaScript is running in this virtual layer above V8 instead of running at the
- 30:58
VM layer. This makes Cloudflare an incredibly cost-effective and memory usage effective platform. And that's why it's so cheap. It's actually cheap to run. They're not just pricing things cheaper because they want to win. They're doing it because their platform allows them to and still have margin. In this sense, Verscell is not overpriced because they want to make more money than Cloudflare. They are charging probably similar margins, but it's
- 31:19
higher cost to them, too. But since Cloudflare is just running in this JavaScript layer, you can't really run an agent in that because you can't write and read files. You can't run a dev server. You certainly can't run TypeScript Go to turn your TypeScript into JavaScript and get feedback on what's wrong to your agent. Oh, huh. Weird. It looks like I did exactly that.
- 31:40
This is the demo I built in Astra, and it has some dumb decisions that are wasting a bunch of memory. So, I'm going to switch to this version that Opus made because it's a better representation of what I have in mind here. Before I can justify the chaos I'm about to show you, I need to show one last reference. There's a library called Just Bash. Just Bash was created by Malta at Verscell and has become a pretty legit library.
- 32:00
The point of just bash is to provide agents a full fake bash environment in memory in JavaScript so that you can import this bash from just bash and then execute most traditional bash commands with a fake file system, fake processes, fake everything, allowing for an agent to operate as though it is working within bash, even if it isn't. But in
- 32:22
order for this to work, you also need to be able to compile and type check and do all the other things that you would do in a normal file system if you wanted to use this to build. If you set this up as a way for your agent to explore fake files to find content, awesome. But you're not going to build a real app with this. You need too many other things, including TypeScript Rust. Well, TypeScript Go or TSC traditional. But now with TypeScript Rust, we can do
- 32:43
really stupid things like here where I had GBD61 Soul build a full app that is entirely functioning all in a V8 isolate. This one is running in node on a server technically, but this could just be run on your browser. There are some limitations, but not as many as you'd think. And this all just ran and worked and did its thing. And the
- 33:05
entirety of utilization for this, everything that was used here, from the agent to the compute it ran on to the compute that the app itself is running on to the type checking and compilation and everything else just ran on my isolate and used under 260 megs of RAM. and it peaked at like 4% CPU.
- 33:25
Do you understand how cool this is? This will allow for you to have an agent build an app where both the agent and the compute it runs on, not the like compute for the inference, but the compute that it's running the code on and building on all together for pennies. And doing this with Go would suck because TSGO wastes a ton of
- 33:46
resources simply because porting that to JS is not viable. WASM, but you get the idea. Web Assembly is not a good fit for Go. It is an incredible fit for Rust. And just to emphasize my point here, when I had Astra build this, it chose to use ESB for the bundler. And just sitting here idle, it's using 2x the RAM of the other version because the
- 34:07
inclusion of a go-based package like ES build inherently just eats memory and resources and the data that has to be loaded in. Like we have to load way more in because of how much bigger the bundle is. I see people confused about how this would work on Cloudflare. Do you know what Cloudflare, Chrome, and Node all have in common? They're all using V8 isolates. The way these things
- 34:28
work is that they are all using the same JavaScript engine. And when you load a Chrome extension or you load a new file or you have some process in Wom, those are all different isolate workers is the term for it. And they can communicate with each other if they're allowed, but they cannot interfere with each other. What I built here isn't impressive because I rebuilt Bolt but used TS Rust.
- 34:49
What I built here is impressive because I could turn my file system on read only and it still works. Nothing here left RAM. Nothing here left V8. I'm not doing anything on the system. This all exists within V8. The whole dev loop exists in V8. There isn't anything comparable. I see people mentioning like random like
- 35:10
database companies and stuff that let you execute code of them. No, no, you're you're not getting it. And if you guys aren't getting it, you're my stream audience. I am not excited to see how others deal with this. Let's talk about how computers work really quick. At the bottom you have hardware, CPU, RAM, whatever else. Above that you have drivers. Drivers are provided by the thing above the kernel. The drivers are
- 35:31
the method in which the kernel interfaces with hardware. Above this we have the operating system. Above this we have applications. I would argue actually we have file system kind of. Then you have applications. So let's figure out how something like Heroku Hetner or any normal cloud works. If we wanted to deploy a Docker image on AWS,
- 35:53
we're not deploying it here because these things have to exist on AWS's hardware so that they can put more systems above. So what they'll do is in the application layer they'll put something like Docker and you know what you do after that you copy paste everything and do this. This is how hosting on normal VPS
- 36:14
providers works on top of the file system and operating system. You have an application, usually something like Docker, and it's not technically running a new kernel instance always. Usually, it's doing a shim kernel. So, I could claim like for this layer here that this is all like the like Docker hypervisor I don't fully understand it, but like docker shim to kernel, which
- 36:36
makes the kernel stuff more efficient for sure. So this shim's there to allow you to skip steps when it makes sense to. And then you're building an OS on top, which is why on something like AWS, you can have one box that AWS is providing that has one person running Ubuntu, one person running Windows Server, and one person running, I don't know, Debian or something. That's all
- 36:56
happening in the application layer of Docker on top of the operating system. Which means every one of these things takes RAM. This is why every time you spin up a new Docker container on your computer, it's using a bunch of resources. It takes resources to run operating systems in your operating system. Let's compare this to how JavaScript works. So, we have our whole stack here, but let's say the
- 37:18
application is Chrome. Inside of Chrome, you have V8, which is the JavaScript runtime. And in V8, you have all of these isolates. And all of these isolates can execute different JavaScript files and functions at the same time. This is way more efficient because now every file or application if it fits in one of these isolates, it
- 37:39
doesn't need to shim out to the kernel, run an operating system, and steal a bunch of resources. Like, like, let's be real here. What takes more resources on your computer? A new Docker instance of Postgress or a new tab in Chrome? Tabs in Chrome, as slow as y'all might think Chroma is, obviously uses way less resources. Absurdly, comically,
- 38:00
obviously. The reason why is the insane billions of dollars that Google has pissed into V8. They wanted to make this engine as efficient as possible so that the web could be a compelling platform so they could finally stop making shitty Windows apps. Google spent a billion dollars so that Chrome would be fast enough so they could stop making Windows versions of Google Earth and That
- 38:21
was a lot of money and effort put in to V8. And this has benefited the whole ecosystem in ways that are hard to comprehend, including all of Cloudflare. Cloudflare's whole platform is built on the idea that virtualizing at the operating system level by putting an OS at the app layer is stupid and VH is a much better abstraction for putting
- 38:42
applications that customers are hosting on top of. The catch is that these are just JavaScript. They don't have full system capabilities. You can't call the kernel because there's no kernel you have access to. You can't write files because there's no file system. It is just JavaScript. Anything that a Chrome tab can't do, a V8 isolate can't do with the exception of cores. I don't even talk about cores now, but remember all
- 39:02
the problems here are you don't have a file system. You don't have native code access. You can't do the things you need to build. Isolates are a good place to put the code once it's built. They are not a good place to build. On Versel, for example, Verscell sandboxes are the same compute and primitives as hosting on Verscell is. on Cloudflare. Cloudflare sandboxes suck because Cloudflare does not know how to make
- 39:23
things work well with Docker because that's not their focus. Their focus is this isolate layer. So, Cloudflare sandboxes run on entirely different compute with entirely different platforms than Cloudflare's traditional like worker hosting. So, the thing I did here, the thing I enabled inspired by Just Bash because the thing that makes Just Bash special is it lies to agents living in this isolate layer. So they
- 39:45
think they have access to the file system and to these other things. And that is what we have built here. If I was to flip a switch on the file system for the server running this demo to be read only. So I literally could not edit files on the computer. I could only do things in RAM in my V8 isolated node. This will still run exactly the same. That is what makes this demo cool. It
- 40:06
would be fully possible to run this entire demo in a Cloudflare worker. So instead of needing half a gig of RAM and a bunch of reserved CPU, I can throw it in here and it costs nothing. Another thing to think about here is how CPU allocation works. If you have a VM that has two CPUs and I have a VPM that has two CPUs and both are idling, we're
- 40:27
still wasting four vCPUs. There are some things that make it easier to like reallocate and have virtual like cores not be guaranteed the same way. But the way it's working on Cloudflare is with the built-in event loop where when your code isn't doing anything, it doesn't cost anything other than a bit of RAM. So when you're waiting for the next token from an inference provider, that's not costing you any compute cuz the CPU is doing nothing. So that CPU is being
- 40:48
used by others. When suddenly data comes in, it will execute the next task because your app, my app, and another person's app are all sharing the same event loop. We're all sharing the same scheduler effectively. So it becomes way more efficient for us to run things here. or even a long running task that takes hours probably spends most of its time idle. Now that only costs money for
- 41:08
when a token comes in, not for the time between tokens. If inference continues to get cheaper, this type of building will become essential because this will enable us to build a gentic developer experiences that are cheap on all sides where running the agent is cheap and giving the agent compute to execute on is also cheap. I see some people in chat
- 41:28
catching on here asking is this related to lakebed? I won't say more yet. I need to give you guys something to be excited about because I just wanted to focus on this. The fact that TS Rust is one of the final pieces I need to take a full endtoend TypeScript developer experience and do it in an isolate instead of doing it in a VM. For those who are curious
- 41:48
and want to see the very complex prompts I use to build TS Rust, here you go. I told it to work in code sandbox ts on this computer. Read the agents MD and the type check or accountability uh files first. Those files included audit notes from the 16-hour run and asked for some things. Make the cargo job count configurable. Add script state and split
- 42:10
yada yada. And then update the agency so we can do more parallel work. To which I said, do it. Great work. I trust your judgment fully. Keep going and doing what you think is best to continue improving compatibility and performance. You can probably move to main soon as well. I did a follow-up at some point saying get whatever the best current state of the work is and set it as the main branch ASAP. I don't care how. I trust your judgment. Then later I did a slashef effort cuz I noticed I was on the wrong effort level. Actually, I I
- 42:31
was on ultra code and it kept bumping me down to medium when it crashed because it was running out of memory and crashing systemd constantly, but once I like added some things at the OS level to fix this, I had to rerun/ef effort to put on ultra code again on XI and then the / goal of complete the port, get as close as possible to 100% compatibility with TSGO, and then also improve performance. Do the efforts in parallel.
- 42:52
Multiple workflows and sub aents should be running at all times to maximize parallelism. Orchestrate how you need to keep them working independently all on this machine. And I noticed it was running out of compute all the time. So I took other machines I had in my fleet, configured them for SSH access, and told it you have SSH access to other machines now like Cup 2 and Alvin, two of my uh VPSs. I noticed it kept stopping because
- 43:13
it like got full compat and then like would make one performance room and be like, look, I fixed performance. So it needed a better end state. So I told it with a new goal, our target is minimum 2x faster than the go build. Continue iterating until we get there. And then it did for two weeks and now we are here. So, as you see, I wrote some very
- 43:33
complex prompts for all of this. Super super high IQ things I was doing here. I'm a mega prompter. Real talk though, the things I did here were not around the prompts. It was around the environment and setup to allow it to do this stuff. So I like I had other threads going where I was auditing constantly figuring out where the blockers were, what's taking a while,
- 43:54
what failure modes are happening again and again, and then updating the environment to allow agents to operate more effectively in parallel without stepping on each other. And after a bit, we built systems that allowed that to work and ended up shipping much faster once we did. But my energy was honestly similar to how I work as a manager where I wasn't spending my time in the trenches writing code. I was spending my
- 44:16
time making sure that my team had everything they needed to be as effective as possible in finding where the slowdown was and helping them get around it so that the slowdown would stop happening. Ended up being really fun. And I know I was making a joke about being a mega prompter, but uh I have been politely requested to mention mega dev a bit more. I am finally doing a paid course for the first time ever. Feels weird and I want to be very clear
- 44:37
about this. Any information that will help you get a job is not going in here. It's just going on my channel for free. The point of this is for employed developers that want to maximize how much they use and abuse AI to ship real software in the real world. I'm working with three other incredible instructors, people who have helped me learn how the web and technologies work for years now. I'll do the sidebar so you can see
- 44:57
Angie, who I love to death, Kent, who taught me half of what I learned about React back in the day, and John Linquist, the creator of Egghhead. If you have to pay for this with your own money, I wouldn't recommend it. But if you can get your boss to sign off on using internal like expenses for educational stuff for it, it'll be fun. And I'm going to do a live workshop with it as well, which should be very fun. So that's where I'll get to talk more about all of these chaotic strategies for
- 45:19
maximizing what your agents are capable of, which should be fun stuff. What a journey this one was. I my AI psychosis was already pretty bad, but the fact that all these truly crazy ideas I've been thinking about for years now are actually doable. Like I've always wanted to have a full stack developer experience that runs purely within V8
- 45:40
back when Bolt was still stack blitz and they first put out their full like JS runtime thing inside of WASM. So you could have a full VS codeike experience with apps bundling in the browser. I was blown away. I wanted to go further with that with Cloudflare and I went as far as trying to reverse engineer Stack Blitz and like decode the JS binary and
- 46:02
their WASM to figure out how they were doing it because they're there was so much cool stuff there. That's how I became friends with Eric back in the day and that stuff was so cool. I've always been a nerd about how the code we write actually runs and executes and how these systems compose and operate. It's such fun stuff and now I get to try out my crazy ideas. Previously, the vast majority of this was just like
- 46:22
imaginary. Like I was playing these games in my head, and if any of those ideas went far enough, I could maybe talk about it with a smarter person and maybe inspire them to build something kind of like it in 5 years in the future. Now, when I have one of these insane ideas, I tell Opus and then I see if it works or not. Even now, I wanted to see if this will actually run on Cloudflare. So, I just told Opus, "Go do it." And it probably will. Absolutely
- 46:44
insane. I on one hand, the engineering we know and love is dead. And very, very much so. these types of YOLO projects should not be viable at all. Like you shouldn't even be able to get it to pass the tests, much less actually work as a drop in replacement for TypeScript. But yeah, if you still think AI dev isn't the future and isn't obviously where all of software dev is going, you deserve
- 47:04
the lack of employment you'll have in 3 months because this is insane. It's actually hard to fathom how capable these tools have gotten. I think this is everything I had to say on this one. I was up till 6:00 in the morning prepping all of these demos and I'm so proud of what Opus and I were able to cook. Not because I think everyone should use this, but because I can find out if these ideas work or not in ways that are
- 47:25
just hard to fathom. I actually feel like we're peering into the future with some of this stuff. And it's so cool to see. I cannot imagine where we will be next year. And I hope you guys are excited to go on this journey with me because it's only going to get more chaotic from here. And with that, until next time, peace nerds.
Description
I ported all of typescript to rust...yes really. ts-rust (aka tsc-rs) is here, and it only cost me $420k.
Thank you Qdrant for sponsoring! Check them out at: https://soydev.link/qdrant
Check it out https://github.com/pingdotgg/ts-rust
Want to sponsor a video? Learn more here: https://soydev.link/sponsor-me
Check out my Twitch, Twitter, Discord more at https://t3.gg
S/O @Ph4seon3 for the awesome edit 🙏
