Better Than LLMs?
BDBen Davis@bmdavis419 · 45.9K subscribers
- Views
- 21.9K
- 21,895
- Outlier
- ×1.0
- Median 21.4K
- Length
- 17:40
- Published
- Oct 10
- Saturday
- Comments
- 34
- 356 likes
Script
How it's written
- Words
- 3,924
- Words a minute
- 222
- Reading time
- 17 min
- Segments
- 12
The hook
0:00–0:21There's this new model that came out a little bit ago called Jev. That's not a normal new model like a new Opus release or something. Instead of being a normal AI model that generates a bunch of texts which can run in loops and do tool calls and agent things, it's a decisions model which will take a bunch of options and then give you the likely probability of each option. And it is extremely cool.
Structure
There's this new model that came out a little bit ago called Jev. That's not a normal new model like a new Opus release or something. Instead of being a normal AI model that generates a bunch of texts which can run in loops and do tool calls and agent things, it's a decisions model which will take a bunch of options and then give you the likely probability of each option. And it is extremely cool. This is a very dope release that a lot of people were very excited about. It does actually add in new abilities. I have a bunch of stuff that I want to show for like where Jev actually makes sense. But there's also some places and things that I've been seeing go very viral that are very stupid, like extremely stupid use cases of Jev that make no sense. But we'll get into that later. I want to just talk about the good first. And there there's a lot of really good stuff here. This is the official announcement for the Jev model. The big thing that it does differently is that while both normal LMS and Jev take in normal unstructured data like whatever you want, the big change is what the output is. The output on an LM is just text. It's a bunch of random text stuff. The output on Jev is type-S safe structured values. It is a clear contract of it will output the same shape every single time, which makes it far more deterministic in nature. And when you combine that with the fact that the input tokens are incredibly cheap, the output tokens are free and you can run a ton of these requests in parallel in like one request. So you send one request that's asking for 10 different decisions. You can just do that and it'll give you all the answers at once in a clear, predictable, structured output that can be parsed by code
Watch from 0:00instead of by a model. It's very useful. And the obvious first example here would be something like classification. So say you have a message come into a support email and you want to have a classifier run on that to mark it as billing, technical, sales or ambiguous. So you get this question. These are the categories with a little description of each one. You run this. It's nearly instant and you get a confidence probability for each answer. So it is 100% confident that this is a billing related request which is basically correct. That's fine. Another useful mode quote unquote this thing has is the null probability which is basically just is this statement true or false? It's a boolean. So you give it some input and then it will return the percentage chance of yes and of no. So you could have something in here like uh is this message urgent or timesensitive? The criteria is true, explicit urgency, false, no, time pressure expressed and you pass in some random request, it'll give you that percentage chance. So you could use this as a quick switch on something to maybe prioritize stuff like this could be a very good important filter within an email or whatever. The third type of request is a scoring request, which this one's kind of the weirdest, but it's still very useful. So, the way it works is you give the model a set of criteria. So, say the zero criteria is calm or neutral, one is frustrated, but civil, two is very angry, expresses intense frustration or hostility. And the instructions here are to judge the expressed emotion. You pass in some user message or whatever. And you will get a score between 0ero and one of how frustrated that user is. So
Watch from 1:30in the example here, the message that the user sent would be about a 35% on the anger scale with a47 confidence. And again, for all three of these request formats, you will get out a fixed JSON blob that it's effectively type safe. So you can do programmatic calls on this for score, for null, for choice. All of them work that way. And as I mentioned earlier, you can send multiple questions at once. So we have two null checks, one score check here. You run them and all three of these pop out from one request. So, you eat the input tokens on all of them. No output token price. It's very quick, very useful. This is a very cool model that unlocks a lot of very novel things. I've seen some dope use cases. I'll show you a couple here. But I've also seen some really, really stupid ones, which I'll show you right after today's sponsor. So, I am very excited about today's sponsor because I successfully tricked Grockbot into paying me. Yes, that's right. This is an ad for Grockbot, the thing that I've talked about in multiple videos and will continue to talk about because it's really good and I run basically all my stuff through it. It is the single best personal agent assistant thing that is out on the market right now. Full stop. It is great for teams. It's great for individuals. It is so powerful to have this always on agent system that has routines, a full computer onto itself. A ton of different bots that are able to communicate with each other in just a really good desktop app, mobile app, and now even in the grock.com website. It's all in there. The whole thing just feels great. And to show you a specific example, this one's tough because like I really can't show too much of how I'm using bot because all of it's like personal private business stuff. But
Watch from 3:01this one is really useful. I have it hooked up so that Amazon is signed in on the Grockbot computer. And you can see right here, it just proactively sent me a message at 8 a.m. today. Hey, here's all what's arriving today. And whenever something does arrive, it goes into an Amazon locker. It'll send me the code for that. I grab it. It all works. If I want to order something, I just tell it to go do it. It'll give me a little confirmation message that looks kind of like this and that's it. It just works. I could keep going on about this for a while, but seriously, just go try it out. Hook up everything you use day-to-day like notion, Gmail, etc., and then talk to the bot. Get everything set up. You'll be very happy with it at dav7.link/bot. Starting with a lot of the dumb use cases here because unfortunately I think the discourse around Jev has gotten polluted by people trying to stick this thing in a lot of places that it just does not belong. This model is not as smart as a normal LLM. Just because it can make decisions does not mean that those decisions are going to be nearly as good as something like Opus 55 with actual reasoning happening with lots more context, actual tool calls doing things. This is just a switch statement. And if you can make a quick gut check decision with narrow context, yeah, it's fantastic. And there's a lot of useful places to put that. This example of using Jev for compaction is stupid. Like extremely stupid. Do not do this. If you want my take on compaction, just use what the labs ship with. They have trained and optimized their models to work well with their own native compaction system. They know when it's coming. They know how to use it. Don't try and complicate it. You will end up shooting yourself in the foot. And that's what this does. This goes through
Watch from 4:32all of the tool calls in your history. It will score them based on whether or not you should keep them or not, and then drop the irrelevant ones. And that's your compaction. You reduce your token price. There's a lot of problems here. First off, like the instant one that should be obvious to most people is that caching is now broken. You will bust cache instantly on this. If you go back and edit the history, like on new anthropic models, they're making it so you can't change the history and tool calls and whatever without busting the cache. And by breaking caching, you are suddenly reading a ton of tokens into context. Input tokens plus cash write tokens get very expensive very fast and you do not want to screw with this. The other problem too is that this will not actually get you useful context in the compaction. you will lose very valuable data almost every single time because it doesn't have the broader context of what's actually happening within the request. When you do normal compaction, the model will take all of the context and pull the useful pieces out, get rid of the stuff that doesn't matter, and then that's what gets compacted forward. Versus this will not have any of that context. When it's checking the read file, it doesn't know if that file was important or not. It just sees, oh, it's a read. Yeah, we should probably keep that. Who cares? Or it's just a test command. Oh, that doesn't matter. we can get rid of that when maybe those did actually matter a lot and since the model is very narrow it doesn't see any of that stuff and when you try and do it this way it just sucks I think that tech one of the co-founders and lead engineers on Hermes agent a very good agent put this very well that when they tested out Jev compaction on their own compaction evals they found that it actually just removed all of the tool
Watch from 6:03calls from their chat history it just wiped them so it ended up being programmatic so you don't even have to pay the Jev API price you could literally just strip all the tool calls And that's your magical compaction, which is not good. And this just kind of leads to a doom loop where you keep removing tool calls from the history and then busting cash over and over and it just makes everything way more expensive and stupid for no good reason. It don't do this. This is a dumb use case. Another pretty dumb use case was the auto router stuff. I think that the model router things, at least right now, are pretty stupid. They don't make much sense at all. Like the Jev router here will use Jev to decide where your request should go. Should it be routed to DeepSeek? Should it be routed to six soul? Should it be routed to 38 Flash? Where should that request go? And maybe on paper that seems like a reasonable thing to do. Maybe if you just see the prompt is, can you go check my email? You would think that that would be a perfectly feasible task for a very dumb small model to go do and it wouldn't have any performance hits. But that doesn't have the actual context of what's happening on the environment. Is the environment in such a state that to go get the emails you have to write a bunch of custom code and use weird connectors? Do you have to use computer use for it? How does all this stuff work? It doesn't have that info. So, you end up getting routed to a dumber model which will take longer, use more tokens, end up actually being more expensive because, I've talked about this a lot, too. The actual price of a model is not the sticker price. 6.1 Soul will end up being much cheaper on a lot of tasks than Deepseek V4.1 Flash will be, even though V4.1 Flash is borderline free.
Watch from 7:35It's absurdly cheap when it runs and runs and runs and runs and runs in circles getting nothing useful done and 61 soul can just do it in five tool calls instantly. That one is going to be better. The cost per task here doesn't end up working out to actually make this better. And as a result, you just end up getting stuck with a much worse model all the time for no good reason. It is not a good use case for something like Jev or almost anything in general. Classifying what model a prompt should go to is a very difficult problem which requires a lot of context, a lot of engineering that a simple router like this just does not have. Honestly, on the topic of routers, one of the things I've wanted to do for a while, I'm not going to do this, so I'll just say it in the video. But it would be a really funny gag if someone like did a add-on launch of a new router and they're like, "Oh, we solved model routing. This is the most powerful router. Look at how well it does on benchmarks and how cheap and efficient and good it is." and all the router is is just a proxy for GPT 6.1 Saul. I would bet money that that would outperform almost all of the other model routers because they don't make much sense. This is not a real thing right now. Don't use Jev for this. And honestly, I think that's enough bashing the bad implementations because I do want to talk about the places where Jev is cool. I again I think this is such a dope useful model. Like to give you an inproduct example, I have the dev server for picking things spun up here and one of the things I recently added is a search command. So you can go in here, you could search for assets or billing or whatever and just quickly jump to it.
Watch from 9:04A command K menu, great feature. I want it in basically every product. One of the things that Jev is really good at is actually searching things because you can pass all of the different places where it could go. There's about 23 options here. And instead of doing a normal text search where if I type something that doesn't quite match, like for example, if I do cancel my uh sub, it will direct me to manage payments. when if I was just doing a normal text search on this, it would not find manage payments. It would order these completely wrong. And the ordering I get here actually makes sense. Manage payments at top billing down here. Then the rest are like whatever. It's basically a 0% chance. And it works great for features, too. If I just type remove, show the remove background, see the assets. There should be like a upload one in here. So yeah, upload an image or add image to site or something. That will also show upload image. Like it just makes the search much smarter. And because the model is so cheap and small and fast, you can kind of get away with doing something like this. In a lot of cases, the quote type- safe nature of Jev is the reason why a weird feature like that can exist. Since when you do your request here, you have the choice here. So, it's doing a choice request. You tell it to rank these. You give it a bunch of different criteria to pick from and the criteria are pages. Then at the end here, you will get out this deterministic state of a completed result with this model with these answers and each of them are ranked by probability. So all the code is actually doing is ranking based off this. It's a simple sort. None of this is using an agent or heavy non-determinism behind the scenes. It is basically a deterministic code function with just a sprinkle of probability in there on the
Watch from 10:25switch statement. Obviously, as far as good use cases go, a switch statement that is intelligent, is just insanely useful for any like internal classification task within a workflow. The stuff I showed earlier of like classifying an email, very useful, makes a ton of sense. Like in Superhuman or Gmail, the inbox filters and tags and folders, those should probably be something like Jev where it can just quickly read the email and then put it exactly where it should be. So you don't have to eat the cost of a full agent, but you also get things sorted in a much smarter way. Another just really cool example I found was uh the founder of browser use, which is a well browser use company, put together a computer use demo of Jev doing computer use stuff. it got access to the browser and was able to do flight booking by interacting with the page and making decisions based on the current state of the page. In these smaller cases, this is actually quite good. You could pull this off in certain scenarios. The problem is as the computer used task gets more complex and requires more external context and it's not just a straight shot. Go find this page, click this exact button, do this thing like the flights page, there's not too much context being stuck in here. If you're doing something like a lot of the things I have computer use with codecs do like go prep a video for editing or go handle a bunch of file downloads and key managements across 10 different tabs and services that level of orchestration is not going to be feasible or possible with Jev. But for again quick smaller things it does work and anything that can be represented as a switch statement you can stick Jev into. Like another one
Watch from 11:56here of a full self-driving truck. Uh Justin posted this one. The model is actually driving the car. It is deciding what path it should take, when it should start, when it should stop, because you can pretty much represent the state of driving as a switch statement. And I think that's kind of the point. Jev works for basically anything that can be represented as a switch statement. And you need to keep the complexity and level of decision-m on that switch statement much lower. Uh like if you're trying to do some deep analysis on something at 32k context window, it is not going to get the job done. But again, small one-off things is really useful. With how insanely well the Jev launch went, there have actually been a lot of companies doing similar things. There's actually one that's been doing this for quite a while that I really like. One of them is OpenAI. They just released their decisions API, which is effectively structured output for GBT6 Luna. You pass in a model. Currently, only GBT6 Luna is supported. You give it some input. You give it some questions. And then it has the three the same three buckets that Jeb has just with different names. Predicate, choice, score. They have a couple examples down here of like you can feed in this cup. What is the probability that it has a dent? 0% 100%. I think one of the reasons why this is the first major example here is actually that since it's using Luna under the hood and Luna has vision capabilities and Jev doesn't, you can do vision classification off of OpenAI's version, but you can't off of Jev. I'm sure that they will add that. that's almost certainly coming soon. But at least for right now, if you want to do this kind of switch statement on vision stuff, you
Watch from 13:28do kind of need to use the OpenAI version here, which is again very good. It's the same general thing. You can see here they have a driving example, the routing example, and a document type example. Like all of these are normal use cases that work well on both of them. Putting these two side by side, the biggest difference is in the input price and also again the image inputs. The price on Jev per million input tokens is about four centsish. Then on the Luna version it's about 10 cents. So it's over twice as expensive. They're both free on output charges. They have the same general capabilities. After running a bunch of examples on both of them, the OpenAI one ended up being about 2x faster versus the Jev one ended up being about 2x cheaper. So pros and cons to both. I'll be honest, most of the time I would probably just find myself reaching for the OpenAI one because it would generally speaking be more convenient since it has image stuff in it, but this cost is a pretty huge difference. And if you're doing weird use cases like the P thing one I showed earlier. The price is quite useful to have lower. That other thing I mentioned earlier that's actually been doing this for a while is BAML. I really like BAML. I found this like I think it was April of LA this year or something like that. It was a while ago. And what they do is really dope. It is just taking normal models. You can stick whatever model in you want and it will hard constrain and make sure that you get the correct output every single time. It is the best way to do structured outputs or at least was the best way to do structured outputs before Jev and before decisions.
Watch from 14:58The syntax for this is actually a new language, a basically madeup language that you give it a class. So the verdict has to have a label. It has to have a confidence and then you just pass in whatever text it is. It will use OpenAI GBT 5.5. You can use whatever model you want. give it a prompt, tell it what the output format is, and every single time you will get the output in this shape. And they have a bunch of SDKs so that it is truly type- safe where this is now basically a deterministic function in your codebase that just classifies things. And I've used this a ton in the past for a bunch of YouTube tooling and that kind of thing. It's a great system, very similar to Jev or decisions, just every single model works with it. So yeah, that's Jev. I think this is an awesome new model and paradigm of doing things that the more creative you get with this, the more weird useful places you could probably find to stick this into existing products. Uh, I really like the search example cuz I think that's one of the quickest UX wins you could get out of this, but there's probably a ton of other stuff that I haven't even thought of. I think this will be huge for sprinkling AI quote unquote into more and more places where it actually makes sense. just making everything into a text box is probably not the right way forward, but sticking more intelligent switch statements into deterministic workflows that has a lot of applications. So yeah, if you haven't played with this yet, I highly recommend it. If you enjoyed the video, make sure you like and subscribe.
Watch from 16:20Full transcript
- 0:00
There's this new model that came out a little bit ago called Jev. That's not a normal new model like a new Opus release or something. Instead of being a normal AI model that generates a bunch of texts which can run in loops and do tool calls and agent things, it's a decisions model which will take a bunch of options and then give you the likely probability of each option. And it is extremely cool.
- 0:21
This is a very dope release that a lot of people were very excited about. It does actually add in new abilities. I have a bunch of stuff that I want to show for like where Jev actually makes sense. But there's also some places and things that I've been seeing go very viral that are very stupid, like extremely stupid use cases of Jev that make no sense. But we'll get into that later. I want to just talk about the
- 0:42
good first. And there there's a lot of really good stuff here. This is the official announcement for the Jev model. The big thing that it does differently is that while both normal LMS and Jev take in normal unstructured data like whatever you want, the big change is what the output is. The output on an LM is just text. It's a bunch of random text stuff. The output on Jev is type-S
- 1:04
safe structured values. It is a clear contract of it will output the same shape every single time, which makes it far more deterministic in nature. And when you combine that with the fact that the input tokens are incredibly cheap, the output tokens are free and you can run a ton of these requests in parallel in like one request. So you send one request that's asking for 10 different decisions. You can just do that and
- 1:25
it'll give you all the answers at once in a clear, predictable, structured output that can be parsed by code instead of by a model. It's very useful. And the obvious first example here would be something like classification. So say you have a message come into a support email and you want to have a classifier run on that to mark it as billing, technical, sales or ambiguous. So you get this question. These are the
- 1:45
categories with a little description of each one. You run this. It's nearly instant and you get a confidence probability for each answer. So it is 100% confident that this is a billing related request which is basically correct. That's fine. Another useful mode quote unquote this thing has is the null probability which is basically just is this statement true or false? It's a boolean. So you give it some input and
- 2:07
then it will return the percentage chance of yes and of no. So you could have something in here like uh is this message urgent or timesensitive? The criteria is true, explicit urgency, false, no, time pressure expressed and you pass in some random request, it'll give you that percentage chance. So you could use this as a quick switch on something to maybe prioritize stuff like
- 2:28
this could be a very good important filter within an email or whatever. The third type of request is a scoring request, which this one's kind of the weirdest, but it's still very useful. So, the way it works is you give the model a set of criteria. So, say the zero criteria is calm or neutral, one is frustrated, but civil, two is very angry, expresses intense frustration or
- 2:49
hostility. And the instructions here are to judge the expressed emotion. You pass in some user message or whatever. And you will get a score between 0ero and one of how frustrated that user is. So in the example here, the message that the user sent would be about a 35% on the anger scale with a47 confidence. And again, for all three of these request
- 3:10
formats, you will get out a fixed JSON blob that it's effectively type safe. So you can do programmatic calls on this for score, for null, for choice. All of them work that way. And as I mentioned earlier, you can send multiple questions at once. So we have two null checks, one score check here. You run them and all three of these pop out from one request. So, you eat the input tokens on all of
- 3:30
them. No output token price. It's very quick, very useful. This is a very cool model that unlocks a lot of very novel things. I've seen some dope use cases. I'll show you a couple here. But I've also seen some really, really stupid ones, which I'll show you right after today's sponsor. So, I am very excited about today's sponsor because I successfully tricked Grockbot into paying me. Yes, that's right. This is an
- 3:51
ad for Grockbot, the thing that I've talked about in multiple videos and will continue to talk about because it's really good and I run basically all my stuff through it. It is the single best personal agent assistant thing that is out on the market right now. Full stop. It is great for teams. It's great for individuals. It is so powerful to have this always on agent system that has routines, a full computer onto itself. A
- 4:13
ton of different bots that are able to communicate with each other in just a really good desktop app, mobile app, and now even in the grock.com website. It's all in there. The whole thing just feels great. And to show you a specific example, this one's tough because like I really can't show too much of how I'm using bot because all of it's like personal private business stuff. But this one is really useful. I have it
- 4:34
hooked up so that Amazon is signed in on the Grockbot computer. And you can see right here, it just proactively sent me a message at 8 a.m. today. Hey, here's all what's arriving today. And whenever something does arrive, it goes into an Amazon locker. It'll send me the code for that. I grab it. It all works. If I want to order something, I just tell it to go do it. It'll give me a little confirmation message that looks kind of
- 4:54
like this and that's it. It just works. I could keep going on about this for a while, but seriously, just go try it out. Hook up everything you use day-to-day like notion, Gmail, etc., and then talk to the bot. Get everything set up. You'll be very happy with it at dav7.link/bot. Starting with a lot of the dumb use cases here because unfortunately I think the discourse around Jev has gotten
- 5:15
polluted by people trying to stick this thing in a lot of places that it just does not belong. This model is not as smart as a normal LLM. Just because it can make decisions does not mean that those decisions are going to be nearly as good as something like Opus 55 with actual reasoning happening with lots more context, actual tool calls doing things. This is just a switch statement.
- 5:35
And if you can make a quick gut check decision with narrow context, yeah, it's fantastic. And there's a lot of useful places to put that. This example of using Jev for compaction is stupid. Like extremely stupid. Do not do this. If you want my take on compaction, just use what the labs ship with. They have trained and optimized their models to work well with their own native
- 5:55
compaction system. They know when it's coming. They know how to use it. Don't try and complicate it. You will end up shooting yourself in the foot. And that's what this does. This goes through all of the tool calls in your history. It will score them based on whether or not you should keep them or not, and then drop the irrelevant ones. And that's your compaction. You reduce your token price. There's a lot of problems here. First off, like the instant one
- 6:16
that should be obvious to most people is that caching is now broken. You will bust cache instantly on this. If you go back and edit the history, like on new anthropic models, they're making it so you can't change the history and tool calls and whatever without busting the cache. And by breaking caching, you are suddenly reading a ton of tokens into context. Input tokens plus cash write tokens get very expensive very fast and
- 6:37
you do not want to screw with this. The other problem too is that this will not actually get you useful context in the compaction. you will lose very valuable data almost every single time because it doesn't have the broader context of what's actually happening within the request. When you do normal compaction, the model will take all of the context and pull the useful pieces out, get rid
- 6:58
of the stuff that doesn't matter, and then that's what gets compacted forward. Versus this will not have any of that context. When it's checking the read file, it doesn't know if that file was important or not. It just sees, oh, it's a read. Yeah, we should probably keep that. Who cares? Or it's just a test command. Oh, that doesn't matter. we can get rid of that when maybe those did actually matter a lot and since the model is very narrow it doesn't see any
- 7:18
of that stuff and when you try and do it this way it just sucks I think that tech one of the co-founders and lead engineers on Hermes agent a very good agent put this very well that when they tested out Jev compaction on their own compaction evals they found that it actually just removed all of the tool calls from their chat history it just wiped them so it ended up being programmatic so you don't even have to
- 7:39
pay the Jev API price you could literally just strip all the tool calls And that's your magical compaction, which is not good. And this just kind of leads to a doom loop where you keep removing tool calls from the history and then busting cash over and over and it just makes everything way more expensive and stupid for no good reason. It don't do this. This is a dumb use case. Another pretty dumb use case was the
- 7:59
auto router stuff. I think that the model router things, at least right now, are pretty stupid. They don't make much sense at all. Like the Jev router here will use Jev to decide where your request should go. Should it be routed to DeepSeek? Should it be routed to six soul? Should it be routed to 38 Flash? Where should that request go? And maybe on paper that seems like a reasonable
- 8:20
thing to do. Maybe if you just see the prompt is, can you go check my email? You would think that that would be a perfectly feasible task for a very dumb small model to go do and it wouldn't have any performance hits. But that doesn't have the actual context of what's happening on the environment. Is the environment in such a state that to go get the emails you have to write a bunch of custom code and use weird
- 8:40
connectors? Do you have to use computer use for it? How does all this stuff work? It doesn't have that info. So, you end up getting routed to a dumber model which will take longer, use more tokens, end up actually being more expensive because, I've talked about this a lot, too. The actual price of a model is not the sticker price. 6.1 Soul will end up being much cheaper on a lot of tasks than Deepseek V4.1 Flash will be, even
- 9:01
though V4.1 Flash is borderline free. It's absurdly cheap when it runs and runs and runs and runs and runs in circles getting nothing useful done and 61 soul can just do it in five tool calls instantly. That one is going to be better. The cost per task here doesn't end up working out to actually make this better. And as a result, you just end up getting stuck with a much worse model all the time for no good reason. It is
- 9:24
not a good use case for something like Jev or almost anything in general. Classifying what model a prompt should go to is a very difficult problem which requires a lot of context, a lot of engineering that a simple router like this just does not have. Honestly, on the topic of routers, one of the things I've wanted to do for a while, I'm not going to do this, so I'll just say it in the video. But it would be a really funny gag if someone like did a add-on
- 9:44
launch of a new router and they're like, "Oh, we solved model routing. This is the most powerful router. Look at how well it does on benchmarks and how cheap and efficient and good it is." and all the router is is just a proxy for GPT 6.1 Saul. I would bet money that that would outperform almost all of the other model routers because they don't make much sense. This is not a real thing right now. Don't use Jev for this. And
- 10:05
honestly, I think that's enough bashing the bad implementations because I do want to talk about the places where Jev is cool. I again I think this is such a dope useful model. Like to give you an inproduct example, I have the dev server for picking things spun up here and one of the things I recently added is a search command. So you can go in here, you could search for assets or billing or whatever and just quickly jump to it.
- 10:25
A command K menu, great feature. I want it in basically every product. One of the things that Jev is really good at is actually searching things because you can pass all of the different places where it could go. There's about 23 options here. And instead of doing a normal text search where if I type something that doesn't quite match, like for example, if I do cancel my uh sub,
- 10:45
it will direct me to manage payments. when if I was just doing a normal text search on this, it would not find manage payments. It would order these completely wrong. And the ordering I get here actually makes sense. Manage payments at top billing down here. Then the rest are like whatever. It's basically a 0% chance. And it works great for features, too. If I just type remove, show the remove background, see the assets. There should be like a
- 11:06
upload one in here. So yeah, upload an image or add image to site or something. That will also show upload image. Like it just makes the search much smarter. And because the model is so cheap and small and fast, you can kind of get away with doing something like this. In a lot of cases, the quote type- safe nature of Jev is the reason why a weird feature like that can exist. Since when you do
- 11:26
your request here, you have the choice here. So, it's doing a choice request. You tell it to rank these. You give it a bunch of different criteria to pick from and the criteria are pages. Then at the end here, you will get out this deterministic state of a completed result with this model with these answers and each of them are ranked by probability. So all the code is actually doing is ranking based off this. It's a
- 11:47
simple sort. None of this is using an agent or heavy non-determinism behind the scenes. It is basically a deterministic code function with just a sprinkle of probability in there on the switch statement. Obviously, as far as good use cases go, a switch statement that is intelligent, is just insanely useful for any like internal classification task within a workflow. The stuff I showed earlier of like
- 12:08
classifying an email, very useful, makes a ton of sense. Like in Superhuman or Gmail, the inbox filters and tags and folders, those should probably be something like Jev where it can just quickly read the email and then put it exactly where it should be. So you don't have to eat the cost of a full agent, but you also get things sorted in a much smarter way. Another just really cool example I found was uh the founder of
- 12:29
browser use, which is a well browser use company, put together a computer use demo of Jev doing computer use stuff. it got access to the browser and was able to do flight booking by interacting with the page and making decisions based on the current state of the page. In these smaller cases, this is actually quite good. You could pull this off in certain
- 12:50
scenarios. The problem is as the computer used task gets more complex and requires more external context and it's not just a straight shot. Go find this page, click this exact button, do this thing like the flights page, there's not too much context being stuck in here. If you're doing something like a lot of the things I have computer use with codecs do like go prep a video for editing or go handle a bunch of file downloads and
- 13:13
key managements across 10 different tabs and services that level of orchestration is not going to be feasible or possible with Jev. But for again quick smaller things it does work and anything that can be represented as a switch statement you can stick Jev into. Like another one here of a full self-driving truck. Uh Justin posted this one. The model is actually driving the car. It is deciding
- 13:35
what path it should take, when it should start, when it should stop, because you can pretty much represent the state of driving as a switch statement. And I think that's kind of the point. Jev works for basically anything that can be represented as a switch statement. And you need to keep the complexity and level of decision-m on that switch statement much lower. Uh like if you're trying to do some deep analysis on something at 32k context window, it is
- 13:57
not going to get the job done. But again, small one-off things is really useful. With how insanely well the Jev launch went, there have actually been a lot of companies doing similar things. There's actually one that's been doing this for quite a while that I really like. One of them is OpenAI. They just released their decisions API, which is effectively structured output for GBT6
- 14:18
Luna. You pass in a model. Currently, only GBT6 Luna is supported. You give it some input. You give it some questions. And then it has the three the same three buckets that Jeb has just with different names. Predicate, choice, score. They have a couple examples down here of like you can feed in this cup. What is the probability that it has a dent? 0% 100%. I think one of the reasons why this is
- 14:38
the first major example here is actually that since it's using Luna under the hood and Luna has vision capabilities and Jev doesn't, you can do vision classification off of OpenAI's version, but you can't off of Jev. I'm sure that they will add that. that's almost certainly coming soon. But at least for right now, if you want to do this kind of switch statement on vision stuff, you do kind of need to use the OpenAI
- 15:00
version here, which is again very good. It's the same general thing. You can see here they have a driving example, the routing example, and a document type example. Like all of these are normal use cases that work well on both of them. Putting these two side by side, the biggest difference is in the input price and also again the image inputs. The price on Jev per million input tokens is about four centsish. Then on
- 15:22
the Luna version it's about 10 cents. So it's over twice as expensive. They're both free on output charges. They have the same general capabilities. After running a bunch of examples on both of them, the OpenAI one ended up being about 2x faster versus the Jev one ended up being about 2x cheaper. So pros and cons to both. I'll be honest, most of the time I would probably just find
- 15:42
myself reaching for the OpenAI one because it would generally speaking be more convenient since it has image stuff in it, but this cost is a pretty huge difference. And if you're doing weird use cases like the P thing one I showed earlier. The price is quite useful to have lower. That other thing I mentioned earlier that's actually been doing this for a while is BAML. I really like BAML. I found this like I think it was April of LA this year or something like that.
- 16:03
It was a while ago. And what they do is really dope. It is just taking normal models. You can stick whatever model in you want and it will hard constrain and make sure that you get the correct output every single time. It is the best way to do structured outputs or at least was the best way to do structured outputs before Jev and before decisions. The syntax for this is actually a new language, a basically madeup language
- 16:25
that you give it a class. So the verdict has to have a label. It has to have a confidence and then you just pass in whatever text it is. It will use OpenAI GBT 5.5. You can use whatever model you want. give it a prompt, tell it what the output format is, and every single time you will get the output in this shape. And they have a bunch of SDKs so that it is truly type- safe where this is now basically a deterministic function in
- 16:47
your codebase that just classifies things. And I've used this a ton in the past for a bunch of YouTube tooling and that kind of thing. It's a great system, very similar to Jev or decisions, just every single model works with it. So yeah, that's Jev. I think this is an awesome new model and paradigm of doing things that the more creative you get with this, the more weird useful places
- 17:07
you could probably find to stick this into existing products. Uh, I really like the search example cuz I think that's one of the quickest UX wins you could get out of this, but there's probably a ton of other stuff that I haven't even thought of. I think this will be huge for sprinkling AI quote unquote into more and more places where it actually makes sense. just making everything into a text box is probably not the right way forward, but sticking
- 17:28
more intelligent switch statements into deterministic workflows that has a lot of applications. So yeah, if you haven't played with this yet, I highly recommend it. If you enjoyed the video, make sure you like and subscribe.
Description
Jev is novel and very useful. There's a ton of cool things to do with it, but there's also a ton of really dumb things people are doing with it.
Thank you Grok Bot for sponsoring! Check them out at: https://davis7.link/bot
jev compaction post - https://x.com/tamarajtran/status/2100...
jev router post - https://x.com/OpenRouter/status/21036...
synthetic data - https://x.com/CompleteSkeptic/status/...
baml also works great - https://boundaryml.com/#observability
My X, Discord, and more: https://davis7.sh/
