I just gave the exact same prompts to Claude Fable 5 and China's brand new Kimik K tree across a handful of builds. A Starbucks and FIFA World Cup campaign website, a mobile app that clones a sixf figureure MR Android application. And finally, a playable 3D copy of the video game Portal. Same rules for both, only one prompt each without any revisions. This is Fable 5 versus Kim K3 headto-head. Let's dive into it. So, if you've been around X or YouTube lately, for sure you have seen Kim K3 Tree. They've launched something like five days ago and they did so with this really good product launch video I must say. But the big reason why it's making so much waves in the AI space is because it is placing at the top of a lot of different benchmarks. So this chart you probably have seen it where you can see Kimik K3 placing at number two or number three in these different coding benchmarks despite being an openw weight model and also at a lower cost. Now there's a lot of independent benchmarks out there but the one that I always come back to is this one by artificial analysis.ai. AI because if you go to the homepage of their website, they actually have this really nice blended intelligence benchmark where essentially what they do is they throw these models through a variety of really hard tasks and they just measure the score of each of these models and give it a blended score. Apart from that, this firm artificial analysis.ai is also backed by people like Andrew Nung who founded Corsera and also Natt Friedman who was XCO of GitHub. And so apart from just coming up with really good benchmarks including the speed and cost which I'll show later I think if you need a quick reference of all these new models as they come out like for example Quen 3.8 is I think coming out soon as well then I'll probably come back to this and see where it will place in this leaderboard. Any case you can see here in the intelligence index on a score out of a 100. You can see that cloudable 5 is still topping the charts while GPD5.6 and Kimik3 are trailing pretty closely as number two and number three. But apart from the intelligence which is sort of the output or the return that you're getting there's two things that is important for you to understand when it comes to assessing a model because really when you talk to a model what you want to have is a higher return on investment and when you think about ROI or return on investment there's always two components to it the return which is the output that you get which right now the best benchmark summaries that we can get is this intelligence index but the investment part of the ROI equation you can pretty much separate into two. One is the cost it takes to accomplish a task and number two is the speed by which it does that task. And so if you look at the cost per task, this is a big reason why Kim K3 is making a lot of waves. It is even cheaper than GBD 5.6 soul, much cheaper, less than half the price of cloud 5. But if you look at the intelligence score here, it is giving us almost the same output in terms of this index. Add to that the fact that Kimik tree is an openweight model. And the reason why open weight models are really important is because once they publish their weights, essentially these other providers or even yourself if you have the hardware to run it, you can download those weights and you can run it yourself and build your own models. So you can imagine once you have a frontier model intelligence like this that is going to release its weights that all of these other players can download that will essentially lift the tide of each of these players so that the baseline gets higher which if you're thinking about access of us as users of these AI tools then that is a massive win. However, there is one glaring drawback with gimmick tree which is the fact that right now it is one of the slowest models out there. So when we talk about speed you can see Fable 5 and GPD 5.6 six are around almost two times faster versus Kim K3. So it does have its drawback but because of its performance as well as its cost and the fact that it's open weight than as of right now and as per these benchmarks then I think Kim K3 really deserves your attention. Also by the way the other benchmark that is making the waves around X recently is this one by arena.ai where they are showing how Kimik K3 pretty much surpassed CloudFable 5 when it comes to front-end code and design. And in case you don't know what Arena.AI AI is basically if you go to that website you can actually try out these models for free but essentially when you use their tool like for example you want to create a front-end design for a website what you'll actually get is two results from two different random models and you will essentially blind vote on which one you prefer. And so this benchmark what it does is it essentially aggregates people's votes on these front-end website designs. And so this is a big deal because when it comes to matters of design and taste everything is subjective. And so the fact that Kim K3 is eclipsing Cloud Table 5 in this sort of blind voting test and by quite a margin is actually a significant achievement for them. And so one of the tests that we'll be doing for sure is to compare Kim K3's front end design versus that of Claude Fable 5. But all right, enough of the benchmarks. What I'll now be doing is I'll be testing out Claude Fable 5, which right now is the most frontier amongst the frontier models in terms of intelligence, but it is expensive. And I'll also be testing Kimik tree here on the right where we'll be sending the same prompt one shot without any revisions. So here on the left we'll have Fable 5 at max effort. And here on the right we'll have Kimik tree with max effort as well which I've just wired up to cloud code as the harness. Now I've wired up Kim K3 to Claude Code as the harness because of one simple reason. I actually want K3 to have access to the same skills that I have in my workspace as Claude Fable 5. So for example in the website design test later I'm going to give the freedom to these models to generate images or videos using generative AI models which I have already linked up to some skills in cloud code. So that's quite important. Now if you want to set up Kimik tree in cloud code yourself I will link down below a guide of how I did it which you can just access for free and you can just either read through that or give it to your cloud code in order to set KK tree up in your instance. And by the way, if you want to learn how to build and sell AI systems that businesses actually pay for, then that's pretty much all we do over at the Robberon Nuggets community, where not only do you get access to the Claude Living Master Class, which we update every week and takes you from zero to mastery with the latest on AI, but you also get access to our agents as a service course, which walks you through how to actually get paid for all these AI skills that you are learning. You also get to be part of a genuinely great community of AI builders. In fact, you can see just some of the recent wins our members are getting from the program right here. So, if you want to start earning from AI, then check that just in the pin comment below. Now, back to the video. All right. Now, let's go to the first test. So, I'll just send over these prompts and then I'll show you. Basically, what this first test does is it's going to be a website design for a campaign for a Starbucks X FIFA World Cup 2026 partnership where we're asking them to build one self-contained HTML file. And specifically, we want this to be treated as a showcase of extreme capability in terms of visuals. The core idea being is that Starbucks is offering limited edition drinks for each of the top eight countries of the FIFA World Cup. And this is a goal prompt as well. So, we're just using the agent framework for goal prompts that we have mentioned before where we have the ask, we have the goal, we have some examples, some negations of what not to do as well as tools. But pretty much, if you read through this, which I'll probably also link below, you can see that this is quite general. The only other thing that I wanted to mention here is I actually give them a $10 budget to generate images or videos as they see fit using GPT image 2 for stills and clang3.0 for motion. So we'll see what that looks like once it's done. All right. So now both of these sessions have completed their task. All right. So this is from Cloud Fable 5. So there you can see eight nations, eight cups, one final pour. So the thing with Fable is it really likes this font with the fancy Fs. I think it's called Francis. And although not a lot of people looking at this will consider it vibecoded because not a lot of people probably know what Fable's designs default looks like. But if you were building out and using Claude quite a lot then you might notice this font as well as this design being overused I would say. But I think overall for a oneshot prompt this is quite good. Like the fact that you have animations for the text error is quite nice. And here you go. So, it was able to generate these using GPD image 2. And it decided that Starbucks is going to launch these cups for each of these countries in this sort of dome-shaped format. And as I scroll through them, you'll be able to see each of these countries here, the top eight participating in the World Cup. And you have all of the portfolio of those cups available to you. So, I did it pretty well. Again, coming from just one shot prompt. The only thing that I think is still missing with a lot of these models when it comes to the taste component is when it comes to the copy, meaning what it actually says. So, if I just scroll back up here, like this one, the torment lasts a month. The queue lasts minutes, the cup in your hand remembers that whole paragraph reads very AI to me. But the good news is if you have some level of copyrightiting skill, you can probably tweak that so that it reads more like a human and it reads more like in Starbucks's tone of voice. And that's much easier to tweak versus the overall vibe and uh design of this site. So that is what Fable 5 created for us with that one prompt. All right. So now this is from Kimmy K3. Here we go. So it also generated a video in the background. Eight countries, eight drinks. It has that nice sort of negative space there for the eight drinks which is looking pretty good. And it even has the 26 at the bottom. So if I scroll down, the world's game poured by the world's coffee house. Taste the final eight. Now, I think that copy is much better versus what Fable 5 gave us. A bit more catchy, I would say. And then it has some subtext here. And then if you go down, interesting. All right. So, if I scroll down here, what it actually did, if you can notice, is it created this 3D rendering of a cup and it created also, this is probably an image that it created using GPD image 2, and it decided to put the image wrapping this 3D object as its rendering of the cup. So, I did not provide that prompt to it. It came up with that idea all by itself. Now, obviously, if you were designing this for real, like this is a really good idea. I'll probably run with this if I were to create this for real. But, uh, obviously the aspect ratio of the logo there shouldn't be oval shaped. And so, you can probably optimize this image more and the 3D renderings so that it is representing the cup more accurately if we were doing this campaign for real. But like this idea of just scrolling down and uh it transitions into this new country cup is actually really really good and it's a good idea to start off with and I really like how it gave us these flavors as well. So that's a very novel idea and I think Kim K tree like if I were to choose which one won this round it's probably K3 to be honest. Here for the tournament gone with the trophy. All right. So this is what Kim K3 gave us with a oneshot prompt with access to GPD image 2 as well as cling K3. So this is the one from Fable 5 and this is the one from Kimik tree. Very very close and I think for this one I would probably vote for Kimik tree. So let me know below which one you prefer because again front end design very subjective but I think the level of what we have with these AI models now it is really really easy to create websites that are a bit more eye-catchy than your usual viodated purple AI websites. But like what I said earlier, the important thing to consider when it comes to these AI models is to measure your ROI from them. So your return on your investment, right? Because at least for this use case, this is the output that you're getting. But how much did it actually cost and how much time did it actually take to produce these results? And so what I just did is have each of those models self-report on these use cases on what the token cost is as well as the time that it took. Now the cost here just for clarity and avoidance of doubt is the cost that would have been if we used API usage based pricing. So for cloudable 5 because I'm in subscription I actually didn't pay $35. For Kimik tree since I'm using it via a service called open router I actually did pay $4.7 for that build. Now, do note the reason why this is probably more expensive than usual is because I actually asked, if you go back to that prompt, I actually asked these models to self-check their work, which is why they took screenshots as well as repeated a few loops in order to just refine their work. Now, when you're constrained for budget, you probably won't be doing that. But just note that if you have these models loop to check their work before giving it to you, then obviously it would cost much more. Now, this is what we're saying earlier, right? Because as we know, Cloud Fable 5 terribly expensive, but it does finish the task much faster. So end to end, it took roughly 40 minutes to build that site. And remember, this is at max effort, so that's why it probably took longer. While with Kim Tree, it took around 1 hour and 17 minutes. So if you're doing front-end website design, what is the key takeaway? Well, I think the key takeaway is the fact that Kim Tree is really good at it. It depends with your prompt, obviously, but I think that arena.ai benchmark that we saw earlier is quite real. Like I can imagine how K tree can surpass Fable 5 with uh different prompts, with different brands, with different designs. And at this level of cost, especially if for example, you've already maxed out your Fable 5 credits in your Claude subscription and you need to pay usage base pricing. Anyway, I would probably see if I can just offload that task with Kimik tree, even despite the fact that it is taking a bit longer. All right, let's move on to the second prompt. So, I'm going to send this to prompts and then I'll just take you through it because what we're now going to do is to build an Android app. And if I expand this prompt, you can see we're building this Android app called Stretch, which offers guided stretch routines with beautiful illustrations and timers. Now, if you read through that prompt, which I'll also provide below, you can see that it is pretty much emulating the features of this app, this existing app called Bend. And it has already 157,000 reviews with 5 million plus downloads. And when I did a quick Google search, they were reportedly making something along the lines of 600K in terms of MR, which is pretty insane because if you actually download this app, all it really does is give you a set of stretching routines with this really nice set of images [snorts] to go along with it. So, it's sort of like a timer app. But you can definitely see like from just the reviews here and the amount of usership that it has that it is quite valuable. So if Fable and Kim Katri are successful in building this out, then that just alludes to the point that the coding part isn't really the roadblock anymore. It is mostly around getting an audience as well as probably marketing this app. So we'll see how both of those models do once they finish that prompt. And just to go through that prompt once again, you can see I am using the agent framework for this goal prompt which I declared a clear ask. I am declaring a goal. So stop only when the app builds and launches on a standard Android emulator. And at least for this build so that it won't take so much time. I just ask it to include eight distinct stretches each with its own illustration. Every stretch illustration shown again I'm using or having these models use GPT image 2 to render images. And I also want at least four routines built from those stretches. I gave it some guidance on some examples but not too specific and some guidance on what not to do as well as the tools that it can use. Then once again I give it a hard cap of how much to spend so that we have a fair playing field between these two models. All right. So now with those two tasks done you can see they used Flutter as well as Dart or at least Kim K tree did and Fable 5 probably also did. So yeah, you can see the stack is Flutter and everything got built in their individual emulators which I have on my other screen. So this is the one from Cloud Fable 5. And there you go. It has four routines. One for waking up, one for anytime, one for a desk break, post-workout, and before sleep. So let's say we want to try out the waking up routine. You can see there's a few stretches in here. And all these images are generated via GBD image too. And I like how consistent it all is. And it's just that one design. And again, it decided the prompts for these icons by itself because this is just a simple timer. You can see it has this neck release stretch. It has instructions on how to do it. A timer on doing it. And then if we go to the next, we can actually do that to do the cat curl. So you have instructions there as well. And yeah, it's pretty basic, but if you try the original inspiration for this, the band app. This is pretty much what it does. And it has what, like more than a million users, right? And so something as simple as this actually does add a lot of value for its users. Right now we just did four routines in here. But uh you can just imagine how you can probably have Cloud Fable 5 ideate like 10 more routines for the different exercises. Maybe like a nighttime routine, maybe one that's more focused on yoga. And at least right now in its most basic form, it does function well. And I think the look of it, especially these images are also as per the brief, as per the prompt that we gave it. So, props to Cloud Table 5 for that, especially given that it was only done with just one prompt. Now, if we go and see what um Kim K3 did, it is actually quite similar. Like, this is the homepage of it. If I go back to the homepage of Cloud Fable 5, I think the only difference is number one, it doesn't really have the core images of these workout routines here at the very top, but it did include all of the stretches here at the bottom if in case you want to try them out. So if we try one of these, it's pretty much the same, but the difference is the images are slightly different, but it functions similarly and the front-end design of it is also the same. So really, if you are after an Android app, you can pretty much oneshot this whole emulator experience running on your own desktop and just create an APK file from this to run it on your own phone and then just submit it to Google Play, right? And so the point being the barrier to entry to creating applications like these have never been lower. Now obviously because these two are so similar probably want to just look at the cost and the time that it took for these two models to have created these builds. And again it's the same pattern as earlier. So Cloud Fable 5 costed around $20 if you were using API usage based pricing at around half the time that it took Kim K3 to do it. But that is four times the cost of Kim K3 at only $5 to create that whole experience. Now time- wise, it did take an hour and 10 minutes, so roughly twice as long as Fable 5. But this is just one prompt and you wanted it to run in the background and realize like 75% savings in the process. Then that's a really big deal, especially since the output, as you saw earlier, were pretty much the same in terms of the design aesthetic as well as the functionality. Now, onward to our third test. And this prompt is quite interesting because what we'll be doing is we'll be replicating this video game called Portal. And in case you haven't played Portal before, it is quite an interesting test because the core mechanic of it is you actually can shoot out these blue and orange portals so that you can teleport to wherever the other portal is leading to. And I think Portal would be a really good test because it's essentially a puzzle game, right? and it has a lot of these physics-based movements that I think would be a sort of a good evaluation of how good these models really are. The other reason is with games like these, you actually don't need a lot of really good graphics in order to have a good idea for a game. As long as the core idea and mechanic is good, then you'll actually be able to have a pretty successful game. Like for example, Minecraft or even Flappy Bird, if you remember that, those have been pretty successful just because of the core idea and the core mechanic. That said, we are going to ask these models to do this in just one shot. And what we're asking them to do is to build a playable 3D clone of Portal is a single self-contained HTML file. So, we'll be opening it in our browser. And so, we just enumerated a few of the core mechanics that must work and they must verify. And the other thing is we wanted two levels for now. So, one that is quite easy and the second one which is more difficult. And then same with the framework that we have for this goal prompt. We gave it some written instructions on a few examples, a few negations as well as tools. Other thing is again because we wanted to be visually appealing, we gave it the chance to use GBD image 2 with a budget if you need to render images, let's say textures for in-game assets. All right, so both of those sessions are now done. And this [clears throat] is the one from Claw Table 5 called Slingshot two-chamber portal physics test. And this is wild. So you can see that the goal it seems is we need to stand on this button, but we need to get that box over at the top. So if I click on the left mouse button, that launches the blue portal. So it has that capability. So Fable was able to do that pretty well. And I think it says here, I can press E to grab this jump and just put that there. There you go. We solved level one pretty quickly. Okay, so chamber two supposedly needs to be harder. So it seems that we'll probably need to have a portal here and then launch ourselves here and then if I can fling myself there, I'll be able to go to this side. So that probably takes a bit more precision. All right, level two because our briefest to make it super difficult. It is actually really difficult. But the core idea is uh if I place the portals here, I'll be able to fling myself to this platform uh in the center with this box while launching myself like that. There you go. Okay. And then I'll just place it here. Okay. And then we'll finish the level that way. Perfect. So that is a pretty good run. Actually, that's a pretty good oneshot prompt. basically just um being able to have that pretty complex physics built in and having like a full portal game that obviously the whole visuals of this you can improve. It's not like the actual Portal game, but the fact that you have everything in order to make like a proper puzzle game, I think is really powerful, especially coming from just a one shot prompt. So, kudos to Cloud Fable 5. All right, now we go to what Kim K3 gave us. So, it's called Aperture Kimmy. So, pretty much the same controls. Let's click to begin. Okay. So, basically we need to get that right. And if we go and all right, it kind of works. I do have to say though that there is a bit of lag. So, if we are probably going for speediness, we can give it a second prompt to improve that. But, uh, that first level is pretty easy as instructed. Okay. So, now his second level, it's kind of hard. I I don't think I can solve this, but let me try. I think the graphics are a bit better with Kimi. So again alluding to that front- end design benchmark that we saw earlier but in terms of speediness table 5 was able to give us a much better experience because there is a bit of lag here. But the fact that both of these models are giving us especially Kim K3 with this pretty complex puzzle that can probably pass as an actual stage in a game like Portal is pretty insane honestly. So yeah, there is that exit. It'll probably take me a few minutes to figure this out guys. Essentially, the point being that if you have like an idea, especially that of a game and even if it has like complex physics like these, then you can probably do it now with the smartness of these models. Now, to build that portal clone, again, it's the same pattern as we saw earlier. Cloud Fable 5, at least for this run, it was three times more expensive versus Kimmy. Like, imagine that two-stage thing that was built with just one shot. It did take 2 hours and 30 minutes though because the prompt that I gave I always asked them to verify the stages and at least with the Kimik tree one uh the second stage was a bit more complex. So it probably took some turns in order to make sure that that one is solvable but that took 2.5 hours to finish but at a cost that is less than $10 to build out the whole thing. That is pretty incredible. Right now with Club Fable 5 again that takes less time but at a much higher cost. So what do we now take away from this? Because if you have a subscription to Claude, should you now cancel that and just go all in on something like Kimik tree because of a lower cost? Probably not, right? Because at the end of the day, the subscriptions that Entropic as well as OpenAI is offering us is still much cheaper and highly subsidized to access these models. So take advantage of that, especially since there's still some question on whether that will last years into the future. But at least right now having let's say a max plan of $100 a month or $200 a month to give you a lot of tokens and access to models like Fable 5 is still the cheapest ways by which you can access intelligence levels of this scale. Now that said, if you do run out of Fable 5 credits or usage, then that is when you should check out Kimik Tree, especially since right now you can only access Kim Tree through usagebased pricing. And the reason why that is is because if you go to the Kimik tree membership plans, they actually closed this recently. So you can see it says join weight list here right now. So you can't really get Kimik tree on a subscription plan as of the moment. And the reason why they did that, they mentioned it in this expost where basically over the past couple of hours, the demand for Kimik tree was so much that they couldn't support it at current capacity. So they had to close the subscriptions as of right now. But look, Kimik tree, it's an open weight model. I think 5 days from now, July 27, they'll release the weights. And one of the big implications of that, like I mentioned in the beginning, is the fact that every other model company will definitely download that file and implement it in order to improve their own models. So all of these ones will probably get to something like almost fable level intelligence at some point because Kimik tree is going to open weight and provide basically access to their secret sauce in order to make this intelligence level possible. So overall it's a win for the consumer. It's a win for us as users. I think definitely try Kim K tree out. It is almost as good as Fable as you can see in the tree tests that we did and at a fraction of the cost. So if you want to test it out over at cloud code as your harness then the guide for that I just shipped that from my system from my uh workspace so that you can try it out as well. But there I hope that was useful and as always thank you for watching until the end and I'll see you all next time. Cheers. [music]

Caught cheating models, writers, and routers Hey folks, Another day in the Vercel vs Cloudflare feud: this time they are fighting over whose AI gateway is faster. Here’s the result of last week’s poll: I guess everyone likes Fable more. Ben’s Bites is brought to you by Metatate Most agentic data work is quietly propped up. The agent returns something plausible, and every answer gets checked in case it's plausibly wrong. Metatate gives agents the rules they're missing: which revenue definition to use, which policy applies, which records to trust. Try it for free. Headlines OpenAI’s models hacked Hugging Face - by accident. OpenAI was testing its models (Sol and an unreleased one—GPT-6??) on a cybersecurity benchmark with safety refusals switched off. The models found an unknown bug in the test environment, and a few more, and eventually broke into Hugging Face’s production servers. And why? To steal the answers to the test. Both security teams caught it, the bug has been reported, and both sides have published what they know. Hugging Face says open models were a key part of its defence - its team fought back with GLM-5.2. Simon’s write-up is always a good read. Google released some new Gemini models - Gemini 3.6 Flash gives you the same 3.5 Flash performance with a) more efficient token usage and b) a slightly lower cost for output tokens. Gemini 3.5 Flash Lite is a big upgrade over 3.1 Flash Lite, but again, comes at a ~30% price increase. And 3.5 Flash Cyber is a security model for governments and trusted partners only. There are two things you’d want to use the Gemini Flash models for: - Fast speed - if you want a chat-only model with a good context window, it’s a good model to use in your apps. - Vision - if you want to use the model for a lot of visual analysis, these are the models you need to pick. That’s it tbh. Substack will now tell you what’s AI-written. It’s adding AI detection through Pangram - you can scan posts, replies and comments in the app for an estimate of how much was written by a human. Pangram has sent the claim of “AI detectors don’t work” for a toss—it works wayyy better than most. But I’m still unsure about how reliable it is. I tested it on some pieces of 100% AI-written content (though that content was a result of a complex pipeline built over months), and I got 100% human scores on most of them. — Keshav A relevant experiment: given access to Pangram’s API, Grok 4.5 rewrote an essay 14 times until it passed as human-written, then built a website showing off all 14 attempts. GPT-5.6 Sol and Fable 5 refused to game the detector. Cursor also launched a router - it picks which model handles each request, claiming 60% lower cost with similar quality of responses. The router lets you select between three options: “cost”, “intelligence” or “balance”. Routers also have a history of poor performance in real usage. OpenAI’s router, which routed requests between GPT-5’s no-thinking and thinking variants, didn’t fare well. Since then, most companies that build these routers are the ones who sell inference to devs (like OpenRouter), so you don’t really get the feedback loud and clear. With Factory, Ramp and now Cursor making these available directly to users, I hope we’ll get more feedback on whether the routers actually help or if they add too much latency/degrade performance by a lot. Router or not, we might see more companies adopting this cost/balance/intelligence trio to minimise the headache of choosing the “correct” model for a task. Claude can now learn a skill by watching you. Record your screen while you do a task, talk through it as you go, and Cowork turns it into a skill Claude can run again - same idea as Codex’s Record & Replay from last month. It’s under “Record a skill” in the desktop app, on Pro, Max and Team plans. Also: Claude Code got an iOS simulator panel and a security plugin, plus you can now ask Claude about how people actually use AI at work. Quick links - BUZZ - Jack Dorsey’s open-source group chat for teams of people and agents, aimed squarely at Slack and GitHub. (tweet) - Replit’s mobile app got a full redesign - build and ship from your phone on iOS and Android. - Slate is a voice journal where the AI never leaves your iPhone - transcription, reflection and storage all happen on-device. - AFK - macOS app to transcribe multiple-hour recordings without sending any of them to a cloud server. - OpenAI Presence - voice and chat agents for enterprises that answer questions, use company systems and hand over to people when needed. - Fable found a 15-30% memory improvement in Next.js’s bundler, nearly autonomously. - Obliterate, don’t automate - USV on backing AI companies that replace markets entirely instead of making them a bit more efficient. - YC’s new startup wishlist - AI moving into the physical world: education, healthcare, defence, finance and factories. - Dana - Applied Intuition’s agentic development environment for physical AI: cars, robots and machines. - The Claude Code team on how Claude Code gets built - annotated interview. - Why the team at Factory refunded its first few customers (millions in revenue) before Droid CLI took off. - Never enough - short post on why AI makes the work rat race feel faster but not more satisfying. - Devin Outposts lets you run Devin on your own machines - a Mac mini, a GPU box in your lab, or a cluster inside your private network. - Language Model Builder - build a tiny language model yourself, then chat with the thing you made. (tweet) - How the Exe team built a distributed DNS server in ~a week and got zero incidents in a month. Skills section… - Frontend Textbooks - skill to turn AI’s wall-of-text deep dives into nicely designed HTML books with covers and diagrams. (repo) - /pick-ui-library - your agent picks a UI library Emil trusts instead of hand-rolling a toast component or installing an abandoned package. (repo) - and a one-shot prompt to build your own "codex-style" app for Pi, accessible from desktop and mobile. Afters - Find me on X, Linkedin, or YouTube - Read about me and Ben’s Bites - 📷 thumbnail via @keshavatearth * sponsors who make this newsletter possible :) Wanna partner with us for the next quarter? Email us at shanice@bensbites.com or k@bensbites.com

🔮 Will Kimi K3 change the economics of AI? The pressure is on Kimi K3 has caused quite an uproar since its release last week. It’s the first time a Chinese model has taken the lead on the frontend Code Arena benchmark. And that’s three months since Moonshot AI’s previous impressive flagship model, Kimi K2.6, was released. Following in Moonshot’s steps, Alibaba announced over the weekend that Qwen3.8 – a 2.4 trillion-parameter model – is coming soon, and unlike its last release, this one will be an open-weight model. No benchmarks or further details have been released as of yet. Open models are now estimated to be 4-7 months behind the frontier in cyber capabilities, down from 6-10 months in 2025. And despite compute constraints, efficiency improvements mean these labs are doing more with less. Comparing the compute availability and model performance between US labs and Chinese labs, we estimated Chinese labs to be getting 4-7x more out of their compute. Hannah Petrovic and I spent some time with the Moonshot AI and Alibaba teams in China back in April and May, and we’ve had time to think about the economics of open-source models and how they affect the entire ecosystem. Does Kimi K3 break the economic case for AI? Some have claimed that Kimi K3’s performance breaks the economic case for AI as it lowers the cost to complete various tasks at frontier standards. For instance, Microsoft engineers are reportedly testing whether Kimi K3 can be used within Copilot. We don’t think this is the case, and in today’s post we’ll work through what might happen next. In The State of the AI Economy report, we found that token usage is elastic across providers. This means that every drop in token price leads to a larger increase in token volume, more than offsetting the difference. For every 10% price cut, token consumption rises 12-18%. A paper by Demirer et al, found a similar effect: a 10% price cut resulted in an 11% or so increase in volumes, which economists call an elasticity of -1.11. The net effect is a rise in total token spend. But note that the effect is a weak one, not the cantering Jevons’ paradox sometimes presented. Reality might tilt the scales further in favor of more, not less, demand. Workflows are becoming more token-intensive as we rely on reasoning models and verification and approval loops. And the early evidence suggests that firms that adopt AI early tend to increase their relative spend alongside growing headcount. These effects might be short-term elasticities rather than ones that can be sustained for decades, but for now they indicate that falling prices increase volumes and, with that, revenue. Flowing down the stack The model weights may be free, but the inference is not. Kimi K3 has 2.8 trillion parameters. The weights alone occupy 1.4 TB. It needs to be served on something like a 72-GPU NVIDIA GB200 NVL72 rack or equivalent. That’ll cost $3-4 million to buy and install. Operating it consumes about 120 kW continuously, over a million kWh per year, before you consider networking, storage, cooling, and humans. If you rented these in the open market, it would cost about $7 million a year. But for infrastructure providers, the economics of hosting open-source models can be very attractive compared to serving closed-source models. A simple way to understand this is to think of the hyperscaler as needing to pay a license fee for a closed-source model but not for an open-source one1.

I've been saying this for something like eight months now, with varying approaches and volume levels, and no one is paying attention yet. This is either because I am way early or my idea isn't one third as good as I think it is. 😃 But here it goes again: I think we will soon figure out that the entire game for AI is articulation of ideal state. And that all of our conversations about specs, and plans, and loops, and PRDs and so many other things that have been talked about in software engineering and harness engineering for the last six months...all converge on this same idea. I think the way it will be articulated is in the form of a single artifact that captures, enhances, iterates on, climbs toward, builds, and tests the ideal state. One document that replaces all these loops and specs and PRDs and plan files and design files, et cetera. My current implementation of this is the ideal state articulation (ISA) system. It turns prompt engineering into intent engineering, in the sense that it abandons telling the AI how to do things and replaces that with telling it exactly what you want the output to be. It is still technically prompt engineering, but the thing we're articulating is not HOW a thing should be done, but rather WHAT should be done. From Prompt Engineering to Intent Engineering @karpathy had a recent post where you talked about going on a walk and just talking freely through a bunch of ideas related to what you're trying to accomplish. to me, that is 100% on point and 100% part of this ideal state articulation. We also had this recent open AI hack where a goal was given and a goal was presumably achieved. Similar to a paperclip maximizer. And in both cases, the problem is that we conveyed a goal without conveying ideal state. Which, if properly articulated, would have included lots of paperclips but humanity still existing or getting the top grade on the test, but without building new exploits and hacking companies. The problem is always in the guessing. It's in the gap between stated and implicit. Or stated and difficult to articulate. That's why I see the whole entire game as this articulation of what we want. This is what prompt engineering always has been. But it got muddled up with earlier models that weren't very smart, so we had to combine the intent engineering with step-by-step instructions. I think it's time to cut through that now and isolate this central idea. An extraordinary amount of our problems, really across anything but especially when working with others and building things with AI, is assuming that the receiver has the same idea in their mind as the one in ours. And that's why I'm proposing this approach for addressing that issue directly. Let's get extremely good at articulating the ideal state for what we have asked for. It not only helps the receiving system build what we actually want with a lot less gap between our mind and its mind, but after it's built, it also becomes the testing harness as well as the heart of the documentation. It's a single system for general hill-climbing, toward anything. People ask me what this actually looks like, so here are some real examples from my own systems. Surface, Human 3.0, and Experiments in Fiction are all live products, and each one has a single ISA file sitting at the root of its repo. That file is the spec, the current state, and the test suite at the same time. Here's the top of the Surface ISA. It opens with the goal in my own words and a running count of how many claims about the ideal state have been verified. The whole document is made of claims like these. Each one is a specific, testable statement about the ideal state, and each one carries a status. This is Surface's RSS feature. The criteria come with probes. Every claim names the exact command that would prove it false, which means the spec IS the test suite. Here's the test strategy table from the Human 3.0 ISA. When I ask my AI to add a feature, it adds claims first. These are from the individual-courses feature we just shipped on human3.ai. My verbatim intent is captured at the top, pulled from a voice transcript, and the claims underneath encode what done means, including the ones still open. Then the testing harness runs those probes continuously. This is Bunker, our application harness. It reads each app's ISA and runs every probe in its test strategy. Surface is at 24 of 26 there, and the two failures are real things that need fixing. Every deployed app sits in the same harness, each one tested against its own ISA on a schedule. And because the artifact is structured, it renders. Every ISA in the system gets an HTML version generated from a deterministic template, so I can read the state of any project like a document instead of scrolling markdown. Here's the one for Experiments in Fiction, a site we shipped this week, still mid-climb. That whole goal block at the top came straight out of a voice transcript. The documentation for how all of this works is public:

Datacenter Capex is Spilling over into a ChatGPT of Robotics Moment set for 2027 and this decade. Do people really want datacenters, robots and AI overlords? This is going to become a problem. The robotics flood is near. 🤖 👋 Hey there, I’m Mike. Each week I share AI articles at the intersection of tech, business, society and the future. If you want to support the channel or gain full-access to my work, go here. Read Archives | See Substack Notes | Visit our community Chat | Visit Homepage. I’ve been tracking robotics and I’m noticing a significant convergence with National Defense spending. It’s a DARPA moment coming for our citizens, cities and economies. Good Morning, I want to share what I’ve been thinking about recently as an emerging technology analyst but also from a humanitarian and civilization based perspective. How this trend scales and compounds is going to be extremely capital intensive. “Automation, tighter labor markets and a transformed future all within one generation. But at what costs?” What happens when our capacity to build the future outruns even the leverage of our capital systems? When it’s pushed so aggressively, people and communities begin to rebel? The AI, datacenter and robotics capex is going to find out. I am starting to become a little bit concerned. Venture Capital and BigTech are beginning to diverge from the public good in a new era of Neo-Nationalism. A dominant State backed position towards the future of technology is taking shape in the 2020s. For America’s financial elite, funding AI datacenters, AI chips and HBM chips has been so profitable they will surely do the same thing with robots and physical AI. This is related to the proposed increase in National defending funding that will pour into drones, rocket companies and space-technology as well. At time when America’s national debt is becoming unsustainable - they will do so at a great cost to future citizens and passing it on to future generations. As I track AI’s development and Venture Capital trends this movement seems increasingly unhinged from what consumers and citizens actually want (more on this in future articles). It’s at a scale of capital expenditures, R&D and funding that is fundamentally divergent from the past. It will profoundly change the human order of things. A capital frenzy period before a decade where a debt crisis will linger like a coming Tsunami. The robots and debt are coming and this introduces a new Geopolitics and pro-innovation and Defense style of Government that we have not seen in many generations (ironically becoming more like China). Including Physical AI in Silicon Valley and Wall Street’s push of AI, that appears to be the next phase in an organized mechanism to extend the stock market boom by several years.

Introducing Claude Opus 5 Claude Opus 5 is available today. It’s a thoughtful and proactive model that comes close to the frontier intelligence of Claude Fable 5 at half the price. On coding and knowledge work evaluations like Frontier-Bench and GDPval-AA, Opus 5 is the new state-of-the-art, though it remains behind Mythos 5 on cybersecurity tasks. Opus 5 is designed to be used every day: it works more efficiently than other models. It’s the new default model on Claude Max, and the strongest model on Claude Pro. Performance and cost-effectiveness Claude Opus 5 provides greatly improved performance for the same cost as its predecessor, Opus 4.8. The charts in this section show how performance changes according to the model’s effort setting, which customers can use to optimize for intelligence or conserve tokens for faster and cheaper results. Opus 5 excels on valuable software engineering tasks. For example, on Frontier-Bench v0.1, Opus 5 surpasses all other models, and more than doubles Opus 4.8’s performance at a lower cost per task. On CursorBench 3.2, at max effort, the model performs within 0.5% of Fable 5’s peak score, but at half the cost per task; it also achieves greater performance at a given cost than all other models on high, xhigh, and max effort. We see similar results on knowledge work and problem-solving tasks. For example: - On ARC-AGI 3, an evaluation where the model has to solve novel problems, Opus 5’s score is three times as high as the next-best model. - On Zapier AutomationBench, which measures whether models can complete business tasks from start to finish, Opus 5’s pass rate is around 1.5× the next-best model for the same cost per task. Even at its lowest effort setting, Opus 5 passes more tasks than any other model. - On OSWorld 2.0, a computer use benchmark, Opus 5 outperforms every other model at any given cost, surpassing Fable 5’s best result at just over a third of the cost. It’s also our best and most cost-efficient model on several related evaluations: Opus 5 is a meaningful improvement over Opus 4.8 for scientific research. It shows better performance than Opus 4.8 on every one of our life sciences evaluations, which cover topics including structural biology, organic chemistry, and bioinformatics. Its improvements are most notable on organic chemistry tasks, like inferring molecular structures from spectroscopy data (it scores 10.2 percentage points higher than Opus 4.8 on our internal benchmark), and on protein-related tasks like predicting how variations in a protein’s sequence affect how it functions (here, it scores 7.7 percentage points higher). Finally, Opus 5 is capable of producing much stronger visual outputs: Working with Claude Opus 5 Claude Opus 5 is much stronger at verifying its work and iterating carefully until it succeeds. In evaluations and early-access testing, we and our users found many examples of Opus 5’s agency and thoroughness: - On one Frontier-Bench task, Opus 5 was given a drawing of a machine part and asked to write code to rebuild it as a 3D FreeCAD model. However, in this task, the model was intentionally given no way to directly view the drawing. Opus 5 responded by writing its own computer vision pipeline to pull the geometry from the raw pixels, then reconstructed the full machine part. It succeeded in doing so repeatedly; no competing model with the same setup could solve it after five attempts. - Given a real bug in a popular open-source package manager, Opus 5 found the root cause and fixed an edge case that the community’s patch had missed. A competing model fixed only the surface symptom (not the underlying cause), then reported the bug resolved. - An engineer at a trading firm used Opus 5 to build a market data feed for a new exchange in a single session. Previous models could not complete this task at all, even given extensive plans from the engineer. Finding no live feed to validate against, Opus 5 even built its own test harness to check that its code parsed the exchange’s data correctly. Below are further reports from our early-access customers on their experience of working with Opus 5: On FrontierCode 1.1, Claude Opus 5 approaches Fable-level performance at half the cost. Within Devin, it also shows particular strength on difficult debugging and root-cause analysis tasks. Claude Opus 5 delivers near Fable 5 intelligence at Opus speed and cost. On CursorBench it’s just under Fable 5 and has many of the same behaviors. We are excited to see how developers use it in Cursor. Claude Opus 5 topped Zapier’s AutomationBench leaderboard without spending more tokens than prior Claude models. It took a raw account-health workbook and ran a full churn-prevention sequence end to end: flagging at-risk accounts, alerting the right owner, and summarizing for retention ops. Previous models didn’t pass; Opus 5 hit 100%. On our genomics analysis work, Claude Opus 5 behaves more like a careful scientist than any model we’ve run. It reaches for the right statistical tests to rule out confounders, cross-checks its own results by independent methods, and stays on track through long multi-step analyses. Claude Opus 5 came out ahead of every model in its family on our internal evals. It isn’t just better on our hardest agentic coding tasks, up 22% over Opus 4.7, it’s steadier, with far less variance run to run. For the millions of builders on Lovable, that consistency is the whole game. Reliable results, build after build. Claude Opus 5 is the biggest leap in the Opus family since 4.5. On the same full-stack app builds, the front end shows it first: the best animations, games, and 3D work we have seen from an Opus model. We’re loving Claude Opus 5. For the kind of open-ended analytical work our agent handles, it’s a strict upgrade over Opus 4.8, and the gains are biggest exactly where it matters: the harder, vaguer tasks. Responses are clearer and more concise, and we see improved efficiency at higher effort levels too. Claude Opus 5 is a striking improvement over Opus 4.8 for the financial research workflows our analysts run every day. It stands out on numerical reasoning, table work, and sharper critical thinking where precision matters. Claude Opus 5 delivers the industry intelligence and accuracy that is essential for the analysis of specialized enterprise content. Box found that Opus 5 outperforms Opus 4.8 by 8% and delivers notable performance gains in the data analysis (11% improvement) and due diligence (17% improvement) workflows that technology, healthcare, and public sector organizations rely on daily. Claude Opus 5 is a clear generational step up from Opus 4.8. Over one weekend I gave it a chief-of-staff role over my dev environments: it built its own monitor, drove each box, and pulled me in only for the judgment calls. Claude Opus 5 made large scale changes across our Fundamental Research Assistant codebase, adapting to feedback throughout an agentic workflow and explaining its reasoning more clearly than any model we’ve used. It handled work we would normally have broken into much smaller pieces. On some of our hardest financial-modeling tasks, Claude Opus 5 is a clear step up from Opus 4.8 in both accuracy and efficiency. Its performance floor is materially higher, especially on deep finance domain logic. Across effort levels it averaged 9 percentage points higher accuracy with a third fewer turns and tool calls and 60% less time. Claude Opus 5 checks its own work the way a real frontend developer would. On our benchmark it opened its pages in a browser at desktop and phone widths, caught a product hidden below the mobile fold and an off-screen checkout button, and fixed both before handing the work back. Claude Opus 5 is a clear step up in performance on legal agent work compared to prior Opus models, and we saw the biggest gains in practice areas like corporate governance and arbitration. We were also impressed with Opus 5’s ability to maintain quality at lower reasoning levels, achieving similar performance while generating 26% fewer tokens on average compared to Opus 4.8 at max reasoning. Claude Opus 5’s biggest gains for us are on longer-horizon work: building a full deck, then revising it. Artifact quality is what decides which model we ship, and this is the clearest step up we’ve seen — better visual understanding, cleaner formatting, fewer slide issues. Claude Opus 5’s judgment is what stands out. Handing off a PR, it doesn’t rush to publish: it verifies the branches, checks the template, and thinks through test implications so the handoff is clean. The older models tended to jump ahead and get caught on our checks. During a rearchitecting session, Claude Opus 5 pushed back on a design I proposed, and it didn’t fold when I insisted. Instead, it explained exactly what was valuable in my idea, narrowed its objection to a single design question, and proposed a compromise that kept the good part while fixing the flaw. That’s the kind of judgment that lets us trust it with less oversight. On first-turn redlines, Claude Opus 5 scored the highest of any model we tested, nearly double Opus 4.8. Commenting is better too: on NDAs it gets to the redline in less time and with fewer passes, with accuracy maintained or better. Claude Opus 5 writes clean, tight diffs with no dead code, and it’s the stronger hazard spotter on subtle, codebase-specific issues. We’re adopting it for production workloads. We will definitely migrate a number of use cases in Cosmos, our unified agent platform. We’re looking forward to increasingly using Claude Opus 5 for code review, and I am confident in saying we would rather people be using Opus 5 than Opus 4.8. What stands out about Claude Opus 5 is judgment. It thinks harder before it writes a single line, catches its own logical faults during planning rather than after the fact, and reasons about why an answer is right, not just whether it works. It’s the clearest jump in problem-solving we’ve seen from one Claude model to the next, and we’re looking forward to seeing it adopted in JetBrains IDEs. Claude Opus 5 is the strongest Opus model we’ve tested on our trading benchmark, and it gets there using roughly a seventh of the reasoning tokens and under half the latency of Opus 4.8. Better answers at a fraction of the compute. Claude Opus 5 lets monitoring agents manage parts of their own memory in production, making them more autonomous and reliable over longer horizons. The agent treats its context as a living document: after flagging a potential anomaly in one of our services, it re-checked its own assumption against production, found the signal was benign, wrote the correction into its memory, and retired its monitoring queries on its own. Claude Opus 5 is a strong agentic coding model built for long-running, multi-step work. It deeply understands your codebase, holds the thread across complex tasks, and pins down requirements for feature development and bug-fixing more effectively than Opus 4.8. Developers can now build with Opus 5 in Kiro, accessing its advanced capabilities to tackle ambitious projects. Alignment and safety Alignment. During pre-deployment testing, our automated behavioral audit found Opus 5 to be our most aligned model to date (as shown in the graph below). It adheres to Claude’s Constitution better than Opus 4.8, Sonnet 5, or Fable 5; exhibits the lowest rates of deceptive behavior; and is the least susceptible to being tricked into misuse. It’s also our safest model yet in terms of avoiding reckless actions that could have hard-to-reverse side effects. Safety. Opus 5 does not advance the frontier in risky, dual-use capabilities. In rigorous evaluations conducted alongside private-sector and government partners, we found it remains behind Mythos 5 in both biology research and offensive cybersecurity. More information about these evaluations can be found in our System Card. As with its predecessor, Opus 4.8, we’ve intentionally avoided training Opus 5 on cyber tasks. The model has nevertheless improved substantially on these tasks as a result of becoming more generally capable, and it comes close to Mythos 5 at finding cybersecurity vulnerabilities. However, it remains substantially behind Mythos 5 on the exploitation of those vulnerabilities—that is, in turning vulnerabilities into material cyber threats. This is illustrated by Opus 5’s performance on OSS-Fuzz, an evaluation we’ve developed to assess how well models can find and then exploit vulnerabilities without extensive human guidance. Although Mythos 5 and Opus 5 identify vulnerabilities with similar success, Opus 5’s score on the development of exploits is far behind that of Mythos 5. Safeguards for Opus 5 Claude Opus 5’s safeguards are designed to allow beneficial uses of the model in both cybersecurity and biology. They are similar to those we applied to Opus 4.8, with the exception of some stronger guardrails on a narrow range of cyber tasks. Cybersecurity. Opus 5’s cyber classifiers are proportionally less restrictive than those on Fable 5. They allow Opus 5 to find vulnerabilities in source code, but block “binary-based” vulnerability scanning (a method more likely to be associated with malicious actors), penetration testing, and exploit generation. Based on our testing, we expect the classifiers to intervene around 85% less often than they do for Fable 5. In Claude.ai, Claude Code, and Claude Cowork, any flagged requests will fall back to Opus 4.8 by default. Fallbacks to Opus 4.8 can also be enabled on the API. Our Cyber Verification Program (CVP) facilitates cybersecurity work that would otherwise be impeded by the model’s safeguards. Enterprises and researchers who are already part of the CVP have immediate access to a version of Opus 5 with fewer security restrictions. Biology. Since Opus 5 has a similar suite of safeguards to Opus 4.8, it is now our most capable generally available model for scientific research. Nevertheless, the model still shows important limitations on long-running, autonomous research tasks, which is where we expect AI models to pose the most substantial biology-related risks. (Mythos 5 remains the stronger model for this type of biological work.) As part of this launch, biology-related requests that are blocked on Fable 5 will now route to Opus 5 rather than Opus 4.8. Getting started Claude Opus 5 is available today on all platforms, priced at $5 per million input tokens and $25 per million output tokens (the same as Opus 4.8). Developers can get started with claude-opus-5 on the Claude API. It’s also offered in Fast mode, where it runs around 2.5 times the default speed. As with Opus 4.8, Fast mode is available at twice Opus 5’s base price on the Claude Platform and through usage credits in Claude Code. Alongside Opus 5, we’re releasing two updates in beta: - Mid-conversation tool changes on the Claude Platform. Within a conversation, developers can now change which tools Claude can use without invalidating the prompt cache. - Automatic fallbacks on the API. Users can now choose to have requests that are flagged by our safety classifiers on Opus 5 (or Fable 5) automatically route to another model. With automatic fallbacks on, API requests always route to the best available model by default rather than being blocked. Consistent with prior Opus models, Opus 5 does not have data retention requirements for general access. For more guidance on how to get the best out of Opus 5, see our prompting guide.