14:09
Aentic AI computers using computers and all the new stories around that. What does it all mean? What does closing the loop mean? We're going to build it at least baby's first version of that. It's a party of mini PCs over here. This is our MSI QBN running Windows. Perfectly ordinary there. But I've got it hooked up to an agent and it's all running locally. But the AI is not running on the QBN. The AI is actually running on DGX Spark, our MSI Edge Expert to be precise, and it is marionetting our MSI QBN. It's part of a triumvirate of technologies that allow you to close the loop as it were. The loop is here's the thing I
would like for you to do. Here is the success criteria for knowing that you've done it. You build that algorithmically and then you've got an agent running and doing things. And there it is, our Pac-Man clone in the browser. This was with the Hermes agent, but I'm also going to show you a full setup with Turnstone. Basically, you set up the AI and then you hook the AI up to the harness. And Turnstone and Hermes do some different things. Open Claw is also really popular, but I'm not going to get into Open Claw in this video, but let's talk more about it.
Okay, bottom line up front, there's a full guide on the level one tech forum that'll walk you through all the steps to set this up. If you just unpack your DJX Spark and you got a mini PC laying around, you can follow the guide and set everything up on the mini PC and everything is great. But if you're here to learn about like all the buzzwords and everything that's been that you know, all the people are saying about Agentic AI and closing the learning loop and that kind of thing. Let's calm down. Take a deep breath. This is a little bit of a hype cycle, I think. Let's start at the beginning. The things that are in play here. One, DGX Spark or MSI Edge Expert. This is where the AI is running.
It's all local. It doesn't have to be local. You can run it in the cloud and do that kind of thing. You're probably familiar with chat bots. You chat with a chatbot. It tells you to do things. Maybe you copy paste commands and then you get errors and you paste it back. Then the next thing that happened after that was a harness. And the first harness that sort of took over the world was OpenClaw. And it's very wild west, very loose. That's the one where we sort of all make fun of it because it's like, oops, I deleted all your emails. Not good. There's also Hermes, which I think is much more mature and adultlike. And then there's Turnstone.
It's it's it's something that like is amazing and you should check out and is on GitHub. It's from Patrick and I've done a little bit of work on it too like I added some interface stuff to it for uh Neotron uh the Neotron Omni model so they can do voice and video and so I've also been working on Turnstone this whole time. So Turnstone is a harness to harness the models that you can chat with and hopefully turn their productivity turn them towards something productivity. Turnstone also sort of funny because Turnstone makes fun of like the claw thing. Yeah. Anyway, and turn is a harness that we can use. Now, the harness gives us some other things.
It gives us a lot of other things. It gives us a way to not just have one chatbot, but to run a bunch of sessions on a model simultaneously so that we have a bunch of instances, chat instances working together to accomplish a goal. Those are called agents. When you're using AI like that, the AI chat things, the agents work together. And so Turnstone can be said to be an orchestrator. It's it's like an orchestra conductor. It's conducting a lot of AI agents. This is why the you know there's been a token explosion. It's like first it was chat bots and then it was reasoning chat bots and that was the first token explosion and then there was another token explosion. There have been several uh when we talk about
that you as an individual are not just talking to one chatbot at a time. You're talking to kind of a supervisor chatbot and it does other things. So if you look at turnstone you look at the orchestrator the setup here there's a drop-own menu that lets you config what role is this going to play orchestrator engineer uh maybe scribe for documentation this is sort of useful for the role that the AI is going to play now I know what you're thinking AI is not deterministic it doesn't behave consistently any which way it is not good to give it access to this that or the other yes totally agree with you. But there are ways of mitigating that and there are ways that you use AI to
build durable infrastructure with a harness like this. So instead of building something where you just turn the AI loose and you say, "Hey AI, babysit this thing," that's not really what we're doing here. It is, "Hey AI, help me build a durable process that may not even involve AI by the time we're done, but is sort of cooperatively built with, you know, human and AI." And so the process for that is it's all kind of wild west. Different people are coming up with different processes. Everybody sort of imagines that their way is the best way. I like Turnstone for its enterprise architecture and how it does output guard and that sort of stuff.
I've got a separate video coming on just turn. Turnstone is really exciting. You can check it out on GitHub, but I don't want to get into that too much in this video. But it does have a lot of safeties and guard rails and enterprise architecture, enterprise think um that Patrick and I have put into thinking about these kinds of problems and how it might exist in an enterprise and how you might give it access to help you configure switches or manage servers or help you manage software. See, even these really advanced AIs, I mean, they have the skill, they don't have the taste. And that does, you know, some people it's like, oh, the goalpost is moving there a little bit, but when you're doing really big enterprise
architecture, you're going to have to step in and do get your hands dirty. You can't just say, "Hey, go do this thing." Um, and the harness is useful for that and spending off the age and that sort of stuff. So, there's also a judge. There's a judge model that's judging what's happening to make sure nothing's going off the rails. And in our case, we're using GMA for that from Google. So, we're running Neotron to do the heavy lifting here that I'm showing you in this video, but we're also running GMA at the same time. And GMA is kind of watching what Neotron is doing. That gives us some resiliency, that gives us some insight, that also gives us a place to pump the brakes. And this
system also has permissions. It's like this is a high-risisk thing, this is a lowrisk thing, this is a safe operation to do. Um, the harness has this idea of skills that explain to the AI how to maybe call a script or to do a particular thing that can be read into the context as it is relevant to understand what it was. Think of it like a quick manual. So like you I don't know you have a piece of office equipment in your office like maybe the coffee machine. It's like the coffee machine requires you to first put in the coffee cup and then turn the dial to whatever mode that it's in because it resets every time and then you decide if you want hot or ice and then you press the
button. So a skill file would explain that to somebody like how to operate the coffee machine. It's really not a lot different. Sometimes there can be Python files bundled with skills. Then there's this idea of MCP model context protocol and there's lots of MCP servers out there. If you do home labbing and you do stuff with Proxmox, there's a Proxmox MCP skill. There's some that's a whole other story that there's some fun stuff coming with that. But the Proxmox MCP skill explains to an AI the kinds of things you can do with Proxmbox like how to create a container, how to create a VM, how to move storage, that sort of stuff. Um, but the MCP exposes those skills as part of the MCP server because
the skills aren't really relevant if you don't have the interface to call it. And MCP can have uh different transports like you can do standard IO stdio that is for local host stuff where everything's running on a single machine and then there's also an HTTP transport which is what we be using this video to control that Windows machine. There's lots of different MCP servers for different use cases in other words. So the MCP server that we're using on Windows is called cursor and it's pretty good. It's a little clunky, but this, you know, again, crawl, walk, run, I think it's a lot of fun to show you what the possibilities are and then maybe you get some ideas for fun stuff that you
can do. The stuff that I'm showing you should not reasonably replace any human being at all. And that it seems to be in corporate America is probably more of a testament to how poorly understood things are with management in a lot of cases. But I digress because, you know, you don't want to offset the multiplication of your productivity by tamping down your productivity because you got too productive. You know what I'm saying? It's sort of weird to set up Turnstone. This is going to be another video. There's really a lot to talk about as far as Turnstone goes. There's a lot of really exciting things, but the quick
start is pretty good and the how-to will walk you through that at the terminal. On our totally fresh Spark system here, we basically just run the commands. It's going to clone the GitHub repository. It's going to ask us how many server nodes we want. I've chosen six because we're going to be in a relatively memory constrained configuration. This is basically how many workers you have that are running here. This is all managed through Docker. When you run Docker ps, it's going to show you these containers. It'll run through build the container and it'll tell you all the stuff that it's doing to set this up here. If you want to customize this, you can modify
your Docker Compose YAML. Then the setup helper will emit this and you're good to go. Now, we went through the, you know, earlier in the how-to when we're talking about setting up the VLM images and that sort of stuff. You can go through the DGX Spark playbook and get there. But there is also a helper, another different an alternate way is what it is described as in the how-to. It's VLM plus light LLM on one unified memory box. And what this is going to do is give you reasoning, perception, and reranking. Ranking is important here. Now, I haven't mentioned reranking before, but it sort of goes without saying that garbage in, garbage out. And that's true of anything in computer science, especially AI. And so, what
re-ranking does is improves the quality of the information sent to the LLM by sorting retrieved documents based on how well they actually answer the user's questions rather than relying on embedding similarity. This means that the model retrieves more relevant context and spends fewer of its limited context tokens on unrelated lower value information. So, as the context grows, you know, we're we're working with local LLMs here. It's tough to do a million token context. It is doable, but you're going to need more than one Spark and uh or other systems that are, you know, have larger memory. Um and so the re-ranker helps keep the quality of the output really high because it is re-ranking the things that are fed into
the model with things that are relevant. It's sort of preloading all the relevant stuff. It's also as a timesaver and a token saver. So, it's pretty awesome and having the reranker run locally, even if you're feeding the cloud AI, is huge. So, these how-tos will walk you through that. Stay tuned for the full Turnstone video, but I feel like this is a really easy installation. Patrick has done an amazing job with this write up and all of the stuff that has gone into making the installation of Turnstone as easy as possible. If you stumble over something or you have feedback, definitely let us know. But early community feedback from early adopters has been pretty positive.
Now from here making sure turnstone is running. We can do docker ps. We can see that all this is running. We see our two LLMs are running. Three LLMs are running. Well, this is a you know. Yeah. So we're up a few hours cuz you know time passes. And so this line here tells us that the web browser part of it is running and it's on port 443. You could change that if you have other stuff running. 8443 is what I use sometimes, but that's just in the Docker Compose file. Uh but this means it should be accessible in the browser. You can get to it from the console on the DGX Spark or from another machine on your LAN. When you go there, you'll get a privacy error. It's fine. We'll just continue.
It'll ask you to set a username and password. I already set mine. And then here we go. So, this is the Turnstone UI. And again, I'll do a different video on this. This is just a fun harness. You know, this could be Hermes or OpenClaw. Although, start with Hermes. Like OpenClaw, it's just OpenClaw is just too wild west. Hermes is pretty good. Hermes is actually amazing and but it's a different thing. solves a different set of problems. It's not solving the exact same set of problems that Turnstone does. All right. So, the first thing that we'll need to do is go to models and hit add model. And when we add a model here, we want to, you know, give it an alias. But first thing is that we'll do is just give it a base
URL. And the API key can just be test. And then we can do detect down here at the bottom. And it says, oh, one model available, 65,000 tokens. And we can come up here and the model ID will autocomplete and the alias will autocomplete. You know, if you were following along, mine already autocomp completed, but it's cuz I had to do this a second time because my context is a little low. The judge model on Turnstone needs kind of a lot of context, 65,000 tokens. If you set this up the alternate way, it'll be 131,000 tokens because the alternate way is a little more efficient because it's using VLM and lock LLM at the same time as opposed to multiple instances of VLM. 128 gigs is not enough memory. It's like this is learning. This is
actually really good stuff for you to learn how this stuff is put together at a larger scale. You also have options here for capabilities, tools, vision, server side, web search, reasoning and effort control, speech to text, transcription, uh text to speech and speech to text, audio input, and a re-ranker capabilities. And so setting these appropriately for the model uh is important, but for this thing, I'm just going to skip over it for now. We'll also want to do add to model for any other models that you have running. You can get two or three of those configured in here. And then you'll want to hit sync to nodes so that it will sync to all of these nodes and they're they're
aware that they have a model there. With the models created, we'll just do a quick smoke test here. Options, you know, we don't need to set a persona. And we want it to run with Neotron 3 Nano. Could use Neotron 3 Nano or Gamma 4 as the judge. You can assign it to a project. And this is the first time this model has been used. It just started up. Uh if you want to, you could do Docker logs from the terminal and see what it's doing. There we go. About 45 tokens per second on Neotron 3 Nano. And this can vary uh if you do the alternate way. Um, I have seen setups on the forum that are up to 27 tokens per second, but VLM right now inv4 and like dense models. The real
limitation there is memory bandwidth. And so this is doing a little bit of a smoke test on all six nodes. And so it'll dispatch things to all six nodes. You'll see it over here on the side. There is a light and dark mode. Like I can toggle between light and dark mode while we wait. Isn't that exciting? Fun times with Turnstone. Now that this is set up, we can set up our MCP, our MCP integration, uh, extensions, skills and MCP server. So, I mentioned skills and how skills work. You can create a skill and it's like you describe what the skill is here. There's there's tons of skills on GitHub. You can go find skills
and paste them. You can set options for the skill, how much thinking or reasoning it needs, a runtime config, which model works best with the skill. Uh, you can also say what tools are allowed in the list of tools that are in the system. In resources, you can also add scripts. So, if you have a Python script or a bash script or something like that is part of this tool call, you can add this. And this is where I've added some of my scripts for doing um certain kinds of tasks. So, I can ask the AI assistance like, hey, can you prep this system by running this script? That way that the AI is not yoloing a list of tasks. It is looking
at this skill and it is saying, okay, here's the stuff that I'm supposed to run. And then in the skill, I say, you know, that this has succeeded. when you run this and verify like you run the package manager and verify that all the packages have been installed or whatever it is and the nice thing about this versus a traditional script is that it's a little less brittle. So like you know installing a bunch of packages with apt is super easy to script. You don't need AI for that. But when canonical does crazy stuff like changes core utils to rust and that sort of thing uh and things break in strange and subtle ways.
This thing can actually troubleshoot that and deal with that. Like the AI can do that and then it can flag it and say hey we might need to adjust the skill or hey we might need to adjust the scripts as a junior intern level assistant might. Um but that's not why we're here today. We're we're here to do the MCP interface to our little QBnuck AI+ and we need to add it here. But first, we have to set it up on the Windows machine. And now the Turnstone is complete. We can take a look here. All six worker nodes are reachable and ready for work. The cluster nodes spawn a dedicated worker on each all six work streams. If you want, I can send the report node ID and time stamp. Get an
actual response. Just let me know what would be fun next step. See, do that. And look, we're only using 14 of our 256,262,000 token context. Approve, deny, approve all, and go ahead and approve. And then we'll see these nodes come online. That's what the icons mean. If we need a little more insight into what a node is doing, we can click on that, of course. So once I hit approve in the corner there, it will create children that is doing the smoke test on all six nodes just to make sure that they're they're there. can also deny. It can also have a conversation with the model to say, "Hey, don't do that." There's actually kind of an extensive thing in Turnstone.
Again, this is probably going to go in the Turnstone video. But, um, under governance, under personas, you have some different roles that it can play. You know, manager role, engineer role. This kind of thing does actually help. Um, in the judge section, there's a lot of options for output guard, heruristic rules, and how these kinds of things work. And so, the output guard is heristic. is not necessarily AI based, but AI is using these tools to achieve an outcome and that is much more effective than just stuffing it all into a prompt. So on our little Spark system, we're running, you know, right now we're
running Neotron models. There's Neotron 3 Omni, which is designed for uh, you know, transcripts like video, understanding video and audio and pictures and multimedia. And it's also capable of reasoning. It's what's called a 30 billion parameter model, but we're using it in NVFP4. But there's other stuff running on our Spark, too. We also have GMA 4. That's from Google. That's not from Nvidia. We're using that as a judge model, and it's a very small model. We're using the 4 billion uh parameter version. Uh there's also Gamma 3. The 12 billion parameter version of GMA 3 is also a pretty good choice for uh a judge model for our harness. And then we also have Neotron 3 Nano which is also a 30 billion
parameter model but is a little better at other things. Now those are it's Neotron and Google but this is also Quinn. Quen a lot of people like Quen 3.6. Quen 3.6 is available in two different formats in NVFP4 and that is uh the 27 billion parameter model. That's called a dense model which means that all 27 billion parameters are active at once. That runs a little slower on our spark system or coin 3.6. 6 35 billion which is 3 billion parameters active at once. That's what the A3B means in the name and that's going to be much faster on the spark because there's only three billion parameters active at once. Now the 27 billion parameter model in an 8bit quant
which means it's going to use you know 27 to 32 gigabytes of your 128 gigs of memory. That is probably the best agentic coding model for uh coding. Uh the Neotron models aren't bad. uh you know of course uh an enterprise model like from OpenAI or Anthropic those are going to be even better and you can mix and match those with Turnstone. But the important thing of running three different models on this is that with the appropriate harness you can control a whole fleet of little AIs that are good at the various things that they're good at, but all of them are running on one piece of hardware. Or if you have two Sparks you can have some models running on one Spark, some models
running on the other one. You can even run really large models across multiple sparks. That's what the connect x4 interface is for. Uh but generally those run a little slower and it's not really as useful. With the agentic harness though, you can run small models to help you plan or to help you do janitorial tasks if you're doing coding tasks, things like cleanup, managing the git repository, looking for common things, and that'll also help you minimize your cloud token spend. So that's really efficient. For this video, we want to also talk about skills, which is part of the harness. The harness can say, "Hey, here's a list of skills, and skills are descriptions of how to do things that
the AI models can read and understand." Keep in mind that everything that I'm showing you is running entirely locally. We're we're doing the AI off of the spark machine, but the spark machine is basically marionetting that uh MSI QBN PC and skills give it some hints about how it does that. The MCP server also will typically publish some skills and the cursor MCP server that's on GitHub uh I think has 18 or 19 built-in skills. things like opening files, manipulating the registry, common things that you would want to do on a Windows machine. So, that's pretty awesome. IMHO. Let's take a closer look at what that looks like on the Windows side of things for MCP and how that interfaces with our
harness. So, this is our Windows workstation. I know chaos abounds. Don't can't help it. This is our MSI QBN AI Plus that I reviewed previously. and it's connected and it's a little pile of wires, but it's running Windows over here and it's running our Windows MCP uh client. This is just a Python program. So, you'll need to install Python before you use this. And then to connect this, you'll just run Python MUV tool run MCP serve. The full guide for this is on the forum. Uh, and I'm using it with an O key, but you could use it with a TLS certificate or something like that. What this does is it opens up a port. It's port 8000 on this machine which will receive authenticated connections
ideally with TLS. In turn under manage under MCP servers we need to set up the connection to this MCP server. The transport is going to be streamable HTTP and then I'm going to give it the IP address of our Qbnuck uh 8000mcp and then authorization with the bearer. uh you have to set that in the header manually. Just follow the how-to. Uh multi-tenant authorization. We're just going to use a static header for this. Uh auto appro approve tools and enabled. And so when you save this and now what this is going to do is expose um this MCP server to the uh to Turnstone, the agents and the orchestrator that are running here. And this Windows MCP server provides some
tools, some basic tools in the description of how to use them. And so I can come now to the dashboard or the orchestrator and I can say um please launch notepad and again like this is kind of a crawl walk example and Notepad is launched. And notepad launched again. And so I told it to say hello from Turnstone, but it messed up a little bit. It took a screenshot, saw that it messed up, and then said hello from Turnstone. Can you tell me anything about this machine? CPU, GPU, Windows version, and any other relevant info. And so this will run a PowerShell command. Inventory the system. Look at device manager.
Uh Intel Panther Lake XE3. So here's our full system overview. MSI Pro PTLH QBnuck AI Plus. That's MSB211. I didn't even know that. Windows 11 Pro 25H2 installed March 9th, 2026. 16 cores, 16 threads, base clock 2.1 gigahertz. uh integrated XE3 graphics, 2 gigs of shared VRAM, displays currently 3840 by 2160 at 60 Hz. It's the integrated XC3GPU with 32 gigs of total memory, one TB f01 TB uh TCCZ, and it is dual channel DDR5600. And we also it's like, oh, by the way, a little while ago, we just ran the ADA 64 memory and benchmark run, and these numbers are, you know, in line with memory bandwidth. Now remember all of this is running locally. This is taking a screenshot and doing stuff with the screenshot but it's entirely locally. I
mean it's uh executing code on our QBN but the screenshots run over the network to the uh the MSI um edge expert DGX Spark. None of this hits the cloud. This is toward the kind of experience Microsoft promised with Copilot except this is something that I'm much more comfortableish running locally because none of this ever hits the cloud. I mean the AI can just take a snapshot and do stuff with the system. Now I wouldn't because the AI has full access to this system. I wouldn't put any privileged information on this even though Turnstone has a lot of guard rails and safeties and that sort of thing. It's really quite innovative in that respect. IMHO but again wait for the video on turnstone for more about that but again it's
compartmentalization that's accessed it's like this is a this is the AI intern would you give the AI intern full access to literally everything no but does that mean that it's not productive or you can't be productive with it also no I think there is something here there is something really interesting here in the way that um you know that some people were making fun of you know home computers when powerful computers mainframe computers were the only computers on Earth. It's like why would anybody want a computer at home? They're inferior. They're not very powerful. You can't really do much with them. And yet enthusiasts took them and ran with the idea of home computing. This is sort of the same thing. We're waiting on the technology
to catch up a little bit andor the hype for the technology is uh driving some of the complexity and expenses here. But this is really kind of fun and interesting that I can marryet this Windows machine from another computer, an AI interface. Now, the Windows MCP GitHub, it talks about interfacing with a claw desktop and services like that, but that means that it's taking a screenshot and doing things with cloud APIs, which, you know, I don't know that I would necessarily recommend. This is all running totally locally, which is fantastic. If you'd like to set up Hermes, the Hermes agent instead of Turnstone, it's pretty much the same steps. You just set up the AI models, you get the models
running on your uh DGX Spark, you can, it's still a good idea to run multiple models for the Judge and that kind of thing, but Hermes doesn't really require it. And the out of the box setup with Hermes, just the very most basic setup. You run the installation from PowerShell. It's paste a command. Basically, you let it run and then the Hermes setup wizard wants you to use their cloud thing. They want to make it as easy as possible, but we don't want to use the cloud thing at all. So, you just sort of let that run, go through the setup, and then you'll have to run Hermes setup again. And then when you run the Hermes setup again, you'll choose option 32. And when you do option 32, you'll enter the IP address
of the DGX Spark of our MSI Edge Expert. And then you can let it auto probe for available models. Now, in the office here, we've also got a two-spark setup running a special NV FB4 version of Deep Seek, which is, you know, crazy, I know, but if you're using Neotron Ultra or Neotron 3 Nano or something like that, it'll auto detect. It makes it pretty easy. And then when that's set up, you can just run Hermes. If you're not sure which model it is or you just want to, you know, do a quick verification, you can do that in Firefox. You can just go to http and then IP address of your machine col 8000/v1/models and it'll tell you which models are available. Remember, we got port 8000, 801, 802 for different models that are
running on the system all the way up to 803 and 4 for the different instances of VLM. And if you wanted to use Turnstone's light LLM setup, but actually use Hermes as the agent, you can do that, too. It's fine. I'm Hermes agent running on your local machine ready to help with whatever you need. So, this verifies that it's working. Hermes solves a different set of problems than Turnstone does. The other thing with these agents is that they create notes and memories by just writing to markdown files. Turnstone has its own facility for storing memories. Meta cognition is what we call it. But in Hermes case, you can have it learn and take over. And that's why
the, you know, it's like notetaking, productivity, research. It's a whole rabbit hole. You're getting ready to go down here with agents. And you close the loop in the same kind of a way. It's like, here's the evaluation criteria. Here's what you look like. Run the thing. Do it. Hermes computer use install. It's like, oh, I need the skill to do to use the computer. There's Notepad. They launched. It's not the active window. It'll take a screenshot to verify it landed. Open in the background without stealing focus from your current work. Nice. And it did type into Hello, this is Hermes agent, but it was into the config gamble that I already opened. Woo! Agents are going to take over. It's like this is
why it's like, oh, layoffs because of AI. Sure, it's not layoffs because of economic downturn. Okay. [snorts] So what's the loop? Like you probably seen or heard the agentic loop or like now I'm not talking about human in the loop. Although human in the loop is a natural consequence of this. But with a skill you can set this is the steps to accomplish X. But you can also put evaluation criteria. Here is the criteria that you run to evaluate whether or not you've been successful. And the loop is the AI will run and do the thing and then see how uh complete it is or how much progress it made. And there's different kinds of loops like you could do a hill climbing loop where the AI will iterate toward success like
it might not be able to get to success immediately but you can give it evaluation criteria to say oh this is more successful this is less successful and then let it run and do things. Now, for the demo that I've showed, you know, it's a little contrived and uh not super effective, and you would probably mock it if I said, "Hey, let's use this for an actual business use case, but as a building block, you probably have some creative ideas, and you could probably put something together." Uh other very talented technical people have used this kind of loop system. This is sort of like here's the goal, do the thing, um with enterprise models to do everything from uh building a hardware interface to an e- in display. That's right. hooking
up a model to a serial port to a physical EI in display and then giving it a webcam to where it can see. In the same way that our MCP server can see the desktop of our Windows machine, the uh the model was able to see the e- in display and help develop a library for controlling the e- in display completely autonomously. So the AI could do some steps, look at it, evaluate, do some more steps, look at it, evaluate, and go on from there. That's why the harness can be so important because you know our context here is only 256,000 tokens. It's going to be important for the harness to compact the context and it's going to be important for the model to maintain focus and be able to
uh not lose the plot of the problem that it's trying to solve. And that's a little bit model and that's a little bit harness depending on again like I say if you're doing coding tasks you definitely want to explore the Quen parts of the how-to as opposed to the Neotron parts. But it just depends on what you're doing. Emotron's good allrounder models for doing things. And don't forget, you can also hook up cloud. Like if you want to do open router and send requests to the cloud there that are complicated and bring them back, you can. But running all of this AOT locally on your Spark will still save you tokens. So you're going to be spending way less tokens than if the whole thing was hooked up to the cloud. And that
could be one of the directives. You can tell the orchestrator like, "Hey, I want to send uh as little tokens as possible." And so it'll spin off an agent that has access to whatever the cloud model is that you tell it. But then that model will uh be used conservatively by the orchestrator whenever the orchestrator feels like maybe one of the agents is looping or one of the agents is has lost the plot of the problem that it's trying to solve. So you have different models working with slightly different goals and slightly different configurations. And that's pretty much all there is to it. There's a lot of promise here. this technology really it's like the first you know sort of meme I know I always refer to it but the first meme of Will
Smith eating spaghetti that's sort of where we are with this technology for what runs locally even on relatively expensive hardware it's pretty early it's pretty nent but look how quickly we went from Will Smith eating spaghetti badly to nano banana that's how quickly this technology is moving otherwise uh not just with local AI because of sovereign reasons but also in the cloud and with different cloud providers. So, it's sort of an interesting time to be alive. But you can experiment with all these technologies right now. Like it's all the same at the enormous multi-million dollar scale as it is in the multi- kilob 10 20 50 kilob range that you have here. And yes, it is expensive for the individual to experiment with this level
of hardware for now. But I think that very quickly it's not going to be as expensive because the technology is going to ramp up and also we're going to have a better understanding of how to use the technology efficiently because right now basically everybody's throwing everything they have at the technology and you know some of it is going to work but a lot of it I don't know it's interesting. Oh yeah, and there's another aspect of this which is DJX Spark has been out for a while now, but the software has improved dramatically. These mixture of experts models that are running on DJX Spark are dramatically faster than they were at launch time. And some of this is because multi-token
prediction. Some of this is because we're better able to access the hardware more efficiently. A lot of it is because of NVFP4. NVFP4 is the star of the show. So, if you get a model that can natively do NVFP4 and has all of the other features that you need for this platform to like fully take advantage of all the hardware and the relatively limited memory bandwidth, then you can get some pretty amazing speeds. That 35 billion parameter Quinn 3.6 model, you can get upwards of 70 tokens per second with all of the stops pulled out to be able to run it in that 4-bit format. The dense format of Quen 3.6, 6. The 27B is more like 15 20 tokens per second max using MTP and all of the features that you
possibly can in that NV FP4 format. But this is kind of where we are. This is kind of how I see it with Aentic AI and harnesses and that sort of stuff. So, if you want to learn more about Turnstone, uh, hit us up in the forum. Let me know what questions you have. That'll help shape the video on Turnstone. I'm level one. This has been actually a lot of fun. sort of cautiously optimistic about the new technology. I'm signing out. You can find me in the level one forums.
