7:53
Holy crap, if you have an R9700 and you've been messing around with the software, uh drop everything because the software thing has changed. Unless you already know about Radiance, in which case hello and welcome. We've got a lot to chat about. It's going to be a little bit of a ramble because Nvidia also just discontinued their 32 gig gamer GPU, the 5090, in favor of much more expensive GPUs. So, the other version of this video could be the CUDA hour is later than Nvidia imagines. We need to talk about this.
I really want to talk about how much the software has improved for the R9700. The R9700 hardware is now firing on all cylinders and it is uh hip and ROCm, but it is really Radiance. And Radiance, as far as I know, the people behind Radiance are not really exactly AMD insiders. RDNA, CDNA AMD's always had the hardware, but the software has been a little bit of a challenge. At the same time, if we switch gears, talk about Nvidia for a second, Nvidia did this to themselves. So, when I say the CUDA hour is later than Nvidia imagines, Nvidia knows that. And it has to do with the software. A lot of people think of CUDA as a moat around what Nvidia is doing and the hardware and that it is difficult to adapt. And just because AMD has been in
a little bit on the struggle bus with RDNA and AI in that context, that has sort of reinforces their opinion. Never mind, Intel is killing it with Int 4 on the B50 and there's some really interesting things coming out of Intel on that side as well. And Intel software and hardware have gone together pretty well. No, no, that the hour is late on CUDA because the tools exist now to be able to move really quickly. And the tools exist now to be able to remove really quickly also with Blackwell. See, Blackwell has also been a problem for people like me that have bought the RTX Pro 6000. It's a fantastic GPU, but that is not the same Blackwell. And the Blackwell that's in DGX Spark is not the same Blackwell
that's in GB200 or GB300 or B200 or B300 base systems. That is also Blackwell. That's enterprise Blackwell. It's basically unobtainium unless you're you're one of the hyperscalers or you're you're spending an obscene amount of money. Uh the DGX Spark system that's laid out on my desk here with some PCIE upgrades that I'm attempting and farting around with. That's SM121. RTX Pro 6000's SM120. And that uses different math kernels. The hardware is a little different than what you get in B200 and B300. And Nvidia calls all of that Blackwell I think because there are some commonalities in terms of like NVFP4 support and Nvidia rolling out NVFP4.
Nvidia betting the farm that NVFP4 on the DGX Spark is going to save it from the 300 GB per second of memory bandwidth, you know, being relatively anemic. AI tools helping develop that kind of software has come through for SM120 time and time again. In my opinion, I don't think it Nvidia has put enough resources in developing SM120 because the amount of breakthroughs that the community has done for SM120 has been incredible compared to some of the releases from Nvidia. Also, local inference lab.ai just popped up on my radar and they have Flare. They're working on their own engine for SM120 to overcome some of the problems with SM SM120. There are a couple of engineers that work at Nvidia on the VL LM side. It almost seems like
the VL LM stuff is a moonlighting project for them on SM120. I mean, SM120 just got it was like some kind of a fused matmul operator just recently and that really improves SM120 performance. And uh you know, some of the labs behind DeepSeek and GLM deep seek 4.1 flash and GLM 5.3. I've done a lot of Herculean things to be able to support the SM120 version of Blackwell. Um and contrast that to AMD. Like AMD has had CDNA and all of their dev work has been going into CDA. Relatively small amount of work has been going into RDNA. Although AMD has moved, you know, we did this demo with this gold system
here which is currently configured with our two R9 700s. And from the initial system, like when these first came out and I first did a review with four RDNA R9 700s in one system, where we are today at the middle of October at the end of October in 2026 is absolutely bonkers insane. Check out the forum thread and like tokens per second and token generation and quality. We're doing qualitative and quantitative analysis on what we have. And again, the AI tooling and like here's the hardware, here's the hardware documentation, here's what we have. AMD is much more open with what their hardware has on the RDNA side and on the CDNA side and what the functionality is. And so tooling and AI models are better able to pick that up and be
able to run with it. And so I think internally Nvidia has sort of realized that and they haven't really wanted to help themselves along so they have it's like, oh you know, SM120 is good but it's too good. It's like the DGX park aspects of it. It's like, oh we're worried about it being too good. So we won't put in a full size 2280 m.2 which is seems insane but why else would you not include a 2280 m.2? It makes no sense at all. Why would, you know, V100 V100s can run reasonable things thanks to the community. 6 7 8 9 year old GPU, the V100, can run relatively modern AI models 27 B dense. Nvidia doesn't really care about that. There's no money in
doing that but the community is able to use tooling and sort of bring that forward. The community has used AI to bring forward this and a bleeding edge kind of a way. And I get why AMD you be a little reluctant to come out and support that or even try to do the same approach as themselves, but then Nvidia has also experienced the same thing. Flare for SM120, not an official Nvidia project, and yet it's sort of crystallizing the shortcomings on SM120. And so if you look at Flare as evidence of the CUDA moat evaporation, then it changes the game a little bit, doesn't it? Because Nvidia created this situation. They didn't make the software stack universal enough. Uh it is universal in the sense that things
run, but it's not universal in the sense that things run as fast as they possibly can. And that was what AMD ran into as well. It's like you could get RDNA running with ROCm on R9 700, and even though they're only 600 GB per second memory bandwidth, the performance wasn't where you would expect it to be based on that. Until now. Uh you know, the Radiance engine on that side, RDNA strict Halo. Halogen. Halogen is another similar project. Totally different software stack. It's like, let's vibe code this, let's put this together. Um some of this would be uh in a official way you describe described as like hill climbing. It's
like, hello algorithm. Our hill that we're going to climb today is this is the hardware that we have. This is the brass tacks of what the performance should be as we iterate over these models. What's stopping us from getting there? And it can hill climb to get there and explain it in a reasonable way. It doesn't necessarily produce incomprehensible code if you don't want it to. And so in all of this, you know, the Nvidia CUDA moat is basically already gone. Now to be clear, that doesn't mean that there's not a bunch more moats that Nvidia has. Nvidia actually really does have a lot more moats, but they're they're playing it kind of close to their chest and they really hope nobody notices. Things like if you have a
thousand GPUs and some of them are idle, better scheduling across GPUs, better job management. Run AI was recently open-sourced, and that's so that they can help, you know, uh protect the moat. Uh hugging face, even if Nvidia doesn't have any protect the moat motivation in Hugging Face, they're going to have excellent data and telemetry from the Hugging Face acquisition as to what people are trying to do, what people are building, what's actually going on. They can keep their finger on the pulse of the community. And so, it doesn't matter that the CUDA mote is actually already gone if you're paying attention because randos should not be able to bang out math kernels that maximize the hardware utilization of GPUs like the R9 700. And
yet, they have. And that is transformational. You also see it in like vibe-coded software. And yeah, the vibe-coded software isn't as good as the state of the art, but think about word processing or photo manipulation software or spreadsheet software 10 or 15 years ago. If you could have that today and it was basically free, that's where we should be. It's just that the ecosystem, not enough programmers. But now, we can. And enough people will get together and enough people will pool their tokens together and their expertise, and we will actually get reasonable, completely free, open-source software that is at least as good as software 5 or 10 years ago. And I think they're probably going to close the gap
on 2 3 4 years. So, how much of a lead does Nvidia have with their software ecosystem? It's It's basically everything not at the point-blank range. It's everything at the macroscopic scale. And that's what AMD and certainly Intel are going to struggle with over the next three or four or five years because of the head start that Nvidia has in that scale of dealing with AI and projects and everything else. Not just like, "Oh, point-blank generate tokens and manage users." But project lifecycle and robotics and vision and all of the stuff that goes with that. And you know, self-driving cars and all this other kind of stuff. And in that
macroscopic scale, there's a lot more stuff that Nvidia has, I think, for their mote. But point-blank math kernels at a hardware level, it ain't it. Uh one other data point I'll give you. This is our GMK Tech X5. This is 192 GB on the Framework has a similar machine. There's a some stuff in the level one tech review pipeline. But check out the review on this if you get a chance. It's basically the Strix Halo platform. It's Gorgon Halo. But 192 GB of memory instead of 128. It's basically the same memory bandwidth as DGX Spark. It doesn't have a QSFP or 100 gigabit network interface. It has 80 gigabit Thunderbolt 5. That is not as
much as I want that to be a replacement for QSFP, it is insanely not. And I've done the work. I've done the kernel hackery. I've done a bunch of stuff on Thunderbolt 4 and Thunderbolt 5. And there's a lot of people that want you to believe that you can do insanely fast low latency networking over Thunderbolt. And you can do fast lower latency over Thunderbolt, but it is not anywhere near the same class as QSFP. And I do think that was short-sided on AMD's part to not bake in some kind of high-speed networking for this for the model. That said, this is 192 GB with the same memory bandwidth. And [snorts] the software gap has closed now. This piece of hardware was not even really designed
for AI. It just has absurdly fast memory and a lot of it locally. The reason it has absurdly fast memory is for gaming and like APU type use cases. For a gaming laptop, APUs in general are the future. It's It's too complicated and too annoying and too dumb to manage discrete GPUs and a portable low power device when we've got, you know, some of these new Panther Lake laptops that are like 7 W or 6 W when you're using it interactively and it's it's so insanely low power. Like you're just You're never going to get there with a dGPU. It's It's hard to do that even with an APU that has this much horsepower. And Intel basically has a high GPU horsepower APU that has the power problem solved basically in the other ver- the LPDDR version of Panther Lake.
So this from AMD with 192 GB has already basically leapfrogged Nvidia. I mean, at 16 cores that are x86 versus 20 arm cores, there's only a couple of benchmarks where the 20 arm cores pull ahead of the x86 cores, and almost all of the benchmarks where the x86 cores fall short of the arm cores have to do with lack of benchmark optimizations or the fact that there's like 72 ways to do things on x86, and asking a human being to figure that out is too onerous. But, guess what? The AI applies there. That doesn't have anything to do with CUDA or anything like that. It's just that there's been enough software breakthroughs on the Strix Halo side of things that Strix Halo in 2026, launching at the end of 2026, the software gap has more or less
been closed. And Halogen really gives you a run for the money on the DGX Spark platform. Also, you know, ROCm SP4, again, community efforts doing a lot of the heavy lifting here. 4-bit formats that run reasonably well on Strix Halo that was never designed for it, whereas DGX Spark really was designed with you know, NVFP4. At the time that I'm doing this video, there's kind of been a refresh of NVFP4 that is a sort of lower loss version of NVFP4. But, if you dig into that and you look at the get commits behind that project and the stuff that you see on GitHub, that is almost all community driven. Nvidia is not focusing on that kind of stuff because this is old news. This is 2-year-old core. They need to get the Vera Rubin hardware out the door for
enterprise customers and everybody else, and probably also a Vera Rubin version of Spark. I would say that a Vera Rubin version of Spark is overdue at this point. Remember, Jensen said, "Yearly cadence." "Oh, global macroeconomic conditions." Yearly cadence. So, again, another data point, the hour for CUDA is later than people imagine when you can just like here, I'm going to run this thing and do this with it. And if that's what you want to do, you can do it, and you can do it competently with a very small team. So, Nvidia discontinuing the 5090, they're probably going to do a 24-gig version of the 5080, and that's that's cool. I mean, that's something. But these GPUs are available now, and
they're 32 GB, and they do a reasonable job with AI now, where uh once they struggled a little bit before. Yeah, the memory bandwidth and the level of compute is not on the same level as a 5090, but if you're going to buy a GPU for AI, and you need 32 gigs of VRAM, these cost a fraction of the 5090, and it has always been a relatively small fraction, even when the 5090 was supposed to be two grand. And now they're like, you know, five or six, which seems absurd. These are on the order of like 16 to 1800 dollars. MSRP was lower. You know, be that as it may, 32 GB here with pretty awesome gaming performance as well, but it's not about gaming.
It's about being able to run local models. I want to level one if you want to chat or hang out or show me cool stuff, come to the forum, forum.level1techs.com. I'm signing out, and I'll see you there. Be sure to check out Radiance and Flare and Halogen, because they are uh they're they're three of the four horsemen of the CUDA apocalypse for Nvidia. If you really sit down and think about it and look at it, and it's like, yeah, I know one of those has to do with Nvidia hardware, but, you know, one of these things is not like the other, cuz you've also Well, maybe even one cat in there, cuz that's the kernel for the V1. I You know, you just got to come hang out at the forum and get right in on this. It's a lot of fun.
It's a lot of exciting. Anyway, I'm signing out, and I'll see you there.
