Achieving PEAK Power-Efficient Performance with PEAK:AIO

Moderator: Welcome to the Tech Arena Data Center Insight series sponsored by Solidigm. Today we're discussing unlocking innovation with high performance, the cost-effective AI infrastructure. Joining us today are Mark Klarzynski, Chief Strategy Officer and Founder of Peak:AIO and Ace Stryker, Director of Market Development with Solidigm. Welcome to both of you. Thanks for joining us, Mark. I will hand it over to you to get us started.
 
Mark Klarzynski: Oh, well, thank you very much and thank you for having me. And Ace, it's good to work with you.

Ace Stryker: Likewise.
 
Mark: Hopefully you can see my screen. And although I'm sure we don't want to turn this into a giant PowerPoint, I think important to this discussion is to look at how we collaboratively have addressed some of the challenges of this new world of AI. And there's just a few slides that I'd like to try and show that help to try and navigate that collaboration and actually some of the challenges that we've dealt with. And the first one we normally start with is this sort of this nice head that tries to demonstrate the fact that although we've grown up with the world of enterprise and traditional IT, and that's how we consider the world to be, the world of artificial intelligence is completely different.

You know, the IT world often uses intelligence, whereas artificial intelligence creates intelligence. And although that sounds similar, I like to often show this as the evolving market. And if we jump back, you know, a handful of years, five, six years or so, NVIDIA will come along with this new technology that most of the world really hadn't seen and didn't know. So although the enterprise market has continued and will continue, there's this whole new evolved market that we consider was AI and GPU that in many ways, there's two completely different trains. And one of the challenges started and was became obvious maybe four or five years or so ago was that although NVIDIA and now peers had pioneered this absolutely amazing new market, the world in many ways had presumed that they could simply adopt and adapt the infrastructure that they'd currently make and it would fit and everything would work. But it doesn't and it didn't.

And we've all began to change on that. And as you know, and I think we've already said, I'm, you know, the founder PEAK:AIO and we’re a software defined storage company that focuses purely on AI and GPU work. And I've had the fortune of working with Solidigm for the past year or so. What we often think of as the AI world is like we see on the screen now large data centers, which they are. There's many large data centers, there's many large hypervisors and quite often AI starts there. And in some cases will live there. There's many a large company that will use cloud and large data centers like this. However, in reality, what we often see, and I think one of the areas worth spending a few moments on and again, not trying to kill you with PowerPoint is where AI often matures to and where it ends to. So if we take the real life example of the Zoological Institute of London, that Solidigm and PEAK:AIO did the webinar, sorry, the case study on you're looking at a rack that doesn't look overly impressive. Now, that's a few million dollars worth of equipment, but what's really important here is that the two NVIDIA DGX supercomputers in there are pretty much close to the amount of energy that that zoo can supply. And it's not a small zoo. So these things demand tremendous amounts of energy. They also demand and need tremendous amounts of performance from storage. They've got a lot of GPUs and I know it's a cliche, but you've got to feed those GPUs with data. Otherwise the outcomes are slow and they don't get where they want. And in this case, these guys are doing some tremendous stuff.

We've all seen the case study with the hedgehog watch, but you know, recently they repopulated into the real world a bird that's been extinct for 40 years. So they are pioneering conservation and what they do tomorrow, we don't know yet. But what we do know is this is the technology that's changing them. Sadly, that technology needs lots of data, needs loads of millions and millions of images of wildlife. And that, when you start thinking about that rack, is fully powered and mostly maxed with just two DGXs actually becomes a challenge. When these guys needed close to three petabytes, we had to really scratch our heads to work out how we A, could fit, make that work with power and density and space, and B, get the performance and the side effects of this. And I know this is in some ways almost funny, but it helps emphasize the challenge is if you look on the outside of that box, you'll see the cooling system. And right next to that cooling system, because it is a zoo, was the Chinese water deer who simply don't like cooling systems turning on and off. And so they actually had to move the water deer. Now I know that's unique to London Zoo, but it helps demonstrate that even a small rack needs tremendous density, requires tremendous performance and tremendous energy, and hasn't got a lot of space and needs a lot of cooling and it's problematic.

And so we've been fortunate enough really over the last few years to deal with Solidigm, get deeper into trying to solve how do we get A, that performance, but B, that level of density within that energy that we can, that the normal user can afford to do. Not the data center that's built from ground up, that's they've got it solved mostly, however, that's obviously going. But certainly the majority of AI customers and not the massage big hypervisors, and the new world that we want to se is to encourage some of these guys. And really being absolutely honest, one of the things that, you know, 30 years of storage has taught me is that much of the magic is actually smoking mirrors.

And that what we do is we actually develop features on top of what is really the magic which is developed by the guys like Solidigm. And so I don't know if this is appropriate, Ace, but I wouldn't mind handing over to you. We know that we have the challenge of how did we manage to get so much capacity in so little a space demanding so little energy and how was that possible when it wasn't possible five years ago and how what that means to this market. So if you're OK, I think it's time that we hear some of the magic.

Ace: Absolutely. Thank you, Mark. I appreciate that. And you could not have teed that up better for the content that I want to share with the group here today. 
 So it's a privilege to be here. Thank you to PEAK:AIO for bringing us in for this. And thank you to our attendees for your time. We want to share a couple of things here related to the role of data infrastructure in things like AI cluster performance, energy efficiency, which Mark talked about, all the things that sort of sum up to business outcomes and the reason we do this kind of work in the first place. So the first chart I want to share with you; every good AI presentation I've seen in the last year starts with a chart and they're all up and to the right.

That's we're all very excited about AI, right? You could be talking about a parameter count, or new models, or investment, or GPU horsepower. For our purposes, I want to focus on a couple of trends that are very pertinent to storage and ultimately to the efficiency of AI infrastructure. So the one on the left is data from MIT that talks about the growth of training data sets for AI over time. You can see this is an exponential curve here. Those who have been following AI for any amount of time probably know that 2017 was kind of our watershed moment. There was a big research paper published by folks at Google at that point, “attention is all you need,” that introduced the idea of the transformer. And since then that really kicked off this this curve right to where we see each year the median size of a training data set just multiplying and getting bigger and bigger and we expect that trend to continue. We see that as one of the primary contributors to model quality and the usefulness of the outputs that you get from models.

And so all that needs to be parked somewhere and that matters a lot to storage. And then on the right hand side, as Mark alluded to earlier, energy consumption is a bigger and bigger concern every day. These conferences I've gone to this year, customer conversations, that's really top of mind for a lot of folks. There are some very scary forecasts out there about energy consumption related to AI data centers. This is one view. This is US centric, but we see it playing out across the globe. What the chart on the right tells you in the numbers that are on the curve there, that's absolute power consumption by data centers in the US from 2023 to 2030, still growing from about 150 terawatt hours to 400 [terawatt hours] over the course of those seven years. The little blue ovals underneath indicate the proportion of overall power in the US that's being consumed by these data centers.

And that's a really striking number growing from 4% to 8% and beyond. And you can see the slope of that line is not getting any flatter. So we anticipate that this is going to continue to be top of mind for folks building AI clusters and doing model development and inference work into the future. Next slide, please, Mark. Thanks. So I mentioned scary forecast. Here's one for you. This is the CEO of ARM who mentioned in April of this year; his prediction was that by 2030 AI data centers would consume more power than the nation of India, which is the world's most populous country at this moment, which is just tough to fathom. Just earlier this month, there was another forecast or, or prediction from I believe the CEO of AWS, who said that within a couple generations, training large language models will consume as much energy as a large city.

And so clearly we need to do things here to get more efficient. We're approaching a scalability ceiling where the thing that's going to dictate the pace of AI innovation is not the good ideas the data scientists are having or the compute horsepower in the GPUs that NVIDIA and others are putting out. It is available power on the planet. And so that's really top of mind for us as well.

Mark: I think that's spot on it. I mean, A, that's quite a scary screen on itself, as you said. But without a doubt, even now we're beginning to see that the slowing factor is simply the amount of energy that they can use. And, you know, that's slowing down AI somewhat and it's still accelerating. But imagine how it would accelerate if we suddenly didn't have the power problem.

Ace: Yeah, that's exactly right. I think you're spot on, Mark. I think the temptation when you look at a prediction like this is to think, oh, that'll be a big problem in 2030 then. And the truth is it's a big problem today. We have folks like Mark Zuckerberg at Meta quoted recently saying we would already be building out bigger AI data centers if we could find the power. So this is it. This is a problem today. This is not a future issue for folks doing this kind of work. What is the storage contribution to the power situation? A lot of people focus on GPUs and rightly so. You know, these high-powered chips from NVIDIA and others, particularly as Blackwell rolls out are increasing power requirements significantly. We're entering a [new] world. We used to have four to six kilowatts of power feeding a rack and now we're potentially looking at single racks that can consume hundreds of kilowatts in the same amount of space. And so that is a significant part of the problem.

But storage, it turns out, plays a bigger role than you might think, as well. So this is a couple of different publicly available white papers. The top left is from Meta and Stanford University. And what they reported in an AI cluster that they had built out that was built on legacy storage for the storage subsystem, and by that I mean mostly mechanical spinning hard drives, was that about 1/3 of the power consumed by their whole infrastructure was attributable to storage. And similar results were reported in a more recent paper by Microsoft Azure and Carnegie Mellon University, where they looked at emissions from data centers, in particular, which is sort of a proxy for power consumption. And they found that 1/3 of emissions from data centers using legacy storage came from the storage subsystem as well.

So it's not as though storage is a very small part of the power consumption or another way to put that is it's not as though storage can't help significantly. Making optimal decisions for storage can have a major impact on overall power consumption in the data center. And we'll look at that in a little more depth in the in the following slides. But the conclusion from Microsoft and Carnegie Mellon is what you see quoted there on the right, which is at the most direct way to reduce those emissions is to use fewer, denser storage devices. Intuitively, that makes sense, right? All else being equal, if you can stuff more data into a single storage device, then you need less of them. That's less power, that's less rack space, that's less cooling requirements. Maybe not quite as disturbing to the Chinese water deer, if we can manage to do that more efficiently. And so that's a finding from that particular research paper that aligns well with Solidigm’s view of the world, and [it’s] where we see an opportunity to help to mitigate some of the power concerns related to AI development.

Mark: The interesting point on this is from my side is actually, this is really what spearheaded the formation of PEAK:AIO. As you know, I've sadly been in storage for 30 plus years and recognizing when AI was growing that all the focus and right to said, like you said, was on the GPU that was doing amazing things. But the but the presumption that we could just take what we knew and then align that and make it work resource in this. And it became clear some years ago to some that it needed not just a new type of storage, but a new way of thinking about storage. Sadly, that's not so easy for really large guys. You can't certainly just change your product line, but it's evolving and this slide makes that point really well and is exactly why PEAK:AIO was founded.

Ace: Right on, agreed. And this is the message in one sentence. This is what we're hearing from our customers, from their customers, from press and analysts who spend a lot of time looking in this space; is that in a world where grid power is a finite resource and in a world where the regulatory environment is evolving to respond to the power consumption from some of these data centers. There's a lot of places in the world where you can't build out any more data centers and certainly other places that have restrictions around power consumption, emissions, noise, heat, this kind of stuff. Every watt and square inch in the data center counts. And so we'll talk about the role that storage plays in that and how you can realize some meaningful savings in those areas by making smart storage choices.

I want to take a minute here and kind of set the stage, talk about what AI work is, relative to the data, and what role data plays in an AI workflow from start to finish. We can spend a lot of time on this slide, but I'll attempt to sum it up for you here in about 3 or 4 minutes. This is Solidigm's view. It's informed by research in our own lab, research from others, customer conversations. But obviously key to making smart storage choices is understanding the work. And so we see AI work really falling into discrete stages and associated with those are discrete storage requirements. And once you understand those, you can have a meaningful conversation about improving data infrastructure. So very briefly, from left to right, a typical, and I'll use typical here advisedly because the truth in AI is always that it depends.

But at a high level, if we're training something like a large language model, for example, we're ingesting a bunch of raw data at the beginning. And that can be potentially a very large data set. Look at something like the common crawl corpus, which is a publicly available set of websites scrapes that's in the 10s of petabytes, I think at this point. You've got to pull all that in and save it to disk to start, and then you'll clean it up. And that's the second stage, data prep or preprocessing, which involves normalizing, deduplicating, vectorizing the data into nice rows and columns so that it can be used to train your model. And in that stage, you can think of it like a funnel, we're starting with a large amount of data and we're ending up with proportionally less after we do the clean-up. So it's more reading, [with] a little bit of writing at the end. And then you got to think about those things from a performance point of view. But you may end up with a training data set that's a 10th the size of the raw data that you ingested upfront. You then feed that into model development in the big orange box in the middle there. And training is, of course, the key activity in model development.

That's where you're exposing your data, your training tokens to the nascent model in random order. And that's by design to avoid creating any kind of over-indexing affecting your weights and biases of your model. So we find that's predominantly almost all a random read activity in terms of the drive, what the drives do in the IO characteristic, and what you need to think about from a performance point of view. And then during that there's checkpointing going on as well in most cases, which is of course important. If you've got a model that takes, in some cases, weeks or months to run a training output, you don't want to lose all that because you have a hardware failure or software issue somewhere halfway down the line. And so you write checkpoints to disk of your in flight state of your model. Those are sequential rights generally. And the important thing about those is while you're writing those checkpoints, your GPUs are idle.

They're not doing much. They're waiting. And so being able to write quickly, do that efficiently, is a key part of kind of maximizing your overall GPU utilization and doing things efficiently and quickly. You get through the model development stage. We won't necessarily speak to every box on the page, but there are some other steps involved in that. And then finally you're ready to deploy your model once it's validated and you're running inference and potentially you're connecting that model to other external data sources that it that it wasn't trained on. That's called RAG or retrieval augmented generation. And in those cases we see mostly random read activity as well. And that'll depend if you have a generative model that deals with richer media types like images and video, that the writes will be proportionally higher. But generally that's a read intensive activity. And then finally, at the very end, you have the archive stage where you're taking all the inputs to the deployed model, all the outputs, any other enhanced context that it grabs from your RAG sources and you're archiving all that stuff. And that may be for training the model and improving it later. It may be for regulatory and compliance reasons. But that tends to be where the data magnitude blows up again at the end. And efficient high-density storage matters a lot in terms of kind of optimizing for your TCO and your energy consumption.

So that's our view of the data pipeline. Data is everywhere, as the title suggests here. And understanding that and understanding where and how it's being used throughout is the first step in making smart decisions and optimizing to try to either improve the quality of your outputs, the times [of] your deliverables, or the cost and energy expenditure to get there.

Mark: I have to say that's possibly, in the last few years, one of the best laid out data flows for AI. I've seen just about every diagram and every arrow. and that is a good representation of probably the majority of them. And interestingly, one that we've [seen], I'm not sure if that's it would be more difficult to put on the picture, but one that also helps emphasize how much data is becoming even bigger is because if we take just an obvious example of medical AI, medical AI have some clearly very obvious use cases. And now that those that have been training those models have begun to tick the boxes and those use cases are becoming more obvious. What we're seeing now is the need for them, during this continuous training process, they need to know that one day when they do an inference and they get the MRI scan wrong, for instance, they can look back at the history of that model. And so we're seeing another level of archive that would be linked more to the orange boxes in the middle where they're saying, OK, we're doing a checkpoint. We've done, one training run, now we need to archive that model, but make sure it links to the model data. So that we can always backtrack historically where that bad decision crept in. So it's interesting, it's a great diagram of the data using and only growing, but interesting the way that it as you can see it's predominantly sequential.

Ace: Yeah, it is through a lot of the pipeline and I appreciate your comments, Mark, you’re spot on. There's many flavors of this, right? But certainly where we've seen what I'll call, I suppose, mission critical application. By that I mean like a health outcomes depend on it or security outcomes depend on it. We've seen the need to feed that that archive data back through much more regularly, right? And continue to [retrain it.] If an airport security system misidentifies somebody, [if] an AI model misidentifies somebody, that's a huge problem. And there's obviously a vested interest on the part of the folks deploying those models to get that right and to correct those issues as quickly as they can. And so, certainly, that would be an example where we see the archive data being reactivated and used much more likely to improve the future inferencing.

Mark: And when you consider we're still in the infancy. So all that, all that data that's being found at the inference that's going to work its way back right at the beginning somewhere. So this is just getting bigger and bigger, as you say.

Ace: Absolutely. OK. So we talked about storage density and what that can mean in terms of power and rack space. I want to give you an example. And this is on a larger scale, but I find it's useful to illustrate the principle here. And then you'd expect to see it play out across deployments of, really, any size. But if you were to take an example AI cluster with A50 MW power budget, which is what you see on the left, you'd spend that power in in roughly this way.

We're making an assumption here that there's a ratio: For every rack of GPU compute that you have, which we're defining as four of those DGX servers that Mark showed you earlier, you need 16 petabytes of data. And that's a reasonable assumption based on inputs we've had from customers. We see deployments with a petabyte or less of data per compute rack. We see deployments with 30 petabytes or more per compute rack. So we feel like this is a reasonable estimate in this scenario. You'd spend that power on about 650 of those compute racks. And to achieve the amount of storage that you need according to that ratio, you'd have a heck of a lot of storage racks.

You'd have the same number of TLC storage racks that would act as sort of a caching layer. And then you'd have the bulk of your data on those mechanical spinning drives [HDDs] that I mentioned earlier about 2000 racks is what it would take to achieve the amount of data that you need in this scenario. Now if you were to take that legacy storage subsystem and replace all that stuff with the high-density NAND of the kind that Solidigm sells and that we use, for example, in that London Zoo example that that Mark showed you earlier, you can accomplish a couple of things. One would be you could reduce your rack space and your overall power consumption by quite a bit. So the graphic you're seeing on the top right is the same compute budget, the same amount of compute horsepower and the same amount of storage on the right. in QLC. QLC is our high density, like the [Solidigm] D5-5336. So what you're looking at here, same capacity, same compute, but you're doing it in obviously much, much less rack space and you're doing it with only about 30-40 megawatts of power.

That's one potential outcome of moving to a modern high density storage subsystem. The other one would be, OK, you have 50 megawatts of power, you can stay within that same envelope and modern high-density storage will buy you a heck of a lot more of that of that energy budget for compute. And so here we're still spending in the bottom right 50 megawatts, the same as what we started with on the left. But now we've scaled and we have 300 additional racks of compute, which obviously is going to drive training much faster. It's going to get the work done and get you to market much faster. And again, you're not spending any extra power to do that. And in fact, you're saving a lot of rack space by moving to the high-density storage there as well.

So we typically see customers who are interested in making that transition from legacy storage to modern high-density stuff come in with one of these two goals, right? Either they want to reduce overall power consumption or they want to scale the proportion of their power budget that's available for the compute components. And this is just one example of how the smart storage choice can enable both of those and result in significant differences in terms of your outcomes.

Mark: I think also, Ace, that's actually a really key point because I'm, I'm luckily enough to work with a whole range of customers. And ultimately when you talk to them about what they want, they want GPUs. They understand GPUs. They know they need the data, but the data is an input and the GPUs will do the work and create the magic for them. So the more GPUs you can buy, or accessories to those GPUs [like] orchestration software, etcetera, the better. What they really don't want to do is spend any more than they need to in terms of energy, space, or budget on storage because that's just going to gobble up what they see as their real return on investment [on] GPUs. So this makes a tremendous difference. And in fairness, this has been a turning point in the last couple of years with Solidigm in AI, in my view,

Ace: Absolutely. Okay, I think I've got one more slide Mark, and then I will toss it back to you. I did want to mention a product launch that occurred this month that we're very, very excited about. So the example I showed you on the last slide was moving from, I think, 24 or 30 terabyte hard drives over to our 60TB SSDs. But what we've announced this month is we've just doubled capacity again and that's not even accounted for in the space and power savings that we just showed you. This is a brand new product. This will be shipping in volume in Q1. But we've now arrived at a point where we're able, thanks to some technical innovations that have been a long time coming, to store over 100TB in a single storage device about the size of a deck of cards. So we expect this to be a compelling part of the solution that goes into PEAK:AIO when we look at everybody from small edge deployments to big data centers doing this work. Just to give you a frame of reference, this is about 5000 times the entire text contents of Wikipedia [that] you could put on this drive. Or another one that I looked up that that was kind of fun was you can take every movie that was theatrically released in the 1990s, which is a favorite decade for film of mine, and get 4K high-quality resolution copies and store them on this drive two-and-a-half times over. So it's a lot of space and we're excited to see how customers take this. And it may be for wildlife conservation like we just talked about, it may be for optimizing manufacturing workflows, it may be for improving patient and provider outcomes in healthcare facilities. But we, certainly have seen the demand for higher density and more density per storage device, and so we're excited for folks to get their hands on this and to hear about the difference that this one can make in terms of AI deployments out there.

Mark: And it will. If you're okay, I'll jump onto the next slide, which is just a real quick introduction into how we put these things together and as I said earlier on, actually, most {a lot) of the magic is done with you guys, [Solidigm]. And as you just explained, you've now got 122TB drives in an off the shelf 24 bay box. That's over three petabytes, maybe, in a single small 1200 watt box. Our job is, then, simply to take those individual units and put them in obviously a protection scheme of some sort, RAID N+2, but maintain the amazing power that they already do so that we generate, you know, close to 200 gigs per second on the inside.

Then [we] get that out to the outside world in a way that this modern world of AI, like [a] solution GPU direct type way, RDMA way. So on one side we have the innovation of Solidigm that gives us the capacity and the performance and of course the density. The other side we've got the [NVIDIA] Mellanox stat that are giving equal performance on the outside. What we're seeing now in a 2U box is probably what we would see in 12U plus only a few years ago. And now you take that across a rack, and like you said before, you can have hundreds of racks of storage, but now you've suddenly got that down to a handful. It's a tremendous difference. And that in itself means that, well let's just take the small example of the London Zoo. No longer is it confined to 20 million pictures of leopards, it's now got the ability to have up 100 million and where that leads, who knows?

And as you get larger and larger, then clearly that that makes a difference. Now, just to get techie for a moment, one of the one of the key challenges that we wanted to overcome was to be able to say, “Hey, you put the work into making the fast drives, then at least we could put the work in to making those fast drives a usable layer without getting too complex.” And this is just a comparison against good old MD, which has been around for many years. And like many things in the Linux stack has to deal with a lot of legacy that includes hard drives. And that's how the stacks wrote and it's a great stack, but obviously that meant that we were producing product that really wasn't going to give the power. It had the density, but not the power. And so we spent a lot of time on that PEAK:PROTECT to ensure that each one of your drives would scale and it would scale as if it was a RAID0 even though we're using M + 2 RAID6. And that's pretty much it from our side.

Moderator: Looks like we have a few questions. One question that came in here. “How does Solidigm deal with the write endurance of QLC?”

Ace: I'm happy to take that one. It's a great question and I'd love to take you on a just a very short journey down memory lane if I could. So our first QLC product, and by our I mean you know our sort of predecessor entity before the birth of Solidigm three years ago, we were part of Intel as [in] Intel's storage and memory group. And so at this time back in 2018, Intel launched the first ever QLC SSD. And there were significant concerns from the market around write endurance of that product. There was a very real question of “is this trade off worth it?” Getting one more bit per cell of storage inside an SSD and losing write endurance as a trade off. That was a fair question. And what that did was laid a foundation where we're now six years down the road and the company is on its fourth generation of QLC NAND. And that's given us an opportunity to refine the technology. It's now mature. And a lot of those concerns that existed in the first generation, that I think created an impression of QLC as something where you've got to worry about write endurance, those just don't exist anymore. If you were to look at our spec sheet for any of our high-density QLC products, it's quite high endurance at this point. And I'll give you one example. We just launched the 122TB SSD which I mentioned. That's the first SSD that I'm aware of that has effectively unlimited random write endurance. And by that I mean you could take that thing and over the course of the five year warranty, if you were to write to that drive at full speed 24/7 for five years straight, you still would not reach the endurance limit. And so this is an area where there's been a lot of improvements made, and the write endurance of this media is quite robust at this point.

Mike: It's interesting because I actually think, Ace, this is more of a perception thing from a previous generation. And in fact, actually I think on a backup slide, I may, yes, this wasn't something we were going to present, but we actually also do an amazing job at writing endurance. So we didn't necessarily write this part of the code for that. What we wrote for was because it was useful for us to structure our writes in a different way. 38:37
 And so we actually have changed a lot of the algorithms where some time ago this would be to buffer the writes to help the endurance. Now we're using that to actually improve write performance. So actually you guys have done a tremendous amount of work on there, but we just don't see it being the problem that a lot of the market still perceives it to be from the old days.

Ace: Good question, Thank you.

Moderator: Looks like we've got one more here. “Do the drives have different power states or are they on, off?” So Ace, if you want to probably address that and Mark, you certainly weigh in.

Ace: Yeah, sure. So if you look at our spec sheet, the two numbers it'll jump out in terms of power are active and idle. And for a lot of, for example, our high-density stuff, the active power rating is around 24 [to] 25 watts and the idle's around 5 watts. So folks are aware of that distinction and they may even use those numbers to arrive at conclusions that could be leading them astray. You know, you may also look at, for example, the hard drive spec sheet and see, “Oh well, you know, the active power rating on that hard drive is much lower,” which is true, but it does not account for what we call duty cycle or how often is the drive actually active versus idle. The much higher performance of an SSD means that the duty cycle is much lower, and it's actually spending a lot more of that time being idle than a hard drive being used for the same purpose. But even within the sort of active and idle states there are there's more granularity to be had. And this is a relatively new development and customers are sort of exploring this, kicking the tires on this. But there is an ability, in some of these newer drives, to set power ratings that are between active and idle; to set a 20 Watt or a 15 Watt power rating if you're willing to accept kind of the commensurate implications in terms of performance and so forth. You say, “OK, I don't need the full performance at the 25 watts gives me, 15 [watts] is enough. And that 10 Watt power savings is a big deal. And that aligns with my goals better.” The newer drives do give you that ability to make those calls as well. And to sort of get really specific with how you tune those for the power performance balance.

Mark: That's actually really topical. Because the one thing that we've probably all learned in AI is that the workloads are never—even if you took a medical app and a medical app—the workload is never the same. It varies greatly. However, what we're beginning to recognize is that we can learn trends. We can start to learn that that particular model has this particular trend. And so actually in that code I was showing a few moments ago, we're beginning to look at exploring the new Solidigm power states so that we can take advantage of knowing when those trends are. And given we've got potentially more performance than we can put onto the network anyway, we're  able to throttle down to bring the power down and use some of the maybe the SLCs very small SLC tool to take the right hits. And then so those power states are actually probably going to have a bigger impact than we probably think at the moment because they’re quite new and we're actively thinking that we're going to redevelop some of the algorithms with you guys purposely to enable us to take advantage of those.

Ace: Yeah, that's a super cool prospect, Mark. And you guys are certainly a leader in that space. You know, there's an element of optimizing your storage by making the right choices on a product basis. You know, do I need SLC or TLC or QLC? What are my read and write performance requirements and how much density? But then there's a whole different set of questions you can start to ask when you when you drill into granular power states that that we think will offer some compelling solutions for certain use cases.

Mark: And, and the reality is most of the users and the people that are looking at these, they don't really need nor want to know those answers of questions. So that's down to us, right, to sort of make that a simple box where that they can feel assured this would do the job and it lets them spend the money on the GPUs. But it would deal with the power problems, it would deal with the density problems and it would deal with the space problems. And that's really our goal. And it's been an absolute pleasure to work alongside Solidigm, and in fact, I don't think it would have been possible without working alongside for the Solidigm.

Ace: Well, I appreciate that. And the feeling is mutual. We've really enjoyed seeing what what's possible when our companies can get together and provide solutions to the market.

Mike: Well, let's keep doing it. And Ace, thank you so much for your time and your expertise. Megan, thank you.

Moderator: Absolutely. Thanks to both of you very much. We appreciate your time and your insight on this topic. Thanks to our attendees.