Anthropic Joins AI Energy Alliance, Apple Launching Servers

0:00My name is Shannon Maldonado, I'm the founder of Yaoie, a gift shop from the lens of artists and handmade objects. I chose Shopify because when I was testing other platforms it was definitely one of the most user-friendly. It was important to me to think about where we would be in the future. All of the tools for reading your sales like planning inventory, so just right there on your dashboard. For anyone starting a small business, the biggest thing I can tell you is it doesn't have to be perfect. Shopify can help you build upon it. Start your free trial on Shopify.com. Welcome to the podcast. Today I want to talk about a bunch of breakthroughs, AI news and build-outs when it comes to AI hardware. I think we've had a lot of talk lately about should we slow down AI, should we not slow down AI, pacing the frontier with Dario Amadeh. But the one thing that I think nobody is debating is the fact that this AI is going to need a lot of power generation, a lot of data centers, a lot of chips, and a lot of memory. And all of those things aren't something you could say let's pace the development of because a data center might take 10 years to get a proper

1:03electrical hookup to it. These things, this infrastructure takes a very long time to get built out, the manufacturing facilities take a very long time, the planning that goes into it. So today I want to talk about some of the areas that we're accelerating on and some of the big advancements we're seeing in the hardware side. The first one I want to talk about is that Apple is going to is basically bringing back their server market, they're kind of back into the server market, they're going to do this in 2029 with their M8 Ultra and they're going to use Nvidia links. This is a market that they left and I'll talk a little bit about that. I also want to talk about the fact that SK HiNex out of South Korea is in talks with Intel to make memory chips in the United States. If you have been falling what's going on with AI, you know that memory is one of the massive biggest bottlenecks and there are only three companies that are currently doing this specific type of memory. At the same time we have Nvidia Google Emerald AI which all launching something called the Alliance for Grid Flexible AI data centers. It's the AI Energy Management Alliance and there's actually 18 members. So Anthropic

2:07is part of it, National Grid is part of it, AES and RG. All of them are coming together to create an interconnected grid. There's some interesting statistics that go along with all of that. Now if you're the kind of person that hates paying for subscriptions to all of those companies I just mentioned like Google and Anthropic and Chat Gbt and everyone else, I'd love for you to check out my startup AIbox.ai. I give you access to over 90 of the top AI models. So you get Anthropic, you get Gemini, you get Claude, you get Grock, you get Deepseek, you get Proplex. So you get all of the top models, Image, Audio, Video, every top model and modality that you could want. There's 90 of them. It is $8.99 to get started and you use it by the token. So if you have a massive project there's higher subscription tiers but if you just want to test stuff out you can chat with all of these different models in the same chat thread and you can also connect them inside. So if you use Claude a lot and you want Claude to generate images you use our MCP connector and inside of Claude images can be generated by Chat Gbt or Video by Google V03 or whatever model you're working with.

3:09It is something that saves me so much time and I hope it's a super useful product. It is AIbox.ai. I'll leave a link in the description. Let's start with what was going on with Apple first. So this is 15 years after they killed X server. If you'll remember Apple is looking to get back into servers and how they'll essentially put this together is they will have four MA Ultra chips and Nvidia's NV link fusion and all of this is how they're going to be building out their new server. Basically a new server offering. I think what's interesting here is we've seen a lot of Apple is basically one of the people that one of the companies that people have criticized heavily. They're like, oh my gosh, Apple's missing the AI boom. They're so slow. They're new series. The Apple intelligence. Everything was so slow. And yet if you look at the stock price of Apple it has not declined over this. They have actually made some incredible progress and it doesn't seem like the the stock is declining. One of the big reasons is because Apple, I don't know if it was accidentally or however you want to say it, but the hardware of Apple has been greatly used and I think one of

4:10the big examples a lot of people will point to is when OpenClock came out basically everyone was running out to go buy a whole stack of Mac minis that they were running OpenClock agents on. Now I think that Mac mini craze was kind of short lived. It was kind of a quick moment. Obviously the Mac mini is just a great, a great product. And so I think a lot of people would just use it as a good computer. But I think they Apple did see a very real, very large explosion in sales because of that. And they've realized, hey look, I mean, you're basically a little server on someone's desk. Why don't we bake actual servers for companies that are paying crazy amounts of money and this is something that's growing up as scaling up as companies are looking to have their own virtual machines. They want their own agents in the cloud. I think Apple caught the vision. They got probably a little taste of that. And so this is kind of the direction that they're they're moving towards again. This isn't going to happen till 2029. We know this because they said that this is going to be the they're going to have four M8 ultra chips. They just announced the M6 generation of chips in August. So the

5:14M8 line is at least two silicone cycles out. And so we know that the you know announcing it on the M8, it's going to probably be 2029. These servers are also going to probably incorporate Nvidia's NV link fusion. This is the same technology that Nvidia uses to stitch multiple chips into one logical unit. So you know, they have their chips, but they're going to probably using Nvidia to stitch them all together. This is interesting because Apple is already kind of they had a pretty rocky relationship with Nvidia for a while, but they've turned back to Nvidia and turned back their chips to help power the rebound series servers. So their relationships have been on the rocks for a while. It looks like they are defying that and getting back into it. And this is going to be a big part of what powers their new server offering. We have some big news out of South Korea. SK high necks is in talks with Intel to make memory chips in the United States. The South Korean memory giant would use Intel's US fabs for the first time. This is probably a big shift in where HBM is going for AI accelerators and where they're going to get built. The reason why this is

6:17an interesting to me is because there's only three major companies that are building this specific type of memory. And by the way, SK high necks is what's making the memory that powers a lot of Nvidia's tech. So this is a major company. We have them. We have Samsung and then we have one other company out of Idaho that is building these specific memory chips. And so this is really kind of a global bottleneck. But I think they're seeing that there's all, you know, most of the demand is coming inside of the United States. Obviously China is going to have a lot of demand, but there's probably could be sanctions or I'm not sure what South Korea's situation is there. I think they see the demand coming out of the United States and they want to move closer to it. They're also making this, the interesting part is the relationship they have with Intel right now, Intel right now, and Intel Foundry has basically been searching for an anchor customer to justify a massive US manufacturing buildout. They've been talking about this for a long time. We know that Intel stock was sliding and kind of, I don't want to say cratering, but Intel has struggled for a long time. It really should be one of the big companies we're talking about all the time. And we have the

7:18old CEO lead. We brought in new CEO and Intel's been on kind of a generational run for the last little bit. They made a lot of partnerships with the United States government. And so it feels like Intel is kind of getting its mojo back and they're making a lot of moves. They recently announced a massive build out US manufacturing build out. And with that, they need customers. And so it looks like SK high nicks might be might be one of the big customers that's going to start building in their US fabrication facilities. This is all according to some reports by routers. This particular deal would be the first time SK high nicks produces memory in the United States. Traditionally, it's all been done in South Korea. SK high nicks is also basically the dominant supplier of high bandwidth memory to Nvidia's AI accelerators. So it's, you know, just kind of speeding up the supply chain on a lot of that. We also have Nvidia Google and Emerald AI, which are launching a alliance for what they're calling grid flexible AI data centers. This is called the AI energy management alliance. There's about eight to different companies that are inside of this. What's interesting with this, it would

8:20just launch to this month and founders Emerald AI, Google and Nvidia are all estimating that flexible flexible data centers could unlock 100 gigawatts in the existing US grid. So what they're saying is that essentially the United States, the US power grid is only running at about 50% utilization on average. And I mean, you can imagine there's peak hours. And sometimes that happens in California, Texas, you hear about like blackouts or brownouts or, you know, and there's a big storm and there parts of the grid go down. Like there's all sorts of issues. But generally speaking on the average day, we're sitting at about 50% utilization is probably like peak hours in the evening time when everyone gets home and is charging their electric cars or, and you know, in the middle of the day when people are at work, it's lower in the middle of the night. It's off, right? So there's ups and downs in this, but it's on average about 50% utilization. Their thesis on this, there's actually something called Brattle Group. And they're estimating that each 10% gain in grid utilization is going to lower electrical rates by about 3.4%. So they're saying, look, we have the capacity to generate

9:24double the electricity. If we went from 50% to 60%, everyone's electrical costs would drop 3.4% because there'd be more abundance and electricity. And so it would just, I mean, I think it's a lot of it gets down to like peak usage times. You're not going to be paying peak prices. And so if there's just more electricity, there's more surplus, the peak, I guess would be smaller. And so you'd save money. So that's their estimation. In video, digital reality and Emerald AI are going to switch on and they're about to switch on about 100 megawatt power flexible AI factory in Virginia later this year. So this is obviously something they're looking at getting power for these data centers and for a lot of these projects is an absolute nightmare. It's very tricky. Apparently US data centers currently wait up to 10 years or more for a grid connection under some of the existing interconnection processes. And this is why you had Elon Musk when he stood up his Memphis facility. He literally went and just purchased every portable diesel generator he could get and shipped it over to the facility. And instead

10:26plugging into the grid just powered the whole thing off of diesel generators, you know, much to the suit, sugar and a whole bunch of lawsuits about like pollution and that kind of stuff. But this is how he got the data center up and spinning. And lest you think it's kind of random. I mean, that data center in particular, he's building much bigger projects, but that particular data center is making a billion dollars a month, about 900 million, a billion dollars a month roughly renting their extra space, their extra compute to anthropics. So anthropics paying them a billion dollars a month. The only reason they can make that much money though is because they kind of threw this together in a, you know, in a sort of not traditional way. And so a lot of these data centers aren't looking to go and rent every portable generator. I think SpaceX or, I guess, at SpaceX now, it was XAI at the time when they built this thing. I'm sure they've kind of built in some more sustainable. I'm hoping they've built in some some more sustainable power generation methods. But at the end of the day, these things are very hard, very tricky. So the goal of this new alliance is to figure out how to plug into the grid and just increase the grid's utilization. The costs should come down for

11:30everyone, but they are also going to be able to plug in a lot of these big data center projects. It's a problem we're going to have to wrestle with. China is absolutely out accelerating us to an insane degree when it comes to power generation. And if we want to power all of the projects that are currently under construction, we need to solve this. So this appears to be the solution that is being looked at by many people. Guys, thank you so much for tuning into the podcast today. If you enjoyed this episode, make sure to go check out aibox.ai. Leave a rating review on the show wherever you get your podcasts. It helps the show out a ton if you wouldn't mind and make sure to go check out aibox.ai. If you want to get all of the different AI models plugged into Chad Chikki or Claude or wherever you do your work, cursor, etc. You can get 90 different AI models plugged into that so you can do audio, video, image, generation all inside of the same all inside of the same workspace. Thanks so much for tuning into the podcast and I'll catch you guys all in the next episode.