The cost/speed to service advantages seem hard to beat for custom SoC vs. commodity when you are in an expansion phase.

Once that peters out, you’ll want to commoditize again, which is what OCI is all about.

Doing the balance will be tough and making sure you really fully depreciate all that stuff before it becomes obsolete, even more so.

Can’t say that I wish M$ good luck, because I want them miles away from any of my data: Copilot, just say no-op!

As with most things IT, there is no single correct choice, because *it depends*.

When you have a green field out there to fill, why go for density if it costs more?

In other places where latencies are really critical, specialized IT service providers will optimize density in a manner that just wouldn’t be competitive in the green field.

As usual, you’ll see diverse architecture, oddly enough enabled by the huge scale of the workloads.

When it comes to connectors and standard placement: unfortunately physics play a role there and with everything copper, trace lengths become critical. There is a lot of cables these days on Gen5 PCIe servers, because running equal length traces on planar mainboards becomes either impossible or too expensive as you add layers. So going for cables rather than slots and connectors may just be dictated by a combination of physics and economy, even if these high precision cables are neither cheap to make nor to deploy.

How that will work out with optical interconnects remains to be seen, but again cables may win, unless the assembly cost become to excessive… where something more robot friendly may emerge.

I guess the main drivers will be a) cheapest assembly cost for the performance, b) near zero maintenance cost, c) disassembly is “for the next guy”

AI in the cloud will always be more powerful than what is possible on a client device. On the other hand, as client devices get more AI hardware and AI algorithms improve, client devices might become good enough for most purposes. Will the AI assistants most people use will stay in the cloud or will they eventually run on client devices? The computer industry has gone through several transitions between centralization and decentralization. I would like to know if AI in the cloud is going to remain the dominant way of implementing AI assistants or is it just a temporary thing (less than 5 years) until client devices become more capable and AI algorithms improve.

Datacenters already do this. HP blades, SAN arrays, ect.

The only people who implement this way are end user customers paying others to do the work for them.

It is peculiar, isn’t it? I have been to Quincy, so cheap space is not a problem. Microsoft could buy 10,000 acres easily.

I think these companies think of racks and rows like we think of servers. Once they roll them in and wired them up, they really don’t want to touch them again unless they have to. But maybe they are leaving space for future capacity. In the past, this was done at the datacenter room level within the datacenter within the region. But maybe there is a reason why you want to fill all the rooms halfway now because of AI loading. But as you say, these are just Arm server CPUs, nothing crazy hot.

Thanks for the catch, and the morale boost. Appreciate you.

The next big change in DC architecture will be when someone defines a way to get rid of all the fibre and other cables by having standard rack designs with plugs in the back for the network/power/ILO support all in one connector in a standard placement.