All kidding aside, a government entity, as you described, could lease space in the data centers of large FAANGs on US mainland and place a few racks in there to do distributed computing folding@home style.

A huge distributed HPC machine has some important advantages:
//1 can be designed and manufactured in a short time
//2 it can use parts from different suppliers
//3 each computing unit can have a different hardware architecture (CPUs, GPUs, ASICs, FPGAs, memory, storage etc.)
//4 can be upgraded in a continuous loop without shutting down the whole machine.
//5 can start small, with a small budget and grow organically, trying to adapt to new requirements as they arise.

Although I don’t have the insight to know why current training techniques need so much bandwidth and memory, it seems to me that it might be possible to train portions of a larger neutral network independently. In fact, I recall attending a thesis defense that touched on such things last year.

Why not?

And use Coca-Cooling systems to keep them from overheating.

100% agreed on the core argument: public investment in supercomputing (and the simulation capabilities that brings, for solving the world’s hardest problems) can and should be multiplied by orders of magnitude.

There is a very large (and literally “powerful”) community of scientists/engineers/… rooted in the tech startup world, that you may know of called HackerNews, who I think will love to read this highly original, provocative, and visionary article. I’ve posted this article there, if you upvote it, then it’s more likely that hundreds of others will see this, and a very interesting discussion will follow: https://news.ycombinator.com/item?id=38907195

By the way, you can thank Google’s Discover algorithm for having recommended this article to me, I guess with the indexing on my end of working closely now with the University of Florida’s HiPerGator supercomputer.

One thing to remember about simulating physical problems in 3-D is that each time we improve spatial resolution (uniformly) by a factor of 2, say in the x, y, and z directions, we need 2x2x2 = 8 times more nodes (simultaneous equations to solve), and possibly twice as many time steps as well (to maintain stability and accuracy constraints). So, improving resolution by a factor of 10x needs essentially 10,000 times the computational power of what is currently available, possibly 10 ZettaFlops/s. We’ll want to be sure to invest in that goal, especially now that the ExaFlop is “more common” and better understood, including leveraging MxP and/or specialized computational stencils (https://www.nextplatform.com/2022/04/25/oil-and-gas-industry-to-get-its-own-stencil-tensor-accelerator/).

I have long joked that every grill in every fast food restaurant should be powered by a supercomputer. These could be home heating units, too!

On the other hand, it looks like the nuclear test ban treaty, which motivated the construction of a number of the largest supercomputers, may be about over, as it’s not really a treaty if only one side abides by it.