AMD 64-Core Rome Deployment: HLRS ‘Hawk’ at 2.35 GHz
by Ian Cutress on November 13, 2018 6:30 PM ESTLast week AMD pre-announced its next-generation EPYC processors, code-named Rome. These new processors are set to be officially announced next year, but will feature up to 64 cores using eight 7nm chiplets surrounding a central 14nm IO die. This week is the annual Supercomputing conference, focusing on high-performance computing, where all the big OEMs with server offerings as well as supercomputing centers get to promote their latest and greatest. One of them is the High Performance Computing Center based in Stuttgart, Germany, who disclosed some information about their new ‘Hawk’ system.
Supercomputers are measured in many metrics: processor type, cores, power, peak throughput, peak performance, interconnect, memory, and many others. The first supercomputer disclosed to be using AMD’s upcoming EPYC Rome processors is ‘Hawk’, based at HLRS, and set to be installed in 2019. The base systems will be developed by HPE, listed as HPE’s ‘Badger’ portfolio.
Details for Hawk were given by the HLRS Director, Prof. Michael Resch, at a talk at the AMD booth here at Supercomputing. The title of the talk was given as ‘AMD in a Large Scale Computing System for Science and Industry’, with a focus on HLRS as a local driving force for compute as well as the work it does. The requirements for the new Hawk system were listed as high performance, professional support/collaboration, lowest possible TCO, and high level of sustainability.
The key slide for us in the talk is this one.
The slide has EPYC misspelled in it, as it is misspelled in a number of places in the slide deck, along with other mistakes – the speaker’s first language is not English.
The slide describes Hawk as a 640,000 core system using 64-core Rome CPUs. That’s 10000 CPUs, listed at a throughput of 24.06 PetaFLOPs. That gives 2.4 TeraFLOPS per CPU, which at 16 SP flops/cycle, works out about correct for a 64 core CPU for the 2.35 GHz frequency mentioned in the slide. This would appear to be a base frequency, although AMD’s Naples processors do offer a ‘constant frequency’ mode to ensure consistent performance. Main memory was listed as 665 TB, with 26 PB of disk space. It was not disclosed the breakdown of how the disk is split between NAND and HDDs.
Unfortunately Prof. Resch was not available for additional questions.
29 Comments
View All Comments
philehidiot - Tuesday, November 13, 2018 - link
The bases of the bases.Evidently, all your base are belong to us.
kartikguha - Tuesday, November 13, 2018 - link
Fl Studio reference?Rocket321 - Tuesday, November 13, 2018 - link
All your base are belong to us.nathanddrews - Wednesday, November 14, 2018 - link
It's a reference to an Sega Genesis/MD video game - Zero Wing - that featured poorly translated engrish. I haven't heard it in a while, people would say it all the time in games like UT99, CS, etc.tommihommi1 - Tuesday, November 13, 2018 - link
2.4TFLOPS per CPU would exactly fit the given clock speed and AMDs AVX implementation. No GPUs needed to boost that number.Only 64 GiB per CPU is interesting, looks like the minimal setup with one 8GiB DIMM for each memory channel was enough for them.
evanh - Wednesday, November 14, 2018 - link
QUOTE: "looks like the minimal setup with one 8GiB DIMM"Possibly. Or they've not filled all channels. I'd guess it's the latter due to very high DRAM prices for the last year or two. Time for DRAM prices to return to normal!
MrSpadge - Wednesday, November 14, 2018 - link
This beast is going to need all the bandwidth it can get, especially for HPC. Not filling all channels is not an option. I guess they'd rather upgrade memory later, if needed.evanh - Wednesday, November 14, 2018 - link
It's not uncommon for HPC to deploy incomplete. Throwing away a huge load of DRAM a year later isn't smart thinking unless they got it for a massive discount.MrSpadge - Wednesday, November 14, 2018 - link
Sure, throwing away would be stupid. I was thinking about 2 DIMMS per channel for the extension. That would drop the frequency a bit, but that's far less harmful than unpopulated channels.evanh - Wednesday, November 14, 2018 - link
It's nerf'd for the moment anyway. It'll run good enough on caching with that little in total.