New Advances
New Cerebras Wafer-Scale ‘Andromeda’ Supercomputer Has 13.5 Million Cores By Jessica Hall on November 21, 2022
Cerebras unveiled its new AI supercomputer Andromeda at SC22. With 13.5 million cores across 16 Cerebras CS-2 systems, Andromeda boasts an exaflop of AI compute and 120 petaflops of dense compute. Its computing workhorse is Cerebras’ wafer-scale, manycore processor, WSE-2.
Each WSE-2 wafer has three physical planes, which handle arithmetic, memory, and communications. By itself, the memory plane’s 40GB of onboard SRAM can hold an entire BERTLARGE. But the arithmetic plane also has some 850,000 independent cores and 3.4 million FPUs. Those cores have a collective 20 PB/s or so of internal bandwidth, across the communication plane’s cartesian mesh.
Each of Andromeda’s wafer-scale processors is the size of a salad plate, 8.5″ on a side. Image: Cerebras
Cerebras is emphasizing what it’s calling “near-perfect linear scaling,” which means that for a given job, two CS-2s will do that job twice as fast as one, three will take a third of the time, and so on. How? Andromeda’s SC-2 systems rely on parallelization, Cerebras said, from the cores on each wafer to the SwarmX fabric coordinating them all. But the supercomputer’s talents extend beyond its already impressive 16 nodes. Using the same data parallelization, researchers can yoke together up to 192 CS-2 systems for a single job.... '
No comments:
Post a Comment