Grace Hopper Superchip Design

Nice to see this effort named after my former Pentagon colleague. See previous mentions of her in this blog at the 'Grace Hopper' label Link.   Technical. 

NVIDIA Grace Hopper Superchip Architecture In-Depth

By Jonathon Evans, Michael Andersch, Vikram Sethi, Gonzalo Brito and Vishal Mehta

CUDA, Grace Hopper Superchip, HPC / Supercomputing, Technical Walkthrough

The NVIDIA Grace Hopper Superchip Architecture is the first true heterogeneous accelerated platform for high-performance computing (HPC) and AI workloads. It accelerates applications with the strengths of both GPUs and CPUs while providing the simplest and most productive distributed heterogeneous programming model to date. Scientists and engineers can focus on solving the world’s most important problems.

In this post, you learn all about the Grace Hopper Superchip and highlight the performance breakthroughs that NVIDIA Grace Hopper delivers. For more information about the speedups that Grace Hopper achieves  over the most powerful PCIe-based accelerated platforms using NVIDIA Hopper H100 GPUs, see the NVIDIA Grace Hopper Superchip Architecture whitepaper.

Performance and productivity for strong-scaling HPC and giant AI workloads

The NVIDIA Grace Hopper Superchip architecture brings together the groundbreaking performance of the NVIDIA Hopper GPU with the versatility of the NVIDIA Grace CPU, connected with a high bandwidth and memory coherent NVIDIA NVLink Chip-2-Chip (C2C) interconnect in a single superchip, and support for the new NVIDIA NVLink Switch System.

Diagram of the NVIDIA Grace Hopper Superchip showing the LPDDR5X, HBM3, NVLink, and I/O bandwidths as well as memory capacities. Hopper has up to 96 GB HBM3 at up to 3000 GB/s bandwidth. Grace has up to 512 GB LPDDR5X at up to 546 GB/s bandwidth. Grace and Hopper are connected with NVLink C2C at up to 900 GB/s bandwidth. The Grace Hopper Superchip has up to 64 PCIe Gen 5 lanes delivering up to 512 GB/s bandwidth and up to 18x NVLink for lanes delivering up to 900 GB/s to the NVLink Switch network.

Figure 2. NVIDIA Grace Hopper Superchip logical overview

NVIDIA NVLink-C2C is an NVIDIA memory coherent, high-bandwidth, and low-latency superchip interconnect. It is the heart of the Grace Hopper Superchip and delivers up to 900 GB/s total bandwidth. This is 7x higher bandwidth than x16 PCIe Gen5 lanes commonly used in accelerated systems.

NVLink-C2C memory coherency increases developer productivity and performance and enables GPUs to access large amounts of memory.CPU and GPU threads can now concurrently and transparently access both CPU– and GPU-resident memory, enabling you to focus on algorithms instead of explicit memory management.

Memory coherency enables  you to transfer only the data you need, and not migrate entire pages to and from the GPU. It also enables lightweight synchronization primitives across GPU and CPU threads by enabling native atomic operations from both the CPU and GPU. NVLink-C2C with Address Translation Services (ATS) leverages the NVIDIA Hopper Direct Memory Access (DMA) copy engines for accelerating bulk transfers of pageable memory across host and device.  ... ' 

Tales of Grace Hopper

Told of my own minor interactions with Grace Hopper at the Pentagon, towards the end of her career,  here is more:

Grace Hopper, Minicomputers, and Megabytes: It's a Fun Career  By Ann Moffatt

Communications of the ACM, November 2021, Vol. 64 No. 11, Page 7 10.1145/3485446

My career started when I joined Kodak in the U.K. in 1959, where I was taught to program by Conway Berners-Lee, father of Sir Tim Berners-Lee, the WWW inventor. At that time, we only knew of about 300 stored program computers in the world, although there were probably 300 more in 'secret' places like the military or government.

By 1963, computing in the U.K. surpassed the rest of the world. The British government decided to make the world's most powerful computer: the mighty Atlas, the first computer with an operating system. The manufacturer, Ferranti, couldn't get it to work and asked would-be buyers to send their best programmers to help. Kodak sent me.

The sales price was about £UK 3 million (about $150 million USD today). The sales manager told his team if Ferranti sold three Atlases to the Russian government it would solve Russian computer needs to the year 2000. It was the time of MAD (Mutually Assured Destruction), when computers guided nuclear weapons and supported the space race to get a man to the moon. Atlas was less powerful than the MAC I'm using today.

As Special Ambassador for Univac, Grace Hopper, the inventor of COBOL (the most widely used programming language in the 1960s to 1980s), toured the world giving lectures about the industry. I had the pleasure of escorting her to various British Computer Society functions whenever she visited the U.K.

After a dinner in 1973, she asked if we would like to see the new computer Univac had loaned her. She dived into her handbag and brought out an object the size of a cigarette packet. We all stared, amazed, as she opened the box and picked up an even smaller object. Grace proceeded to tell us the impossibly small computer had a 64-kilobyte COBOL complier. We wanted to see it in action, so someone brought over a teletype with a printer, and from the side of the device Grace pulled out a fine cable the width of a human hair and a transformer with an adaptor for the fine cable to plug into as the power supply.

The group watched as Grace ran a simple COBOL program. We didn't know it then, but we had just witnessed an early silicon chip-based computer. In Grace's opinion, the mainframe was dead and would be replaced by 'multitudes of minicomputers' that would be linked by telephone lines, all working together. It was quite possible she had seen a demonstration of the U.S. Department of Defense's ARPANET, the precursor of the Internet.

In 1974, I moved to Australia. I was the only woman executive at AMP, the largest company in terms of assets. Part of my job was to buy all the hardware. Computers still used core memory, which was sensitive to heat fluctuations. Our Univac salesman told me there was a new type of memory called Metal Oxide on Silica (MOS), which was not affected by heat fluctuations. It cost $1.5 million a megabyte. My boss was flummoxed, not only by the price but also by the size of the memory saying we already had two of the largest computers in the southern hemisphere, each with half a megabyte. I explained the system we were developing was to run real-time database systems, and more memory would be a distinct advantage. I was told if I could get the salesman to bring the cost down to $1 million, we would buy it.

I wish I saved the invoice because it would be of great historical and hysterical value now. Today we refer to MOS memory as computer chips and a megabyte costs a fraction of a cent. Although manufacturing processes for chips have changed considerably, they are not very different from that first megabyte I bought for AMP in the mid 1970s.

And so it goes. Always new things to learn, always tremendous increases in technology accompanied by drop in costs. Always exciting new applications.

It's a wonderful career, especially for women. Come and join us.  ... ' 

Cobol Still in Use

I started coding well after COBOL had declined, but key parts of the company were still being run by COBOL code.  It still did many of the logical things required, and because it was compiled, it was very fast for the computers of the day.    If you knew coding you could learn it a few days.   After that I was involved with the Y2K event, and we had to wade through huge amounts of COBOL and other code, to determine if there was code that would fail based on date encoding. 

We only found only a few examples of suspect date coding, but found quite a few more examples where there were possible problems not related to Y2K.   So it helped after all, and the remaining code was fixed, recompiled and allowed to carry on.    I do wonder how much different it would have been if all the COBOL code had been done in Python?  Though lots of shared code design might have helped.  The article below also mentions my colleague Grace Hopper, at the Pentagon who I have talked about here before

COBOL Turns 60: Why It Will Outlive Us All   By ZDNet

I cut my programming teeth on IBM 360 Assembler. This shouldn't be anyone's first language. In computing's early years, the only languages were machine and assembler. In those days, computing science really was "science." Clearly, there needed to be an easier language for programming those hulking early mainframes. That language, named in September 1959, became Common Business-Oriented Language (COBOL).

The credit for coming up with the basic idea goes not to Grace Hopper, although she contributed to the language and promoted it, but to Mary Hawes. She was a Burroughs Corporation programmer who saw a need for a computer language. In March 1959, Hawes proposed that a new computer language be created. It would have an English-like vocabulary that could be used across different computers to perform basic business tasks.

Hawes talked Hopper and others into creating a vendor-neutral interoperable computer language. Hopper suggested they approach the Department of Defense (DoD) for funding and as a potential customer for the unnamed language. 

Business IT experts agreed, and in May 1959, 41 computer users and manufacturers met at the Pentagon. There, they formed the Short Range Committee of the Conference on Data Systems Languages (CODASYL).

Drawing on earlier business computer languages such as Remington Rand UNIVAC's FLOW-MATIC, which was largely the work of Grace Hopper, and IBM's Commercial Translator, the committee established that COBOL-written programs should resemble ordinary English. ...

Classifying Images with TensorFlow

From the Google research blog.  A non technical post on how to use their now open source TensorFlow capability for image classification.  Instructive as to capabilities.  Links to simple tutorials there. I see my former colleague, Admiral Grace Hopper is assisting.

Creating Computing Superheroes

Most interesting view of computing history.   Examining Walter Isaacson's The Innovators: How a Group of Hackers, Geniuses, and Geeks Created the Digital Revolution.  Regarding the treatment of Ada Lovelace and Grace Hopper.  I briefly encountered Grace Hopper at the Pentagon in the 70s. Have written about it in this blog.  See tags below. We have a long history of creating heroes for our needs.

Grace Hopper at 107

I see that it is the 107th anniversary of computer Pioneer Grace Hopper's birth.  I wrote about my brief encounter with her while I was working at the Pentagon,  in an early post in this blog.   It was an honor to be connected to an early pioneer in anything that became so important to technology.  She has been honored by inclusion in Googles search page today  with a cute animation. (Today only)  Also by the declaration of an hour of code, made to encourage students to take up the science and art of coding.  " ... Be a maker, a creator, an innovator. Get started now with an Hour of Code. ... ".    I grate a bit at it being promoted in a celebrity based way, but it is still worth sharing with a student who may be interested in the craft.

Grace Hopper Interview

An interview with computer pioneer Grace Hopper, who I met while working at the Pentagon. A reminder of how far we have come.

Grace Hopper's 100th Birthday

I see that this past Friday was Grace Hopper's 100th birthday. Here is a personal reminiscense from my first professional IT experience:

In 1975 I was freshly graduated from the University of Pennsylvania with a degree in astrophysics. There were few opportunities for someone with an undergraduate degree in that field, so I took the civil service exam, and ended up taking the train from Philadelphia to Washington for the first time. I had hired on as an analyst for the national Military Command Systems Support Center at the Pentagon. We were some number of floors below the ground, responsible for keeping several military computers running. We were managed by the Navy, so even though we were far below dry earth, when a new officer arrived they were 'piped on board' in shipboard fashion.

Just around the corner from my cubicle was the very humble office of then Captain Grace Hopper. Having always been interested in the history of technology I recognized the name mainly due to her involvement in the development of compilers for the computer language Cobol. She gave a talk a few months after my arrival and I was impressed by her vision. Her talks gave a nod to computer history, which she had been a key part of, but even more so she was sincerely interested in the future of computing. In her talks then and for many years afterward she always carried a length of wire with her as a prop, this wire, she would declare, is a nanosecond ... equating time and computing power. She then would talk about where she thought computing was going. Her views were mainly about numerical rather than symbolic computing, but even in this realm she understood computers were stretching far beyond just numbers.

She was very rarely in her office, even then acting as an ambassador for military computing. I made it a point to stop by one day when she was there and introduced myself, a slightly longhaired young computer neophyte. She was gracious and invited me in. We hit it off because most of my work in school had been about solving differential equations, which had been an interest of hers. I was particularly interested in the pre-compiler world of the computer, when developing code was more like wiring a machine than writing logic. She also spoke about her involvement with Eckert and Mauchly, early pioneers of computer hardware. Looking back I should have had far more questions, but our chat ended far too soon. It was our only meeting, I moved on soon afterward, though we said hi a few more times in the hall.

She was appointed to admiral in 1985 and died in 1992, just as the Web had been developed. What might she have thought about that development? Many millions of non-technical people using 'Hypertext Transfer Protocol' to communicate? Her development of the compiler was a first step from numerical computing to using computers to manipulate text and communicate thoughts freely between ordinary people. She showed the way in the very early years and helped make it happen.

Good short overview of her life and achievements.

FA Dill June, 2004