On Ray and Path Tracing

Representing 3D Objects.

What Is Path Tracing?

March 23, 2022 by Brian Caulfield

Turn on your TV. Fire up your favorite streaming service. Grab a Coke. A demo of the most important visual technology of our time is as close as your living room couch. Propelled by an explosion in computing power over the past decade and a half, path tracing has swept through visual media. It brings big effects to the biggest blockbusters, casts subtle light and shadow on the most immersive melodramas and has propelled the art of animation to new levels.

Path tracing is going real time, unleashing interactive, photorealistic 3D environments filled with dynamic light and shadow, reflections and refractions.

So what is path tracing? The big idea behind it is seductively simple, connecting innovators in the arts and sciences over the span half a millennium.

What’s the Difference Between Rasterization and Ray Tracing?

First, let’s  define some terms, and how they’re used today to create interactive graphics — graphics that can react in real time to input from a user, such as in video games.

The first, rasterization, is a technique that produces an image as seen from a single viewpoint. It’s been at the heart of GPUs from the start. Modern NVIDIA GPUs can generate over 100 billion rasterized pixels per second. That’s made rasterization ideal for real-time graphics, like gaming.

Ray tracing is a more powerful technique than rasterization. Rather than being constrained to finding out what is visible from a single point, it can determine what is visible from many different points, in many different directions. Starting with the NVIDIA Turing architecture, NVIDIA GPUs have provided specialized RTX hardware to accelerate this difficult computation. Today, a single GPU can trace billions of rays per second.

Being able to trace all of those rays makes it possible to simulate how light scatters in the real world much more accurately than is possible with rasterization. However, we still must answer the questions, how will we simulate light and how will we bring that simulation to the GPU?

What’s Ray Tracing? Just Follow the String

To better answer that question, it helps to understand how we got here.

David Luebke, NVIDIA vice president of graphics research, likes to begin the story in the 16th century with Albrecht Dürer — one of the most important figures of the Northern European Renaissance — who used string and weights to replicate a 3D image on a 2D surface.

Dürer made it his life’s work to bring classical and contemporary mathematics together with the arts, achieving breakthroughs in expressiveness and realism.

The string’s the thing: Albrecht Dürer was the first to describe what’s now known as “ray tracing,” a technique for creating accurate representations of 3D objects on a 2D surfaces in Underweysung der Messung (Nuremberg, 1525),f15

In 1525 with Treatise on Measurement, Dürer was the first to describe the idea of ray tracing. Seeing how Dürer described the idea is the easiest way to get your head around the concept.

Just think about how light illuminates the world we see around us. Now imagine tracing those rays of light backward from the eye with a piece of string like the one Dürer used, to the objects that light interacts with. That’s ray tracing.

Ray Tracing for Computer Graphics    .... 

Rethinking the Computer Chip in the Age of AI

 New designs for Computer chips. 

Rethinking the Computer Chip in the Age of AI,    via U of Penn

Posted on September 29, 2022   Author Devorah Fischler 

The transistor-free compute-in-memory architecture permits three computational tasks essential for AI applications: search, storage, and neural network operations.

Artificial intelligence presents a major challenge to conventional computing architecture. In standard models, memory storage and computing take place in different parts of the machine, and data must move from its area of storage to a CPU or GPU for processing.

The problem with this design is that movement takes time. Too much time. You can have the most powerful processing unit on the market, but its performance will be limited as it idles waiting for data, a problem known as the “memory wall” or “bottleneck.”

When computing outperforms memory transfer, latency is unavoidable. These delays become serious problems when dealing with the enormous amounts of data essential for machine learning and AI applications.

As AI software continues to develop in sophistication and the rise of the sensor-heavy Internet of Things produces larger and larger data sets, researchers have zeroed in on hardware redesign to deliver required improvements in speed, agility and energy usage.

A team of researchers from the University of Pennsylvania’s School of Engineering and Applied Science, in partnership with scientists from Sandia National Laboratories and Brookhaven National Laboratory, has introduced a computing architecture ideal for AI.

Deep Jariwala, Xiwen Liu and Troy Olsson

Co-led by Deep Jariwala, Assistant Professor in the Department of Electrical and Systems Engineering (ESE), Troy Olsson, Associate Professor in ESE, and Xiwen Liu, a Ph.D. candidate in Jarawala’s Device Research and Engineering Laboratory, the research group relied on an approach known as compute-in-memory (CIM).

In CIM architectures, processing and storage occur in the same place, eliminating transfer time as well as minimizing energy consumption. The team’s new CIM design, the subject of a recent study published in Nano Letters, is notable for being completely transistor-free. This design is uniquely attuned to the way that Big Data applications have transformed the nature of computing.

“Even when used in a compute-in-memory architecture, transistors compromise the access time of data,” says Jariwala. “They require a lot of wiring in the overall circuitry of a chip and thus use time, space and energy in excess of what we would want for AI applications. The beauty of our transistor-free design is that it is simple, small and quick and it requires very little energy.”

The advance is not only at the circuit-level design. This new computing architecture builds on the team’s earlier work in materials science focused on a semiconductor known as scandium-alloyed aluminum nitride (AlScN). AlScN allows for ferroelectric switching, the physics of which are faster and more energy efficient than alternative nonvolatile memory elements.

“One of this material’s key attributes is that it can be deposited at temperatures low enough to be compatible with silicon foundries,” says Olsson. “Most ferroelectric materials require much higher temperatures. AlScN’s special properties mean our demonstrated memory devices can go on top of the silicon layer in a vertical hetero-integrated stack. Think about the difference between a multistory parking lot with a hundred-car capacity and a hundred individual parking spaces spread out over a single lot. Which is more efficient in terms of space? The same is the case for information and devices in a highly miniaturized chip like ours. This efficiency is as important for applications that require resource constraints, such as mobile or wearable devices, as it is for applications that are extremely energy intensive, such as data centers.”  ... ' 

Grace Hopper Superchip Design

Nice to see this effort named after my former Pentagon colleague. See previous mentions of her in this blog at the 'Grace Hopper' label Link.   Technical. 

NVIDIA Grace Hopper Superchip Architecture In-Depth

By Jonathon Evans, Michael Andersch, Vikram Sethi, Gonzalo Brito and Vishal Mehta

CUDA, Grace Hopper Superchip, HPC / Supercomputing, Technical Walkthrough

The NVIDIA Grace Hopper Superchip Architecture is the first true heterogeneous accelerated platform for high-performance computing (HPC) and AI workloads. It accelerates applications with the strengths of both GPUs and CPUs while providing the simplest and most productive distributed heterogeneous programming model to date. Scientists and engineers can focus on solving the world’s most important problems.

In this post, you learn all about the Grace Hopper Superchip and highlight the performance breakthroughs that NVIDIA Grace Hopper delivers. For more information about the speedups that Grace Hopper achieves  over the most powerful PCIe-based accelerated platforms using NVIDIA Hopper H100 GPUs, see the NVIDIA Grace Hopper Superchip Architecture whitepaper.

Performance and productivity for strong-scaling HPC and giant AI workloads

The NVIDIA Grace Hopper Superchip architecture brings together the groundbreaking performance of the NVIDIA Hopper GPU with the versatility of the NVIDIA Grace CPU, connected with a high bandwidth and memory coherent NVIDIA NVLink Chip-2-Chip (C2C) interconnect in a single superchip, and support for the new NVIDIA NVLink Switch System.

Diagram of the NVIDIA Grace Hopper Superchip showing the LPDDR5X, HBM3, NVLink, and I/O bandwidths as well as memory capacities. Hopper has up to 96 GB HBM3 at up to 3000 GB/s bandwidth. Grace has up to 512 GB LPDDR5X at up to 546 GB/s bandwidth. Grace and Hopper are connected with NVLink C2C at up to 900 GB/s bandwidth. The Grace Hopper Superchip has up to 64 PCIe Gen 5 lanes delivering up to 512 GB/s bandwidth and up to 18x NVLink for lanes delivering up to 900 GB/s to the NVLink Switch network.

Figure 2. NVIDIA Grace Hopper Superchip logical overview

NVIDIA NVLink-C2C is an NVIDIA memory coherent, high-bandwidth, and low-latency superchip interconnect. It is the heart of the Grace Hopper Superchip and delivers up to 900 GB/s total bandwidth. This is 7x higher bandwidth than x16 PCIe Gen5 lanes commonly used in accelerated systems.

NVLink-C2C memory coherency increases developer productivity and performance and enables GPUs to access large amounts of memory.CPU and GPU threads can now concurrently and transparently access both CPU– and GPU-resident memory, enabling you to focus on algorithms instead of explicit memory management.

Memory coherency enables  you to transfer only the data you need, and not migrate entire pages to and from the GPU. It also enables lightweight synchronization primitives across GPU and CPU threads by enabling native atomic operations from both the CPU and GPU. NVLink-C2C with Address Translation Services (ATS) leverages the NVIDIA Hopper Direct Memory Access (DMA) copy engines for accelerating bulk transfers of pageable memory across host and device.  ... ' 

Latest NVIDIA Developer Resources

 Impressive look,  just a sampler below. 

Latest Releases and Resources: Feb. 3-10, NVIDIA By Michelle Horton

Tags: Automotive / Transportation, Computer Vision / Video Analytics, Conversational AI / NLP, CUDA, Data Science, Education & Training, featured, Inference, Metropolis, News, Riva, TensorRT, Webinar

Our weekly roundup covers the most recent software updates, learning resources, events, and notable news. 

Software releases, Courses, Webinars, Software releases

GPU-Accelerated Asymmetric Numeral Systems with nvCOMP v2.2.0

The redesigned nvCOMP 2.2.0 interface provides a single nvcompManagerBase object that can do compression and decompression. Users can now decompress nvcomp-compressed files without knowing how they were compressed. The interface also can manage scratch space and split the input buffer into independent chunks for parallel processing.

What’s new:

The redesigned, high-level interface enhances the user experience by storing metadata in the compressed buffer.

All compressors are available through both low-level and high-level APIs.

Proprietary entropy encoder based on Asymmetric Numeral Systems.

GDeflate: Entropy-only variant

Windows support

Download now: nvCOMP version 2.2.0   .... 

A GPU can Brute Force Your Passwords

Faster GPUs are eroding security.  Using a password manager, which give you a larger number of characters,  and/or a multifactor link up makes much sense.   

The Nvidia RTX 3090 GPU Can Probably Crack Your Passwords   By Ryan Whitwam

The new Nvidia GeForce RTX 3090 is a gaming powerhouse, but that’s not all it can do. According to the makers of a popular password recovery application, the RTX 3090 is also good at brute-forcing passwords. That’s great if you forget an important password, but that’s probably not why people are using such tools. The latest Nvidia cards could make cracking someone else’s files almost trivially easy. 

The RTX 3090 is Nvidia’s latest top-of-the-line GPU with a GA102 graphics processor sporting 10,496 cores and 24GB of GDDR6X memory. It is monstrously, obscenely powerful by today’s gaming standards, and comes with a correspondingly high price of $1,500, give or take a few hundred depending on supply. With a focus on high core counts, GPUs are also great for parallel computing. That’s why you couldn’t even buy a GPU for several months when Bitcoin was at its peak. In the same vein, GPUs are very good at cracking passwords.   .... "

Faster Learning and Operations in Theater

Though the examples are military, the use of the same methods could be in many sensory rich, real time application of machine learning and operations.   IOT too.

Artificial intelligence helps Soldiers learn many times faster in combat
 -- New technology allows U.S. Soldiers to learn 13 times faster than conventional methods and Army researchers said this may help save lives.

At the U.S. Army Research Laboratory, scientists are improving the rate of learning even with limited resources. It's possible to help Soldiers decipher hints of information faster and more quickly deploy solutions, such as recognizing threats like a vehicle-borne improvised explosive device, or potential danger zones from aerial war zone images.

The researchers relied on low-cost, lightweight hardware and implemented collaborative filtering, a well-known machine learning technique on a state-of-the-art, low-power Field Programmable Gate Array platform to achieve a 13.3 times speedup of training compared to a state-of-the-art optimized multi-core system and 12.7 times speedup for optimized GPU systems. .... " 

AI Chips Advance Smarter Devices

Very good overview of the nature of chips that enable AI on the edge capabilities.  New capabilities for the IOT.  And who is doing what in delivering new hardware.  I recall mention back in the 90s it was said that faster hardware would be needed to make neural nets to drive solutions.  But at the time, there were few direct applications.   It was said then that software would drive solutions.   Now hardware is coming to the forefront.  Still expect software skills to be needed to communicate that power.

New AI systems on a chip will spark an explosion of even smarter devices  By James Kobielus in SiliconAngle

" .... That’s why mass-market mobile and IoT edge devices are increasingly coming equipped with systems-on-a-chip that are optimized for local AI processing. What distinguishes AI systems on a chip from traditional mobile processors is that they come with specialized neural-network processors, such as graphics processing units or GPUs, tensor processing units or TPUs, and field programming gate arrays or FPGAs. These AI-optimized chips offload neural-network processing from the device’s central processing unit chip, enabling more local autonomous AI processing and reducing the need to communicate with the cloud for AI processing..... "

Executing Intelligence at the Edge

You would like to get the execution of deep learning neural nets to the edge of the network.  It is ultimately where we want to execute learned knowledge.    Training them has evolved to new efficiencies by using adapted net architectures and parallel chips, but now we need new ways to make them work at this edge, in mobile devices.

Google Researcher Invented New Technology to Bring Neural Networks to Mobile Devices  by Roland Meertens in InfoQ

Recently many companies released applications that use deep neural networks. Neural networks can require a large amount of computation. For this reason, they often appear in software as a service applications on GPU powered servers. For applications that should run without internet access, must be fast and responsible, or in which privacy is a concern using networks on servers is not possible. ... " 

Why Andrew Ng Teaches AI

Why AI Visionary Andrew Ng Teaches Humans to Teach Computers 
Associated Press  by Ryan Nakashima

Stanford University professor Andrew Ng's current area of concentration is teaching next-generation artificial intelligence (AI) specialists to train computers. As a pioneer in such AI-training courses, Ng says he envisions AI as a tool to "free humanity from repetitive mental drudgery." He expects AI to have a transformative effect on virtually every industry, and eventually enable any job that takes less than a second of thought to become automated. Ng is an advocate of machine-learning neural networks, and his breakthrough in the field of AI was discovering a new way to give neural networks a significant performance boost by using chips typically found in video-game machines. Neural networks function much better if they can run thousands of calculations concurrently, an operation well suited to graphics-processing units. Ng and his Stanford team's research in this field was able to accelerate machine learning by as much as 70 times in 2008.  ... " 

Powering new Chips with Better Algorithms

A continued race between hardware and software ...

We Need Next Generation Algorithms To Harness The Power Of Today's AI Chips
 by Greg Diamos is the Head of Systems Research at Baidu Silicon Valley AI Lab.

At the GTC technology conference this year, NVIDIA launched their latest and most advanced GPU called Volta. At the center of this chip is Tensor Core, an Artificial Intelligence accelerator that that is poised to usher in the next phase of AI applications. However, our current AI algorithms are not fully utilizing this accelerator, and for us to achieve another major breakthrough in AI, we need to change our software. ... " 

IBM Research Writes about Progress in Deep Learning

Recently have used deep learning from IBM to solve a difficult classification problem.

From IBM AI Research: 
New IBM Research distributed deep learning software achieves record performance for large neural network, large data set ... 

Deep learning is a widely used AI method to help computers understand and extract meaning from images and sounds and other data types using neural networks, a brain-inspired approach to computing. It holds promise to fuel breakthroughs in everything from consumer mobile app experiences to medical imaging diagnostics. Progress in accuracy and deploying deep learning at scale is limited by technical challenges that slow processing time to days and weeks. IBM Research AI experts have created distributed deep learning software, achieving record performance for image recognition accuracy and large neural networks composed of up to 250 GPUs, a special processor for large amounts of data. Developers and data scientists can now preview this technical milestone in version 4 of the PowerAI enterprise deep learning software.  ... " 

Towards the AI Data Center

GPU's may soon rule, but the models still need to be built correctly.

Nvidia Steps Up AI Data Center Push
By Jon Markman ,   Contributor in Forbes

Analyzing tech stocks through the prism of cultural change.  
Recently, Nivida unveiled Volta, the most advanced data-center graphics-processing unit ever built. With 21.1 billion transistors and a massive 815 mm2 footprint, it will facilitate the next generation of artificial intelligence.

Nvidia is still pushing the envelope, leading from way out in front.

Jen-Hsun Huang, Nvidia’s CEO, boasts that Volta is “pushing the limits of photolithography”. He also understands, ultimately, this is only the beginning of what is possible for AI. ... " 

TensorFlow Update for Mobile

Noted that Google was coming up with versions for mobile application, so revisited their offerings.

An open-source software library for Machine Intelligence
TensorFlow™ is an open source software library for numerical computation using data flow graphs. Nodes in the graph represent mathematical operations, while the graph edges represent the multidimensional data arrays (tensors) communicated between them. The flexible architecture allows you to deploy computation to one or more CPUs or GPUs in a desktop, server, or mobile device with a single API. TensorFlow was originally developed by researchers and engineers working on the Google Brain Team within Google's Machine Intelligence research organization for the purposes of conducting machine learning and deep neural networks research, but the system is general enough to be applicable in a wide variety of other domains as well. .... " 

GPU Technology Conference

The keynote of the GPU Tech conference was brought to my attention. Truncated from Engadget, below.  It has long been known that GPUs could be used for analytics, its only most recently that the application has exploded.  There might be a similar effect with quantum computing as it matures. Could that drive us to more intelligence?

Watch the highlights of NVIDIA's GPU Technology Conference keynote
Deep learning, self-driving cars and GPU processors galore.  ... " 

Microsoft Reveals Hololens Progress

VRFocus on Upcoming applications to be shown at the Microsoft Build Event.  Good indication of what is being worked on

Microsoft Build 2017 Sessions Revealed
The conference takes place later this week.  May 10.
 By Peter Graham  Last updated May 8, 2017 

HoloLensMicrosoftMicrosoft Build 2017
For virtual reality (VR), augmented reality (AR) and mixed reality (MR) tech enthusiasts and developers this week is quite a big one. Starting today and finishing Wednesday is the NVIDIA GPU Technology Conference (2017) featuring the latest immersive news and insights from the company. Wednesday also sees the start of Microsoft Build 2017, set to include even more of the very latest tech, and today the sessions have now been unveiled.

As VRFocus readers may expect, Microsoft Build 2017 will be far more focused on MR and AR than VR, with sessions revolving around HoloLens and the Universal Windows Platform (UWP). .... " 

( At link, session titles of work underway)  .....

Cheap not Deep Learning is the Future

Had not heard of the term 'cheap-learning'  before, but have been thinking of the concept for a long time.  Building higher levels of abstraction is a matter of better leverage.  Only a few people need to know the low level math.  These people are absolutely necessary, but just a few of them.     And very few as we creep towards automation.    A matter of car mechanics vs Phds in thermal mechanics.

In Datanami: 

" ... Higher Levels of Abstraction

In the new cheap learning paradigm, the combination of sophisticated frameworks like Theano and Tensorflow and the powerful but simple languages like Python will help to create a new layer of abstraction that eliminates the need for big data application developers to understand the nitty gritty details of high-level math and low-level execution models to get stuff done.

Developers will be able to tell the computer what to do at a high level, Dunning says, and the computer will take care of the implementation details, whether it’s running on a 1,000-node cluster, a gaggle of GPUs, or just a laptop.

(Ted) Dunning (of MapR)  uses a car analogy to communicate his vision for how the cheap learning metaphor will evolve.

“I don’t understand cars anymore. I understand what combustion is and what gasoline is, but the actual details of how cars work escaped my grasp many years ago,” Dunning says. “But I have a mental model, when I want to get to work or talk to somebody about the car. We need to have that same sort of loss of detail, and a higher abstraction level, when talking about these parallel programs.” .... ' 

HPC and Deep Learning by Baidu

That Baidu opening up here is interesting, read the full article at the link.  Openness of Baidu is intriguing.  See an intro to OpenMPI.

HPC Technique Propels Deep Learning at Scale   By HPC Wire 

Baidu's Silicon Valley Artificial Intelligence Lab (SVAIL) has released a modified implementation of the ring all-reduce OpenMPI algorithm for the deep-learning community, which will enable faster training of neural networks across graphical-processing unit (GPU) nodes.

Unlike the OpenMPI version, the SVAIL modification avoids making extraneous copies between the central processing unit (CPU) and the GPU.

Although commonplace in high-performance computing, the technique has been underused within AI and deep learning, according to Baidu. Compared with using a single GPU, the ring all-reduce algorithm is about 31 times faster at 40 GPUs.

The algorithm has enabled the SVAIL team to get linear GPU scaling up to 128 GPUs and to parallelize the training of Deep Speech 2, its speech-recognition mode.

Two years after the approach was initially developed, the researchers have issued two non-proprietary implementations, one for TensorFlow and one for more general applications. ... " 

Q&A on PowerAI

For research I took a look at the PowerAI offering.  See the complete article:

Putting the “AI” in PowerAI
A Q&A with IBM Research’s Hillery Hunter

IBM’s latest Power servers come with an AI twist. Optimized for deep learning, a new so-called PowerAI toolkit will “help train the systems to think and learn in a more human-like way, at a faster pace,” as announced at SC16, the International Conference for High Performance Computing, Networking, Storage and Analysis.

I spoke with Hillery Hunter, IBM Research’s director of Systems Acceleration and Memory and Memory Strategist, about her team’s contribution to the software behind the world’s fastest deep learning servers.

How was the AI added, and what does that mean for how a Power system functions?
 Today’s launch represents our first public offering in hardware-software co-optimization for deep learning. Researchers have worked closely with IBM’s systems engineers to create code that is optimized to the Power S822LC (IBM’s highest-performing OpenPOWER server).

We used the S822LC’s unique design – including high-bandwidth NVLink interconnect, not just between pairs of GPUs, but also between the GPUs and the CPUs – to deliver higher deep learning training performance.

This means that the co-designed PowerAI hardware and software can build learned models from images, speech, or other media in less time than prior generations of hardware and software. Deep learning training time is a key metric for developer productivity in this domain. It enables innovation at a faster pace, as developers can invent and try out many new models, parameter settings, and data sets.    .... " 

Arup and 3D Printing

We did some work with the Engineering and  design firm Arup in the innovation space.   Was interesting to see they are doing work with 3D printing.

This Intelligent 3D Printer Is Building Big, Beautiful Structures  by Alison E. Berman

Ai Build, a London-based startup, aims to pave the way to 3D printing on large scales.

The company is equipping industrial-grade Kuka robotic arms with artificial intelligence and "3D printing guns" to 3D print large structures that focus on maximizing efficiency with labor and materials.

Founder and CEO Daghan Cam dreamed up the technology while considering traditional commercial construction and wondering what a more efficient and automated process might look like.

In October, the company partnered with engineering consulting firm Arup Engineers to debut the 3D printed “Daedalus Pavilion” at the GPU Technology Conference in Amsterdam. The structure is roughly 16 feet wide and 14 feet tall. Its 48 parts were printed in 15 days and assembled in less than one.

Ai Build's system uses video cameras outfitted with machine learning algorithms to allow robots to learn from their mistakes—meaning they can operate more quickly, correcting for errors on the fly instead of moving slowly to prevent them. According to Cam, Ai Build's arms can print in half the time it would take using standard techniques.   .... " 

New Chip Usage to Power AI

Re-using chip designs to perform the methods used for AI applications.  Now new chip designs are in play.

Intel Looks to a New Chip to Power the Coming Age of AI
Microsoft Researchers recently built an artificially intelligent system that seems to recognize conversational speech as effectively as a human. Yes, this research comes with caveats, but it’s part of a very real and very rapid leap in artificial intelligence over the past several years, a leap driven by deep neural networks.

These sweepingly complex algorithms can teach themselves very particular tasks by analyzing vast amounts of data. Microsoft’s system learned to recognize words by looking for patterns in old tech support calls. But it’s not just the algorithms that are driving the recent revolution in AI. It’s also the hardware behind these algorithms. Microsoft’s speech rec system relies on large farms of GPU processors, chips that were originally designed for rendering graphics but have proven remarkably adept at running artificial intelligence models. .... 

Internet giants like Microsoft, Google, Facebook, and Baidu typically train their deep neural nets using GPUs. But they’re moving towards other, more specialized chips that can help accelerate not just the training but the execution of these systems. Google recently built its own AI processor. IBM is building another.

So Intel, the world’s largest chip maker, is doing the same. Yesterday, the company unveiled a new AI processor called Nervana, saying it plans on testing prototypes by the middle of next year, and if all goes well, a finished chip will come to market by the end of 2017. At the moment, the market for AI chips is dominated by nVidia, the primary supplier of GPUs.  ... "