Data Preprocessing with NVIDIA Dali

Came across DALI in the NVIDIA developers blog.    Nice idea, below is the intro, much more tech detail at the link, with coding usage examples.  As in all data-hungry methods, its often about how to get the data ready and verified to do testing and and then ongoing for delivery.    Especially if the data resides on an  Edge Application.   Approach here is intriguing.

Fast AI Data Preprocessing with NVIDIA DALI  By Joaquin Anton Guirao, Krzysztof Łęcki, Janusz Lisiecki, Serge Panev, Michał Szołucha, Albert Wolant and Michał Zientkiewicz    Tags: DALI, data preprocessing, Deep Learning, featured, machine learning and AI, Storage

 Training deep learning models with vast amounts of data is necessary to achieve accurate results. Data in the wild, or even prepared data sets, is usually not in the form that can be directly fed into neural network. This is where NVIDIA DALI data preprocessing comes into play. .... 

 ... DALI To the Rescue
NVIDIA Data Loading Library (DALI) is a result of our efforts find a scalable and portable solution to the data pipeline issues mentioned above. DALI is a set of highly optimized building blocks plus an execution engine to accelerate input data pre-processing for deep learning applications, as diagrammed in figure 1. DALI provides performance and flexibility for accelerating different data pipelines. .... "