Optical Algorithm Simplifies Analog AI Training

Thinking this, possibilities and  implications.  

Optical Algorithm Simplifies Analog AI Training When backpropagation loses its bite, this approach may substitute  ...    in IEEE Spectrum

Researchers have developed a range of analog and other unconventional machine learning systems in the expectation that they will prove vastly more energy efficient than today’s computers. But training these AIs to do their tasks has been a big stumbling block. Researchers at NTT Device Technology Labs and the University of Tokyo now say they’ve come up with a training algorithm (announced by NTT last month) that goes a long way toward letting these systems meet their promise.

Their results, established on an optical analog computer, represent progress towards obtaining the potential efficiency gains that researchers have long sought from “unconventional” computer architectures.

Modern AI programs use a biologically-inspired architecture called an artificial neural network to execute tasks like image recognition or text generation. The strength of connections between artificial neurons, which control the outputs of the computation, must be modified or trained using standard algorithms. The most prominent of these algorithms is called backpropagation, which updates the connection strengths to reduce the network’s errors, while it processes trial data. Because adjustments to some parameters depend on adjustments to others, there is a need for active information passing and routing by the computer.

As Spectrum has elsewhere explained, “Error backpropagation is like running inference in reverse, moving from the last layer of the network back to the first layer; weight update then combines information from the original forward inference run with these backpropagated errors to adjust the network weights in a way that makes the model more accurate.”

Alternative computing architectures, which trade complexity for efficiency, often cannot perform the information passing required by the algorithm. As a consequence, the trained parameters of the network must be obtained from an independent physics simulation of the entire hardware setup and its information processing. But creating simulations of sufficient quality can itself be challenging.  ... '