Limitations of Deep Learning

About limitations of deep learning problems with higher dimensional data. Technical.

Geometric Deep Learning Advances Data Science   By Samuel Greengard

Communications of the ACM, January 2021, Vol. 64 No. 1, Pages 13-15

10.1145/3433951

Deep learning has transformed numerous fields. In tackling complex tasks such as speech recognition, computer vision, predictive analytics, and even medical diagnostics, these systems consistently achieve—and even exceed—human-level performance. Yet deep learning, an umbrella term for machine learning systems based primarily on artificial neural networks, is not without its limitations. As data becomes non-planar and more complex, the ability of the machine to identify patterns declines markedly.

At the heart of the issue are the basic mechanics of deep learning frameworks. "With just two layers, a simple perceptron-type network can approximate any smooth function to any desired accuracy, a property called 'universal approximation'," points out Michael Bronstein, a professor in the Department of Computing at Imperial College London in the U.K. "Yet, multilayer perceptrons display very weak inductive bias, in the sense that they assume very little about the structure of the problem at hand and fail miserably if applied to high-dimensional data."  .... '