Computer Vision That Works More Like a Brain Sees More Like People Do

Towards better vision. 

ACM TECHNEWS

Computer Vision That Works More Like a Brain Sees More Like People Do

By MIT News, July 13, 2023

When deep Learning computer vision systems establish efficient ways to solve visual problems, they end up with artificial circuits that work similarly to the neural circuits that process visual information in our own brains.

Researchers made a computer vision model more robust by training it to work like a part of the brain that humans and other primates rely on for object recognition.

Credit: iStock

James DiCarlo and colleagues at the Massachusetts Institute of Technology trained an artificial neural network to function more like the human and primate brain's inferior temporal (IT) cortex to improve computer vision.

The researchers constructed a computer vision model based on neural data from primate vision-processing neurons, and tasked it to recognize objects.

DiCarlo said this made the artificial neural circuits process visual information differently.

The researchers found the biologically informed model IT layer aligned better with the IT neural data than a similarly-sized network model that lacked neural-data training.

They also discovered the neurally aligned model was more resilient against adversarial attacks for assessing computer vision and artificial intelligence systems.

From MIT News

View Full Article      

Accelerating Optical Communications with AI

 Accelerating Optical Communications with AI     By Chris Edwards

Communications of the ACM, July 2023, Vol. 66 No. 7, Pages 13-15  10.1145/3595957

Photonic computing has seen its share of research breakthroughs and deep research winters, much like the history of artificial intelligence (AI). Now, with the resurgence of AI, the huge amounts of energy today's large neural-network models need when running on electronic computers is reawakening interest in uniting the two.

More than 30 years ago, during one of the booms in research into artificial neural networks, Demetri Psaltis and colleagues at the California Institute of Technology demonstrated how techniques from holography could perform rudimentary face recognition. The team members showed they could store as many as one billion weights for a two-layer neural network using the core elements from a liquid-crystal display. Similar spatial light modulators became the foundation of several attempts to commercialize optical computing technology, including those by U.K.-based startup Optalysys, which has focused in recent years on applying the technology to accelerating homomorphic encryption to support secure remote computing.

Though some groups are using spatial light modulators for AI, they represent just one category of optical computer suitable for the job. There are also decisions as to which form of neural networks best suits optical computing. Some techniques focus on the matrix-arithmetic operations of mainstream deep learning pipelines, while others focus squarely on emulating the spike trains of biological brains.

What all the proposed systems have in common is the possibility that, by using photons to communicate and calculate, they will deliver major advantages in density and speed over systems based purely on electrical signaling. A 2021 study of inferencing based on matrix arithmetic by Mitchell Nahmias, now CTO of startup Luminous Computing, and colleagues at Princeton University argued the theoretical efficiency of AI inferencing in the optical domain could far surpass that of conventional accelerators based on existing electronics-only architectures.

The key issue for machine learning is the amount of energy needed to move data around accelerators. Electronic accelerators often employ strategies to cache as much data as possible to reduce this overhead, with major trade-offs concerning whether temporary results or weights are held in the cache depending on the model's structure. However, the energy cost of delivering photons over distances larger than the span of a single chip is far lower than it is for electrical signaling.

A second potential advantage of photonic AI comes from the ease with which it can handle complex operations in the analog domain, though the power savings achievable here are less certain than for communications. Whereas matrix arithmetic relies on highly parallelized hardware circuits for performance in conventional systems, simply passing photons through an optical component such as a Mach-Zehnder interferometer (MZi) or micro-ring resonator will perform arithmetic requiring hundreds or even thousands of logic gates in a digital circuit.

In the MZi, coherent beams of light pass through a succession of couplers and phase shifters. At each coupling point, the interference between the beams results in phase shifts that can be interpreted as part of a matrix multiplication. A 4x4 matrix operation requires just four inputs that feed into six coupling elements, with four output ports providing the result. The speed of the operations is limited only by the rate at which coherent pulses can be passed through the array.

In analog architectures, noise presents a significant hurdle. Work by numerous groups on accelerating inference operations has shown that deep neural networks can work successfully at an effective resolution of 4 bits, but hardware overhead and energy rise quickly as resolution increases. These effects may limit the practical energy advantage of photonic designs.

Estimates by Alexander Tait, assistant professor at Queen's University, Ontario, found the potential power-savings easily eroded by the practical limitations of today's optical devices. Tait calculated just 500 micro-ring resonators acting as neurons in a fully connected layout could fit onto a single 1cm2 die using early 2020s technology. But operating at 10GHz, it would require a kilowatt of power. Tait stresses the example shows the impact of the current need for heaters to tune optical properties. Scaling and design changes could bring the energy down dramatically. "The heaters are certainly a solvable problem," he says.  .... ' 

NeuralAngelo from NVIDIA via Neural Networks

Impressive, high detail transformation.   Could have used this in several past applications.  See videos at link. Architectural and other 3D vision apps.

 https://www.youtube.com/watch?v=PQMNCXR-WF8

Digital Renaissance: Neuralangelo by NVIDIA Research Reconstructs 3D

60,044 views  Jun 1, 2023

Neuralangelo, a new AI model by NVIDIA Research for 3D reconstruction using neural networks, turns 2D video clips into detailed 3D structures — generating lifelike virtual replicas of buildings, sculptures and other real-world objects. 

Like Michelangelo sculpting stunning, life-like visions from blocks of marble, Neuralangelo generates 3D structures with intricate details and textures.

Neuralangelo’s ability to translate the textures of complex materials — including roof shingles, panes of glass and smooth marble — from 2D videos to 3D assets significantly surpasses prior methods. The high fidelity makes its 3D reconstructions easier for developers and creative professionals to rapidly create usable virtual objects for their projects using footage captured by smartphones. 

Neuralangelo is one of nearly 30 projects by NVIDIA Research to be presented at the Conference on Computer Vision and Pattern Recognition (CVPR), taking place June 18-22 in Vancouver. The papers span topics including pose estimation, 3D reconstruction and video generation.

Read more: https://nvda.ws/3oF87HA

Learn more about NVIDIA Research at CVPR: https://www.nvidia.com/en-us/events/c... 

NVIDIA Research: https://www.nvidia.com/en-us/research/ 

Join the NVIDIA Developer Program: https://nvda.ws/3OhiXfl 

Read and subscribe to the NVIDIA Technical Blog: https://nvda.ws/3XHae9F

3D, Artificial Intelligence, AI, NVIDIA Research, Graphics, CVPR, CVPR2023, Neural Networks, Computer Vision  ... '

AI, Neural Nets for Boundary Design.

Applications for agriculture work here as well as ,  archeology and even city design and redesign

NEURAL NETWORKS FOR FIELD BOUNDARIES DETECTION

April 29, 2023 

Using Neural Networks for Field Boundaries Detection

Today, agriculture faces many challenges, many of which are related to climate change. At the same time, the industry has a severe impact on the climate and the environment, as it is a source of pollution and greenhouse gas emissions. 

Neural Networks Boundaries Detection

Innovative technologies, including geospatial data analytics are helping to develop solutions that reduce negative impacts on wildlife and biodiversity. Defining field boundaries is one such solution. The decision is intended to protect uncultivated land from agricultural inputs, including pesticides and fertilizers.

The technological basis for automatically determining field boundaries is AI and machine learning, one of the most critical elements is artificial neural networks. They are designed to the likeness of the structure of the human brain, and the range of their application in agriculture and other industries is quite broad. 

This technology is used to predict yields, track diseases and pests, control weeds, and more. AI helps in optimizing many farm processes, decision-making and management. Special software is required to implement machine learning methods on a farm because a massive amount of data should be processed. The development of digital technologies and precision farming leads to more and more growers turning to tools based on artificial intelligence.

Deep Learning with Artificial Neural Networks

Machine learning aims to allow machines to learn from data and extract information without being explicitly programmed. Such algorithms can analyse and interpret large amounts of data. Deep learning algorithms (a field of machine learning) are complex and are applied as practical tools for image recognition. The most popular of these algorithms are convolutional neural networks.

In the geospatial data analytics market, companies use AI and machine learning to develop their products and create valuable features. EOS Data Analytics provides AI-powered satellite imagery analyticsand uses innovations to build its software products. EOSDA Crop Monitoring is a precision farming platform that helps growers to make data-based decisions, enhance farm management and decrease the negative impact of agriculture on the environment. 

EOSDA solutions also help clients solve various prediction and classification problems due to the ability of the neural network to detect patterns. Data scientists train neural networks on large sets of images to recognize and distinguish between objects on the Earth’s surface. It enables the classification of crop types, the study of land cover, and obtaining information about soil and vegetation health from satellite imagery.

Main Stages in the Building Process of a Neural Network

The development of a neural network consists of three main stages. The first step is to create an image database to train the network. It is important to note that the fundamental step in machine learning is data collection. The process will be smooth with proper preparation, but it is possible to carry out further actions effectively. It is vital to have many high-quality images to provide the network with what it will observe during its application. Like geospatial data analysis in agriculture, this complex process brings many benefits.

Then it would help if you chose the network architecture. Models with proven effectiveness already exist and are used as a basis for further application development. The third step is network training, on which its specialization and, accordingly, the tasks to be performed will depend. The first three stages are the most time-consuming.    ... 

Neural Nets Speed Simulations

 Very interesting piece, Often the speed of complex simulations can be key for their use.  Below an introductory text, makes a good case.  Think of it as a way to insert learned knowledge into  a model.  More at the link.

Neural Networks Learn to Speed Up Simulations  By Chris Edwards

Communications of the ACM, May 2022, Vol. 65 No. 5, Pages 27-29   101145/3524015

Physical scientists and engineering research and development (R&D) teams are embracing neural networks in attempts to accelerate their simulations. From quantum mechanics to the prediction of blood flow in the body, numerous teams have reported on speedups in simulation by swapping conventional finite-element solvers for models trained on various combinations of experimental and synthetic data.

At the company's technology conference in November, Animashree Anandkumar, Nvidia's director of machine learning research and Bren Professor of Computing at the California Institute of Technology, pointed to one project the company worked on for weather forecasting. She claimed the neural network that team created could achieve results 100,000 times faster than a simulation that used traditional numerical methods to solve the partial differential equations (PDEs) on which the model relies.

Nvidia has packaged the machine learning techniques that underpin the weather-forecasting project into the Simnet software package it provides to customers. Its engineers have used the same approach to model the heat-sinks that cool the graphics processing units (GPUs) that power many other machine learning systems.

Other engineering companies are following suit. Both Ansys and Siemens Digital Industries Software are working on their own implementations to support their mechanical simulation product lines, adding to a growing body of open source initiatives such as the DeepModeling community.

A key reason for using machine learning for scientific simulations is that a collection of fully connected artificial neurons can act as a universal function approximator. Though training those neurons is computationally intensive, during the inference phase the neural network often will provide faster results than running simulators based on finite-element or numerical approximations to PDEs.

One approach to training a neural network for scientific simulation is to record experimental data and augment that with simulated data using numerical methods. For example, a simulation of the motion of a shock wave in a fluid-filled pipe might use a combination of sensor recordings and the solutions of the Bateman-Burgers equation.

The simulated data can be used to supply usable data for points where it is impossible to place a sensor to record pressure or simply to provide a higher density of data points. In principle, the machine learning model then will interpolate reasonable values for points where no data has been supplied. But the learned approximation can easily diverge from reality when checked against traditional models. The neural network likely will not learn the underlying patterns, just those that let it approximate the data points used for training.  .... ' 

Brain Like Chips

 And how might  do we apply them?

Reconfigurable Brain-Like Chips Top Deep Neural Net

By R. Colin Johnson, Commissioned by CACM Staff, April 26, 2022

Today's deep neural networks (DNNs) must be taken off-line to update their learned skills, but real brains update their skill set by reconfiguring their neurons and synapses constantly on the fly—a feat that can now be performed with reconfigurable brain-like chips using quantum materials.

Quantum materials—such as superconductors, topological insulators and even graphene—have remarkable electronic properties that manifest macroscopic quantum mechanical principles that traditional semiconductors only exhibit at the microscopic scale. Now researchers have found a way to form a nickelate (nickel-based) quantum material from the rare earth element neodymium bonded to nickel oxide, NdNiO3 (also known as NNO), with remarkable room-temperature electronic properties that make it ideal for lifelong neural learning hardware, according to a team of researchers from Purdue University, Argonne National Laboratory, the University of Illinois Chicago, Brookhaven National Laboratory, and the University of Georgia.

"The brain is continuously learning," said Purdue University professor Shriram Ramanathan. The formation of new neurons and synapses, a process known as neurogenesis, is ongoing in human brains from birth to death, while today's DNNs are routinely fixed after a lengthy training period, during which they learn a rote function. Ramanathan believes the new quantum material NNO will enable DNNs to learn continuously using neurogenesis, just like human brains.

"Neurogenesis and synaptic rewiring play a crucial role in the formation of memory, and a dynamic brain engaged in life-long learning," said Ramanathan. "Inspired by dynamic reconfiguration in the brain, we hypothesized: if we could mimic neurogenesis behaviors in electrical hardware, we can make AI machines that learn throughout their lifespan."

Funded by the U.S. Department of Energy Office of Science, the U.S. Air Force Office of Scientific Research, and the National Science Foundation (NSF), the researchers formed thin films of NNO on silicon-on-insulator (SOI) -compatible substrates to create proof-of-concept microchips that outperform traditional silicon in deep neural learning hardware. The NNO material demonstrates novel quantum properties that enable it to learn throughout its lifetime, forming new classification categories and deleting old unused ones. It seems ideal for Internet of Things devices on the network's edge, autonomous cars, and many other real-world functions where the world's raw data is in constant flux. .... ' 

Maintaining Neural Networks

 Models will degrade, how do you maintain them?  

Researchers Discover How to Predict Degradation of Neural Network

DiariDigital URV Activ@, February 16, 2022

Researchers at Spain's Universitat Rovira i Virgili (URV) have identified the theoretical underpinnings for predicting how neural networks will function and degrade over time. These findings indicate how much damage a system can endure before it will completely degrade and lose its functionality, known as the phase transition of percolation degradation. URV's Alex Arenas said, "We have been able to find this transition and we have also been able to calculate the homeostatic response [i.e., the ability to find alternatives and continue functioning] of the network." He added that a set of mathematical tools "that can be very useful not only in neuroscience but in any type of network" is now available to the scientific community.  ... ' 

Robustly Labeled Graphs

Interesting and Technical piece in the Google AI Blog. 

Robust Graph Neural Networks  in The Google Blog

Tuesday, March 8, 2022

Posted by Bryan Perozzi, Research Scientist and Qi Zhu, Research Intern, Google Research

Graph Neural Networks (GNNs) are powerful tools for leveraging graph-structured data in machine learning. Graphs are flexible data structures that can model many different kinds of relationships and have been used in diverse applications like traffic prediction, rumor and fake news detection, modeling disease spread, and understanding why molecules smell.

As is standard in machine learning (ML), GNNs assume that training samples are selected uniformly at random (i.e., are an independent and identically distributed or “IID” sample). This is easy to do with standard academic datasets, which are specifically created for research analysis and therefore have every node already labeled. However, in many real world scenarios, data comes without labels, and labeling data can be an onerous process involving skilled human raters, which makes it difficult to label all nodes. In addition, biased training data is a common issue because the act of selecting nodes for labeling is usually not IID. For example, sometimes fixed heuristics are used to select a subset of data (which shares some characteristics) for labeling, and other times, human analysts individually choose data items for labeling using complex domain knowledge.  .... '

Insights into Brain Functions

Insights into Brain Functions

STANN Reveals Insights Into How the Brain Functions

Baylor College of Medicine, Ana Maria Rodriguez, December 21, 2021

Baylor College of Medicine (BCM)'s Dr. Abul Hassan Samee and colleagues developed the Spatial Transcriptomics cell-types Assignment using Neural Networks (STANN) model to obtain novel insights into brain function. Samee said the team applied STANN and other computational techniques to brain datasets of the mouse olfactory bulb, and determined "the precise location of different cell types, whether they communicated with each other, and by which means." The researchers theorize that the brain's morphological layers contain different spatially localized clusters of cell types, and district subtypes executing location-specific functions. BCM's Dr. James Martin said STANN offers an "instruction manual" for scientists to analyze other brain regions or organs.

Full Article.

Neural Network Representations

Interesting, technical.  See link for useful supporting visuals.

How should we compare neural network representations?

Frances Ding and Jacob Steinhardt    Nov 8, 2021  Berkeley 

Cross-posted from Bounded Regret.  

To understand neural networks, researchers often use similarity metrics to measure how similar or different two neural networks are to each other. For instance, they are used to compare vision transformers to convnets [1], to understand transfer learning [2], and to explain the success of standard training practices for deep models [3]. Below is an example visualization using similarity metrics; specifically we use the popular CKA similarity metric (introduced in [4]) to compare two transformer models across different layers:

Figure 1. CKA (Centered Kernel Alignment) similarity between two networks trained identically except for random initialization. Lower values (darker colors) are more similar. CKA suggests that the two networks have similar representations.

Unfortunately, there isn’t much agreement on which particular similarity metric to use. Here’s the exact same figure, but produced using the Canonical Correlation Analysis (CCA) metric instead of CKA:

Figure 2. CCA (Canonical Correlation Analysis) similarity between the same two networks. CCA distances suggest that the two networks learn somewhat different representations, especially at later layerss.

In the literature, researchers often propose new metrics and justify them based on intuitive desiderata that were missing from previous metrics. For example, Morcos et al. motivate CCA by arguing that similarity metrics should be invariant to invertible linear transformations [5]. Kornblith et al. disagree about which invariances a similarity metric should have, and instead argue that metrics should pass an intuitive test - given two trained networks with the same architecture but different initialization, layers at the same depth should be most similar to each other - and their proposed metric, CKA, performs the best on their test [4].

Our paper, Grounding Representation Similarity with Statistical Testing, argues against this practice. To start, we show that by choosing different intuitive tests, we can make any method look good. CKA does well on a “specificity test” similar to the one proposed by Kornblith et al., but it does poorly on a “sensitivity test” that CCA shines on.

To move beyond intuitive tests, our paper provides a carefully-designed quantitative benchmark for evaluting similarity metrics. The basic idea is that a good similarity metric should correlate with the actual functionality of a neural network, which we operationalize as accuracy on a task. Why? Accuracy differences between models are a signal that the models are processing data differently, so intermediate representations must be different, and similarity metrics should notice this.

Thus, for a given pair of neural network representations, we measure both their (dis)similarity and the difference between their accuracies on some task. If these are well-correlated across many pairs of representations, we have a good similarity metric. Of course, a perfect correlation with accuracy on a particular task also isn’t what we’re hoping for, since metrics should capture many important differences between models, not just one. A good similarity metric is one that gets generally high correlations across a couple of functionalities.

We assess functionality with a range of tasks. For a concrete example, one subtask in our benchmark builds off the observation that BERT language models finetuned with different random seeds will have nearly identical in-distribution accuracy, but widely varying out-of-distribution accuracy (for example, ranging from 0 to 60% on the HANS dataset [6]). Given two robust models, a similarity metric should rate them as similar, and given one robust and one non-robust model, a metric should rate them as dissimilar. Thus we take 100 such BERT models and evaluate whether (dis)similarity between each pair of model representations correlates with their difference in OOD accuracy.  ..... ' 

Saw Some Examples of Tunable and more Efficient Models for IOT

 Cutting 'Edge': A Tunable Neural Network Framework Towards Compact, Efficient Models

Tokyo Institute of Technology News (Japan)    August 23, 2021

A sparse convolutional neural network (CNN) framework and training algorithms developed by researchers at Japan's Tokyo Institute of Technology (Tokyo Tech) can seamlessly integrate CNN models on low-power edge devices. The 40-nanometer sparse CNN chip yields high accuracy and efficiency through a Cartesian-product multiply and accumulate (MAC) array and pipelined activation aligners that spatially shift activations onto a regular Cartesian MAC array. Tokyo Tech's Kota Ando said, "Regular and dense computations on a parallel computational array are more efficient than irregular or sparse ones. With our novel architecture employing MAC array and activation aligners, we were able to achieve dense computing of sparse convolution." ... 

Dealing with Autism's Difficulty with Facial Expression

 Analysis of pattern 

ACM TECHNEWS

Neural Network Model Unravels People with Autism's Difficulty with Facial Expressions By News-Medical.net, August 11, 2021

Researchers at Japan's Tohoku University developed an artificial neural network model that can help explain the difficulty people with autism spectrum disorder have in interpreting facial expressions.

The model takes into account predictive processing theory, which states that the brain predicts the next sensory stimulus and adapts using such sensory information as facial expressions to reduce errors in its predictions. The model learned to predict the movement of parts of the face using videos of facial expressions and was able to generalize facial expressions not provided during training.

However, the model's ability to generalize decreased with the heterogeneity of activity in the neural population, restraining emotional cluster formation in higher-level neurons, similar to what occurs with autism spectrum disorder.

The researchers describe their work in "Neural Network Modeling of Altered Facial Expression Recognition in Autism Spectrum Disorders based on Predictive Processing Framework," published in Scientific Reports.

"The study will help advance developing appropriate intervention methods for people who find it difficult to identify emotions," says Yuta Takahashi of the Department of Psychiatry at Tohoku University Hospital.

From News-Medical.net

View Full Article  

Bio Neurons versus Computational

When we first learned of the use of the patterns of brain neurons to potentially use as reasoning devices, we took a course from actual neuroscientists.  We very quickly learned that human bio neurons were very much more complex that neurons in our feeble  'neural networks'.   Always been intrigued by the concept, so how could they be more useful 'reasoners"? Tried to augment  our nets with this new complexity.  Here is another case why they are very different.  We can ask ourselves, can we use these aspects of neurons to improve reasoning?  Note the indication that timing matters.  What embedded information  could make us learn faster?   Still unclear, 

Neurons Unexpectedly Encode Information in the Timing of Their Firing

Elena Renken  Quanta Mag     Contributing Writer

A temporal pattern of activity observed in human brains for the first time may explain how we can learn so quickly.

For decades, neuroscientists have treated the brain somewhat like a Geiger counter: The rate at which neurons fire is taken as a measure of activity, just as a Geiger counter’s click rate indicates the strength of radiation. But new research suggests the brain may be more like a musical instrument. When you play the piano, how often you hit the keys matters, but the precise timing of the notes is also essential to the melody.

“It’s really important not just how many [neuron activations] occur, but when exactly they occur,” said Joshua Jacobs, a neuroscientist and biomedical engineer at Columbia University who reported new evidence for this claim last month in Cell.  ... ' 

Turing Lecture on Deep Learning

 Quite good, relatively non technical.   Worth a look.

Turing Lecture

Deep Learning for AI

By Yoshua Bengio, Yann Lecun, Geoffrey Hinton    from ACM

Communications of the ACM, July 2021, Vol. 64 No. 7, Pages 58-65   10.1145/3448250

Yoshua Bengio, Yann LeCun, and Geoffrey Hinton are recipients of the 2018 ACM A.M. Turing Award for breakthroughs that have made deep neural networks a critical component of computing.

Research on artificial neural networks was motivated by the observation that human intelligence emerges from highly parallel networks of relatively simple, non-linear neurons that learn by adjusting the strengths of their connections. This observation leads to a central computational question: How is it possible for networks of this general kind to learn the complicated internal representations that are required for difficult tasks such as recognizing objects or understanding language? Deep learning seeks to answer this question by using many layers of activity vectors as representations and learning the connection strengths that give rise to these vectors by following the stochastic gradient of an objective function that measures how well the network is performing. It is very surprising that such a conceptually simple approach has proved to be so effective when applied to large training sets using huge amounts of computation and it appears that a key ingredient is depth: shallow networks simply do not work as well.

 We reviewed the basic concepts and some of the breakthrough achievements of deep learning several years ago.  Here we briefly describe the origins of deep learning, describe a few of the more recent advances, and discuss some of the future challenges. These challenges include learning with little or no external supervision, coping with test examples that come from a different distribution than the training examples, and using the deep learning approach for tasks that humans solve by using a deliberate sequence of steps which we attend to consciously—tasks that Kahneman56 calls system 2 tasks as opposed to system 1 tasks like object recognition or immediate natural language understanding, which generally feel effortless.   ... " 

Faster Neural Networks

Speed for solutions and speed for alternative maintenance updates of models.

Latest Neural Nets Solve World’s Hardest Equations Faster Than Ever Before

Two new approaches allow deep neural networks to solve entire families of partial differential equations, making it easier to model complicated systems and to do so orders of magnitude faster.

Partial differential equations, such as the ones governing the behavior of flowing fluids, are notoriously difficult to solve. Neural nets may be the answer.

Alexander Dracott for Quanta Magazine,  Anil Ananthaswamy

In high school physics, we learn about Newton’s second law of motion — force equals mass times acceleration — through simple examples of a single force (say, gravity) acting on an object of some mass. In an idealized scenario where the only independent variable is time, the second law is effectively an “ordinary differential equation,” which one can solve to calculate the position or velocity of the object at any moment in time.

But in more involved situations, multiple forces act on the many moving parts of an intricate system over time. To model a passenger jet scything through the air, a seismic wave rippling through Earth or the spread of a disease through a population — to say nothing of the interactions of fundamental forces and particles — engineers, scientists and mathematicians resort to “partial differential equations” (PDEs) that can describe complex phenomena involving many independent variables.

The problem is that partial differential equations — as essential and ubiquitous as they are in science and engineering — are notoriously difficult to solve, if they can be solved at all. Approximate methods can be used to solve them, but even then, it can take millions of CPU hours to sort out complicated PDEs. As the problems we tackle become increasingly complex, from designing better rocket engines to modeling climate change, we’ll need better, more efficient ways to solve these equations.

Now researchers have built new kinds of artificial neural networks that can approximate solutions to partial differential equations orders of magnitude faster than traditional PDE solvers. And once trained, the new neural nets can solve not just a single PDE but an entire family of them without retraining.

To achieve these results, the scientists are taking deep neural networks — the modern face of artificial intelligence — into new territory. Normally, neural nets map, or convert data, from one finite-dimensional space (say, the pixel values of images) to another finite-dimensional space (say, the numbers that classify the images, like 1 for cat and 2 for dog). But the new deep nets do something dramatically different. They “map between an infinite-dimensional space and an infinite-dimensional space,” said the mathematician Siddhartha Mishra of the Swiss Federal Institute of Technology Zurich, who didn’t design the deep nets but has been analyzing them mathematically.  ... '

Machine Learning Security

 Long look at the topic covering many aspects of the idea. 

Machine learning security needs new perspectives and incentives  By Ben Dickson  in BDTechTalks

At this year’s International Conference on Learning Representations (ICLR), a team of researchers from the University of Maryland presented an attack technique meant to slow down deep learning models that have been optimized for fast and sensitive operations. The attack, aptly named DeepSloth, targets “adaptive deep neural networks,” a range of deep learning architectures that cut down computations to speed up processing.

Recent years have seen growing interest in the security of machine learning and deep learning, and there are numerous papers and techniques on hacking and defending neural networks. But one thing made DeepSloth particularly interesting: The researchers at the University of Maryland were presenting a vulnerability in a technique they themselves had developed two years earlier.

In some ways, the story of DeepSloth illustrates the challenges that the machine learning community faces. On the one hand, many researchers and developers are racing to make deep learning available to different applications. On the other hand, their innovations cause new challenges of their own. And they need to actively seek out and address those challenges before they cause irreparable damage.  ...  '

AI for Personalized Cancer Vaccines

 Seems quite a big move forward.    Will we be able to learn which amino acid sequences will work best to fight cancer? 

Using Machine Learning to Develop Personalized Cancer Vaccines

University of Waterloo Cheriton School of Computer Science (Canada)  via ACM

Researchers at Canada's University of Waterloo Cheriton School of Computer Science are applying machine learning to identify tumor-specific neoantigens, which could lead to personalized cancer vaccines. Cheriton's Hieu Tran said the team used a model similar to natural language processing to ascertain neoantigens' amino acid sequences based on one-letter amino acid codes. The researchers used the DeepNovo recurrent neural network to predict amino acid sequences, which Tran said expanded the predicted immunopeptidomes of five melanoma patients by 5% to 15%, based solely on data from mass spectrometry—and personalized the neoantigens to each patient.

Extending Insight from Neural Networks

 Some thoughts about how trained networks can be used to further analyse chemical structure.

Opening the Black Box of Neural Networks

Pacific Northwest National Laboratory, Allan Brettman

Pacific Northwest National Laboratory (PNNL) researchers used deep learning neural networks to model water molecule interactions, unearthing data about hydrogen bonds and structural patterns. The PNNL team employed 500,000 water clusters from a database of more than 5 million water cluster minima to train a neural network, relying on graph theory to extract structural patterns of the molecules' aggregation. The method provides additional analysis after the network has been trained, allowing comparison between measurements of the water cluster networks' structural traits and the predicted neural network, enhancing the network's understanding in subsequent analyses. PNNL's Jenna Pope said, "If you were able to train a neural network, that neural network would be able to do computational chemistry on larger systems. And then you could make similar insights in computational chemistry about chemical structure or hydrogen bonding or the molecules’ response to temperature changes.”

Optimizing Neural Networks on a Brain-Inspired Computer

Considerable challenge.   Will the biomimicry provide enough value to adjust our methods to what we now know about the brain?

How to Optimize Neural Networks on a Brain-Inspired Computer
HPCwire
July 28, 2020

A study by scientists at Germany’s Heidelberg University and the Max Planck Institute for Dynamics and Self-Organization reveals how "critical states" can be used to optimize artificial neural networks running on brain-inspired neuromorphic hardware. Critical states are the points at which systems can quickly and fundamentally change their overall characteristics. Although they are widely assumed to be optimal for computation in recurrent neural networks, the researchers found that criticality is not beneficial for every task. In an experiment performed on a prototype of the analog neuromorphic BrainScales-2 chip, the researchers found that changing input strength permits easy adjustment of the distance to criticality. They also showed a clear relationship between criticality and task performance, finding that only complex, memory-intensive tasks benefited from criticality. ... "

Sensors Hacking for Neural Net Activity

Thinking this. Another use of sensors to examine activity, and potentially determine 'secrets'.

Preventing AI From Divulging Its Own Secrets
IEEE Spectrum
Jeremy Hsu

North Carolina State University (NC State) researchers have demonstrated the first countermeasure for shielding artificial intelligence from differential power analysis attacks. Such attacks involve hackers exploiting neural networks' power signatures to reverse-engineer the inner mechanisms of computer chips that are running those networks. The attack relies on adversaries physically accessing devices in order to measure their power signature, or analyze output electromagnetic radiation. Attackers can repeatedly have the neural network run specific computational tasks with known input data, and eventually determine power patterns associated with the secret weight values. The countermeasure is adapted from a masking technique; explains NC State's Aydin Aysu, "We use the secure multi-part computations and randomize all intermediate computations to mitigate the attack." ... '