The Brain Rotates Memories

How might we utilize this this? Beyond just tagging memories with a time stamp?

The Brain ‘Rotates’ Memories to Save Them From New Sensations  By Jordana Cepelewicz,  Senior Writer, Quanta Magazine

Some populations of neurons simultaneously process sensations and memories. New work shows how the brain rotates those representations to prevent interference.

During every waking moment, we humans and other animals have to balance on the edge of our awareness of past and present. We must absorb new sensory information about the world around us while holding on to short-term memories of earlier observations or events. Our ability to make sense of our surroundings, to learn, to act and to think all depend on constant, nimble interactions between perception and memory.

But to accomplish this, the brain has to keep the two distinct; otherwise, incoming data streams could interfere with representations of previous stimuli and cause us to overwrite or misinterpret important contextual information. Compounding that challenge, a body of research hints that the brain does not neatly partition short-term memory function exclusively into higher cognitive areas like the prefrontal cortex. Instead, the sensory regions and other lower cortical centers that detect and represent experiences may also encode and store memories of them. And yet those memories can’t be allowed to intrude on our perception of the present, or to be randomly rewritten by new experiences.

A paper published recently in Nature Neuroscience  may finally explain how the brain’s protective buffer works. A pair of researchers showed that, to represent current and past stimuli simultaneously without mutual interference, the brain essentially “rotates” sensory information to encode it as a memory. The two orthogonal representations can then draw from overlapping neural activity without intruding on each other. The details of this mechanism may help to resolve several long-standing debates about memory processing.  ...' 

Seeing the way we do

New ways of perceptive seeing, now with Texture and Shape

GLOM: Teaching Computers to See the Way(s) We Do  By John Delaney,  Commissioned by CACM Staff   September 14, 2021

At the virtual Collision technology  earlier this year deep learning pioneer Geoffrey Hinton explained how he conceived of a new type of neural network that, he said, would be able to perceive things the way people do.

Hinton, an emeritus distinguished professor in the department of computer science of the Faculty of Arts & Science at Canada's University of Toronto, and also an Engineering Fellow at Google, is responsible for some of the biggest breakthroughs in deep learning and neural networks. He was honored as co-recipient of the 2018 ACM A.M. Turing Award, along with Yoshua Bengio and Yann LeCun, for conceptual and engineering breakthroughs that have made deep neural networks a critical component of modern computing.

In Hinton's Collision talk, he pointed out that the representations used by most neural networks performing object classification are produced by convolutional neural networks, which work well at classifying objects such as images or words, even winning competitions such as the ImageNet Large Scale Visual Recognition Challenge, but they perceive images in a very different way than people do, which can sometimes lead to "crazy errors."

"They use lots of texture information, which people are insensitive to," Hinton said, "but they fail to use a lot of shape information, which people are very sensitive to."  .... '

AI versus Human Perception Performance

Interesting challenge because we always emphasize that we need to be able to measure something to use/improve it.   Which leads to our design of the measurement system. My response is that you can build measurement systems for particular use contexts.    Reading the noted paper which emphasises " .. compare deep neural networks and the human vision system ... "   which is a very broad statement for the problem.  Like the discussion here. 

Why AI and human perception are too complex to be compared   By Ben Dickson in Tnw
Human-level performance. Human-level accuracy. Those are terms you hear a lot from companies developing artificial intelligence systems, whether it’s facial recognition, object detection, or question answering. And to their credit, the recent years have seen many great products powered by AI algorithms, mostly thanks to advances in machine learning and deep learning.

But many of these comparisons only take into account the end-result of testing the deep learning algorithms on limited data sets. This approach can create false expectations about AI systems and yield dangerous results when they are entrusted with critical tasks.

In a recent study, a group of researchers from various German organizations and universities has highlighted the challenges of evaluating the performance of deep learning in processing visual data. In their paper, titled, “The Notorious Difficulty of Comparing Human and Machine Perception,” the researchers highlight the problems in current methods that compare deep neural networks and the human vision system.  ... "

In their research, the scientist conducted a series of experiments that dig beneath the surface of deep learning results and compare them to the workings of the human visual system. Their findings are a reminder that we must be cautious when comparing AI to humans, even if it shows equal or better performance on the same task.  ... "

Expectation Influences Perception

Known for some time.   Now how do we best  make use of this in AI interactions?   Can our brains be primed with signals to make them ready for interaction?  Can we measure what is needed using Bayesian methods?

How expectation influences perception

Neuroscientists find brain activity patterns that encode our beliefs and affect how we interpret the world around us.

MIT neuroscientists have identified patterns of brain activity that underlie our ability to interpret sensory input based on our expectations and past experiences.

By Anne Trafton | MIT News Office 

For decades, research has shown that our perception of the world is influenced by our expectations. These expectations, also called “prior beliefs,” help us make sense of what we are perceiving in the present, based on similar past experiences. Consider, for instance, how a shadow on a patient’s X-ray image, easily missed by a less experienced intern, jumps out at a seasoned physician. The physician’s prior experience helps her arrive at the most probable interpretation of a weak signal.

The process of combining prior knowledge with uncertain evidence is known as Bayesian integration and is believed to widely impact our perceptions, thoughts, and actions. Now, MIT neuroscientists have discovered distinctive brain signals that encode these prior beliefs. They have also found how the brain uses these signals to make judicious decisions in the face of uncertainty.

“How these beliefs come to influence brain activity and bias our perceptions was the question we wanted to answer,” says Mehrdad Jazayeri, the Robert A. Swanson Career Development Professor of Life Sciences, a member of MIT’s McGovern Institute for Brain Research, and the senior author of the study. .... "

Cutting Out of Stocks: Reality and Perception

A favorite topic.   Availability is essential to support marketing, else whats a shelf placement for?   Thus too while online works well.

Stores have cut out-of-stocks. Why don’t consumers know that? in Retailwire  by Andrew Blatherwick

Research shows that online retailers have significantly lower service levels when compared to traditional retailers that operate warehouses and stores. Still, the perception persists that Amazon.com can quickly and efficiently deliver any product to your doorstep, while brick-and-mortar retailers constantly run out of stock on items. Why the disconnect and, more importantly for traditional merchants, how do you overcome that perception?

The perception disconnect is rooted in the fact that customers have grown accustomed to high availability as retailers have improved their supply chains and technology over the last 20 years. For physical retailers, an out-of-stock is as clear as the hole on the shelf. It’s tangible, and shoppers experience a time delay in acquiring the out-of-stock item — whether by visiting a competitor’s store or placing an order online.

While ecommerce customers may be unimpressed by poor availability, they are far less inconvenienced because switching to a competing online retailer takes just a moment’s time and a few mouse clicks. The negative impact on long-term loyalty in the ecommerce scenario is lower because it’s easier to shop multiple online retailers than multiple traditional retailers.  ... ."

Faster and Smaller Neural Nets

Fascinating development.   Smaller usually means faster with training nets.   Smaller can also mean easier implementation at the IOT edge.  Now will they be as accurate?   It is all about more efficient perception.  Closer to human.   Technical piece in Google AI.  Intro below, more at the link:


MorphNet: Towards Faster and Smaller Neural Networks  in Google AI.   Wednesday, April 17, 2019

Posted by Andrew Poon, Senior Software Engineer and Dhyanesh Narayanan, Product Manager, Google AI Perception 

Deep neural networks (DNNs) have demonstrated remarkable effectiveness in solving hard problems of practical relevance such as image classification, text recognition and speech transcription. However, designing a suitable DNN architecture for a given problem continues to be a challenging task. Given the large search space of possible architectures, designing a network from scratch for your specific application can be prohibitively expensive in terms of computational resources and time. Approaches such as Neural Architecture Search and AdaNet use machine learning to search the design space in order to find improved architectures. An alternative is to take an existing architecture for a similar problem and, in one shot, optimize it for the task at hand.  .... " 

Gestures, Perception and Meaning

Long followed gesture as alternate interface, is it far beyond that?

How the Brain Links Gestures, Perception and Meaning in QuantaMagazine

Neuroscience has found that gestures are not merely important as tools of expression but as guides of cognition and perception.  By Raleigh McElvery    Contributing Writer

Remember the last time someone flipped you the bird? Whether or not that single finger was accompanied by spoken obscenities, you knew exactly what it meant.

The conversion from movement into meaning is both seamless and direct, because we are endowed with the capacity to speak without talking and comprehend without hearing. We can direct attention by pointing, enhance narrative by miming, emphasize with rhythmic strokes and convey entire responses with a simple combination of fingers.

The tendency to supplement communication with motion is universal, though the nuances of delivery vary slightly. In Papua New Guinea, for instance, people point with their noses and heads, while in Laos they sometimes use their lips. In Ghana, left-handed pointing can be taboo, while in Greece or Turkey forming a ring with your index finger and thumb to indicate everything is A-OK could get you in trouble ..... "

Recreating Seen Images

A long term goal, and have seen many such claims as to the precision to which this might be possible. Looked at it for product recognition at one time, but was then very inadequate.   The use of EEG in particular is interesting because it is much researched for applications in brain research.

This A.I. literally reads your mind to re-create images of the faces you see  in DigitalTrend

Google’s artificial intelligence technology may sometimes seem like it’s reading our mind, but neuroscientists at Canada’s University of Toronto Scarborough are literally using A.I. for that very purpose — by reconstructing images based on brain perception using data gathered by electroencephalography (EEG).

In a test, subjects were hooked up to EEG brainwave-reading equipment and shown images of faces. While this happened, their brain activity was recorded and then analyzed using machine learning algorithms. Impressively, the researchers were able to use this information to digitally re-create the face image stored in the person’s mind. Unlike basic shapes, being able to re-create faces involves a high level of fine-grained visual detail, showcasing a high level of sophistication for the technology.

While this isn’t the first time that A.I. has been used to read people’s minds, it’s the first time this has been achieved using EEG data. .... " 

Vision, Real and Virtual

There have been many years of research about how vision works, but not enough about how this effects virtual and augmented reality.    Ultimately it drives perception.  An extended look at the topic:

 Perception is Reality -- and Virtual Reality  By Rebecca Guenard
Virtual reality is here. In fact, it’s everywhere. Beyond video games, it is helping therapists to treat PTSD, allowing medical students to do virtual operations, and letting engineers test vehicle safety before the car is built. 

But in order to provide a genuine experience through virtual reality (VR), manufacturers must first understand how vision works in the real world. And understanding a complex system like vision requires advances on multiple fronts. At Penn Arts and Sciences, psychologists and physicists are looking more closely at the basics of how we see. ... "

Human Perception and Data Visualization

A look at what we know about the topic.  In Medium.  Research and art.  Nice outline of what we think we know and where to find out more ...

" ... There is visualization in practice and there is visualization in theory and research. Each should inform the other, but it typically doesn’t happen that way. Kennedy Elliot, a graphics editor at the Washington Post, provides a rundown of one branch from the research side of things: human perception. There are quite a few studies. ... "