Reading Minds with AI: Researchers Translate Brain Waves to Images

Quite a considerable capability suggested.   in KDNuggets

Reading Minds with AI: Researchers Translate Brain Waves to Images

Two researchers from Osaka University were able to reconstruct highly accurate images from human brain activity obtained by fMRI. Read this article if you are curious to find out what all the hype is about.

By Kanwal Mehreen, Software Engineering Student at NUST on March 28, 2023 in Artificial Intelligence    

Decoding Canine Cognition

 New ways to link to animal toughts? 

Decoding Canine Cognition

By Emory University, September 15, 2022

Emory University researchers have decoded visual images from a dog's brain.

The researchers captured functional magnetic resonance imaging (fMRI) neural data of two dogs as they viewed videos for 90 minutes in total, then analyzed their brain-data patterns using a machine learning algorithm.

They time-stamped the video data into classifiers, including object-based classifiers and action-based classifiers; the time stamps mapped the brain data onto the classifiers. The Ivis algorithm decoded the action classifiers, but not the object classifiers, with 75% to 88% accuracy.

"We showed that we can monitor the activity in a dog's brain while it is watching a video and, to at least a limited degree, reconstruct what it is looking at," explained Emory's Gregory Berns.

From Emory University

Full article:  

Brain Activity Turned into Images

 AI, Images, MRI

‘Mind-Reading’ Technology Can Turn Brain Activity Into Images   By Adrianna Nine on August 26, 2022 at 8:37 am

Researchers at Radboud University in the Netherlands have developed technology that can “read minds” by turning a person’s neurological activity into stunningly accurate pictures.

The system, devised by neurologists, AI researchers, and cognitive scientists at Radboud University, combines AI with medical imaging techniques. It begins with a more sophisticated version of the magnetic resonance imaging (MRI) scanner called a functional magnetic resonance imaging (fMRI) scanner. While a conventional MRI machine facilitates imaging of a person’s anatomy to diagnose trauma or disease, an fMRI machine detects tiny changes in metabolic function. This includes neuron activity and the minuscule changes in blood flow within the brain.  ... ' 

Neuroimaging Data with Varying Results

We examined fMRI for possible uses in neuromarketing applications, and also found such variation in analysis results.   Standards in the analysis approach are important.  Another general caution for the results you achieve, even with very large databases.

Neuroimaging Results Altered by Varying Analysis Pipelines
Nature

A survey of neuroimaging studies found that nearly every study used a different analysis pipeline, and the analytical choices of individual researchers significantly impacted findings gleaned from a functional magnetic resonance imaging (fMRI) dataset. The team provided the same dataset to 70 independent research groups and asked them to test nine hypotheses, each of which asserted that activity in a specific brain region correlated with a specific task feature. There were considerable variations between each team's results, even when their underlying maps were highly correlated. The finding highlights the potential consequences of a lack of standardized pipelines for processing complex data.   ... " 

Dopamine and Motivation Scan

We spend much time trying to understand how fmri scans could be used to judge potential engagement with product. Could this new Dopamine 'scan' be used to understand this more effectively?

 How dopamine drives brain activity
A specialized MRI sensor reveals the neurotransmitter’s influence on neural activity throughout the brain.

By Anne Trafton | MIT News Office

Using a specialized magnetic resonance imaging (MRI) sensor, MIT neuroscientists have discovered how dopamine released deep within the brain influences both nearby and distant brain regions.

Dopamine plays many roles in the brain, most notably related to movement, motivation, and reinforcement of behavior. However, until now it has been difficult to study precisely how a flood of dopamine affects neural activity throughout the brain. Using their new technique, the MIT team found that dopamine appears to exert significant effects in two regions of the brain’s cortex, including the motor cortex.

“There has been a lot of work on the immediate cellular consequences of dopamine release, but here what we’re looking at are the consequences of what dopamine is doing on a more brain-wide level,” says Alan Jasanoff, an MIT professor of biological engineering, brain and cognitive sciences, and nuclear science and engineering. Jasanoff is also an associate member of MIT’s McGovern Institute for Brain Research and the senior author of the study.

The MIT team found that in addition to the motor cortex, the remote brain area most affected by dopamine is the insular cortex. This region is critical for many cognitive functions related to perception of the body’s internal states, including physical and emotional states.

MIT postdoc Nan Li is the lead author of the study, which appears today in Nature.   ... "

AI Studying Human Learning

Seen the basic idea proposed before, and not disssimilar to neuromarketing, but fMRI still messy to implement in general.

AI could study your brain to help teachers improve their courses
Machine learning can determine if you understand a concept.

By Jon Fingas, @jonfingas in Engadget
1h ago in Medicine

Teachers don't always know how well their methods work. They can ask questions and hand out tests, of course, but it's not always clear who's at fault if the message doesn't get through. AI might do the trick before long, though. Dartmouth College researchers have produced a machine learning algorithm that measures activity across your brain to determine how well you understand a given concept.

The team started out by having rookie and intermediate engineering students both take standard tests as well as answer questions about pictures while sitting in an fMRI scanner. From there, they had the algorithm generate "neural scores" that could predict a student's performance. The more certain parts of the brain lit up, the easier it was to tell whether or not a student grasped the concepts at play.    .... " 

What can fMRI Do?

Useful for an understanding of the challenges of reading minds:

A neuroscientist explains the limits and possibilities of using technology to read our thoughts
Brain activity doesn’t tell us what someone is experiencing
By Angela Chen@chengela

” Poldrack is a neuroscientist at Stanford University and the author of The New Mind Readers: What Neuroimaging Can and Cannot Reveal about Our Thoughts (out now from Princeton University Press). His research focuses on what we can learn from brain imagining techniques such as fMRI, which measures blood activity in the brain as a proxy for brain activity. And one of the clearest conclusions, he writes, is that activity in a particular brain region doesn’t actually tell us what the person is experiencing.

The Verge spoke to Poldrack about the limits and possibilities of fMRI, the fallacies that people commit in interpreting its results, and the limits of its widespread use. This interview has been lightly edited for clarity.  .... " 

Algorithm Reads Brain Reaction to Music

We experimented with fMRI to attempt to discern consumer reactions to product.    So assume this is a measure of the reaction of people to different kinds of music.  How would that end in their behavior to listening or buying or engaging with background music?

Algorithm Allows for Potential 'Brain-Reading' 
Digital Journal via the ACM
By Tim Sandle

Researchers at the D'Or Institute for Research and Education in Brazil have developed a machine learning algorithm capable of identifying pieces of music from functional magnetic resonance imaging (fMRI) scans of the listener. The team first mapped brain responses triggered by listening to the music, and then used the collected information to identify novel musical pieces based on fMRI imaging data alone. FMRI visualizes cerebral blood flow and neuronal activation, because when an area of the brain is in use, blood flow to that region increases. The implication of the research is that by interpreting the right mapping of musical features to the brain, scientists can predict and decode any unique musical piece. The researchers say the model was based on analyzing six participants who listened to 40 distinct pieces of music. Through this method, the algorithm encoded the listeners' fMRI responses for individual pieces of music, evaluating specific features such as tonality, dynamics, rhythm, and timbre.  ... " 

First Brain MRI, 30 Years Ago

Heady Times: This Scientist Took The First Brain Selfie And Helped Revolutionize Medical Imaging
Early one October morning 30 years ago, GE scientist John Schenck was lying on a makeshift platform inside a GE lab in upstate New York. The itself lab was put together with special non-magnetic nails because surrounding his body was a large magnet, 30,000 times stronger than the Earth’s magnetic field. Standing at his side were a handful of colleagues and a nurse. They were there to peer inside Schenck’s head and take the first magnetic resonance scan (MRI) of the brain.

Schenck is one of the GE scientists whose work is featured in Breakthrough, the new six-part science TV series developed by GE and National Geographic Channel. The episode, titled Decoding the Brain and directed by Brett Rattner, aired last Sunday. ... " 

Includes more details and Video link of program

More on the MRI tag from GE.

Brain Marketing

In the HBR:  Using the activity of the brain to understand purchasing and marketing decisions.   We called this neuromarketing, but that term is never used in the article.   " ... Brain Marketing: Is the Product Worth the Price?  Are consumers more likely to buy if they see the price before the product, or vice versa? Uma Karmarkar and colleagues scan the brains of shoppers to find out. ... " .  

The approaches described uses fMRI, while today most experimentation by retailers and manufacturers use very different electroencephalogram methods.

" ... By the time you decided to buy a product, you knew both what you were buying and how much it cost. But was your decision affected by whether you saw the price or the product first? That's the question at the heart of new experimental research that uses neuroscience tools to shed light on how our brains make purchasing decisions. ... " 

Brainwashed, the Overreach of Neuroscience

Just received, Brainwashed: The Seductive Appeal of Mindless Neuroscience  by  Sally Satel  and Scott O. Lilienfeld  I was interviewed for this book and read an early version.  Will read and remark on in full soon.   Their description:  " ... In recent years, the advent of MRI technology seems to have unlocked the secrets of the human mind, revealing the sources of our deepest desires, intentions, and fears. As renowned psychiatrist and scholar Sally Satel and psychologist Scott O. Lilienfeld demonstrate in Brainwashed, however, the explanatory power of brain scans in particular and neuroscience more generally has been vastly overestimated. Although acknowledging its tremendous potential, the authors argue that the overzealous application of the burgeoning field of brain science has put innocent people in jail, prevented addicts from healing themselves, and undermined notions of free will and responsibility.  A provocative challenge to the use and abuse of a seductive science, Brainwashed offers an essential corrective to determinist explanations of human behavior.... " 

Updated: See also a balanced WSJ review.

Predicting Images in Dreams

In the Smithsonian Magazine.  Apparently we can now predict to some degree what people are seeing with a combination of fMRI and EEG sensors.  I observed a number of fMRI sessions that sought to determine consumer reactions to commercials.  Now can we reverse adapt the dreaming idea to load people with dreams of our products?  Hopefully not.  " ... Although it’s only capable of relatively crude predictions, the system demonstrates something surprising: Our dreams might seem like subjective, private experiences, but they produce objective, consistent pieces of data that can be analyzed by others. The researchers say this work could be an initial foray into scientific dream analysis, eventually allowing more sophisticated dream interpretation during deeper stages of sleep.... " 

True Impact Neuromarketing

Recently connected with Diana Lucaci of True Impact Marketing in Toronto. A Canadian firm that does neuromarketing studies.   I was impressed with their list of clients and analytical perception of the problem.

See also, their blog for examples.

" ... Our company is a leading Neuromarketing firm in Canada, with two main businesses segments: Strategy and Research. Neuromarketing Strategy involves alignment with your marketing and corporate objectives, identifying the gap in customer understanding, and delivering actionable insights of customer emotional engagement.   Neuromarketing Research is conducted with a specific, target audience and studies communications that appeal to any of the 5 senses: sight, sound, smell, taste, touch. Research grade equipment and protocols are used to ensure the highest level of ethical standards. Our technologies of choice include functional Magnetic Resonance Imaging (fMRI), Electroencephalogram (EEG) and Eye-Tracking.... " 

Twenty Year Look at fMRI

A view of the only twenty year history of fMIRI scanning of the brain.

Mindsign and 3D Ads

In ReadWriteWeb:Mindsign uses fMRI to analyze Cooliris ads with neuromarketing methods. So what do they get from it?

AdWeek Exploration of Neuromarketing

A good general exploration of neuromarketing by AdWeek. Primarly about fMRI and which supports its further exploration and combining with classical market research techniques.

Brandwashed Read and Reviewed

Just completed a review copy of : Brandwashed: Tricks Companies Use to Manipulate Our Minds and Persuade us to Buy by Martin Lindstrom.   To be available September 20.  Have enjoyed Lindstrom's earlier books such as Buyology: Truth and Lies About Why We Buy.

I will be posting some review snippets of this book here.  I highly recommend every marketer and consumer read it.  It is very much an updated version of Vance Packard's 1958 book: Hidden Persuaders.  Completely updated and containing lots of interesting examples of how marketing manipulates.   In many, but not all cases, he names specific companies and agencies.  Of course today, unlike in the 50s, we have many more marketing channels than just  TV, Radio, Billboards and Print.  The Internet is now here as well.  Shoppers are armed with their own mobile devices, and data about their activities is being recorded at a furious rate.

People who work in marketing or merchandising will know many of the examples mentioned, but I found some I had not heard of.  Lindstrom also commissions some specific studies for the book, such as in the final chapter where he constructs a test of human influencers, using the Morgenson family.   He also mentions  commissioned studies using brain study methods like fMRI.

Since I worked with the design and use of  mock retail innovation centers,  I will start by quoting the book on just such a concept:

" ... Very few people know this, but most major consumer goods companies, including Unilever, Kraft, Pepsico and Coca-Cola have set up 'fake supermarkets' ...  They stock the shelves with their own products and those of their competitors, then late at night  ... they invite people to come and shop.  While they are browsing the aisle, cameras and brain-scanning equipment are measuring what happens in real time while they select and reject various brands and items.  Not unlike in the film  'Minority Report', these supermarkets generally have a control room lined with TV screens on which reps can actually measure the changes in consumers brain waves as they encounter different positioning of products.  Based on this data, the company develops what in the business is called a Planogram, a model showing where each product should be placed to generate the highest sales .... "  

This is mostly correct, and looks ahead as we further seek to understand the conscious and especially non-conscious interactions of shoppers with the shelf and packaging,  using all of their senses.  The store laboratory allows for new retail designs to be both virtually and  physically examined.  The future is here today.

See also Martin Lindstrom's site.  As one review suggests, Lindstrom is very much an insider, and that makes this book all the more telling and interesting.  In today's world, unlike in Packard's, the industry changes very  quickly.  Advances like Neuromarketing, interactive signage, and electronic shelf labels, mentioned in the book, are changing quickly.  The book provides great examples of their use, but cannot hope to keep up with them.  Go to Lindstrom's site, and follow others like this site to keep up to date.

Brain and Brand Vibrations

Martin Lindstrom in his latest article writes about phantom cellphone vibrations That have us reaching for our pockets even without any conscious cues. He relates how this work relates to fmri studies he did concerning the influence of cigarette warning images. Our brains are powerful and very malleable instruments. Thus the increasing interest in how our brains really react to marketing messages. And how we ultimately respond to those cues under alternative scripts of behavior. Big sophisticated marketers are all investigating. I will be reviewing colleague Lindstrom's forthcoming book 'BrandWashed, Tricks Companies Use to Manipulate Our Minds and Persuade Us to Buy' here soon.

Too Much Information, A Brain activity study

Some interesting questions to ask, from an article in Newsweek:

I Can’t Think! ... The Twitterization of our culture has revolutionized our lives, but with an unintended consequence—our overloaded brains freeze when we have to make decisions.

Is there too much information?  Too many choices?  A neuro study using fMRI is described:  " ... As the information load increased, she found, so did activity in the dorsolateral prefrontal cortex, a region behind the forehead that is responsible for decision making and control of emotions. But as the researchers gave the bidders more and more information, activity in the dorsolateral PFC suddenly fell off, as if a circuit breaker had popped ... "

Neuromarketing Movies

In Gizmodo, an article about how a number are using neuromarketing methods to predict how people react to movies. In particular the use of fMRI methods. It is very natural to examine new methods like these to look at very high cost and risk efforts like the making and marketing of a film. We examined the use of agent-based models to understand the social dynamics of film marketing in the 90s.