Deceiving AI

 Made me think, usually models we create are to determine some state, current or future, to be more accurate.   But now we can make models that have more precisely deceptive results.     Yes, can see why DARPA is interested.  Includes  a visual overview.

Deceiving AI   By Don Monroe

Communications of the ACM, June 2021, Vol. 64 No. 6, Pages 15-16 10.1145/3460218

Over the last decade, deep learning systems have shown an astonishing ability to classify images, translate languages, and perform other tasks that once seemed uniquely human. However, these systems work opaquely and sometimes make elementary mistakes, and this fragility could be intentionally exploited to threaten security or safety.

In 2018, for example, a group of undergraduates at the Massachusetts Institute of Technology (MIT) three-dimensionally (3D) printed a toy turtle that Google's Cloud Vision system consistently classified as a rifle, even when viewed from various directions. Other researchers have tweaked an ordinary-sounding speech segment to direct a smart speaker to a malicious website. These misclassifications sound amusing, but they could also represent a serious vulnerability as machine learning is widely deployed in medical, legal, and financial systems.

The potential vulnerabilities extend to military systems, said Hava Siegelman of the University of Massachusetts, Amherst. Siegelman initiated a program called Guaranteed AI Robustness against Deception (GARD) while she was on assignment to the U.S. Defense Advanced Research Projects Agency (DARPA). To illustrate the issue to colleagues there, she said, "I showed them an example that I did, and they all started screaming that the room was not secure enough." The examples she shares publicly are worrisome enough, though, such as a tank adorned with tiny pictures of cows that cause an artificial intelligence (AI)-based vision system to perceive it to be as a herd of cows because, she said, AI "works on the surfaces."

The current program manager for GARD at DARPA, Bruce Draper of Colorado State University, is more sanguine. "We have not yet gotten to that point where there's something out there that has happened that has given me nightmares," he said, adding, "We're trying to head that off."

Researchers, some with funding from DARPA, are actively exploring ways to make machine learning more robust against adversarial attacks, and to understand the principles and limitations of these approaches. In the real world, these techniques are likely to be one piece of an ongoing, multilayered security strategy that will slow attackers but not stop them entirely. "It's an AI problem, but it's also a security problem," Draper said. ... '

Towards Deception Detection? In Humans, in Machines? In Crowds?

Following this in the Language Log Blog for some time.  We even looked at large databases of human comments on products. But could such comments really be marked as 'deception'?    And when our brands started to converse with customers, relating their experiences, concerns and needs,  could such conversations be closer to speaking 'truth'?  Or not?  How about if we linked it to other behavioral  cues?   Or is that a violation of privacy?    All that detail, though discussed, was never implemented.  But can it be done better now?   Is there truth in a crowd response?   See the tags under 'deception'  here, which I am about to review.

New directions in deception detection?

March 28, 2021 @ 11:23 am · Filed by Mark Liberman under Nonverbal communication, Psychology of language

Jessica Seigel, "The truth about lying", Knowable Magazine 3/25/2021

You can’t spot a liar just by looking — but psychologists are zeroing in on methods that might actually work

The featured research is a review by Aldert Vrij, Maria Hartwig, and Pär Anders Granhag, "Reading Lies: Nonverbal Communication and Deception", Annual Review of Psychology 2019   :... '

Further Deception Detection

See other models, for example the U of MD, which also been working on this.  Of course lie detection using biometics has been around for a long time, but is still not generally accepted in US courts of law.  Will this be treated similarly?  But then if this detection is integrated with other facial recognition data and rolled into an algorithm?  Implications unclear.

Using Data Science to Tell Which of These People Is Lying    By University of Rochester

University of Rochester researchers are applying data science and an online crowdsourcing framework to read facial and verbal cues for signs of deception.

The Automated Dyadic Data Recorder framework was used to generate the largest publicly available deception dataset currently in existence. Participants sign up on Amazon Mechanical Turk to be assigned the roles of describer or interrogator. The former is displayed as an image they must memorize thoroughly, and the computer instructs them to either lie or truthfully relate the image details. The interrogator then asks the describer a set of irrelevant baseline queries to record individual behavioral differences that are fed to a "personalized model."

The researchers have culled 1.3 million frames of facial expressions from 151 pairs of individuals conducting this experiment, analyzing the information with data science. Among their findings is the detection of five types of smile-related expressions people make in response to questions, including one most frequently associated with lying.    .... "

From University of Rochester

Detecting Deception

Skeptical, like to see more.

A New AI That Detects 'Deception' May Bring an End to Lying as We Know It    Futurism   By Dom Galeon in Futurism in ACM News

Researchers at the University of Maryland (UMD) have developed the Deception Analysis and Reasoning Engine (DARE), which uses artificial intelligence (AI) to autonomously detect deception in courtroom trial videos. The team trained DARE to seek out and classify human micro-expressions, such as "lips protruded" or "eyebrows frown," and analyze audio frequency for vocal patterns that signal whether a person is lying or not. DARE then was tested with a training set of videos in which actors were told to either lie or be honest. UMD's Bharat Singh says DARE outperformed the average person in detecting lies, and notes "a remarkable observation was that the visual AI system was significantly better than common people at predicting deception." Singh estimates DARE scored an area under the curve (AUC) of 0.877, which rose to 0.922 when combined with human annotations of micro-expressions, while ordinary people score an AUC of 0.58. .... " 

Decepticons

Research at Ga Tech is using 'deception' methods used in nature to enhance behavioral models.  Have always been interested in biological models.  They have often been used in simulation of competitive systems.