Are ChatGPT's Good at 'Not'?

Are ChatGPT's Good at 'Not'  in QuantaMagazine

Max G. Levy,    Contributing Writer, May 12, 2023

Nora Kassner suspected her computer wasn’t as smart as people thought. In October 2018, Google released a language model algorithm called BERT, which Kassner, a researcher in the same field, quickly loaded on her laptop. It was Google’s first language model that was self-taught on a massive volume of online data. Like her peers, Kassner was impressed that BERT could complete users’ sentences and answer simple questions. It seemed as if the large language model (LLM) could read text like a human (or better).

But Kassner, at the time a graduate student at Ludwig Maximilian University of Munich, remained skeptical. She felt LLMs should understand what their answers mean — and what they don’t mean. It’s one thing to know that a bird can fly. “A model should automatically also know that the negated statement — ‘a bird cannot fly’ — is false,” she said. But when she and her adviser, Hinrich Schütze, tested BERT and two other LLMs in 2019, they found that the models behaved as if words like “not” were invisible.

Since then, LLMs have skyrocketed in size and ability. “The algorithm itself is still similar to what we had before. But the scale and the performance is really astonishing,” said Ding Zhao, who leads the Safe Artificial Intelligence Lab at Carnegie Mellon University.

But while chatbots have improved their humanlike performances, they still have trouble with negation. They know what it means if a bird can’t fly, but they collapse when confronted with more complicated logic involving words like “not,” which is trivial to a human.

“Large language models work better than any system we have ever had before,” said Pascale Fung, an AI researcher at the Hong Kong University of Science and Technology. “Why do they struggle with something that’s seemingly simple while it’s demonstrating amazing power in other things that we don’t expect it to?” Recent studies have finally started to explain the difficulties, and what programmers can do to get around them. But researchers still don’t understand whether machines will ever truly know the word “no.”

Nora Kassner in a blue shirt against a black background.

Nora Kassner has tested popular chatbots and found they typically can’t understand the concept of negation.

Courtesy of Nora Kassner

Making Connections

 It’s hard to coax a computer into reading and writing like a human. Machines excel at storing lots of data and blasting through complex calculations, so developers build LLMs as neural networks: statistical models that assess how objects (words, in this case) relate to one another. Each linguistic relationship carries some weight, and that weight — fine-tuned during training — codifies the relationship’s strength. For example, “rat” relates more to “rodent” than “pizza,” even if some rats have been known to enjoy a good slice.

In the same way that your smartphone’s keyboard learns that you follow “good” with “morning,” LLMs sequentially predict the next word in a block of text. The bigger the data set used to train them, the better the predictions, and as the amount of data used to train the models has increased enormously, dozens of emergent behaviors have bubbled up. Chatbots have learned style, syntax and tone, for example, all on their own. “An early problem was that they completely could not detect emotional language at all. And now they can,” said Kathleen Carley, a computer scientist at Carnegie Mellon. Carley uses LLMs for “sentiment analysis,” which is all about extracting emotional language from large data sets — an approach used for things like mining social media for opinions.

So new models should get the right answers more reliably. “But we’re not applying reasoning,” Carley said. “We’re just applying a kind of mathematical change.” And, unsurprisingly, experts are finding gaps where these models diverge from how humans read.

No Negatives.. Unlike humans, LLMs process language by turning it into math. This helps them excel at generating text — by predicting likely combinations of text — but it comes at a cost.

“The problem is that the task of prediction is not equivalent to the task of understanding,” said Allyson Ettinger, a computational linguist at the University of Chicago. Like Kassner, Ettinger tests how language models fare on tasks that seem easy to humans. In 2019, for example, Ettinger tested BERT with diagnostics pulled from experiments designed to test human language ability. The model’s abilities weren’t consistent. For example:

He caught the pass and scored another touchdown. There was nothing he enjoyed more than a good game of ____. (BERT correctly predicted “football.”)

The snow had piled up on the drive so high that they couldn’t get the car out. When Albert woke up, his father handed him a ____. (BERT incorrectly guessed “note,” “letter,” “gun.”)

And when it came to negation, BERT consistently struggled.  ... '

BERT for Unsupervised Training

 A nice, fairly compact, somewhat technical view of how to use BERT for unsupervised training problems.    And some unexpected uses.  True, everyone should know how to use this method.  Click through for full detail. 

For unsupervised task solving ... 

BERT is a prize addition to the practitioner’s toolbox   By Ajit Rajasekharan  in TowardsDataScience

Figure 1. Few reasons why BERT is a valuable addition to a practitioner’s toolbox apart from its well known use of fine-tuning for downstream tasks. (1) BERT’s learned vocabulary of vectors (in say 768 dimensional space) serve as targets that masked output vectors predict and learn from prediction errors during training. After training, these moving targets settle into landmarks that can be clustered and annotated (a one-time step) and used for classifying model output vectors in a variety of tasks — NER, relation extraction etc. (2) A model pre-trained enough to achieve a low next sentence prediction loss (in addition to the masked word prediction loss) yields quality CLS vectors representing any input term/phrase/sentence.  ... ' 

Detecting Sarcasm

Even in the early days of reading consumer reactions to product communications  we knew it was useful to have an idea of how much sarcasm was being used.  To determine human reactions to statements or images.   Often we just scrubbed potential sarcasm. but we knew we were losing something by not analyzing it.

This work on Detecting Sarcasm  also Discussed in  VentureBeat  

Modeling Intra and Inter-modality Incongruity for Multi-Modal Sarcasm Detection

By Hongliang Pan, Zheng Lin, Peng Fu, Yatao Qi, Weiping Wang

Abstract

Sarcasm is a pervasive phenomenon in today’s social media platforms such as Twitter and Reddit. These platforms allow users to create multi-modal messages, including texts, images, and videos. Existing multi-modal sarcasm detection methods either simply concatenate the features from multi modalities or fuse the multi modalities information in a designed manner. However, they ignore the incongruity character in sarcastic utterance, which is often manifested between modalities or within modalities. Inspired by this, we propose a BERT architecture-based model, which concentrates on both intra and inter-modality incongruity for multi-modal sarcasm detection. To be specific, we are inspired by the idea of self-attention mechanism and design inter-modality attention to capturing inter-modality incongruity. In addition, the co-attention mechanism is applied to model the contradiction within the text. The incongruity information is then used for prediction. The experimental results demonstrate that our model achieves state-of-the-art performance on a public multi-modal sarcasm detection dataset. ... 

Shrinking BERT Networks to Model Language

Considerable shrinking of neural networks, more likely for applications at the Edge.

Shrinking massive neural networks used to model language

A new approach could lower computing costs and increase accessibility to state-of-the-art natural language processing.

Daniel Ackerman | MIT News Office

Researchers at the Massachusetts Institute of Technology (MIT), the University of Texas at Austin, and the MIT-IBM Watson Artificial Intelligence Laboratory identified lean subnetworks within a state-of-the-art neural network approach to natural language processing (NLP). These subnetworks, found in the Bidirectional Encoder Representations from Transformers (BERT) network, could potentially enable more users to develop NLP tools using less bulky and more efficient systems, like smartphones. BERT is trained by repeatedly attempting to fill in words omitted from a passage of writing, using a massive dataset; users can then refine its neural network to a specific task. By iteratively trimming parameters from the BERT model, then comparing the new subnetwork's performance to that of the original model, the team found effective subnetworks that were 40% to 90% leaner, and required no task-specific fine-tuning to identify "winning ticket" subnetworks that executed tasks successfully.  .... 

Trick for Teaching AI the Meaning of Language

More advances in machine language learning and understanding.

Baidu Has Trick for Teaching AI the Meaning of Language
By MIT Technology Review

Baidu, Chinas closest equivalent to Google, recently outperformed Google and Microsoft in a competition known as the General Language Understanding Evaluation, or GLUE.

Chinese technology company Baidu outperformed Microsoft and Google in the General Language Understanding Evaluation, a benchmark for artificial intelligence (AI)'s comprehension of human language.

Researchers built the Enhanced Representation through kNowledge IntEgration (ERNIE) model specifically for the Chinese language, using a method that also makes the model better understand English.

The researchers built on the Bidirectional Encoder Representations from Transformers (BERT) model, which predicts and interprets the meaning of words by considering context before and after words simultaneously. BERT randomly hides 15% of words in a given passage of text and attempts to predict those words from the remaining ones, and Baidu modified this technique for ERNIE, hiding strings of characters rather than single characters; ERNIE also differentiates between meaningful and random strings to mask the right character combinations.

Baidu uses ERNIE to produce more applicable search results, eliminate duplicate stories in its news feed, and make AI-facilitated responses to requests more accurate.

From MIT Technology Review ... " 

IBM Upgrades Debate AI Tool to Better Derive Evidence

Intriguing approach to mining information to support a goal directed conversation.   A key aspect to making conversational systems more powerful.  Note also the crowdsourcing integrated here to grade evidence.  Noting that the report here does not mention 'Watson', it seems IBM is using their AI trademark much less these days.

IBM's Debating AI Just Got a Lot Closer to Being a Useful Tool
By MIT Technology Review via CACM

The IBM Debater system taking part in a debate at the University of Cambridge last year.
IBM upgraded the neural networks used by its Project Debater system, to improve the quality of evidence the argument-mining system uncovers.

IBM upgraded the neural networks used by its Project Debater system to improve the quality of evidence the argument-mining system uncovers.

One new add-on for the debating system is BERT (Bidirectional Encoder Representations from Transformers), a network designed by Google for natural language processing and answering queries.

IBM Research scientists trained the AI on 400 million documents from the LexisNexis database, providing a natural language dataset of roughly 10 billion sentences; the researchers combined the dataset with claims about several hundred different topics, then had crowdsourced workers label the sentences based on the quality of their evidence for or against specific claims.

A supervised learning algorithm digested this data, allowing BERT to manage queries on a wide range of subjects and to yield more relevant sentences compared to previous systems.

Project Debater was 95% accurate for the top 50 sentences across 100 distinct topics, according to IBM researcher Noam Slonim,  ... " 

Advances in Conversational Search

Here a long time interest in intelligent conversation, and one of the most used uses of conversation on line these days is search.   Intelligent conversation gets us much closer to general AI.    Has this now been considerably improved?  Are we getting much closer, or is this one more demo that cannot be broadly delivered?   Note too how Google mentions people adapting to machines with 'keyword-ese'  and avoiding natural conversation. 

Google now understands more conversational search queries
The tech giant says it's one of the biggest Search updates in the product's history.
By Mariella Moon, @mariella_moon  in Engadget ..... 

Understanding searches better than ever before
Pandu Nayak in the Google Blog
Google Fellow and Vice President, Search

If there’s one thing I’ve learned over the 15 years working on Google Search, it’s that people’s curiosity is endless. We see billions of searches every day, and 15 percent of those queries are ones we haven’t seen before--so we’ve built ways to return results for queries we can’t anticipate.

When people like you or I come to Search, we aren’t always quite sure about the best way to formulate a query. We might not know the right words to use, or how to spell something, because often times, we come to Search looking to learn--we don’t necessarily have the knowledge to begin with. 

At its core, Search is about understanding language. It’s our job to figure out what you’re searching for and surface helpful information from the web, no matter how you spell or combine the words in your query. While we’ve continued to improve our language understanding capabilities over the years, we sometimes still don’t quite get it right, particularly with complex or conversational queries. In fact, that’s one of the reasons why people often use “keyword-ese,” typing strings of words that they think we’ll understand, but aren’t actually how they’d naturally ask a question. 

With the latest advancements from our research team in the science of language understanding--made possible by machine learning--we’re making a significant improvement to how we understand queries, representing the biggest leap forward in the past five years, and one of the biggest leaps forward in the history of Search.   .... " 

AI for Reading Understanding

Been reading and exploring about what reading understanding means.  Here an update from Quanta Magazine on the toic.      We still have far to go when we have to deal with changing context, common sense and even inferring things like implications of cause and effect.   We did lots of work with 'sentiment analysis' long ago,  and its much easier to do now, with lots of easy to plug in capabilities, but the result is still statistically weak.  Shows how difficult a building a semi general purpose chatbot is.    We discovered that during several efforts.     Good read here at the link:

Machines Beat Humans on a Reading Test. But Do They Understand?
A tool known as BERT can now beat humans on advanced reading-comprehension tests. But it's also revealed how far AI has to go.

In the fall of 2017, Sam Bowman, a computational linguist at New York University, figured that computers still weren’t very good at understanding the written word. Sure, they had become decent at simulating that understanding in certain narrow domains, like automatic translation or sentiment analysis (for example, determining if a sentence sounds “mean or nice,” he said). But Bowman wanted measurable evidence of the genuine article: bona fide, human-style reading comprehension in English. So he came up with a test.

In an April 2018 paper coauthored with collaborators from the University of Washington and DeepMind, the Google-owned artificial intelligence company, Bowman introduced a battery of nine reading-comprehension tasks for computers called GLUE (General Language Understanding Evaluation). The test was designed as “a fairly representative sample of what the research community thought were interesting challenges,” said Bowman, but also “pretty straightforward for humans.” For example, one task asks whether a sentence is true based on information offered in a preceding sentence. If you can tell that “President Trump landed in Iraq for the start of a seven-day visit” implies that “President Trump is on an overseas visit,” you’ve just passed.

The machines bombed. Even state-of-the-art neural networks scored no higher than 69 out of 100 across all nine tasks: a D-plus, in letter grade terms. Bowman and his coauthors weren’t surprised. Neural networks — layers of computational connections built in a crude approximation of how neurons communicate within mammalian brains — had shown promise in the field of “natural language processing” (NLP), but the researchers weren’t convinced that these systems were learning anything substantial about language itself. And GLUE seemed to prove it. “These early results indicate that solving GLUE is beyond the capabilities of current models and methods,” Bowman and his coauthors wrote.   .... " 

BERT for Natural Language Understanding

Was just introduced to this again, worth a a look:

BERT Technology introduced in 3-minutes   By Suleiman Khan, Ph.D.  in Medium

Google BERT is a pre-training method for natural language understanding that performs various NLP tasks better than ever before.

BERT works in two steps, First, it uses a large amount of unlabeled data to learn a language representation in an unsupervised fashion called pre-training. Then, the pre-trained model can be fine-tuned in a supervised fashion using a small amount of labeled trained data to perform various supervised tasks. Pre-training machine learning models have already seen success in various domains including image processing and natural language processing (NLP). .... " 

Finishing Your Sentences with Common Sense

Yes, its one way to tailor common sense driven sentences.   Here some snippets of work underway in the space, starting with a NYT article, then links to technical details.  The overall challenge for common sense natural language understanding research at this level is well described, but the solutions are technical:

Finally, a Machine That Can Finish Your Sentence

Completing someone else’s thought is not an easy trick for A.I. But new systems are starting to crack the code of natural language.    By Cade Metz  in the WSJ.   ... "

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova

We introduce a new language representation model called BERT, which stands for Bidirectional Encoder Representations from Transformers. Unlike recent language representation models, BERT is designed to pre-train deep bidirectional representations by jointly conditioning on both left and right context in all layers. As a result, the pre-trained BERT representations can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks, such as question answering and language inference, without substantial task-specific architecture modifications.  ..."

And of course,   a data challenge for this problem,with early success results:

A Large-Scale Adversarial Dataset for Grounded Commonsense Inference

Rowan Zellers. Yonatan Bisk, Roy Schwartz, Yejin Choi
Paul G. Allen School of Computer Science & Engineering, University of Washington
Allen Institute for Artificial Intelligence

Further description of the data challenge in Swag:

Given a partial description like “she opened the hood of the car,” humans can reason about the situation and anticipate what might come, next (“then, she examined the engine”). In this paper, we introduce the task of grounded commonsense inference, unifying natural language inference and commonsense reasoning.

We present Swag, a new dataset with 113k multiple choice questions about a rich spectrum of grounded situations. To address the, recurring challenges of the annotation artifacts and human biases found in many existing datasets, we propose Adversarial Filtering (AF), a novel procedure that constructs a de-biased dataset by iteratively training an ensemble of stylistic classifiers, and using them to filter the data. To account for the aggressive adversarial filtering, we use state-of-theart language models to massively oversample a diverse set of potential counterfactuals.

Empirical results demonstrate that while humans can solve the resulting inference problems with high accuracy (88%), various competitive models struggle on our task. We provide comprehensive analysis that indicates significant opportunities for future research.  ... "