Detecting Object Permanence and more

Reinforcement learning problem. Learning like children do.

AI Agents Learned Object Permanence by Playing Hide and Seek

By IEEE Spectrum  February 17, 2021

Researchers at the Allen Institute for AI (AI2) demonstrated that artificial intelligence agents learned the concept of object permanence — that objects hidden from view are still there — by playing hide and seek.

The agents, playing as both hiders and seekers, learned the game "Cache" via reinforcement learning. The agents began learning about the environment by taking random actions, like pulling on drawers, and dropping objects in random places. Their game play improved as they learned from outcomes, with the hider, for instance, learning that it had selected a good hiding place when the seeker failed to find the object.

Subsequent testing showed that the agents understood the principles of containment and object permanence and were able to rank images based on how much free space they contained.

The agents performed as well or better than models trained on the gold-standard ImageNet.

From IEEE Spectrum

Finishing Your Sentences with Common Sense

Yes, its one way to tailor common sense driven sentences.   Here some snippets of work underway in the space, starting with a NYT article, then links to technical details.  The overall challenge for common sense natural language understanding research at this level is well described, but the solutions are technical:

Finally, a Machine That Can Finish Your Sentence

Completing someone else’s thought is not an easy trick for A.I. But new systems are starting to crack the code of natural language.    By Cade Metz  in the WSJ.   ... "

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova

We introduce a new language representation model called BERT, which stands for Bidirectional Encoder Representations from Transformers. Unlike recent language representation models, BERT is designed to pre-train deep bidirectional representations by jointly conditioning on both left and right context in all layers. As a result, the pre-trained BERT representations can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks, such as question answering and language inference, without substantial task-specific architecture modifications.  ..."

And of course,   a data challenge for this problem,with early success results:

A Large-Scale Adversarial Dataset for Grounded Commonsense Inference

Rowan Zellers. Yonatan Bisk, Roy Schwartz, Yejin Choi
Paul G. Allen School of Computer Science & Engineering, University of Washington
Allen Institute for Artificial Intelligence

Further description of the data challenge in Swag:

Given a partial description like “she opened the hood of the car,” humans can reason about the situation and anticipate what might come, next (“then, she examined the engine”). In this paper, we introduce the task of grounded commonsense inference, unifying natural language inference and commonsense reasoning.

We present Swag, a new dataset with 113k multiple choice questions about a rich spectrum of grounded situations. To address the, recurring challenges of the annotation artifacts and human biases found in many existing datasets, we propose Adversarial Filtering (AF), a novel procedure that constructs a de-biased dataset by iteratively training an ensemble of stylistic classifiers, and using them to filter the data. To account for the aggressive adversarial filtering, we use state-of-theart language models to massively oversample a diverse set of potential counterfactuals.

Empirical results demonstrate that while humans can solve the resulting inference problems with high accuracy (88%), various competitive models struggle on our task. We provide comprehensive analysis that indicates significant opportunities for future research.  ... "

Paul Allen, Microsoft Co-Founder

Our group met with Allen early on to discuss the future of personal computing.   He gave an early example of buying and adapting versus building.  See also his Allen Institute for AI.

Microsoft co-founder Paul Allen dies, aged 65

He was also a philanthropist and drove Seattle's development into a cultural centre.  Mr Allen came up with the company's name Micro-Soft, for a company that made software for small computers. The term personal computer (PC) would become commonplace later  ... "

Microsoft co-founder Paul G. Allen died Monday in Seattle, WA, from complications of non-Hodgkin's lymphoma. Allen, 65, helped usher in the personal computing era in the 1980s by arranging a deal to acquire (and later tweak) an operating system that served as the basis for the Microsoft Disk Operating System (MS-DOS), a forerunner of today’s Windows operating system. Said Microsoft CEO Satya Nadella, "In his own quiet and persistent way, he created magical products, experiences and institutions, and in doing so, he changed the world." In addition, over the course of his life, Allen donated more than $2 billion toward nonprofit groups dedicated to the advancement of science, technology, education, the environment, and the arts. He also funded scientific research organizations such as the Allen Institute for Brain Science and the Allen Institute for Artificial Intelligence.  ... " 

Common Sense - Project Alexandria

See my tag links below on our past looks at common sense based reasoning. Ultimately essential to get us to machine intelligence. 

Paul Allen Wants to Teach Machines Common Sense

Project Alexandria   By Cade Metz in The New York Times

Microsoft co-founder Paul Allen on Wednesday announced he would pour another $125 million into the non-profit Allen Institute for Artificial Intelligence to fund a project to teach computers common sense. He notes the additional funding should help to underwrite existing research as well as the common sense initiative, called Project Alexandria. Artificial intelligence (AI) "recognizes objects, but can't explain what it sees," says Allen Institute CEO Oren Etzioni. "It can't read a textbook and understand the questions in the back of the book." The Allen Institute wants to compile a database of fundamental knowledge that humans take for granted but which machines have always lacked, which will be fed into efforts such as Project Alexandria. "To make real progress in AI, we have to overcome the big challenges in the area of common sense,” Allen contends. .... 

Common sense is the everyday knowledge   that virtually every person has but no machine does.

More on AI2:

AI2 was founded in 2014 with the singular focus of conducting high-impact research and engineering in the field of artificial intelligence, all for the common good. AI2 is the creation of Paul Allen, Microsoft co-founder, and is led by Dr. Oren Etzioni, a world-renowned researcher and professor in the field of AI and computer science.

Situated on the shores of Lake Union, AI2 employs over 60 of the world’s best scientific talent in the field of AI, attracting individuals of varied interests and backgrounds from across the globe. AI2 prides itself on the diversity and collaboration of its team, and takes a results-oriented approach to complex challenges in AI.

AI2 has undertaken four main projects, Aristo, Semantic Scholar, Euclid and Plato to drive fundamental advances in science and medicine through AI. Meet our team, review our projects, and see our advisors and collaborators.

Based on these project efforts:

Alexandria,  Common Sense Knowledge and Reasoning
Aristo, Machine Reading and Reasoning
Semantic Scholar, AI-Based Search
Euclid, Natural Language Understanding
Plato, Computer Vision
AllenNLP,  Deep Semantic NLP Platform
Incubator,   Launching AI Powered Startups

Modeling the Visual Intelligence of Dogs

Quite a remarkable claim.   Can many visuals replace the ability to query a system about about what it knows.  Could this work with humans, entities like firms and their data?  Examining.

Via the University of Washington and Allen Institute for AI.  Who used neural networks to understand the behavior of dogs.   And suggest that animals provide data to train AI systems, including robotics.  What other visual agents?  And how accurate is the model?  Examining.

Who Let The Dogs Out? Modeling Dog Behavior From Visual   in arXIV
Kiana Ehsani, Hessam Bagherinezhad, Joseph Redmon, Roozbeh Mottaghi, Ali Farhadi

We introduce the task of directly modeling a visually intelligent agent. Computer vision typically focuses on solving various subtasks related to visual intelligence. We depart from this standard approach to computer vision; instead we directly model a visually intelligent agent. Our model takes visual information as input and directly predicts the actions of the agent. Toward this end we introduce DECADE, a large-scale dataset of ego-centric videos from a dog's perspective as well as her corresponding movements. Using this data we model how the dog acts and how the dog plans her movements. We show under a variety of metrics that given just visual input we can successfully model this intelligent agent in many situations. Moreover, the representation learned by our model encodes distinct information compared to representations trained on image classification, and our learned representation can generalize to other domains. In particular, we show strong results on the task of walkable surface estimation by using this dog modeling task as representation learning. ..."

Building Intelligent Machines

Cautionary review of intelligent machines.   Are they thinking like people, or just with people?  And what are the implications for value and bias?

AI savants, recognizing bias, and building machines that think like people

Despite impressive advances, three speakers at EmTech Digital show how far there is to go in the AI world.  by David Rotman  March 26, 2018  in MIT Tech Review.

Even with all the amazing examples of progress in artificial intelligence, such as self-driving cars and the victories of AlphaGo, the technology is still very narrow in its accomplishments and far from autonomous. Indeed, says Oren Etzioni, chief executive officer of the Allen Institute for Artificial Intelligence, today’s machine-learning systems are “AI savants.”

Etzioni, speaking today at MIT Technology Review’s annual EmTech Digital conference in San Francisco, explained that self-driving cars and speech recognition are based on machine learning. And even today, he said, 99 percent of machine learning is based on human work.

Etzioni pointed out that machine learning needs large amounts of data, all of which needs to be labeled: this is a dog; this is a cat. And then people need to supply the appropriate algorithms. All this relies on manual labor.  ... " 

Assistants and Common Sense

Its actually not too often that assistants speak gibberish, they more often just admit to not knowing what was asked, when a human would understand readily.  That's usually better to diminish risk.  See my previous posts on Common Sense reasoning, which we worked on in the enterprise.  The idea of a challenge is good, it will at least scope the problem in contextual and current terms.   See the examples at the link below.  Also the notes on what 'fundamental limitations' are. 

AI assistants don’t have the common sense to avoid talking gibberish
A new test could prove that when it comes to language, today’s best AI systems are fundamentally limited.   by Will Knight in Technology Review

Siri and Alexa are clearly far from perfect, but there is hope that steady progress in machine learning will turn them into articulate helpers before long. A new test, however, may help show that a fundamentally different approach is required for AI systems to actually master language.

Developed by researchers at the Allen Institute for AI (Ai2), a nonprofit based in Seattle, the Arc Reasoning Challenge (ARC) will pose elementary-school-level multiple-choice science questions. Each question will require some understanding of how the world works.  .... " 

Reasoning with Common Sense

Can we teach machines common sense?  Which points back to our own look at the work Doug Lenat was doing with the Cyc Project. We wanted to create a means by which an agent could understand the basic terminology and goals of an organization like a company, and reason about it with the right language and direction.  That did not happen,  but the goal is still there.   And Cycorp has been active for 34 years, and produced a stable software release just three months ago.  We talked to them back then, but it didn't meet our needs. And every time we make some progress, like our broad use of assistants, now in many homes, we see the need again.

See the Cyc Wikipedia Article.  and

The Limits of Explainability
Academics, economists, and AI researchers often undervalue the role of intuition in science. Here's why they're wrong.
by Joi Ito in Wired ....

Paul Allen Wants to Teach Machines Common Sense 
By Cade Metz in the NYT. .... "