Insights for AI from the Human Mind

Good thoughts by Gary Marcus, aiming at the difficulty of creating intelligence, even though we have very rich models around we can do testing with.  

Insights for AI from the Human Mind   By Gary Marcus, Ernest Davis

Communications of the ACM, January 2021, Vol. 64 No. 1, Pages 38-41  10.1145/3392663

What magical trick makes us intelligent? The trick is that there is no trick. The power of intelligence stems from our vast diversity, not from any single, perfect principle.

—Marvin Minsky, The Society of Mind

Artificial intelligence has recently beaten world champions in Go and poker and made extraordinary progress in domains such as machine translation, object classification, and speech recognition. However, most AI systems are extremely narrowly focused. AlphaGo, the champion Go player, does not know that the game is played by putting stones onto a board; it has no idea what a "stone" or a "board" is, and would need to be retrained from scratch if you presented it with a rectangular board rather than a square grid.

To build AIs able to comprehend open text or power general-purpose domestic robots, we need to go further. A good place to start is by looking at the human mind, which still far outstrips machines in comprehension and flexible thinking.

Here, we offer 11 clues drawn from the cognitive sciences—psychology, linguistics, and philosophy.

No Silver Bullets

All too often, people have propounded simple theories that allegedly explained all of human intelligence, from behaviorism to Bayesian inference to deep learning. But, quoting Firestone and Scholl,4 "there is no one way the mind works, because the mind is not one thing. Instead, the mind has parts, and the different parts of the mind operate in different ways: Seeing a color works differently than planning a vacation, which works differently than understanding a sentence, moving a limb, remembering a fact, or feeling an emotion."

The human brain is enormously complex and diverse, with more than 150 distinctly identifiable brain areas, approximately 86 billion neurons, hundreds if not thousands of different types; trillions of synapses; and hundreds of distinct proteins within each individual synapse.

Truly intelligent and flexible systems are likely to be full of complexity, much like brains. Any theory that proposes to reduce intelligence down to a single principle—or a single "master algorithm"—is bound to fail.

Rich Internal Representations

Cognitive psychology often focuses on internal representations, such as beliefs, desires, and goals. Classical AI did likewise; for instance, to represent President Kennedy's famous 1963 visit to Berlin, one would add a set of facts such as part-of (Berlin, Germany), and visited (Kennedy, Berlin, June 1963). Knowledge consists in an accumulation of such representations, and inference is built on that bedrock; it is trivial on that foundation to infer that Kennedy visited Germany.

Currently, deep learning tries to fudge this, with a bunch of vectors that capture a little bit of what's going on, in a rough sort of way, but that never directly represent propositions at all. There is no specific way to represent visited (Kennedy, Berlin, 1963) or part-of (Berlin, Germany); everything is just rough approximation. Deep learning currently struggles with inference and abstract reasoning because it is not geared toward representing precise factual knowledge in the first place. Once facts are fuzzy, it is difficult to get reasoning right. The much-hyped GPT-3 system1 is a good example of this.11 The related system BERT3 is unable to reliably answer questions like "if you put two trophies on a table and add another, how many do you have?" .... '   (much more follows) 

Are We at the Narrow Edge of General AI?

And what is even the definition of 'General AI'?   Say a means to repeatedly solve a non trivial problem in a business or scientific domain, with varying data and context.  One that could be 'intelligent' enough to explain its method, ethics and risks to groups of humans to assure them of its net value of implementation.  That can also learn when given new data.  Also one that could measure and report on its net value over time.  Its not to say that narrow AI is not valuable.  It is,  but its not general.   We are not close yet, and not close to a transition point either.

Are We at the edge of general AI?
We’re entering the AI twilight zone between narrow and general AI
Gary Grossman, Edelman    @garyg02

With recent advances, the tech industry is leaving the confines of narrow artificial intelligence (AI) and entering a twilight zone, an ill-defined area between narrow and general AI.

To date, all the capabilities attributed to machine learning and AI have been in the category of narrow AI. No matter how sophisticated – from insurance rating to fraud detection to manufacturing quality control and aerial dogfights or even aiding with nuclear fission research – each algorithm has only been able to meet a single purpose. This means a couple of things: 1) an algorithm designed to do one thing (say, identify objects) cannot be used for anything else (play a video game, for example), and 2) anything one algorithm “learns” cannot be effectively transferred to another algorithm designed to fulfill a different specific purpose. For example, AlphaGO, the algorithm that outperformed the human world champion at the game of Go, cannot play other games, despite those games being much simpler.  ... " 

Singularity Hub Looks at GPT-3

Out of SingularityHub.   A look at the writing AI called GPT-3.  Scopes some of the possibilities and limitations.

OpenAI’s GPT-3 Wrote This Short Film—Even the Twist at the End  By Vanessa Bates Ramirez

OpenAI’s text generating AI has gotten a lot of buzz since its release in June. It’s been used to post comments on Reddit, write a poem roasting Elon Musk, and even write an entire article in The Guardian (which editors admitted they worked on and tweaked just as they would a human-written op ed).

When the system learned to autocomplete images without having been specifically trained to do so (as well as write code, translate between languages, and do math) it even got people speculating whether GPT-3 might be the gateway to artificial general intelligence (it’s probably not).

Now there’s another feat to add to GPT-3’s list: it wrote a screenplay.

It’s short, and weird, and honestly not that good. But… it’s also not all that bad, especially given that it was written by a machine.

The three-and-half-minute short film shows a man knocking on a woman’s door and sharing a story about an accident he was in. It’s hard to tell where the storyline is going, but surprises viewers with what could be considered a twist ending.  ... " 

Human-Like too Low a Bar

Good cautious point is made here.  But I respond that in most cases we are assisting a human,  or doing something rather narrow than a human does today.    But quicker and without many kinds of human-like errors.  So the bar is still a useful general measure, but it is not close to general intelligence.   See the pointer to the NYU paper below.

Why ‘human-like’ is a low bar for most AI projects

Artificial Intelligence – The Next Webby by Tristan Greene in TNW 
Awww, look! It thinks it's people!

Show me a human-like machine and I’ll show you a faulty piece of tech. The AI market is expected to eclipse $300 billion by 2025. And the vast majority of the companies trying to cash in on that bonanza are marketing some form of “human-like” AI. Maybe it’s time to reconsider that approach.

The big idea is that human-like AI is an upgrade. Computers compute, but AI can learn. Unfortunately, humans aren’t very good at the kinds of tasks a computer makes sense for and AI isn’t very good at the kinds of tasks that humans are. That’s why researchers are moving away from development paradigms that focus on imitating human cognition.

A pair of NYU researchers recently took a deep dive into how humans and AI process words and word meaning. Through the study of “psychological semantics,” the duo hoped to explain the shortcomings held by machine learning systems in the natural language processing (NLP) domain. According to a study they published to arXiv:  https://arxiv.org/pdf/2008.01766.pdf   ... "

Artificial Neural Nets More similar to Brains than we thuoght

In our earliest uses of neural methods we actually worked with some brain scientists.  And they were quick to point out that neural methods, though inspired by the brain, were nothing like them.   So our thoughts of biomimicry were dangerous.    But it seems this idea has changed some,  but in a glance at this non technical piece, there is still much we do not know.  But is the brain a reasonable place to start for general AI?

Artificial neural networks are more similar to the brain than they get credit for  By Ben Dickson in BDTechtalks

This article is part of our reviews of AI research papers, a series of posts that explore the latest findings in artificial intelligence.

Consider the animal in the following image. If you recognize it, a quick series of neuron activations in your brain will link its image to its name and other information you know about it (habitat, size, diet, lifespan, etc…). But if like me, you’ve never seen this animal before, your mind is now racing through your repertoire of animal species, comparing tails, ears, paws, noses, snouts, and everything else to determine which bucket this odd creature belongs to. Your biological neural network is reprocessing your past experience to deal with a novel situation.

Our brains, honed through millions of years of evolution, are very efficient processing machines, sorting out the ton of information we receive through our sensory inputs, associating known items with their respective categories. ... " 

Is AI Already Conscious?

Regards the definition of consciousness,and its necessity for intelligence.

Is AI already conscious?
 Nicole Gray in ThenextWeb

The ultimate goal of most high-level AI research is the development of a general artificial intelligence (GAI). In essence, what we want is a synthetic mind that could function the same as a human were it placed into a physical vessel of similar capability.

Most experts – not all – believe we’re decades away from anything of the sort. Unlike other incredibly complex problems such as nuclear fusion or readjusting the Hubble Constant, nobody really understands yet what GAI actually looks like.

Some researchers think Deep Learning is the path to machines that think like humans, others believe we’ll need an entirely new calculus to create the necessary “master algorithm,” and still others think GAI is probably impossible.

But the fact of the matter is that scientists don’t truly understand intelligence as it relates to the human brain, or consciousness as it relates to anything. We’re just scratching the gray-matter surface when it comes to understanding how intelligence and consciousness emerge in the human brain.

As far as AI goes, in lieu of a GAI all we have is patchwork neural networks and clever algorithms. It’s hard to make an argument that modern AI will ever have human intelligence and even harder to demonstrate a path towards actual robot consciousness. But it’s not impossible.

In fact, AI might already be conscious.

Mathematician Johannes Kleiner and physicist Sean Tull recently pre-published a research paper on the nature of consciousness that seems to indicate, mathematically speaking, that the universe and everything in it is imbued with physical consciousness.

Basically the duo’s paper sorts out some of the math behind a popular theory called the Integrated Information Theory of Consciousness (ITT). It says that everything in the entire universe exhibits the traits of consciousness to some degree or another.

This is an interesting theory because it’s supported by the idea that consciousness emerges as a result of physical states. You’re conscious because of your ability to “experience” things. A tree, for example, is conscious because it can “sense” the sun’s light and bend towards it. An ant is conscious because it experiences ant stuff, and on and on it goes.  ... "

Decade of Voice Assistants

General but simplified history, worth a look. Platforms have been constructed, with lots of users.  Now how will they filled with meaningful assistance? How close to general AI will that evolve to?  How will it change the workplace and home?

The Decade of Voice Assistant Revolution
 By Eric Hall Schwartz in Voicebot.ai

The last 10 years have utterly transformed how people think about voice technology. From limited uses in just a few outlets, voice assistants are now integrating into every part of people’s lives. To encapsulate everything that has happened in ten years, we’ve picked a notable event from each year of the last decade to highlight and show how they marked a milestone in the way voice assistants have evolved and spread.  .... " 

Causation to Provide the Why

Basic causation is a great start.   Why did this happen? We are doing it all the time.  Its one of our basic knowledge processing capabilities that lead to learning.    We can figure out the answer by observation and combing observations to build rules of operation.   Or we can be taught specific rules, or even  imprecise rules of thumb, to help us process knowledge.   Combining things we have observed or not.   They must include things like causation, space and time relationships.  It is this kind of knowledge we need to do general AI.  Not just more data.   Its more than just unstructured data.  Its about combining all learning experiences we experience into an interacting data rich architecture that we can use.   Like the direction of the below:

An AI Pioneer Wants His Algorithms to Understand the 'Why'
  Will Knight, in Wired

Yoshua Bengio, a researcher at the University of Montreal in Canada who is co-recipient of the 2018 ACM A.M. Turing Award for contributions to the development of deep learning, thinks artificial intelligence will not realize its full potential until it can move beyond pattern recognition and learn more about cause and effect, which would make existing AI systems smarter and more efficient. A robot that understands dropping things causes them to break, for example, would not need to toss dozens of vases onto the floor to see what happens to them. Bengio is developing a version of deep learning that can recognize simple cause-and-effect relationships. His team used a dataset that maps causal relationships between real-world phenomena in terms of probabilities. The resulting algorithm essentially forms a hypothesis about which variables are causally related, and then tests how changes to different variables fit the theory.  ... " 

Building More General, Trustable AI: Deeper Understanding?

I have just been thinking about the idea of what is called 'deep understanding'  here.  That is more generally applicable AI.    Agree that deep learning is impressive, but still very narrow  I don't agree that deep understanding, more general AI would make AI safer, could make it less transparent, prone to tricks and misuse, and dangerous.

Book:  Rebooting AI,  Building Artificial Intelligence we can Trust   By Gary Marcus and Ernest Davis  Reading ...

We can’t trust AI systems built on deep learning alone 
Gary Marcus, a leader in the field, discusses how we could achieve general intelligence—and why that might make machines safer.   by Karen Hao  in Technology Review 

Gary Marcus is not impressed by the hype around deep learning. While the NYU professor believes that the technique has played an important role in advancing AI, he also thinks the field’s current overemphasis on it may well lead to its demise.

Marcus, a neuroscientist by training who has spent his career at the forefront of AI research, cites both technical and ethical concerns. From a technical perspective, deep learning may be good at mimicking the perceptual tasks of the human brain, like image or speech recognition. But it falls short on other tasks, like understanding conversations or causal relationships. To create more capable and broadly intelligent machines, often referred to colloquially as artificial general intelligence, deep learning must be combined with other methods. ... "

Solving the Problem of AI

Stanford's Fei Fei Li Looks to Solve the Problem of Artificial Intelligence

Right after the Startup of the Year competition finished, Fei Fei Li, the director of the Artificial Intelligence Lab at Stanford University, took the stage at Innovate! and Celebrate to welcome entrepreneurs, founders, and attendees to the conference with an in-depth discussion of artificial intelligence. And when we say in-depth, that is exactly what we mean.

Learning Visual Intelligence:

Li started the discussion off by discussing the importance of vision when it comes to general intelligence. Going back more than 500 million years, she explained that there was no bigger evolutionary expansion than the development of one particular body part: the eyes.  ... "