Challenge and Workshop for Open Domain Question Answering

Answer questions based on open domain Knowledge.  Good general statement of the most important part of useful AI.   Details at the link.

Presenting a Challenge and Workshop in Efficient Open-Domain Question Answering 
\
Posted by Eunsol Choi, Visiting Faculty Researcher and Tom Kwiatkowski, Research Scientist, in Google Research Blog 

One of the primary goals of natural language processing is to build systems that can answer a user's questions. To do this, computers need to be able to understand questions, represent world knowledge, and reason their way to answers. Traditionally, answers have been retrieved from a collection of documents or a knowledge graph. For example, to answer the question, “When was the declaration of independence officially signed?” a system might first find the most relevant article from Wikipedia, and then locate a sentence containing the answer, “August 2, 1776”. However, more recent approaches, like T5, have also shown that neural models, trained on large amounts of web-text, can also answer questions directly, without retrieving documents or facts from a knowledge graph. This has led to significant debate about how knowledge should be stored for use by our question answering systems — in human readable text and structured formats, or in the learned parameters of a neural network.

Today, we are proud to announce the EfficientQA competition and workshop at NeurIPS 2020, organized in cooperation with Princeton University and the University of Washington. The goal is to develop an end-to-end question answering system that contains all of the knowledge required to answer open-domain questions. There are no constraints on how the knowledge is stored — it could be in documents, databases, the parameters of a neural network, or any other form — but entries will be evaluated based on the number of bytes used to access this knowledge, including code, corpora, and model parameters. There will also be an unconstrained track, in which the goal is to achieve the best possible question answering performance regardless of system size. To build small, yet robust systems, participants will have to explore new methods of knowledge representation and reasoning. ... " 

Alexa Asks for Answers

A kind of admission that its hard to create general answers to questions?   Further relating to my last post about this kind of approach.   Do note its similarity to what you get with Google search, linking to previous questions with similar keywords and structure.     I have noted recently an increased number of answers to questions.   Attribution indication also interesting.

Amazon Alexa is beta testing crowdsourced answers
Customers in the invitation-only program can start contributing answers to Alexa today  By Dami Lee@dami_lee in TheVerge.

Amazon announced recently that it’s beta testing Alexa Answers, an invitation-only program for users to add responses for questions that Alexa can’t answer. Amazon says that in the last month of its internal Alexa Answers beta program, 100,000 responses have been added and served to customers “millions of times.”

Starting today, selected customers who have received email invitations can participate via the Alexa Answers website. Users can browse through various topic categories, like science and geography, and choose to answer questions that have been asked by other customers that Alexa doesn’t know the answers to. Those answers may then be served to other Alexa customers, and their answer will be attributed to “an Amazon customer.”    ... " 

Google Acquires Superpod

Relating to an examination of improved question and answer interacton.

Google secretly acquires Q&A startup Superpod
 By Maria Deutscher  in SiliconAngle

Google LLC usually publicizes its acquisitions, but there are exceptions.

The search giant has secretly bought a startup called Superpod Inc. that developed a crowdsourced question-and-answer app in the same vein as Quora. Axios broke the news this morning, citing an unnamed source, and a Google spokesperson confirmed the deal.

The tipster told the publication that the search giant paid less than “less than $60 million” for Superpod. Even if the actual deal size was only a fraction of that figure, it still represents an impressive exit for the Palo Alto, California-based startup. Superpod launched its app into private beta only about a year ago and didn’t attract much in the way of public attention during its brief run.

The startup shut down the service in September, which is presumably when the acquisition was signed. Superpod co-founders Sophia Yang and William Li hinted at the deal in a note still up on the startup’s website.  ... "

Asking Questions to Gather Knowledge

The Game of Questions, once again looking at a means of questioning to gather knowledge ... we explored this idea from a game perspective.

Back in 2013, posted about this:

https://eponymouspickle.blogspot.com/2013/09/asking-questions-to-tell-story.html

https://althouse.blogspot.com/2013/09/questions-is-game-that-is-played-by.html

AI Question Answering: Slides for Talk Today

Slides for talk today.  Recording will follow.

Sept. 13 10:30 AM ET Engineered AI Still Matters for Question Answering, By J. William Murdock, IBM

Zoom meeting Link: https://zoom.us/j/7371462221
Slides and recording will end up here:  http://cognitive-science.info/community/weekly-update/

Description: Many question-answering systems rely on a significant amount of engineering effort. They often require both knowledge bases and rules, which can be very expensive to create. Even when there is significant statistical machine learning involved in these systems, there is also an enormous amount of effort spent on identifying what features are useful for the machine learning and implementing capabilities (often using knowledge bases and rules) to assign values to those features. However, in recent years an alternative approach has been growing in popularity: single-strategy systems in which one statistical model is used to address the entire task. In this presentation, I will describe work in which we pursue both approaches and also integrate the two together. I describe results across two different data sets and show that purely statistical approaches are an excellent fit for some data, but that engineered knowledge and rules remain useful for more realistic and open-ended tasks. For additional details, see our paper at http://www.cogsys.org/papers/ACSvol6/article06.pdf   ..... 

Engineered AI Still Matters for Question Answering

Upcoming CGI Meeting on a crucial area of AI application.

Via Karolyn Schalk, Manager
Executive and Technical Expertise, Cloud, AI and IT Operations

Sept. 13 10:30 AM ET Engineered AI Still Matters for Question Answering, By J. William Murdock, IBM

Zoom meeting Link: https://zoom.us/j/7371462221
Slides and recording will end up here:  http://cognitive-science.info/community/weekly-update/

Description: Many question-answering systems rely on a significant amount of engineering effort. They often require both knowledge bases and rules, which can be very expensive to create. Even when there is significant statistical machine learning involved in these systems, there is also an enormous amount of effort spent on identifying what features are useful for the machine learning and implementing capabilities (often using knowledge bases and rules) to assign values to those features. However, in recent years an alternative approach has been growing in popularity: single-strategy systems in which one statistical model is used to address the entire task. In this presentation, I will describe work in which we pursue both approaches and also integrate the two together. I describe results across two different data sets and show that purely statistical approaches are an excellent fit for some data, but that engineered knowledge and rules remain useful for more realistic and open-ended tasks. For additional details, see our paper at http://www.cogsys.org/papers/ACSvol6/article06.pdf

Bio: J. William Murdock is a researcher at IBM's Watson Research Center and a development manager for the IBM Watson Discovery product. In 2001, he received a Ph.D. in Computer Science from Georgia Tech, where he was a member of Ashok Goel's Design & Intelligence Laboratory. He worked as a post-doc with David Aha at the United States Naval Research Laboratory. Dr. Murdock has been working on the IBM Watson question answering system since the project began in 2007. He was the guest editor of the 2012 special issue of the IBM Journal of Research and Development entitled "This is Watson." His research interests include question answering, natural-language semantics, analogical reasoning, knowledge-based planning, machine learning, and self-aware artificial intelligence.  ... " 

Alexa will be a Concierge

Its the ultimate kind of intelligence.  We called it a concierge skill.  Its not what you know, its who you know, or what you have that knows, and can apply it to your need.    Its the simplest and most useful kind of dialog.  The beginnings of AI.  Now how well will it do that?

Alexa will recommend third-party skills for things it can’t do
The feature is limited for now but should get better over time.  By Rob LeFebvre, @roblef

When Amazon's Alexa doesn't know how the answer to a question, the digital assistant typically just says so. Now, though, instead of "Hmmmm. I don't know that," Alexa may start to recommend third-party skills to help you out more effectively.

In a video shot by Voicebot's Bret Kinsella, you can clearly hear Alexa recommending a non-Amazon skill to answer his question about Apple stock prices. According to Kinsella, after Alexa asked if he wanted to use a skill to answer his stock query, it went straight to a genera listing of prices instead of addressing the specific question he originally asked.  ... " 

Bots Asking Questions

A natural next step.  In any conversation, questions go in both directions.  With initiative from both sides. To test understanding, but also to establish context and progress towards a goal.  To gather information to feed algorithms.   To link with expertise, human and artificial.  Still not easily modeled well, but will be crucial for assistants.  How natural must such interaction be?

Inquisitive bot asks questions to test your understanding
By Matthew Reynolds

Inquisitive artificial intelligence that asks questions about things it reads could be used to quiz students in class. The question-asking ability would also help chatbots with the back and forth of human conversation.

AI is usually on the receiving end of queries, says Xinya Du at Cornell University in Ithaca, New York. Du and his colleagues have turned the tables by building a system that has learned to ask questions of its own.

This is something that people have been wanting to do for a long time, says Karen Mazidi at the University of Dallas in Texas. Previous attempts by other people using hand-coded rules haven’t been particularly successful. .... " 

Simple Bots for Questions and Answers

This was brought to my attention via Mary J Foley, who has a new site for innovation community.  Here is a current entry:   https://www.petri.com/community-champs/gary-pretty

QnA Maker Dialog for Bot Framework  By Gary Pretty

 Microsoft Bot Framework
The QnA Maker service from Microsoft, part of the Cognitive Services suite, allows you to quickly build, train and publish a question and answer bot service based on FAQ URLs or structured lists of questions and answers. Once published you can call a QnA Maker service using simple HTTP calls and integrate it with applications, including bots built on the Bot Framework.

Right now, out of the box, you will need to roll your own code / dialog within your bot to call the QnA Maker service. The new QnAMakerDialog which is now available via NuGet aims to make this integration even easier, by allowing you to integrate with the service in just a couple of minutes with virtually no code.   ... " 

    

Fielding a Hundred Thousand Questions

Via O'Reilly.   We did something similar when creating our Extempo bot system, but only included a few hundred questions.  Nice to see Microsoft pushing research with this.    How many parameterized questions does a human ask in normal conversation?   How does this relate to context? To get to useful answers?  Good, largly non-technical article.  If you are at all interested in bot like interaction, worth a look.  Thinking of feeding selected elements of this through some current assistants.

100,000 questions (and answers)
Microsoft has released a set of 100,000 questions and answers that you can use to create AI systems that can read and answer questions as well as a human. The team behind the dataset hopes that releasing it will spur the kind of breakthroughs in machine reading that are already happening in image and speech recognition.  ... " 

Biologically Inspired Question Answering

 Via Jim Spohrer.    See the previous CSIG talk on this work.

In Kurzweil AI: IBM’s Watson shown to enhance human-computer co-creativity, support biologically inspired design ... Watson Engagement Advisor AI system was trained to "learn" about biologically inspired design from biology articles, then answer questions.

Georgia Institute of Technology researchers, working with student teams, trained a cloud-based version of IBM’s Watson called the Watson Engagement Advisor to provide answers to questions about biologically inspired design (biomimetics), a design paradigm that uses biological systems as analogues for inventing technological systems.

Ashok Goel, a professor at Georgia Tech’s School of Interactive Computing who conducts research on computational creativity. In an experiment, he used this version of Watson as an “intelligent research assistant” to support teaching about biologically inspired design and computational creativity in the Georgia Tech CS4803/8803 class on Computational Creativity in Spring 2015. Goel found that Watson’s ability to retrieve natural language information could allow a novice to quickly “train up” about complex topics and better determine whether their idea or hypothesis is worth pursuing.  ... " 

What Kinds of Questions Can We Answer?

What kinds of questions can data science answer?  Fascinating piece by Microsoft Data Scientist. Very fundamental idea.   Good way to present the idea to management.

" ... This post, authored by Brandon Rohrer, Senior Data Scientist at Microsoft, is the second in a three-part series introducing data science with no jargon. The first post covered What Can Data Science Do For Me?

Machine learning (ML) is the motor that drives data science. Each ML method (also called an algorithm) takes in data, turns it over, and spits out an answer. ML algorithms do the part of data science that is the trickiest to explain and the most fun to work with. That’s where the mathematical magic happens. 

ML algorithms can be grouped into families based on the type of question they answer. These can help guide your thinking as you are formulating your razor sharp question. ... " 

Read it all for the examples, will read the other parts of the series myself.

Cognitive Agents in the Wild... the Jeopardy Experience

Excellent CSIG talk today: Slides here.   " ... Our speaker for this week is Eric Nyberg from Carnegie Mellon University, who will present " From Jeopardy! To Cognitive Agents: Effective Learning in the Wild". ... "  They have made considerable progress since we worked with them in the 90s.

Eric Nyberg's home page, with many more links.