IBM Enables AI Enabled Debate

This could lead to AI being able to make 'arguments' as parts of business process.  Could be used broadly, for example in courts and as parts of smart contract testing.   Or to test advertising pitches in context.  Truly a newly emergent part of AI.  An improved model for crowd sourcing?

IBM showcases latest A.I. advancements on Bloomberg's "That's Debatable" TV show

Software, which IBM hopes to sell to businesses, distills 'key points' from thousands of individual comments  ... "

More from IBM on Project Debater  ....    Related Blog on Project Debater

Code as Evidence in Contracts, Disputes

A UK piece on the topic  of computing and dispute resolutions, code as evidence, which came up with respect to smart contracts.  Click through for useful and detailed links. 

The role of usability, power dynamics, and incentives in dispute resolutions around computer evidence  in Bentham’s Gaze by Alexander Hicks  

As evidence produced by a computer is often used in court cases, there are necessarily presumptions about the correct operation of the computer that produces it. At present, based on a 1997 paper by the Law Commission, it is assumed that a computer operated correctly unless there is explicit evidence to the contrary.

The recent Post Office trial (previously mentioned on Bentham’s Gaze) has made clear, if previous cases had not, that this assumption is flawed. After all, computers and the software they run are never perfect.

This blog post discusses a recent invited paper published in the Digital Evidence and Electronic Signature Law Review titled The Law Commission presumption concerning the dependability of computer evidence. The authors of the paper, collectively referred to as LLTT, are Peter Bernard Ladkin, Bev Littlewood, Harold Thimbleby and Martyn Thomas.

LLTT examine the basis for the presumption that a computer operated correctly unless there is explicit evidence to the contrary. They explain why the Law Commission’s belief in Colin Tapper’s statement in 1991 that “most computer error is either immediately detectable or results from error in the data entered into the machine” is flawed. Not only can computers be assumed to have bugs (including undiscovered bugs) but the occurrence of a bug may not be noticeable.  ... "

IBM Upgrades Debate AI Tool to Better Derive Evidence

Intriguing approach to mining information to support a goal directed conversation.   A key aspect to making conversational systems more powerful.  Note also the crowdsourcing integrated here to grade evidence.  Noting that the report here does not mention 'Watson', it seems IBM is using their AI trademark much less these days.

IBM's Debating AI Just Got a Lot Closer to Being a Useful Tool
By MIT Technology Review via CACM

The IBM Debater system taking part in a debate at the University of Cambridge last year.
IBM upgraded the neural networks used by its Project Debater system, to improve the quality of evidence the argument-mining system uncovers.

IBM upgraded the neural networks used by its Project Debater system to improve the quality of evidence the argument-mining system uncovers.

One new add-on for the debating system is BERT (Bidirectional Encoder Representations from Transformers), a network designed by Google for natural language processing and answering queries.

IBM Research scientists trained the AI on 400 million documents from the LexisNexis database, providing a natural language dataset of roughly 10 billion sentences; the researchers combined the dataset with claims about several hundred different topics, then had crowdsourced workers label the sentences based on the quality of their evidence for or against specific claims.

A supervised learning algorithm digested this data, allowing BERT to manage queries on a wide range of subjects and to yield more relevant sentences compared to previous systems.

Project Debater was 95% accurate for the top 50 sentences across 100 distinct topics, according to IBM researcher Noam Slonim,  ... " 

Cloud Based CyberCrime Evidence Detection

Links to other approaches I have recently seen in this area.

CIT Researchers Develop System to Detect Cloud-Based Cybercrime Evidence
Purdue University News
John O'Malley

Researchers at the Purdue Polytechnic Institute's Department of Computer and Information Technology (CIT) have developed a forensic model that uses machine learning to collect digital evidence related to unlawful activities on cloud storage apps. The technology performs real-time identification and analysis of cybercrime-related incidents via transactions uploaded to the apps. An app user uploading a media file causes the system to implement deep learning models to scan images for cybercrime evidence and report illegal activities through a forensic evidence collection system. Cloud service providers can compile alert logs, block associated accounts, and notify law enforcement based on a cloud search warrant request. CIT's Fahad Salamh said, "It is important to automate the process of digital forensic and incident response in order to cope with advanced technology and sophisticated hiding techniques and to reduce the mass storage of digital evidence on cases involving cloud storage applications."  ... ' 

Knowledge Engineering Book for Cognitive Assistants

So how do you actually engineer knowledge to integrate into an assistant system? Its a mix of knowledge architecture, reasoning and effective delivery.  Many of our early attempts struggled with the idea. This book takes a look at approaches.  I will be reviewing.  From GMU.

Knowledge Engineering: Building Cognitive Assistants for Evidence-based Reasoning    Authors: Gheorghe Tecuci, Dorin Marcu, Mihai Boicu, David A. Schum
Hardcover: 456 pages
Publisher: Cambridge University Press (September 2016)
ISBN: 9781107122567

This book presents a significant advancement in the theory and practice of knowledge engineering, the discipline concerned with the development of intelligent agents that use knowledge and reasoning to perform problem-solving and decision-making tasks. It covers the main stages in the development of a knowledge-based agent: understanding the application domain, modeling problem solving in that domain, developing the ontology, learning the reasoning rules, and testing the agent. The book focuses on a special class of agents: cognitive assistants for evidence-based reasoning that learn complex problem-solving expertise directly from human experts, support experts, and nonexperts in problem solving and decision making, and teach their problem-solving expertise to students. A powerful learning agent shell, Disciple-EBR, is included with the book, enabling students, practitioners, and researchers to develop cognitive assistants rapidly in a wide variety of domains that require evidence-based reasoning, including intelligence analysis, cybersecurity, law, forensics, medicine, and education.  .... " 

Discussion in Cognitive Systems Institute,  Linkedin.

Intelligence Analysis

New Book of Of interest.  How are these kinds of structure embedded into AI?

Intelligence Analysis as Discovery of Evidence, Hypotheses, and Arguments: Connecting the Dots
by Gheorghe Tecuci, David A. Schum, Dorin Marcu, Mihai Boicu 

This unique book on intelligence analysis covers several vital but often overlooked topics. It teaches the evidential and inferential issues involved in "connecting the dots" to draw defensible and persuasive conclusions from masses of evidence: from observations we make, or questions we ask, we generate alternative hypotheses as explanations or answers; we make use of our hypotheses to generate new lines of inquiry and discover new evidence; and we test the hypotheses with the discovered evidence. 

To facilitate understanding of these issues and enable the performance of complex analyses, the book introduces an intelligent analytical tool, called Disciple-CD. Readers will practice with Disciple-CD and learn how to formulate hypotheses; develop arguments that reduce complex hypotheses to simpler ones; collect evidence to evaluate the simplest hypotheses; and assess the relevance and the believability of evidence, which combine in complex ways to determine its inferential force and the probabilities of the hypotheses. .... " 

Judea Pearl on Engines of Evidence

A favorite researcher on the topic.  How do we understand how evidence models results?  In the Edge: 

Engines of Evidence,  A Conversation With Judea Pearl
A new thinking came about in the early '80s when we changed from rule-based systems to a Bayesian network. Bayesian networks are probabilistic reasoning systems. An expert will put in his or her perception of the domain. A domain can be a disease, or an oil field—the same target that we had for expert systems. 

The idea was to model the domain rather than the procedures that were applied to it. In other words, you would put in local chunks of probabilistic knowledge about a disease and its various manifestations and, if you observe some evidence, the computer will take those chunks, activate them when needed and compute for you the revised probabilities warranted by the new evidence.

It's an engine for evidence. It is fed a probabilistic description of the domain and, when new evidence arrives, the system just shuffles things around and gives you your revised belief in all the propositions, revised to reflect the new evidence.         

JUDEA PEARL, professor of computer science at UCLA, has been at the center of not one but two scientific revolutions. First, in the 1980s, he introduced a new tool to artificial intelligence called Bayesian networks. This probability-based model of machine reasoning enabled machines to function in a complex, ambiguous, and uncertain world. Within a few years, Bayesian networks completely overshadowed the previous rule-based approaches to artificial intelligence.

Leveraging the computational benefits of Bayesian networks, Pearl realized that the combination of simple graphical models and probability (as in Bayesian networks) could also be used to reason about cause-effect relationships. The significance of this discovery far transcends its roots in artificial intelligence. His principled, mathematical approach to causality has already benefited virtually every field of science and social science, and promises to do more when popularized. 

He is the author of Heuristics; Probabilistic Reasoning in Intelligent Systems; and Causality: Models, Reasoning, and Inference. He is the winner of the Alan Turing Award.  .... " 

Stephen Hawking Opens British AI Hub

Will be interesting to see the varying views of Hawking and colleagues on different aspects of AI.  Building these 'Centers' for studying complex and disruptive tech seems to be growing.  Following.

Stephen Hawking Opens British Artificial Intelligence Hub
Agence France-Presse (10/19/16) 

Scientist Stephen Hawking on Wednesday opened an artificial research (AI) center at the U.K.'s Cambridge University. Funded by a $12.3-million grant from the Leverhulme Trust, the Leverhulme Centre for the Future of Intelligence (CFI) will bring together researchers, industry representatives, and policymakers to make sure AI technology is used to benefit humanity. The ethics of AI is a core concern for Hawking, who has warned the technology's misuse could pose serious risks to civilization. "It will bring disruption to our economy," Hawking says. "And in the future, AI could develop a will of its own--a will that is in conflict with ours." Researchers will be tasked with developing systems that have goals aligned with human values and are sufficiently trustworthy. The center also will pursue projects ranging from the regulation of autonomous weapons to the impact of AI on democracy. "We don't need to see AI as replacing us, but can see it as enhancing us: we will be able to make better decisions, on the basis of better evidence and better insights," says Stephen Cave, the center's director. "AI will help us to learn about ourselves and our environment--and could, if managed well, be liberating."  ... " 

Bots Fighting Online

The Growing Problem of Bots That Fight Online

The way software agents interact on the Web is poorly understood. Now evidence shows that they fight each other for years.

by Emerging Technology from the arXiv  September 20, 2016

Software agents, or bots, permeate the Web. They gather data about Web pages, they correct vandalism on Wikipedia, they generate spam and even emulate humans.

And their impact is growing. By some measures, bots account for 49 percent of visits to Web pages and are responsible for over 50 percent of clicks on ads. This impact is set to increase as the number of bots rises exponentially.

“An increasing number of decisions, options, choices, and services depend now on bots working properly, efficaciously, and successfully,” say Taha Yasseri and pals at the University of Oxford in the U.K.  “Yet, we know very little about the life and evolution of our digital minions.”

This raises an interesting question. How do bots interact with each other? And how do these interactions differ from the way humans interact?  ... " 

RetroScope

RetroScope Opens Doors to the Past in Smartphone Investigations
By Purdue University News

August 4, 2016

Researchers at Purdue University are developing a technique that could help law enforcement recover evidence from smartphones when investigating crimes.

The technique, RetroScope, gathers data from a device's random-access memory, which is more volatile than the information stored on a phone's hard drive.

"We argue this is the frontier in cybercrime investigation in the sense that the volatile memory has the freshest information from the execution of all the apps," says Purdue professor Dongyan Xu. "Investigators are able to obtain more timely forensic information toward solving a crime or an attack."  ... " 

Data and Decisions

Its not only about data, ever.  Its about decisions in the context of process and environment. Cisco's take.

In Cisco Blog: 
In a recent article,  CEOs Must Up Level Their Digital-Decisions Skills, Thorton May highlights that executives need to work on improving the digital-decision making process, affirming that EVERYONE is a digital decision maker.  The underlying message being that organizations must improve digital decision making of non-IT executives.

With more emphasis on evidence-based management, getting the right data to the right people at the right time is critical to empowering everyone to be a decision maker.

The two most critical elements to quality decisions are: the amount of time available to make a decision and the information available to the decision maker. ... " 

Why and How is Watson a Cognitive System?

Jim Spohrer sends along a link to a talk by Rob High, IBM Chief Technology Officer, Fellow and Vice President,  Does a good job of discussing why Watson is cognitive, unique, and why it differs from previous attempts at delivering expertise.   Non technical.

I ask: So how you convince a complex organization this uniqueness is useful, stable and scalable? Only through clear operational examples in varying contexts.  That appears to be what they are working on.

Abstract

IBM® Watson™ represents a first step into cognitive systems, a new era of computing. Watson builds on the current era of programmatic computing but differs in significant ways. The combination of the following capabilities makes Watson unique:

Natural language processing by helping to understand the complexities of unstructured data, which makes up as much as 80 percent of the data in the world today

Hypothesis generation and evaluation by applying advanced analytics to weigh and evaluate a panel of responses based on only relevant evidence

Dynamic learning by helping to improve learning based on outcomes to get smarter with each iteration and interaction

Although none of these capabilities alone are unique to Watson, the combination delivers a powerful solution:

To move beyond the constraints of programmatic computing
To move from reliance on structured, local data to unlock the world of global, unstructured data
To move from decision tree-driven, deterministic applications to probabilistic systems that co-evolve with their users

To move from keyword-based search that provides a list of locations where an answer might (or might not) be located, to an intuitive, conversational means of discovering a set of confidence-ranked responses

This IBM Redguide™ publication describes how Watson combines natural language processing, dynamic learning, and hypothesis generation and evaluation to give direct, confidence-based responses.

For additional information about how Watson can transform how organizations think, act, and operate in the future, see the IBM Redbooks Point-of-View publication "Transforming the Way Organizations Think with Cognitive Systems", REDP-4961:

Table of contents:
What language is and why it is hard for computers to understand
IBM Watson understands language
Understanding language is just the beginning
Problems come in different shapes
Accuracy is improved through generalization  ... 

Layoffs Hurt Companies

The cost of layoffs to companies. In Knowledge@Wharton:  

' ... Contrary to popular belief, there’s not much evidence that layoffs are a cure for weak profits, or, to use the current euphemism, that they reposition a firm for growth going forward. “It’s very difficult to sort out the relationship because firms that are laying off are almost by definition in trouble,” says Peter Cappelli, Wharton management professor and director of the school’s Center for Human Resources. “The research evidence has not found any support for the overall idea that layoffs help firm performance. There is more support for the idea that where there is overcapacity, such as a market downturn, layoffs help firms. There is no evidence that cutting to improve profitability helps beyond the immediate, short-term accounting bump.”  ... '  

Exploring the Uncanny Valley of Bots

Have long been a student and practitioner of engineering how people react to intelligent machines. And here we mean by a depth beyond just posing and answering questions, but how people actually engage, trust and build some relationship with machines.

One aspect of this,  that came out of robotics, is the idea of an 'uncanny valley', where people are averse to machines that seem too human-like.  Can also be applied to bots, as described below.  From the CACM:

The Edge of the Uncanny By Gregory Mone 
Communications of the ACM, Vol. 59 No. 9, Pages 17-19

" ... Mitsuku is quick-witted, occasionally confusing, and strangely engaging. She is also a chatbot, built from the A.L.I.C.E. (Artificial Linguistic Internet Computer Entity) platform originally developed by Richard Wallace in 1995. She conducts hundreds of thousands of conversations daily, according to Lauren Kunze, principal of Pandorabots, the Oakland, CA-based company behind the technology. "She doesn't really do anything," Kunze says. "She's not designed to assist you. She can tell you the weather or perform an Internet search, but she's really just there to talk to you, and she's wildly popular with teens. People say, 'I love you' and 'you're my best friend.'"

The appeal is not accidental. The designers of chatbots like Mitsuku and the engineers of physical social robots have made significant advances in their understanding of how to build more engaging machines. Yet there are still many challenges, one of which is the unpredictability of humans. "We just don't understand how people are going to react to physical or software robots," says University of Southern California computer scientist Yolanda Gil, chair of SIGAI, ACM's Special Interest Group on Artificial Intelligence. "This is one kind of technology where people continue to surprise us."

While there are no absolute guidelines for building effective social robots or engaging chatbots, a few common themes have emerged.

Uncanny Expectations

One frequently cited theory in social robotics is the Uncanny Valley, first described by Japanese roboticist Masahiro Mori in 1970. The Uncanny Valley contends there is a risk in building machines that are too human, that instead of attracting people, realistic androids can have a repulsive effect because of their "uncanny" resemblance to real humans. The reasons for the aversion are varied. Researchers have found evidence that highly capable androids bother people because they represent a threat to human uniqueness, or that on a subconscious level, they actually remind us of corpses. ... " 

Foundational Course for Statistical Thinking

My interaction with Columbia University led me to the following course.  This is meant as a preparatory foundation course for those taking graduate professional level Analytics classes. like the ones I will be teaching.  More information to follow.

Statistical Thinking for Data Science and Analytics  .... 

Learn how statistics plays a central role in the data science approach.

This statistics and data analysis course will pave the statistical foundation for our discussion on data science.  You will learn how data scientists exercise statistical thinking in designing data collection, derive insights from visualizing data, obtain supporting evidence for data-based decisions and construct models for predicting future trends from data.

This is the first course in the three-part Data Science and Analytics XSeries.  ....   " 

Half of Web Traffic is non-Human

In CWorld: I recall a statement like this before, and have also seen evidence of this when examining peaks in web analytics.   The volume still amazes me.   It must be very easy to do, and have real incentive. You do need to be careful about using web stats.

" ...  "We used to think of bots as passive ambient noise," Cremin said. "That's now changed to the point where they actually interact with the sites they visit and mimic human traffic exactly." ... Bots are commonly used to generate "clicks" and false ad revenue, but in some cases, they make purchases online with the goal of influencing prices, Cremin said. ... " 

Multimodal Robotic Drones

From the University of PA Grasp lab, in IEEE Spectrum.    Bimodal, here meaning it can walk or fly.   Applications that make sense?     Bi specialization for specific environments or needs.

" ... Some of the most versatile and adaptable robots also exhibit multimodal characteristics: they can fly and climb, or jump and glide, or even fly and swim. But flying and walking seems to be by far the most useful combination, as evidenced by the variety of animals that can do it, and researchers at the University of Pennsylvania’s GRASP Laboratory have designed a new robot called Picobug that can fly, walk, and even (soon) grab on to stuff. ... " 

Caution with the Statistical P Value

Statistical measures like P-Values and R Squares are dragged out to prove a number of things.  But caution should be considered.  This Nature article does a good job of explaining the needed cautions:

" ... Scientific method: Statistical errors
P values, the 'gold standard' of statistical validity, are not as reliable as many scientists assume.
by Regina Nuzzo     ... " 

The final quote in the article brings us back to how any study, analytic or statistical should be considered.  It is about the process involved.  

  "  .... Statistician Richard Royall of Johns Hopkins Bloomberg School of Public Health in Baltimore, Maryland, said that there are three questions a scientist might want to ask after a study: 'What is the evidence?' 'What should I believe?' and 'What should I do?' One method cannot answer all these questions, Goodman says: “The numbers are where the scientific discussion should start, not end.  .... " 

A Crusade Against Multiple Regression Analysis

In the Edge
Yes, well known, but often ignored,  is all the context in the model?  Again in the realm of misusing statistics.  A lengthy conversation with Richard Nisbett.  A crusade, he says, that may be worth taking note of.

" .... The thing I’m most interested in right now has become a kind of crusade against correlational statistical analysis—in particular, what’s called multiple regression analysis. Say you want to find out whether taking Vitamin E is associated with lower prostate cancer risk. You look at the correlational evidence and indeed it turns out that men who take Vitamin E have lower risk for prostate cancer. Then someone says, "Well, let’s see if we do the actual experiment, what happens." And what happens when you do the experiment is that Vitamin E contributes to the likelihood of prostate cancer. How could there be differences? These happen a lot. The correlational—the observational—evidence tells you one thing, the experimental evidence tells you something completely different. ... " 

Does Gamification Work?

In K@W:  A look at the evidence. Everyone agrees that games are engaging, and they can be designed,  built and delivered.  They can get you to a place where your presence has value.  But can the concepts in games be built into things that people dislike?   Our own experience is yes.  But its about culture, context and design.   Technology is only a framework.  Not necessarily easy.   Good piece.  (See the next article in the tag below for an alternate view)