Measuring Results

How Accurate Is Your AI? 
from Kyoto University

A researcher at Kyoto University in Japan has developed a new technique that evaluates artificial intelligence's (AI) performance based solely on the input data. In typical AI development, a performance evaluation is trusted if there is an equal number of positive and negative results, and data biased toward either value means the current system of evaluation will distort the system's ability. "The novelty of this technique is that it doesn't depend on any one type of AI technology, such as deep learning," says Kyoto's J.B. Brown. "It can help develop new evaluation metrics by looking at how a metric interplays with the balance in predicted data. We can then tell if the resulting metrics could be biased." Brown's work breaks down the AI utilization and analyzes the nature of the statistics used for reporting an AI's ability, while also producing a probability of the performance level, given evaluation data. .... "