Microsoft Makes AI Debugging Tool Open Source

And with similarities to Google's What-If tool, Microsoft has made its debugging tool open source.  Have not seen this one directly as yet,  but this is NOT a No-code solution, likely making it harder for management to understand.

Microsoft Open-Sources TensorWatch AI Debugging Tool   by Anthony Alford in InfoQ

Microsoft Research open-sourced TensorWatch, their debugging tool for AI and deep-learning. TensorWatch supports PyTorch as well as TensorFlow eager tensors, and allows developers to interactively debug training jobs in real-time via Jupyter notebooks, or build their own custom UIs in Python.

In a recent blog post, a research team led by Shital Shah announced the open-source release of TensorWatch. TensorWatch is a Python library for visualizing data from all phases of the deep-learning model-development cycle: from model structure, to training metrics, to explanation of model predictions. TensorWatch is designed to be used as an interactive tool in Jupyter notebooks or JupyterLab dashboards, but as a Python library it can also be included in custom tools and UIs. According to the development team:

"We like to think of TensorWatch as the Swiss Army knife of debugging tools with many advanced capabilities researchers and engineers will find helpful in their work."  ... " 

More from the MS Research Blog.

Google What-IF Tool for Code Free ML Visualization

Like the idea of visual tools that map with specific process, resource needs and output results.  Leads to better understandable and resilient results.

Google's What-If Tool And The Future Of Explainable AI
Kalev Leetaru Contributor in Forbes
AI & Big Data

(Excerpt)

" ..... As deep learning has matured sufficiently to find widespread adoption in industry and as developers require increasingly greater understanding of their creations in order to pioneer new advances, the AI community has begun investing heavily in explainable AI as a way to render their black boxes transparent.

Google has been an early leader in emphasizing interpretability and how practitioners can build more understandable, representative and resilient AI solutions. Last year the company unveiled its What-If Tool, which offers a range of interactive visualizations and guided explorations of a TensorFlow model, allowing developers to explore how their model interpreted its training data and how subtle changes to a given input would change its classification, yielding insights into the model’s robustness. .... " 

Google's Description:

The What-If Tool: Code-Free Probing of Machine Learning Models
Tuesday, September 11, 2018
Posted by James Wexler, Software Engineer, Google AI

Building effective machine learning (ML) systems means asking a lot of questions. It's not enough to train a model and walk away. Instead, good practitioners act as detectives, probing to understand their model better: How would changes to a datapoint affect my model’s prediction? Does it perform differently for various groups–for example, historically marginalized people? How diverse is the dataset I am testing my model on?

Answering these kinds of questions isn’t easy. Probing “what if” scenarios often means writing custom, one-off code to analyze a specific model. Not only is this process inefficient, it makes it hard for non-programmers to participate in the process of shaping and improving ML models. One focus of the Google AI PAIR initiative is making it easier for a broad set of people to examine, evaluate, and debug ML systems.


Today, we are launching the What-If Tool, a new feature of the open-source TensorBoard web application, which let users analyze an ML model without writing code. Given pointers to a TensorFlow model and a dataset, the What-If Tool offers an interactive visual interface for exploring model results.  .... " 

Analyze an ML Model without More Coding

Once a system has been taught it acts as an experimental model of its learning.  When we built such neural models you had to write code or use data manipulation tools to test against new data sets.  Classic method for all analytic models, not only ML.  This new tool should make it easier.

Google's new What-If Tool "allows users to analyze a machine learning model without the need for writing any further code. Given pointers to a TensorFlow model and a dataset, the What-If Tool offers an interactive visual interface for exploring model results."

What If...
you could inspect a machine learning model, with no coding required?
Building effective machine learning systems means asking a lot of questions. It's not enough to train a model and walk away. Instead, good practitioners act as detectives, probing to understand their model better.

But answering these kinds of questions isn't easy. Probing "what if" scenarios often means writing custom, one-off code to analyze a specific model. Not only is this process inefficient, it makes it hard for non-programmers to participate in the process of shaping and improving machine learning models. For us, making it easier for a broad set of people to examine, evaluate, and debug machine learning systems is a key concern.

That's why we built the What-If Tool. Built into the open-source TensorBoard web application - a standard part of the TensorFlow platform - the tool allows users to analyze an machine learning model without the need for writing any further code. Given pointers to a TensorFlow model and a dataset, the What-If Tool offers an interactive visual interface for exploring model results.... "