More on the topic, and some additional background information and links. Very powerful concept that that should continue to expand. Automation is the word, See also Google AutoML, at tag below.
MIT Researchers Open-Source AutoML Visualization Tool ATMSeer by Anthony Alford ...
A research team from MIT, Hong Kong University, and Zhejiang University has open-sourced ATMSeer, a tool for visualizing and controlling automated machine-learning processes.
Solving a problem with machine learning (ML) requires more than just a dataset and training. For any given ML tasks, there are a variety of algorithms that could be used, and for each algorithm there can be many hyperparameters that can be tweaked. Because different values of hyperparameters will produce models with different accuracies, ML practitioners usually try out several sets of hyperparameter values on a given dataset to try to find hyperparameters that produce the best model.
This can be time-consuming, as a separate training job and model evaluation process must be conducted for each set. Of course, they can be run in parallel, but the jobs must be setup and triggered, and the results recorded. Furthermore, choosing the particular values for hyperparameters can involve a bit of guesswork, especially for ones that can take on any numeric value: if 2.5 and 2.6 produce good results, maybe 2.55 would be even better? What about 2.56 or 2.54?
Enter automated machine learning, or AutoML. These are techniques and tools for automating the selection and evaluation of hyperparameters (as well as other common ML tasks such as data cleanup and feature engineering). Both Google Cloud Platform and Microsoft Azure provide commercial AutoML solutions, and there are several open-source packages such as auto-sklearn and Auto-Keras. ...."
Showing posts with label ATMSeer. Show all posts
Showing posts with label ATMSeer. Show all posts
Tuesday, June 18, 2019
Friday, May 31, 2019
Advances in Automated Machine Learning
Automated machine learning is inevitable. How good will it be, and how much human oversight needs to be applied to ensure confidence in their results is important. This article is a good overview of work underway. Its no only about searching for the right model, in context of needed goals, its also about maintaining the interaction between data and solutions. Not too unlike the use of any kind of analytics optimization, which has been studied for years.
Cracking open the black box of automated machine learning
Interactive tool lets users see and control how automated model searches work.
By Rob Matheson | MIT News Office
Researchers from MIT and elsewhere have developed an interactive tool that, for the first time, lets users see and control how automated machine-learning systems work. The aim is to build confidence in these systems and find ways to improve them.
Designing a machine-learning model for a certain task — such as image classification, disease diagnoses, and stock market prediction — is an arduous, time-consuming process. Experts first choose from among many different algorithms to build the model around. Then, they manually tweak “hyperparameters” — which determine the model’s overall structure — before the model starts training.
Recently developed automated machine-learning (AutoML) systems iteratively test and modify algorithms and those hyperparameters, and select the best-suited models. But the systems operate as “black boxes,” meaning their selection techniques are hidden from users. Therefore, users may not trust the results and can find it difficult to tailor the systems to their search needs.
In a paper presented at the ACM CHI Conference on Human Factors in Computing Systems, researchers from MIT, the Hong Kong University of Science and Technology (HKUST), and Zhejiang University describe a tool that puts the analyses and control of AutoML methods into users’ hands. Called ATMSeer, the tool takes as input an AutoML system, a dataset, and some information about a user’s task. Then, it visualizes the search process in a user-friendly interface, which presents in-depth information on the models’ performance.
“We let users pick and see how the AutoML systems works,” says co-author Kalyan Veeramachaneni, a principal research scientist in the MIT Laboratory for Information and Decision Systems (LIDS), who leads the Data to AI group. “You might simply choose the top-performing model, or you might have other considerations or use domain expertise to guide the system to search for some models over others.”
In case studies with science graduate students, who were AutoML novices, the researchers found about 85 percent of participants who used ATMSeer were confident in the models selected by the system. Nearly all participants said using the tool made them comfortable enough to use AutoML systems in the future. ... "
Also discusses Auto-Tuned Models ATMs ... "
Cracking open the black box of automated machine learning
Interactive tool lets users see and control how automated model searches work.
By Rob Matheson | MIT News Office
Researchers from MIT and elsewhere have developed an interactive tool that, for the first time, lets users see and control how automated machine-learning systems work. The aim is to build confidence in these systems and find ways to improve them.
Designing a machine-learning model for a certain task — such as image classification, disease diagnoses, and stock market prediction — is an arduous, time-consuming process. Experts first choose from among many different algorithms to build the model around. Then, they manually tweak “hyperparameters” — which determine the model’s overall structure — before the model starts training.
Recently developed automated machine-learning (AutoML) systems iteratively test and modify algorithms and those hyperparameters, and select the best-suited models. But the systems operate as “black boxes,” meaning their selection techniques are hidden from users. Therefore, users may not trust the results and can find it difficult to tailor the systems to their search needs.
In a paper presented at the ACM CHI Conference on Human Factors in Computing Systems, researchers from MIT, the Hong Kong University of Science and Technology (HKUST), and Zhejiang University describe a tool that puts the analyses and control of AutoML methods into users’ hands. Called ATMSeer, the tool takes as input an AutoML system, a dataset, and some information about a user’s task. Then, it visualizes the search process in a user-friendly interface, which presents in-depth information on the models’ performance.
“We let users pick and see how the AutoML systems works,” says co-author Kalyan Veeramachaneni, a principal research scientist in the MIT Laboratory for Information and Decision Systems (LIDS), who leads the Data to AI group. “You might simply choose the top-performing model, or you might have other considerations or use domain expertise to guide the system to search for some models over others.”
In case studies with science graduate students, who were AutoML novices, the researchers found about 85 percent of participants who used ATMSeer were confident in the models selected by the system. Nearly all participants said using the tool made them comfortable enough to use AutoML systems in the future. ... "
Also discusses Auto-Tuned Models ATMs ... "
Subscribe to:
Posts (Atom)