Doctors Train in Virtual Reality

Doctors Train in Virtual Reality

By Jake Widman

Commissioned by CACM Staff, June 22, 2023

Surgeons collaborate during virtual surgery.

In addition to relevant tools and a patient, the simulations can provide highlights and tool tips or floating diagrams to guide a trainee through performing the surgery.

Healthcare is just one of the many fields finding new uses for extended reality (XR), comprising both augmented and virtual reality.

Virtual reality (VR) in particular is being used for everything from hosting remote support groups for teen cancer patients to giving medical students a taste of the grisly reality of trauma medicine.

Those projects typically are one-off purpose-built implementations of VR, though. The technology has continued to advance and its availability to increase, enabling the creation of more general-purpose applications.

The potential is not lost on VR developers and investors. According to the June 2022 Global AR-VR in Healthcare Market report from Research and Markets, the sector was valued at $2,748 million in 2021, and is expected to reach $9,796 million by 2027.

Two applications identified by Research and Markets as drivers of that growth are medical training and surgery planning.

The U.K.'s Royal College of Surgeons has set up an commission to examine where surgery is headed, and one of its commissioners, plastic surgeon and medical entrepreneur Dr. Nadine Hachach-Haram, told MobiHealthNews, "We can see the effective use of virtual reality in simulation, in terms of preparation and rehearsal. The challenge here is the cost of scaling, as there are tens of thousands of different procedures with nuances in each of them."

Several companies are tackling the scaling problem.

How surgeons now train

A main value of VR simulation, as exemplified by its use to expose medical students to trauma situations, is to give surgeons an idea of what they can expect to encounter in actual surgery.

Surgery is traditionally taught as a mentorship program, explains Dr. Michael Ast, an orthopedic surgeon and Chief Medical Innovation Officer at New York's Hospital for Special Surgery. "We do didactic teaching in a modified classroom setting, and then we guide our trainees through the operations. Eventually, they start to participate in the procedure."

The problem with that approach is the limited number of those "tens of thousands of different procedures with nuances in each of them" in which a trainee might actually get to participate. Also, those 'nuances' are not about just the mechanics of the surgery, says Dr. Ast; "The farther along into training, the more our trainees come to realize that our job is not just to fix the problem, but also to preserve the rest of the anatomy. The questions tend to change from 'how does this drill work' to 'how do I make sure that when I'm using this drill, I'm not causing damage to other things?'"

Training on generic patient models   ... ' 

Robot Uses Fake Raspberry to Practice Picking Fruit

Robot Uses Fake Raspberry to Practice Picking Fruit

By Popular Science, June 26, 2023

Lab equipment with the raspberry mockup.

A growing number of farmers are interested in using robots for time-intensive tasks such as harvesting strawberries, sweet peppers, apples, lettuce, and tomatoes.

Credit: EPFL Create

Scientists at the Swiss Federal Institute of Technology, Lausanne (EPFL) Computational Robot Design & Fabrication Laboratory engineered a robot that practiced raspberry-picking on a silicone raspberry mockup with an artificial stem to ensure it learned how to handle the fragile fruit.

The researchers said the stem "can 'tell' the robot how much pressure is being applied, both while the fruit is still attached to the receptacle and after it's been released."

Experiments showed the robot could harvest 60% of the fruits while keeping them intact versus the human average of 90%.

An improved raspberry model could help enhance the robot's performance, while an extended setting that simulates "environmental conditions such as lighting, temperature, and humidity could further close the Lab2Field reality gap," according to the researchers.

From Popular Science

View Full Article   

Harvard's New Computer Science Teacher is a Chatbot

Once people get used to using particular chat framework, and comfortable with it use, they can be ready for any kind of training.   See my previous post on this where Bing makes the case. 

Harvard's New Computer Science Teacher is a Chatbot

By The Independent (U.K.),  June 23, 2023

Harvard University plans to use an AI chatbot similar to ChatGPT as an instructor on its flagship coding course.

Students enrolled on the Computer Science 50: Introduction to Computer Science (CS50) programme will be encouraged to use the artificial intelligence tool when classes begin in September.

The AI teacher will likely be based on OpenAI's GPT 3.5 or GPT 4 models, according to course instructors.

From The Independent (U.K.) 

Training Machines for Uncertain Real-World Simulations

 Uncertainty a key aspect.  

Training Machines for Uncertain Real-World Situations

MIT News

Adam Zewe, May 31, 2023

An algorithm developed by researchers at the Massachusetts Institute of Technology (MIT) and Technion—Israel Institute of Technology can determine automatically and independently whether and when imitation learning or reinforcement learning is more effective for training a "student" machine. The algorithm is adaptive, allowing the machine to move between both types of learning throughout the training process based on which would achieve better, faster results. MIT's Idan Shenfeld said, "This combination of learning by trial-and-error and following a teacher is very powerful. It gives our algorithm the ability to solve very difficult tasks that cannot be solved by using either technique individually."

Dark Web ChatGPT Unleashed: Meet DarkBERT

 Training on the Dark Web?

Dark Web ChatGPT Unleashed: Meet DarkBERT      in CACM   By Tom's Hardware, May 19, 2023

To train the model, the researchers crawled the Dark Web through the Tor network, then filtered the raw data (applying techniques such as deduplication, category balancing, and data pre-processing) to generate a Dark Web database.

Researchers at South Korea's Korea Advanced Institute of Science and Technology (KAIST) and data intelligence company S2W have created a large language model (LLM) trained on Dark Web data.

The researchers fed the RoBERTa framework a database they compiled from the Dark Web via the Tor network to create the DarkBERT LLM, which can analyze and extract useful information from a new piece of Dark Web content composed in its own dialects and heavily-coded messages.

They demonstrated DarkBERT's superior performance to other LLMs, which should enable security researchers and law enforcement to delve deeper into the Dark Web.

From Tom's Hardware

View Full Article   

Training Machines to Learn More Like Humans Do

Training computer Vision models

MIT News, By Adam Zewe  May 9, 2023

Scientists at the Massachusetts Institute of Technology (MIT) and Toyota subsidiary Woven Planet found computer vision models can be trained to produce more stable, predictable visual representations, similar to those humans learn through perceptual straightening. The researchers taught the models millions of examples via adversarial training, which enhanced their perceptual straightness while reducing their reactivity to slight errors within images. They discovered the models trained on more perceptually straight representations could correctly classify objects in videos with greater consistency. MIT's Vasha DuTell said, "One of the take-home messages here is that taking inspiration from biological systems, such as human vision, can both give you insight about why certain things work the way that they do and also inspire ideas to improve neural networks."

Full Article  

Retraining, Essential.

Examined in practice. 

Published in  Towards Data Science,      By Claire Longo

Embracing Automated Retraining

How to move away from retraining at a set cadence (or not at all) in favor of a dynamic approach

This piece was co-authored by Trevor LaViale

While the industry has invested a lot in processes and techniques for knowing when to deploy a model into production, there is arguably less collective knowledge on the equally important task of knowing when to retrain a model. In truth, knowing when to retrain a model is hard due to factors like delays in feedback or labels for live predictions. In practice, many models are in production with no retraining at all, use manual retraining methods, or are retraining without optimizing or studying the cadence.

This post is written to help data scientists and machine learning engineering teams embrace automated retraining.

Approaches for Retraining

There are two core approaches to automated model retraining:

Fixed: Retraining a set cadence (e.g., daily, weekly, monthly)

Dynamic: Ad-hoc triggered retraining based on model performance metrics.

While the fixed approach is straightforward to implement, there are some drawbacks. Compute costs can be higher than necessary, and the frequent retraining can lead to inconsistencies from one model to another, while infrequent retraining schedules can lead to a stale model.

The dynamic approach can prevent models from going stale, and optimize the compute cost. While there are numerous approaches to retraining, here are some recommended best practices for dynamic model retraining that will keep models healthier and performant.

Generalized Retraining Architecture

Optical Algorithm Simplifies Analog AI Training

Thinking this, possibilities and  implications.  

Optical Algorithm Simplifies Analog AI Training When backpropagation loses its bite, this approach may substitute  ...    in IEEE Spectrum

Researchers have developed a range of analog and other unconventional machine learning systems in the expectation that they will prove vastly more energy efficient than today’s computers. But training these AIs to do their tasks has been a big stumbling block. Researchers at NTT Device Technology Labs and the University of Tokyo now say they’ve come up with a training algorithm (announced by NTT last month) that goes a long way toward letting these systems meet their promise.

Their results, established on an optical analog computer, represent progress towards obtaining the potential efficiency gains that researchers have long sought from “unconventional” computer architectures.

Modern AI programs use a biologically-inspired architecture called an artificial neural network to execute tasks like image recognition or text generation. The strength of connections between artificial neurons, which control the outputs of the computation, must be modified or trained using standard algorithms. The most prominent of these algorithms is called backpropagation, which updates the connection strengths to reduce the network’s errors, while it processes trial data. Because adjustments to some parameters depend on adjustments to others, there is a need for active information passing and routing by the computer.

As Spectrum has elsewhere explained, “Error backpropagation is like running inference in reverse, moving from the last layer of the network back to the first layer; weight update then combines information from the original forward inference run with these backpropagated errors to adjust the network weights in a way that makes the model more accurate.”

Alternative computing architectures, which trade complexity for efficiency, often cannot perform the information passing required by the algorithm. As a consequence, the trained parameters of the network must be obtained from an independent physics simulation of the entire hardware setup and its information processing. But creating simulations of sufficient quality can itself be challenging.  ... ' 

ChatGPT: The Google Killer, Distributed Training with PyTorch and Azure ML, and Many Models Batch Training

Discovered in Medium, for exploration:

ChatGPT: The Google Killer?, Distributed Training with PyTorch and Azure ML, and Many Models Batch Training

Distributed Training with PyTorch and Azure ML 

Many Models Batch Training at Scale with Ray Core 

Ray Core and stateless Ray tasks.

Five Ways to Safely Use Generative AI  

From workers using chatbots as research assistants to creating art through image generators and more, here are a few ways that you can safely use generative AI.

What to Expect in 2023: A Data Scientist’s Top 5 AI Predictions  

Between improved NLP and increased use of AI in finance, here are one data scientist’s 2023 AI predictions.

Why Synthetic Data Is Key to Paving the Way for Smart Cities

Good thoughts here for other applications as well

Why Synthetic Data Is Key to Paving the Way for Smart Cities

By Spiceworks, November 18, 2022

Illustration of a futuristic smart city.

Gartner predicts synthetic data will overshadow real-world data in artificial intelligence (AI) models by 2030. It saves valuable time and energy while also providing the opportunity to counteract incumbent bias in the base data.

Of course, model training should not be limited to one data source or the other. Synthetic and real-world data can be complementary in efficiently progressing training at a lower cost, with greater applicability and without privacy or bias issues. But, it stands that synthetic data is essential for developing smarter AIs capable of recognizing and preventing corner cases in smart cities.

The potential for uniting synthetic and real-world data in the pursuit of smart cities is highly promising for the future of smarter AI.

From Spiceworks

View Full Article    

New Book on Synthetic Data

The data is most often,  the thing.    Look forward to taking a look.

New Book: Synthetic Data

Published on November 28, 2022, By Vincent Granville

Author of Intuitive Machine Learning and Explainable AI.

84 articles

Synthetic data is used more and more to augment real-life datasets, enriching them and allowing black-box systems to correctly classify observations or predict values that are well outside of training and validation sets. In addition, it helps understand decisions made by obscure systems such as deep neural networks, contributing to the development of explainable AI. It also helps with unbalanced data, for instance in fraud detection. Finally, since synthetic data is not directly linked to real people or transactions, it offers protection against data leakage. Synthetic data also contributes to eliminating algorithm biases and privacy issues, and more generally, to increased security.

Terrain generation, evolution and morphing (in chapter 11)

This book is the culmination of years of research on the topic, by the author. Not only it integrates all the material from his previous book “Intuitive Machine Learning and explainable AI”, but it also contains all but the most advanced math from his book on stochastic simulations. The author also added more recent advances with applications to terrain generation (with animated data), synthetic universes and experimental math. The latter is an infinite source of synthetic data to build and benchmark new machine learning techniques. Conversely mathematics benefits from these techniques to uncover new insights related to the most famous unsolved math problems. Chapter 14 on the Riemann Hypothesis illustrates this point, with new state-of-the-art research results on the subject.  ... ' 

Reports on Computing, Big and Small

Training and More

New Records for the Biggest and Smallest AI Computers Nvidia H100 and Intel Sapphire Rapids Xeon debut on ML Perf training benchmarks   By SAMUEL K. MOORE  in Spectrum IEEE 

The machine-learning consortium MLCommons released the latest set of benchmark results last week, offering a glimpse at the capabilities of new chips and old as they tackled executing lightweight AI on the tiniest systems and training neural networks at both server and supercomputer scales. The benchmark tests saw the debut of new chips from Intel and Nvidia as well as speed boosts from software improvements and predictions that new software will play a role in speeding the new chips in the years after their debut.

Training Servers

Training AI has been a problem that’s driven billions of dollars in investment, and it seems to be paying off. “A few years ago we were talking about training these networks in days or weeks, now we’re talking about minutes,” says Dave Salvator, director of product marketing at Nvidia.

There are eight benchmarks in the MLPerf training suite, but here I’m showing results from just two—image classification and natural-language processing—because although they don’t give a complete picture, they’re illustrative of what’s happening. Not every company puts up benchmark results every time; in the past, systems from Baidu, Google, Graphcore, and Qualcomm have made marks, but none of these were on the most recent list. And there are companies whose goal is to train the very biggest neural networks, such as Cerebras and SambaNova, that have never participated.

Another note about the results I’m showing—they are incomplete. To keep the eye glazing to a minimum, I’ve listed only the fastest system of each configuration. There were already four categories in the main “closed” contest: cloud (self-evident), on premises (systems you could buy and install in-house right now), preview (systems you can buy soon but not now), and R&D (interesting but odd, so I excluded them). I then listed the fastest training result for each category for each configuration—the number of accelerators in a computer. If you want to see the complete list, it’s at the MLCommons website.  .... ' 

Training Neural Nets on Small Devices

 Good direction.

We Can Train Big Neural Networks on Small Devices

IEEE Spectrum

Matthew Hutson, September 20, 2022

A new training method expands small devices' capabilities to train large neural networks, while potentially helping to protect privacy. The University of California, Berkeley's Shishir Patil and colleagues integrated offloading and rematerialization techniques using suboptimal heuristics to reduce memory requirements for training via the private optimal energy training (POET) system. Users feed POET a device's technical details and data on the architecture of a neural network they want to train, specifying memory and time budgets; the system generates a training process that minimizes energy usage. Defining the problem as a mixed integer linear programming challenge was critical to POET's effectiveness. Testing showed the system could slash memory usage by about 80% without significantly increasing energy consumption.  ...

Deepmind AI Learns to Play Soccer

Another useful game example, learned,  trained and automated 

DeepMind AI Learns to Play Soccer Using Decades of Match Simulations

By New Scientist, September 8, 2022

Researchers at artificial intelligence (AI) research company DeepMind trained an AI to play a computer simulation of soccer using an athletic curriculum resembling an accelerated model of a human baby maturing into a soccer player.

The algorithm controlled digital humanoids with lifelike body masses and joint movements, who in the first phase learned to run naturally by mimicking motion-capture video clips of human soccer players.

The humanoids learned in the second phase to dribble and shoot the ball through trial-and-error machine learning, and more complex behaviors manifested after five simulated years of training.

Teamwork skills emerged in the third training phase as the humanoids learned to score goals in two-on-two matches, which improved their off-ball scoring opportunity ratings. .... 

The digital humanoids were trained on simplified rules that allowed fouls, provided a wall-like boundary around the pitch, and avoided set pieces such as throw-ins or goal kicks.... 

From New Scientist  

Robot Dogs Learning Tough Terrain

Been following this for some time, and potential uses. 

Robot Dog Learns to Walk Tough Terrain in 20 Minutes

New Scientist, Alex Wilkins,  August 26, 2022

Researchers at the University of California, Berkeley (UC Berkeley) developed a machine learning algorithm that enabled a robot dog to learn to navigate difficult terrain in only 20 minutes. The Q-learning algorithm does not need a model of the target terrain. As a result, said UC Berkeley's Sergey Levine, "We don't need to understand how the physics of an environment actually works, we just put the robot into an environment and turn it on." The algorithm teaches the robot by rewarding it for each successful action until reaching its ultimate goal. The researchers demonstrated that the robot was able to walk on terrains it had not previously encountered, including grass, a layer of bark, a memory foam mattress, and a hiking trail, after about 20 minutes of training on each. ... ' 

College in a MetaVersity?

Change is coming,  but the metaverse is also not ready.     Based on our own experiences,  there is much yet to be done to prepare content, delivery and users of training and interaction in this way.  Meta-reality is still hard.  

College in the Metaverse Is Here. Is Higher Ed Ready?

By Inside Higher Ed, August 8, 2022

This fall, students at 10 universities will attend metaversities, immersive virtual reality platforms where remote faculty and students don virtual reality headsets and meet synchronously as they would on a physical campus.

This fall, students at 10 U.S. universities will attend metaversities, virtual reality (VR) platforms where educators and students wear VR headsets and interact synchronously.

Advocates claim VR increases student engagement, achievement, and satisfaction, but some scholars worry about metaversity technology licensors putting revenues above academic freedom, exploiting students' data, or replicating biased narratives in VR.  Many such challenges can be addressed by aligning educational best practices, commercial incentives, and political resolution, while students may find metaversities better than engagement via remote, two-dimensional screens.

Last year, Morehouse College tested a proof-of-concept metaversity with courses in world history, biology, and chemistry, and participating world history students improved their grade point averages 10% compared to grades in the same class facilitated through Zoom and face-to-face. ... 

The University of Massachusetts' Nir Eisikovits believes on-campus education will ultimately supplement metaversities, not vice-versa. ... 

From Inside Higher Ed

View Full Article   

We are Training Much Faster Now

Faster, better training is here outlined and described. 

We’re Training AI Twice as Fast This Year as Last New MLPerf rankings show training times plunging     By SAMUEL K. MOORE    in IEEE Spectrum

According to the best measures we’ve got, a set of benchmarks called MLPerf, machine-learning systems can be trained nearly twice as quickly as they could last year. It’s a figure that outstrips Moore’s Law, but also one we’ve come to expect. Most of the gain is thanks to software and systems innovations, but this year also gave the first peek at what some new processors, notably from Graphcore and Intel subsidiary Habana Labs, can do.

The once-crippling time it took to train a neural network to do its task is the problem that launched startups like Cerebras and SambaNova and drove companies like Google to develop machine-learning accelerator chips in house. But the new MLPerf data shows that training time for standard neural networks has gotten a lot less taxing in a short period of time. And that speedup has come from much more than just the advance of Moore’s Law.

This capability has only incentivized machine-learning experts to dream big. So the size of new neural networks continues to outpace computing power.

Called by some “the Olympics of machine learning,” MLPerf consists of eight benchmark tests: image recognition, medical-imaging segmentation, two versions of object detection, speech recognition, natural-language processing, recommendation, and a form of gameplay called reinforcement learning. (One of the object-detection benchmarks was updated for this round to a neural net that is closer to the state of the art.) Computers and software from 21 companies and institutions compete on any or all of the tests. This time around, officially called MLPerf Training 2.0, they collectively submitted 250 results.

Very few commercial and cloud systems were tested on all eight, but Nvidia director of product development for accelerated computing Shar Narasimhan gave an interesting example of why systems should be able to handle such breadth: Imagine a person with a smartphone snapping a photo of a flower and asking the phone: “What kind of flower is this?” It seems like a single request, but answering it would likely involve 10 different machine-learning models, several of which are represented in MLPerf.

To give a taste of the data, for each benchmark we’ve listed the fastest results for commercially available computers and cloud offerings (Microsoft Azure and Google Cloud) by how many machine-learning accelerators (usually GPUs) were involved. Keep in mind that some of these will be a category of one. For instance, there really aren’t that many places that can devote thousands of GPUs to a task. Likewise, there are some benchmarks where systems beat their nearest competitor by a matter of seconds or where five or more entries landed within a few minutes of each other. So if you’re curious about the nuances of AI performance, check out the complete list.  .... '   (much more at link) 

Beware Allure of Training Tech

 As it says, context matters a great deal when training. Will the VR actually change results? 

Beware the allure of training technology   By Tim Marler in Defense News

The allure of training technology can often overshadow its value. Today, virtual reality, or VR, is a hot topic in the military training community, but training tools must be developed and selected according to their anticipated use. Context matters, and sometimes the best and most cost-effective training tool may just be a book.

While VR offers many benefits, the Department or Defense could seek to ensure that virtual training content derives from operational needs, integrates with existing processes and curricula, and is validated. Furthermore, this process could be assessed and refined continuously because although we cannot predict the future of technology, we can be confident that what we need and what we want will change.

The United States military has always derived great benefit from novel technological advances, and today is no exception. Rapid acquisition and innovation are focal points for the DoD. This focus extends to training as well, where emerging technologies can present many opportunities. But too often the tie from various capabilities to actual training objectives can be tenuous at best.

Within the training arena, there is now considerable focus on virtual reality and augmented reality. VR involves a user being completely immersed in a virtual environment, and AR involves overlaying virtual entities on real items. The Army, for example, is developing the Integrated Visual Augmentation System that will allow soldiers to see what combat vehicles see, have 3D terrain maps projected onto their real field of view and enable other capabilities for increased situational awareness.

In addition, most of the services are developing aspects of live, virtual and constructive, or LVC, capabilities that can, for example, allow pilots in real jets (live) to train with pilots in flight simulators (virtual) in different locations, all interacting with computer-based (constructive) representations of adversary jets ..... ' 

IFIF Design Futures Class

 I worked with IFTF for years as part of their connection to our big enterprise,   connecting with our  top corporate level.    Were always very thought provoking and useful.  Here an intro, click through for much more details and registration. 

IFTF Design Futures

Institute for the Future (IFTF) is pleased to offer a training with our team of award-winning design futurists to learn to imagine and prototype informed alternatives to the future, and design new offerings and policies for the next decade.   Creating artifacts from the future is one way IFTF uses design t ostart the futures thinking process. They are the first prototypes of a world to come. The artifact from the future, at left, creates a plausible vision of what direct-action climate activism might look like in 10 years.  .... 

Expand your design practice with foresight:

Combine professional design practices with foresight methods and tools to identify forces of change, signals, and alternative futures.

Recognize biases and assumptions about the future:

Identify the visions embedded in products, strategies, and media, and learn how to redesign them.

Build future worlds:

Describe future contexts with evidence-based world building and user journeys to anticipate problems, dilemmas and opportunities of the next decade.

Prototype experiences of a future world: Design experiential scenarios to feel life in the next decade, and engage audiences in tangible ways.

Design a new offering for the future:

Use a future values canvas to create innovative revenue and impact models for people and systems  ...

Live, multi-session instruction conducted by IFTF researchers and award-winning design futurists, Jake Dunagan, Ilana Lipsett, and Jacques Barcia

Immersion in design futures, thinking skills, methodologies, and processes

IFTF Design Futures Toolkit

Full set of proprietary PowerPoint presentations, Personalized strategic Foresight-Insight-Action plan for post-training application, Networking with a diverse cohort of foresight practitioners and designers, Official certificate of completion of IFTF Design Futures Training  .... 

Making Pizza etc. with Robots

Here a long time amateur cook, very interested in how this could all be well automated.  Starting to see it in fast food. Like also the connection to things like clothing.  

Solving the challenges of robotic pizza-making

A new technique could enable a robot to manipulate squishy objects like pizza dough or soft materials like clothing.

Adam Zewe | MIT News Office   Publication Date: March 31, 2022

Imagine a pizza maker working with a ball of dough. She might use a spatula to lift the dough onto a cutting board then use a rolling pin to flatten it into a circle. Easy, right? Not if this pizza maker is a robot.

For a robot, working with a deformable object like dough is tricky because the shape of dough can change in many ways, which are difficult to represent with an equation. Plus, creating a new shape out of that dough requires multiple steps and the use of different tools. It is especially difficult for a robot to learn a manipulation task with a long sequence of steps — where there are many possible choices — since learning often occurs through trial and error.

Researchers at MIT, Carnegie Mellon University, and the University of California at San Diego, have come up with a better way. They created a framework for a robotic manipulation system that uses a two-stage learning process, which could enable a robot to perform complex dough-manipulation tasks over a long timeframe. A “teacher” algorithm solves each step the robot must take to complete the task. Then, it trains a “student” machine-learning model that learns abstract ideas about when and how to execute each skill it needs during the task, like using a rolling pin. With this knowledge, the system reasons about how to execute the skills to complete the entire task.

The researchers show that this method, which they call DiffSkill, can perform complex manipulation tasks in simulations, like cutting and spreading dough, or gathering pieces of dough from around a cutting board, while outperforming other machine-learning methods. .... '