Preserving the Past with Immersive Technologies

Beyond the museum.  Much more to follow.  

NEWS

Preserving the Past with Immersive Technologies,  By Esther Shein

Communications of the ACM, December 2022, Vol. 65 No. 12, Pages 15-17   10.1145/3565978

The Skin & Bones augmented reality app brings to life the skeletons in the Bone Hall of the Smithsonian's National Museum of Natural History.

Inside the U.S. Holocaust Memorial Museum in Washington, D.C., is an exhibit called the Tower of Faces (https://bit.ly/3cxI3Ik), which uses augmented reality to tell stories behind some of the 1,041 photos of people from the small town of Eishishok, in what is now Lithuania. The tower soars 50 feet high across 30 rows, displaying the faces of the town's inhabitants, nearly 4,000 of whom were massacred when the Germans invaded during World War II.

When visitors walk into the tower, they can pick up one of several iPads and hold it up to an image on the wall, which will then play a video that transports them into the town. The video first appears in color and then fades to black and white, while a narrator reads a brief script about the person. One tells the story of Szeina, an actress who was fluent in five languages and owned a hotel on the market square the Nazis took over to use as their local headquarters.

"It's a beautiful experience … there are many, many photos looking out at you—people riding bikes, outside in the snow, at a wedding banquet, and on the stairs of their houses," says Sarah Lumbard, director of museum experience and digital media at the Holocaust Museum. The photos depict people of all ages prior to the massacre in September 1941.

Many of the photos were supplied by Yaffa Sonenson Eliach, whose grandmother was a photographer in Eishishok and is herself a survivor. The immersive experience opened in April 2022 and the idea was to "create a spark of life" so the residents of Eishishok will be remembered, Lumbard says.

"Our question was: How do we bring them to life, just for a moment, and have it not just be a memorial but have victims of the Holocaust come to life and treat them with respect and engage visitors to really see them as people," Lumbard says.

Figure. The interactive Heroes and Legends attraction at NASA's Kennedy Space Center Visitor Complex in Florida, featuring the U.S. Astronaut Hall of Fame.

Digital technologies such as virtual reality (VR), augmented reality (AR), and three-dimensional (3D) graphics are making it possible for museums and other institutions to preserve historical events and tell the stories of those events in an engaging way. In the case of VR, the technology actually takes them to another time or place away from where they physically are.

As research firm Gartner says, "the future of digital experience is immersive" (see https://gtnr.it/3BqmqU2).

"In this way, we can experience historical events and places that are long gone in an immersive way—kind of like IMAX taken to the next level," explains Tuong Nguyen, a senior principal analyst at Gartner. "So instead of just seeing it on a flat screen, like TV or movies, or seeing it all around you like an IMAX movie, VR can potentially enable people to explore that space with 360 degrees of freedom.  

AR can change how someone experiences the world in front of them or around them, usually in a visual way, integrating information such as text, graphics, and audio with real-world objects. "The idea is to show users how things looked in the past with actual video and photos from these time periods," Nguyen says.

For the Smithsonian's National Museum of Natural History, also in Washington D.C., the impetus behind developing a mobile app called "Skin and Bones" (https://s.si.edu/3wGy0aT) using 3D augmented reality and 3D tracking was to attract more visitors to its Bone Hall by telling stories about some of the specimens on display.

Visits and dwell time in the hall and the experience people were having in the Bone Hall "fell far short of any measure of what a visitor experience should be in a modern-day exhibition," says Robert Costello, national outreach program manager, who developed the mobile experience. In fact, most visitors were using the Bone Hall as a passageway from one section of the museum to another, rather than a destination, Costello says. The average time spent in a modern exhibition at the museum is between 10 and 20 minutes, and most of those visitors were spending less than two minutes in the hall, which has a storied history, he says.  .... '