Picture this: A guy is sitting on a bench, and it starts raining. Along comes a cute giraffe carrying an umbrella and kindly offers to share so the poor human doesn’t get soaked.
It’s not yet a ride on Aladdin’s magic carpet, but it is a peek at where storytelling at Disney is headed: combining augmented reality (AR) and mixed reality (MR) experiences.
Disney Research's Magic Bench, which previewed at SIGGRAPH 2017, demonstrates the magical possibilities of mixed reality by combining physical space—a bench—with virtual space, allowing a single user or a group to share the same experience and see, hear, and feel animated characters.
Current virtual reality experiences require users to employ hardware, such as wearing a head-mounted display or directing a handheld device like a phone or tablet. Disney researchers are exploring ways to instead instrument the environment, not an individual, to create a seamless walk-up-and-play experience. In the video announcement, Moshe Mahler, principal digital artist at Disney Research, explains, "Our mantra for this project was: hear a character coming, see them enter the space, and feel them sit next to you."
In the demonstration, the bench is essentially a stage facing a display. The Magic Bench contains haptic actuators that provide vibrations that let participants feel the presence of computer-generated characters and objects. A RGB camera and depth sensor create a real-time, HD-video-textured 3D reconstruction of the bench, surroundings, and participants. Those sitting on the Magic Bench view themselves mirrored on a large screen in front of them, creating a third-person point of view.
The bench itself serves as a controller; the mixed reality show doesn’t begin until someone sits down. Those sitting on the bench are in the middle ground, along with a foreground and a background. The various formations of the seated participants dictate the experiences. Participants can not only see and hear the characters, but they can also feel them on the bench through haptic feedback.
Here’s a video about the Magic Bench. Or, download the Magic Bench — A Multi-User & Multi-Sensory AR/MR Platform for more technical specifications.
As part of The Walt Disney Company, Disney Research is a network of research laboratories with facilities in Los Angeles, San Francisco, Pittsburgh, and Zürich. Research topics include computer graphics, video processing, computer vision, robotics, radio and antennas, wireless communications, human-computer interaction, displays, data mining, machine learning, and behavioral sciences.
