For centuries, cadavers were the primary way medical students learned about anatomy. Technology like the Microsoft Hololens is changing that as mixed reality offers an interactive experience.
For a medical student studying anatomy, there’s really no substitute for the real thing. Since the sixteenth century or earlier, students have relied on the dissection of human cadavers to identify and locate muscles, organs, bones, and everything that comprises the human anatomy. Elaborate theaters were built to ensure every student had an unobstructed view: Italy’s University of Padua’s is the oldest, dating to 1594, and boasts six viewing tiers carved from walnut.
While venues have changed over time, the underlying course structure revolves around cadavers—some are now synthetically fabricated—or 2D drawings and illustrations. That is now changing, and quickly, as educators discover holographic computing. In Cleveland, Case Western Reserve University is embracing Microsoft’s Hololens, a headset that brings mixed reality images to the classroom. Proponents label it disruptive, changing the way anatomy has been traditionally taught, using interactive 3D images that can be manipulated through gesture, gaze, and hand commands. Further, it allows remote users to view the same images and interact with others in a central location.
“It is life-changing, and I don’t say that lightly,” says Mark Griswold, professor of radiology with the Case Center for Imaging Research. “It is completely revolutionary.”
Hololens is a wearable holographic computer that runs on an updated version of Windows 10. Its holographic processing unit calculates where 3D images exist in a user’s physical space and projects them onto the lenses of the headset. There, the images are integrated with physical elements in a real-world environment.
Multiple users interacting in mixed reality. Image: Microsoft.
That is where mixed reality diverges from virtual reality. With VR, programs immerse the users in a virtual environment completely separated from the real world. Augmented reality enhances that by overlaying data on the virtual images. Mixed reality goes even further, projecting 3D images and integrating them with the physical world. The images are interactive, and can be viewed by multiple users fitted with headsets. Remote users, such as a teacher, can see where other users are pointing and interact with the group.
CWRU and the Cleveland Clinic are partnering on a new teaching campus for the clinic’s and the university’s medical schools where mixed reality will be a featured teaching tool. In 2015, Microsoft invited the school’s faculty to partner in developing the then-unreleased and secret product after an initial visit by Cleveland Clinic CEO Toby Cosgrove.
“We had no idea what was going on,” says Griswold. “We became part of a collaboration. With the Cleveland Clinic, we were two of twelve partners with Microsoft.”
The collaboration benefited both sides as Microsoft wanted real-world input on Hololens development and university and medical school faculty, recognizing the technology as game-changing, got a jump on learning how to apply it. One early result is an app allowing users to view and explore the human anatomy at their own pace and from any perspective.
Developed by a CWRU team, viewers can look at the body as a whole, and extract layers or details from various systems using voice or hand commands. For instance, the digestive system can be pulled from the body and isolated, giving the viewer an unobstructed 3D look at the body’s major organs. Viewers can walk around the image, looking at it from any angle. Remote users can view the same image at the same time, and interact with the others.
Some organs may still remain obstructed. The pancreas, for instance, is located behind the liver. When using a cadaver, the instructor has to remove the liver to give students a view of where the pancreas is located and what it looks like. Using mixed reality, the liver can be moved with a pinch of the fingers, exposing a 3D image of the pancreas. It can be extracted to get an isolated view, and made larger to get detail. Other organs, such as the heart, can similarly be extracted, but students also can see and hear it beating, and view how valves and chambers work.
Griswold says app development was fairly straightforward, “unbelievably quick, actually.” He says a team of developers, artists and 16 students produced the app, with much of the work being done in Unity programming.
“The bigger challenges are how to synchronize multiple devices and providing the level of detail for the anatomy package,” he says. “The questions are all around, “How do you actually use it?”
Besides the processing units, a Microsoft spokesperson says the headgear is fitted with numerous pieces of hardware: one intertial measuring unit; four environment-understanding cameras; one depth camera to assist with spatial mapping; a 2-MP photo/HD video camera; mixed reality capture, enabling users to record video and photos of their mixed reality experiences to share with others; four microphones to enable accurate voice commands; and an ambient light sensor.
Griswold notes this is the first version, and believes the headset can be lighter and less obtrusive. Future versions should add more computing power and a larger field of view. Microsoft now offers the device to developers and commercial organizations. Consumer sales are still in the future.
For now, CWRU faculty are happy to use the current version and are designing the new medical school campus with that in mind. Located at Cleveland Clinic’s sprawling campus, the new buildings will incorporate what may be a first for a medical school. There will not be a traditional anatomy lab, with a dissection table for cadavers. Students will learn anatomy completely through holograms and mixed reality.
“In 2019 we move into the new building,” Griswold says. “This will be the only way to teach anatomy.”
But the faculty is still working out how best to use the new tool to teach the curriculum, and work in large groups. He also points to its advantages in overall design.
“If you design a classroom, you have to look at blueprints and that can be hard to visualize,” Griswold says. “But if you look through the Hololens, it becomes immediately clear. The power of design and visualization is so strong and it is collaborative. One of the things that makes it unique is it is untethered. People can walk around the image in a collaborative way. It really is revolutionary.”