Skip to content

Telestration: How Helena Mentis Applies Design Thinking to Surgery

Telestration: How Helena Mentis Applies Design Thinking to Surgery

Telestration: How Helena Mentis Applies Design Thinking to Surgery

Helena Mentis is the director of the Bodies in Motion Lab at University of Maryland, Baltimore County (UMBC) with research spanning human-computer interaction (HCI), computer supported cooperative work (CSCW) and medical informatics. During a recent visit to the Design Lab at UC San Diego, Mentis talked about her research on surgery in the operating room.

She examines the medical world through surgical instruments and the workflow inside the operating room. Mentis hones in on minimally invasive surgery and the reliance toward images.  She is particularly interested in how medical professionals see and share visual information in a collaborative fashion, which has grown over the past several years. She asks, “What happens if surgeons were given greater control over the image? What would happen to the workflow? Would it change anything?”

In one study at Thomas Hospital in London, surgeons were using a lot of pointing gestures to direct the operation. Confusion would arise and the surgeon would need to repeat his exact intention with others. This break in the workflow inspired Mentis’ team to ask: what if we were to build a touchless illustration system that responded to the surgeon’s gestures? Her team set out to build what she calls “telestration,” which enables surgeons to use gestures to illustrate their intentions through an interactive display.

During another operation, the surgeon encountered a soft bone and had to stop the procedure. As a result, the surgeon had take off their gloves to re-examine the tissue on the visual display. Mentis notes, “There is a tight coupling between images on display and feeling with the instrument in hand.” If the image on display could be more closely integrated with the workflow, would this save time in the operating room?After publishing her findings, people raved over how voice narration rather than gesture aided imaging and collaboration in surgery. Consequently Mentis asked, “If given the opportunity would doctors use voice or gesture?” The ensuing observations revealed that while doctors stated their preference for voice, gesture was more frequently used for shaping telestration images. While voice narration and gestures provided greater interaction with the image, surgeons actually spent more time in surgery. Mentis reasons, “There is more opportunity for collaborative discussion with the information.” Interestingly, this did add time to the overall operation, but it also yielded greater opportunities to uncover and discuss critical information.

About Helena Mentis, Ph.D.

Assistant Professor, Department of Information Systems
University of Maryland, Baltimore County

Helena Mentis, Ph.D., is an assistant professor in the Department of Information Systems at the University of Maryland, Baltimore County. Her research contributes to the areas of human-computer interaction (HCI), computer supported cooperative work (CSCW), and health informatics. She investigates how new interactive sensors can be integrated into the operating room to support medical collaboration and care. Before UMBC, she was a research fellow at Harvard Medical School, held a joint postdoctoral fellowship at Microsoft Research Cambridge and the University of Cambridge, and was an ERCIM postdoctoral scholar at Mobile Life in Sweden. She received her Ph.D. in Information Sciences and Technology from Pennsylvania State University.

Helena Mentis is the director of the Bodies in Motion Lab at University of Maryland, Baltimore County (UMBC) with research spanning human-computer interaction (HCI), computer supported cooperative work (CSCW) and medical informatics. During a recent visit to the Design Lab at UC San Diego, Mentis talked about her research on surgery in the operating room.

She examines the medical world through surgical instruments and the workflow inside the operating room. Mentis hones in on minimally invasive surgery and the reliance toward images.  She is particularly interested in how medical professionals see and share visual information in a collaborative fashion, which has grown over the past several years. She asks, “What happens if surgeons were given greater control over the image? What would happen to the workflow? Would it change anything?”

In one study at Thomas Hospital in London, surgeons were using a lot of pointing gestures to direct the operation. Confusion would arise and the surgeon would need to repeat his exact intention with others. This break in the workflow inspired Mentis’ team to ask: what if we were to build a touchless illustration system that responded to the surgeon’s gestures? Her team set out to build what she calls “telestration,” which enables surgeons to use gestures to illustrate their intentions through an interactive display.

During another operation, the surgeon encountered a soft bone and had to stop the procedure. As a result, the surgeon had take off their gloves to re-examine the tissue on the visual display. Mentis notes, “There is a tight coupling between images on display and feeling with the instrument in hand.” If the image on display could be more closely integrated with the workflow, would this save time in the operating room?After publishing her findings, people raved over how voice narration rather than gesture aided imaging and collaboration in surgery. Consequently Mentis asked, “If given the opportunity would doctors use voice or gesture?” The ensuing observations revealed that while doctors stated their preference for voice, gesture was more frequently used for shaping telestration images. While voice narration and gestures provided greater interaction with the image, surgeons actually spent more time in surgery. Mentis reasons, “There is more opportunity for collaborative discussion with the information.” Interestingly, this did add time to the overall operation, but it also yielded greater opportunities to uncover and discuss critical information.

About Helena Mentis, Ph.D.

Assistant Professor, Department of Information Systems
University of Maryland, Baltimore County

Helena Mentis, Ph.D., is an assistant professor in the Department of Information Systems at the University of Maryland, Baltimore County. Her research contributes to the areas of human-computer interaction (HCI), computer supported cooperative work (CSCW), and health informatics. She investigates how new interactive sensors can be integrated into the operating room to support medical collaboration and care. Before UMBC, she was a research fellow at Harvard Medical School, held a joint postdoctoral fellowship at Microsoft Research Cambridge and the University of Cambridge, and was an ERCIM postdoctoral scholar at Mobile Life in Sweden. She received her Ph.D. in Information Sciences and Technology from Pennsylvania State University.

Helena Mentis is the director of the Bodies in Motion Lab at University of Maryland, Baltimore County (UMBC) with research spanning human-computer interaction (HCI), computer supported cooperative work (CSCW) and medical informatics. During a recent visit to the Design Lab at UC San Diego, Mentis talked about her research on surgery in the operating room.

She examines the medical world through surgical instruments and the workflow inside the operating room. Mentis hones in on minimally invasive surgery and the reliance toward images.  She is particularly interested in how medical professionals see and share visual information in a collaborative fashion, which has grown over the past several years. She asks, “What happens if surgeons were given greater control over the image? What would happen to the workflow? Would it change anything?”

In one study at Thomas Hospital in London, surgeons were using a lot of pointing gestures to direct the operation. Confusion would arise and the surgeon would need to repeat his exact intention with others. This break in the workflow inspired Mentis’ team to ask: what if we were to build a touchless illustration system that responded to the surgeon’s gestures? Her team set out to build what she calls “telestration,” which enables surgeons to use gestures to illustrate their intentions through an interactive display.

During another operation, the surgeon encountered a soft bone and had to stop the procedure. As a result, the surgeon had take off their gloves to re-examine the tissue on the visual display. Mentis notes, “There is a tight coupling between images on display and feeling with the instrument in hand.” If the image on display could be more closely integrated with the workflow, would this save time in the operating room?After publishing her findings, people raved over how voice narration rather than gesture aided imaging and collaboration in surgery. Consequently Mentis asked, “If given the opportunity would doctors use voice or gesture?” The ensuing observations revealed that while doctors stated their preference for voice, gesture was more frequently used for shaping telestration images. While voice narration and gestures provided greater interaction with the image, surgeons actually spent more time in surgery. Mentis reasons, “There is more opportunity for collaborative discussion with the information.” Interestingly, this did add time to the overall operation, but it also yielded greater opportunities to uncover and discuss critical information.

About Helena Mentis, Ph.D.

Assistant Professor, Department of Information Systems
University of Maryland, Baltimore County

Helena Mentis, Ph.D., is an assistant professor in the Department of Information Systems at the University of Maryland, Baltimore County. Her research contributes to the areas of human-computer interaction (HCI), computer supported cooperative work (CSCW), and health informatics. She investigates how new interactive sensors can be integrated into the operating room to support medical collaboration and care. Before UMBC, she was a research fellow at Harvard Medical School, held a joint postdoctoral fellowship at Microsoft Research Cambridge and the University of Cambridge, and was an ERCIM postdoctoral scholar at Mobile Life in Sweden. She received her Ph.D. in Information Sciences and Technology from Pennsylvania State University.

Read Next

San Diego/Tijuana is finalist to become a World Design Capital

San Diego Union Tribune

The San Diego/Tijuana region is a finalist to become a World Design Capital that could mean a year-long promotion of the binational region.

Winners are chosen based on how each region effectively incorporates design across their economic, technological, social, cultural, political and environmental sectors . More than just having a fancy title, winning means a year of events to promote the region, including a street festival, a one-day celebration highlighting the winner’s designs and a design conference that should bring people from around the globe.

“This is an incredibly exciting opportunity to not only showcase our bi-national region as a longstanding design and innovation powerhouse,” she wrote, “but to also shape the narrative around what it means to be a 21st century metropolis, [says Michèle Morris, President of the Design Forward Alliance and Associate Director of UCSD Design Lab]."

Telestration: How Helena Mentis Applies Design Thinking to Surgery

Helena Mentis is the director of the Bodies in Motion Lab at University of Maryland, Baltimore County…

Trolley Stops Designathon UCSD Design Lab

Designathon Seeks to Reimagine Trolley Stops

On April 6+7, 2019, on UC San Diego campus Warren Mall, over 200+ students, neighbors, designers, technologists, and media-makers will come together for the Pepper Canyon Mobility Hub Designathon, an event developing proposals that will support the transformation Pepper Canyon Trolley Station at UCSD campus, currently under construction, into a dynamic, multimodal mobility hub. The event is produced through a partnership between The UC San Diego Design Lab, SANDAG, UC San Diego Campus Planning, the UC San Diego Young Planners’ Society, Sixth College Culture, Art, Technology program, and UC San Diego Urban Planning Program.

Design Lab Students Swarm CHI Conference in Denver

In May, many UC San Diego Design Lab members and students swarmed the largest human-computer interaction conference in the world, ACM CHI 2017. Affiliated with ACM SIGCHI, the premier international society for professionals, academics and students who are interested in human-technology and human-computer interaction (HCI), the conference brings together people from multiple disciplines and cultures to explore new ways to practice, develop and improve methods and systems in HCI.

“I love the mix of people at CHI—chatting with people making new sensor technologies, new theoretical approaches, new architectural construction techniques -- it has incredible diversity but is still brought together with a common set of ideas and expectations,” said former Design Lab Fellow Derek Lomas, who presented at the conference.

This year, the mega-HCI conference, which was sponsored by tech-industry giants such as Facebook, Google, IBM, Microsoft and Yahoo! was held in Denver near the foothills of the Rocky Mountains. Organizers selected the site, which is full of scenic trees, mountains and valleys to serve as a motivation for the theme of “Motivate, Innovate, Inspire.”
Seda Evis

Designer in Residence: Seda Evis

While Seda Evis participates as a Designer-in-Residence at UC San Diego with The Design Lab, and is VP of Strategy & Growth at FreshForm Interactive—an experience design and innovation consultancy—she also claims to have what she refers to as a superpower: her hybrid mind, which she describes as the combination of two worlds: the business side and the design side.

As a Designer-in-Residence at The Design Lab,  Evis enjoys working in an interdisciplinary setting, which she says enhances her existing skillset. “Academia tends to be quite separate from how the practice is done,” she explains. “I find ways of doing interdisciplinary work, as well as cross industry work, very important for innovation because that’s how you actually get seeds from different places.” As of now, most of Evis’s work at The Design Lab has been working with the Community Team on the now winning bid for the World Design Capital 2024 (WDC) designation. Her work dovetails with her role on the Board of  Directors for the Design Forward Alliance–a non-profit organization started by the Lab in partnership with the regional design community and one of the partners co-leading the HOME 2024 WDC efforts alongside the Design Lab, Burnham Center for Community Advancement, the City of San Diego and the City of Tijuana.   The designation puts the San Diego-Tijuana region in a prestigious international cohort recognized for the “effective use of design to drive economic, social, cultural and environmental development.”  Even the proposal theme is significant, Evis says. “Home” not only refers to the immense and diverse communities of San Diego and Tijuana that form one, but also serves as an acronym for Human-centered, Open, Multi-disciplinary/Multi-cultural, and Experimental. For Evis, participating in HOME2024 signifies her career “coming full circle.” 
UC Adopts Recommendations For The Responsible Use Of Artificial Intelligence

UC adopts recommendations for the responsible use of Artificial Intelligence

Camille Nebeker, Ed.D., associate professor with appointments in the UC San Diego Herbert Wertheim School of Public Health and Human Longevity Science and the Design Lab

The University of California Presidential Working Group on Artificial Intelligence was launched in 2020 by University of California President Michael V. Drake and former UC President Janet Napolitano to assist UC in determining a set of responsible principles to guide procurement, development, implementation, and monitoring of artificial intelligence (AI) in UC operations.

“The use of artificial intelligence within the UC campuses cuts across human resources, procurement, policing, student experience and healthcare. We, as an organization, did not have guiding principles to support responsible decision-making around AI,” said Nebeker, who co-founded and directs the Research Center for Optimal Digital Ethics Health at UC San Diego, a multidisciplinary group that conducts research and provides education to support ethical digital health study practices.
Back To Top