BY MATT WHEELER    October 20, 2016

In the Newhouse Media, Interface and Network Design (MIND) Lab, Leanne Hirshfield and her collaborative team of students are using technology to read and analyze human emotion.

For decades, medical researchers have understood which areas of the brain are devoted to abilities, memories and emotions. And, physicians have long been able to closely examine our brains with magnetic fields in MRI scanners for health purposes. Newhouse professor Leanne Hirshfield’s research takes things to a whole new level.

In the Newhouse Media, Interface and Network Design (MIND) Lab, Hirshfield and her collaborative team of students from SU’s College of Engineering and Computer Science, School of Information Studies (iSchool) and College of Arts and Sciences are using technology to read and analyze human emotion.

The team’s work centers around one key piece of scientific equipment—a functional near-infrared spectroscopy device. Like something out of a movie, it is strapped to a subject’s head to measure brain activity non-invasively. When activated, an image of the subject’s brain illuminates a nearby display. Red, orange, green and yellow swell and fade like on a weather map as blood flows to different areas of the brain. This provides insight into what a person is experiencing when they are exposed to different stimuli—say a tear-jerking ad, a contentious political debate or an overwhelming task. Hirshfield and student researchers Danushka Bandara G'12 and Natalie Sommer, Sarah Bratt G'14 and Trevor Grant '17 measure and record the subject’s mental state, which includes their emotion, cognitive workload and stress level.

Hirshfield, a trained computer scientist, was inspired to enter this field as a graduate student at Tufts University, where she and her research team conducted various studies that involved surveying people. They found that the problem with surveys is that the person taking it makes a conscious choice about how to respond. Their response could be influenced by fear of social consequences—like looking dumb—or a desire to remain consistent with their established public persona. These factors can make self-reporting inaccurate. That’s when a light bulb went on. It occurred to them—“What if we could measure a person’s response in the brain, and figure out how they felt objectively about what they were experiencing in real time?”

The possible implications are striking. By uncovering this data, it may be possible to predict someone’s cognitive and emotional state, market products in a way that appeals to consumers scientifically and determine if someone can be trusted to be given security clearances or access to classified materials.

The collaborative nature of this compelling research makes Syracuse University a unique and ideal place for it to occur. As with most notable scientific advances, the research is occurring at the intersection of many different fields. Here, computer engineering, data science and neuroscience—three programs from three SU Colleges—converge to contribute to something groundbreaking.

Danushka Bandara, a computer engineering Ph.D. student with a degree in electrical engineering, is a crucial contributor to Hirshfield’s research. Following guidance from Hirshfield and his advisor, Senem Velipasalar, he uses machine learning to develop new ways to analyze the data that the team collects. He explains, “This research produces a tremendous amount of data and traditional methods cannot make sense of it all. I’m building computational models that are able to recognize and learn the patterns that exist within the data and automate their identification.”

By turning the task over to a computer, Bandara gives the team the ability to uncover meaning and context from the copious amounts of data. Without his efforts, the rest of their work would be for naught.

In the end it’s fitting that research that can, in a sense, “read your mind” takes a diverse set of minds to bring it to life. As the leader of this work, Hirshfield believes that the interdisciplinary nature of their research is key. “Some of the best research comes from the fringes of disciplines, and that is where you tend to find niches that no one else has looked into. That is a big part of the success we are having in our research. When everyone communicates between these different areas of expertise, it opens up a whole new world of knowledge.”