Paul Zak spent over two decades developing the science beneath his company, Immersion. His tastemaker software aims to measure and predict how people respond to music, movies and other experiences by tracking their brain activity through a smartwatch or fitness tracker.
With a newly launched software-as-a-service platform, his mind-reading tool is now available to the masses for as little as $199 a month.
But the wide release of a technology that purports to know people better than they know themselves is worrying to some, who say it could limit artistic expression, perpetuate unconscious biases and, in the wrong hands, subject unwilling people to spying and manipulation.
Zak's tool measures emotional resonance, or what he calls "immersion." It uses sensors to track attention levels and infer brain levels of oxytocin, the so-called "love hormone" known for its association with bonding that works as a neurotransmitter. The sensors monitor how a person's brain responds to a given stimulus, moment by moment. Software generates a readout that provides real-time feedback potentially useful for everything from testing Hollywood audiences to understanding what resonates during a work presentation.
Through sensors Immersion measures someone's pulse, which Zak says is correlated with attention levels.
"Attention is the necessary condition; immersion is the sufficient condition," he added, noting that the latter is measured by oxytocin: a naturally occurring chemical in the brain that, he says, "is why people cry at movies when the boy kisses the girl."
Paul Zak directs Claremont Graduate University's Center for Neuroeconomic Studies
Zak, who directs Claremont Graduate University's Center for Neuroeconomic Studies, researches how brain activity corresponds to decision-making. His papers have been cited by academic publications over 15,000 times. He is not without his critics, though, some of whom have argued that oxytocin, in addition to correlating with feelings like empathy and trust, can also correlate with envy and tribalism.
As wearable sensors improved over time, Zak says he became able to map the data gathered by noninvasive, everyday items like smartwatches back to the brain-activity readouts he's gathered for years in his lab from blood-draws and expensive medical equipment.
"We created the first democratized platform for neuroscience where anybody could measure what the brain loves in real time," he said. And with this week's release of his company's SaaS platform, just about anyone can use it.
After three years working in stealth, Immersion ramped up its marketing right around the onset of the pandemic. The company began as a service focused primarily on helping entertainment companies create better content.
It has worked with a handful of Hollywood studios, including Paramount and Warner Bros, to help produce movie trailers, determine slates for the upcoming fall TV lineup and ascertain whether during lockdown people preferred to watch new or familiar content.
Zak said Immersion can predict hits with over 80% accuracy. Music streamer Pandora has used the service to study which songs listeners would enjoy, he said.
The idea of using mind-interpreting software on the masses to shape what we experience offers intriguing possibilities, but some say it could also amplify biases and distort creative output by favoring content that scores well on brain-activity metrics.
Traditional focus groups rely on surveys to gather feedback. When people fill out questionnaires on what they liked and disliked about a given experience, they have time to counteract their subconscious biases that may instinctively cause them to recoil from certain concepts they find unappealing, such as homosexuality, said Patrick Lin, director of Ethics + Emerging Sciences Group at Cal Poly in San Luis Obispo. Relying on real-time brain activity, though, doesn't give people the opportunity to self-correct for those biases.
"You can hide that in a survey, but you might not be able to hide it from a technology like this," said Lin. That could skew productions away from edgier or more provocative fare that Lin says can be useful for dislodging society from its comfort zones.
But to Zak, being able to measure how people really feel offers tremendous potential for improving and even lengthening lives. He is in talks with smartwatch makers to include Immersion on their devices out of the box. The reason someone would want that, he said, is to learn from their data what frustrates them and what makes them happy.
"Then you can begin to curate people's lives for greater happiness," he said. "And we know that individuals who are happier live longer."
When the world went remote, film production slowed down and face-to-face contact dwindled, and other kinds of businesses began looking to Immersion for help. Companies needed ways to monitor the effectiveness of their attempts to adapt to a distributed world where social cues like body language were no longer available and surveys were unreliable. Zak said he has signed on three of the five FAANG companies as clients to help them make meetings and employee trainings more engaging.
Immersion is designed so that, in these situations, employers can only match data to specific employees if they have consented to having their identities revealed. Zak said Immersion does not store any personally identifiable information online, and noted that his company has worked with European firms and was deemed compliant with the EU's strict data privacy-protection laws.
But going deeper into workers' minds not only raises privacy questions but could also make employees' lives more difficult.
One could easily imagine unscrupulous companies using the technology to squeeze out every last drop of employee productivity, said Michael Karanicolas, executive director of the UCLA Institute for Technology, Law and Policy. He pointed to news reports of Amazon employees struggling to find time to use the bathroom during their shifts as an example of the danger.
When asked about potential ethical concerns, Zak emphasized his company's policy of requiring consent before people's brain activity is tracked.
Zak has been a trailblazer in the field of neuroeconomics. He has received grants totaling over $1 million from the U.S. Department of Defense and Intelligence community to research what motivates people to make decisions and take action. His 2011 TED talk on oxytocin has nearly 2 million views. He was even once named one of the 10 sexiest geeks by WIRED Magazine.
Zak formed Immersion when the university where his lab is based grew uncomfortable with commercial applications of his research, he said.
In March this year, his company landed a $1.7 million seed investment led by Silicon Valley billionaire investor Tim Draper.
"The real arc of my professional life has been to create technologies to help people live more fulfilled and happier lives," he said. "And so this is really the culmination for me of 25 years of my life."
Yet with his company's lofty goals comes the possibility, as with any technology, of unintended consequences.
"These mindreading technologies are going to chip away at the last fig leaf we have," said Lin, "–the privacy inside our own head."
Editor's note: This story was updated to clarify role of oxytocin
- Corto Aims to Bring AI to Hollywood Script Writing - dot.LA ›
- L.A. Startup Immersion Launches to Measure Audience 'Love' - dot.LA ›
Bob Gerard was sitting in an office room, having just watched a public service announcement while wearing a sensor that Dr. Paul Zak had wrapped around his arm.
"When it was done," said Gerard, who had been introduced to Zak by a mutual friend who thought Zak could help Gerard improve his internal training sessions at Accenture, a professional services firm, "Paul showed me the graph that the sensor had generated, and he replayed the video. As it was replaying, he was telling me exactly what I was feeling as he was reading it off the chart. My first thought was: I need this."
What this is, is called Immersion: a software program that, in the words of the L.A.-based startup's Chief Executive Scott Brown, "measures love." The sensor monitors how a person's brain responds to a given stimulus, moment by moment. And the readout that the software generates, for up to dozens of people at a time, provides real-time feedback that could be useful for everything from testing Hollywood audiences to understanding what resonates during a presentation.
A sensor measures the brain's response to content and uploads the data
Based on 20 years of scientific research, the company said, the technology can be used to understand which moments in a piece of content — a trailer, an advertisement, a lecture — are emotionally resonant, and which moments aren't. Creators can then adjust accordingly, and gain insight from algorithmic predictions about what actions people will take in response.
Immersion, Zak explained, specifically measures two factors in the brain: attention, and emotional engagement. When these two levels are high, Zak said, a person is immersed in the material at hand.
The company, which was founded less than three years ago and has mainly been in stealth mode, grew from Zak's work as a social sciences researcher and professor at Claremont Graduate University. He is also the director of the Center for Neuroeconomics Studies, which "develops neuroscience technologies to understand how people make decisions." Zak said the two main client uses of Immersion have been entertainment — such as for film, TV shows or advertisements — and corporate trainings.
Dr. Paul Zak runs the Center for Neuroeconomics Studies at Claremont Graduate College
Zak's research, which has received funding from the U.S. Department of Defense and Intelligence community, produced a set of findings that eventually formed into Immersion's commercial product. He found that oxytocin, a molecule produced in mammals, strongly correlates with emotional resonance, and he figured out how to measure oxytocin levels and attention non-invasively. Immersion's sensors — and, as of this week, other sensors like the Apple Watch or Fitbit Versa — can capture that data, then upload it into Immersion's software, which displays people's neurologic responses in a way that is designed to allow users like Gerard to get insight about their content "without having to have a PhD in data science," Zak said.
At Accenture, Gerard estimates that he's used Immersion for hundreds of classes and thousands of trainees since signing up for the subscription service. The program has helped to reveal which parts of trainings are engaging and should be expanded, and which parts are snoozers and need to be improved. Gerard has also experimented with using Immersion for sales training workshops, wherein two employees role-play a sales pitch while the one playing the customer wears a sensor that monitors her reactions and displays them to the class in real time.
An Immersion readout
Brown and Zak said Immersion has also been designed to enable clients to predict future behavior. In fact, they claim, because it relies on brain measurement rather than what people say, Immersion can understand audience response and intent better than audiences understand themselves.
When someone's levels of emotional resonance and attention are high, Zak explained, the brain "tags" those moments as "more relevant." These moments become "more easily recalled, and therefore more likely to be acted upon," Zak said. That is, the more that one is "immersed" in the content, the more likely they are to, say, watch the film (ie, if the piece of content is a trailer) or buy the product (if it's an ad). Immersion can also, Brown said, predict whether a piece of content will be a hit. Based on numerous studies, including with big companies like Facebook, Paramount and Warner, Immersion claims predictive accuracy of over 80%.
One hurdle that Immersion faces to growing is skepticism about a "big brother" omniscience about people's "true" feelings. Zak emphasized, however, that users must opt in to having their brain activity tracked, and that clients decide how much data to collect. Accenture, for example, chooses not to gather any personally identifiable information, Gerard said. Zak also suggested to dot.LA that being able to identify specific individuals who were highly immersed could be beneficial, as those people could be used as evangelists or for peer support.
The coronavirus pandemic is not exactly well timed for the company that according to Zak is "just now taking the lid off" and doesn't have a full-time marketing team, but the founder is confident his company will survive. While some clients, including a major studio and a large appliance-maker, have had to put their subscriptions or negotiations on hold, interest is perking up elsewhere. As companies and society experiment with distributed work, Zak is hopeful that his distributed neuroscience firm – subjects need not congregate in the same room – can fill a void. He's in talks with several potential new clients that are working from home at a distance.
"You want to test content and see what the brain's responding to?" Zak asked. "Right now, we're the only game in town."
Sam Blake covers media and entertainment for dot.LA. Find him on Twitter @hisamblake and email him at samblake@dot.LA