The Science and Cinema of the Brain

Sloan Foundation gets cerebral at the Sundance Film Festival, going into the science and psychology of motion pictures.

Published February 5, 2006

By Adrienne Burke
Academy Contributor

How is your mind like a movie? Will new technologies enhance the way films convey cognitive experience? How will the ancient human capacity for processing emotions keep pace with rapidly accelerating cognitive experiences?

These and other questions were tackled by a panel of four scientists and three filmmakers recently at the Sundance Film Festival in Park City, Utah. An audience of 250 filmmakers, journalists, and film enthusiasts attended the event called “What’s on Your Mind? The Science and Cinema of the Brain,” hosted by New York’s Alfred P. Sloan Foundation on January 27, to engage in a discussion about how movies can be tools for exploring the mind, for fulfilling the human need to vicariously experience emotion, or for mimicking the editing process in which our brains engage.

Meet the Panel

Moderating the panel was John Underkoffler, an MIT-trained engineer who has consulted as a science and technology advisor on films such as Steven Spielberg’s “Minority Report” and “The Hulk,” in which Nick Nolte plays a mad scientist.

Panelists, in order of appearance, were:

• Lynn Hershman Leeson, artist and director of the films “Conceiving Ada,” about the contributions of the Countess of Lovelace to early computer science, and “Teknolust,” which won the Sloan Award at the 2002 Hamptons Film Festival;
• Hal Haberman and Jeremy Passmore, the directing and writing team that created a film screened this year at Sundance called “Special,” about a man who enters a clinical trial and suffers a breakdown and thinks he is a superhero;
• Antonio Damasio, a neurologist and neuroscientist who directs the University of Southern California Institute for the Study of the Brain and Creativity;
• Martha Farah, director of the University of Pennsylvania’s Center for Cognitive Neuroscience; and
• Kay Jamison, professor of psychiatry at Johns Hopkins University School of Medicine and author of several books on manic depression and bipolar disorder, including her autobiography, An Unquiet Mind.

Storytelling and Technology

Underkoffler kicked off the discussion pointing out that new technologies such as functional MRI are enabling neuroscientists to see where in the operating mind different activities are taking place, and to address for the first time questions that were previously the domain of philosophers, only answerable through intuitive thought, not scientific analysis. Considering that film is a unique vehicle for conveying states of mind, Underkoffler asked, “Is film privileged as a tool for exploring these ideas of mind and brain?”

*Here is an abridged version of the conversation that followed.*

Leeson: The technology always has some kind of way of altering the way we think. Some people have said that iPods are restructuring the way we create narratives. The advent of multidimensional possibilities with DVDs or other aspects of Internet use has created varying levels of how we communicate and what stories we tell and how we develop ideas of fractured intelligence, identity, and even artificial intelligence as characters and character subplots.

Haberman: For me, technology influences how we make movies, but in terms of changing the actual stories we’re telling and the structure of the stories we’re telling, I don’t think those are much different from the way I would have told the story in a movie if I had been alive to make one 30 years ago.

Passmore: I’d agree with that. The film doesn’t happen on the screen or in the speakers; the film happens when it’s synthesized by your brain when you’re sitting in the audience. Film is inherently the medium by which you experience alternate realities. As the technology evolves, whatever is after cinema is going to become even more so.

Frames in the Mind

Damasio: Film, and before it theatre and literature in general, have been historically means of inquiry into the human mind. Greek theatre was doing things similar to what filmmakers are doing today: using narrative you’re looking into the human mind and human behavior.

There’s something privileged about cinema that is different from the other modalities, [because] it’s probably so far the closest we can have to the kind of subjective experience we have of our own mind. It has to do with the fact that there is a frame in our minds when we’re looking at the world, whether we’re looking at the actual world, or into our minds with our eyes closed. The visual and the auditory are very powerful and are the bread and butter of film making. They bring us much closer to the experience of our own mind.

It’s as if film has [copied] some of the characteristics of the human mind. Editing is something we do all the time when we apportion attention differently to one image or another. We are constantly running an editing machine in our own mind by bringing a character into focus more strongly, by reframing it, or by the duration for which we allow the image of that character to linger.

It’s quite interesting that there are very close connections between the mind process and what our eyes are doing. John Huston might have been the first to point out that you cut on the blink in filmmaking. It’s something that shows film to be very privileged in its connection to brain and mind science, far more so than literature or theatre of any kind I can think of.

Simulating Experiences

Farah: I think the film “Being John Malkovich” illustrates your point well — that through film we can simulate the subjective experience of another person. “Special” does the same thing with this ambiguity between Les’s perception of what is going on and the reality. It’s a seemingly unbridgeable gulf that cognitive neuroscientists are continually trying to bridge, between subjective mental experience and objective observable things.

Haberman: “Being John Malkovich” is interesting also because it shows how you can illustrate things cinematically for a broader audience than scientists. A lot of people probably don’t know what a feedback loop is, but when they walk down the tunnel and there are John Malkoviches everywhere, I think intuitively [the audience] understands what’s happening. It illustrates a scientific principle without feeling like it’s telling or explaining to you.

Redefining Film

Leeson: I think the whole definition of film is radically changing right now, in a way that we haven’t seen in the last hundred years. We’re developing different options for how we look at moving images and therefore the whole definition of what film is and dealing with possibilities for entering virtual realities … We’ve never been able to have these possibilities before.

Jamison: If you’re trying to convey mood or desolation or despair or psychosis, or madness or ecstasy or expansive mood, it’s so much in the acting and directing and writing. The technology is not my bailiwick, but it seems to me that tremendous portrayal has been done so well since the beginning of film. If you’re trying to convey a mood such as desolation or despair, what is it in the technology recently that has made any difference in how well that would come across now to an audience as opposed to 30 years ago?

Underkoffler: Technologically, it seems like nothing. The digital resolution, sound, would have no bearing.

Leeson: Some artists are using PDAs to create environments that do alter moods when one goes there. They create installations and environments that are addressing these very particular issues.

A Wider Domain

Haberman: I think the most obvious example is video games that are so popular right now. That experience couldn’t have happened 10 years ago. They’re playing a narrative. It’s a whole way of watching a story.

Passmore: It’s kind of like antidepressants. It’s our version of “we don’t really know what the long term effects of it will be.”

Leeson: We’ve never had the connectedness that we have now. We’re able to interpret and hear so many points of view that it seems like we’re congealing things beyond a particular culture to a wider domain.

Haberman: But that’s something people have been thinking has been going on for years and years. Even if you look at things people were writing in the 1960s, it was all about connectedness and different cultures coming together. And all the poststructuralist film theory from the 1980s is the same thing: People always want to feel they’re more and more connected with each other and that technology does that, but I’m not convinced it does.

Transhumanists Thinking Like Bats

Underkoffler: I’m also interested in technologically expanded options for what cinema might become. It’s interesting to wonder what else is possible. Peter Greenway famously and cantankerously said sometime in the early 1990s that film had done nothing but produce illustrated 19th century novels in the sense that they follow a comprehensible narrative. What else could film do to map our cognitive or mental states onto other possibly even nonhuman or transhuman artifacts or situations? Might we elicit some kind of state that is impossible to elicit in any other way?

Farah: Well, it’s like the famous article “What Is It Like to Be a Bat?” by the philosopher Thomas Nagel, who ends up concluding that you can’t know what it’s like to be a bat because you don’t have a bat brain, you don’t have a bat experience.

Underkoffler: And you don’t have a bat body.

Passmore: What we need is a bat filmmaker.

The Essence of the Subjective Experience

Farah: How close could you get to a bat experience by watching a film? I’m going to say not very. If you can’t get the essence of the subjective experience of being a bat by walking around in the world having light impinge on your retina because it’s reflecting off surfaces around us, I don’t see how having light impinge on your retina because it’s coming from a movie screen is going to make a difference.

But one thing that might make a difference is a sort of wacky idea that Ray Kurzweil describes in his new book The Singularity is Near: When Humans Transcend Biologyall about how changes in computer- and nanotechnology are going to increasingly be incorporated into our bodies, including our central nervous systems. Eventually we’ll gradually transform ourselves into these cyborg creatures that won’t resemble much the humanity version 1.0, which is what we are sitting around here today.

One interesting scenario he describes is the use of nanotechnology to penetrate our nervous systems. We would first use nanotechnology to get a highly detailed, three-dimensional image of the state of somebody else’s brain. A nanobot would go into John’s ear and infiltrate his brain and get the picture and then I could inhale them into my brain and they could simulate the same state and thereby let me know what it’s like to be John Underkoffler. And maybe they could do the same thing with a bat.

The Cyborgian Age

Leeson: I think we already are posthuman and we’ve already entered the cyborgian age. More and more symbiosis with technology is altering the way we’re thinking. And as far as projections into the future, I think one that’s very close is how we distribute narratives, not just only on screens in dark rooms, but on computers and through software programs that incorporate moving images and build memory.

Damasio: I think with the Kurzweil scenario, there’s no need for immediate worry. It’s far into the distant future. If the Kurzweil scenario comes to pass it will lead to different relationships within ourselves and with technology, and I don’t know if it will illuminate our experience with nonhuman species, but I don’t think it will affect film as it is in itself. Film could portray all of this, but it doesn’t follow that it will alter it necessarily and change that fundamental technique.

How Movies Nourish Emotions

Passmore: My opinion is that this technology is great, it will help bring new ways of telling stories to people, but I think there’s a reason the narrative structure hasn’t changed over 1,000 years. It’s because we want to experience someone else’s life, someone else’s reality. We want to see a character and view the world through that character’s eyes and I think that’s the basis of narrative and I don’t see that changing anytime soon.

At the end of the day, you have an audience that wants someone they can identify with. There are always going to be people trying to beat their heads against the wall trying new things, but eventually the strength of the narrative in its current form is going to carry on forever.

Damasio: That has a lot of do with our own needs to experience vicariously emotional states. There are a lot of things going on in movies traditionally and in classical novels and theatre that is a way to experience emotion we would like to have and sometimes experience emotions that we would not like to have.

I don’t think anybody would choose to be in situations that cause extreme horror and terror and so on, but the fact is that people flock to movies that have suspense and show fear and that lead you to experience enormous horror sometimes. I think there’s one reason that continues, and that is that we rehearse. In some way we get rid of the need to worry about them, because we are going through that experience in a way that we know once the lights come up we’re not going to get killed or nothing terrible is going to happen to us.

Our Own Mortality

Passmore: It tricks us into thinking that we’ve dealt with our own mortality.

Damasio: Exactly. We need to have nourishment for our own emotions. And here I would point out biology. There is a big disconnect between the way our brain and our organism processes emotions, and the way our organism processes what people call straight cognition. Cognition is like lightning. Cognition is very rapid, and has the potential to become more rapid.

It’s quite likely that people in the world who are growing up with new technologies are going to have even more rapid cognition. But that doesn’t mean that they’re going to have faster emotional processes, because the emotional processes are very old, in terms of evolution, and they’re probably much more rigid and difficult to change at least over a course of a relatively limited period of time.

Leeson: Do you think there’s a difference in generational cognition and that it’s changing?

Jamison: I would address the emotional side, which is the more ancient side, and that probably is not changing nearly so rapidly. The thinking process probably is, but the moods and the fears and so forth are not changing so rapidly, so it’s a fascinating time in human evolution.

Also read: Music on the Mind: A Neurologist’s Take