Gil Sperling

video, stage and music

Why computation?

September 9, 2019 by gilsperling


The kinds of work I like to consume and to make are work that provokes new thoughts, that raises questions which linger with me long after the work has ended, or that opens new possibilities unimagined before about how to create or how to look at the world.

A non-computational example: The short film Un Chien Andalou by Luis Bunuel, one of the first applications of the surrealist aesthetic to cinematic language. Bunuel uses the non-linear nature of film editing to open new paths into understanding the unconscious and representing it through art. It’s amazing how inventively the work uses the grammar of film, which is such a young medium at the time. It also demonstrates why the medium is especially attuned to portraying the human condition at that moment in history (post-WWI, post-enlightenment, pre-sexual revolution).

A computational example: Yesterday Tomorrow by Annie Dorsen. This performance piece, which I saw live in 2016, uses an algorithm (credited to Pierre Godard) to transform music and text over time, from the song “Yesterday” by the Beatles into the song “Tomorrow” from the musical Annie. Each iteration of the transformation is relayed in real time (visualized by ITP alum Ryan Holsopple!) to a vocal ensemble, who have to sing the music as it goes from the familiar source towards an increasingly non-sensical and “non-musical” middle, then back to the familiar destination. While I found the musical choices and staging a bit cheesy, compared to the complexity of the algorithmic element, this work did leave me with many penetrating thoughts such as:

  • The ability of computation to represent and embody the unknown and the chaotic
  • How will we as humans deal with a world where our actions are increasingly determined by AI, designed by a logic we can’t comprehend?
  • Where does the creative power lie – is it with the algorithm, that yields different and unexpected paths with each performance, or with the writer of the code, who allowed for that specific kind of unpredictability?

My own work

In 2014 I did projection design for the theater piece Uriel Acosta: I want that man! by Target Margin Theater. One element of the design was a “wand”, actually a hose spouting dry ice fog, which acted as a living projection surface – catching different images as an operator waved it around the stage.

The “wand”: directing fog to catch projections on a stage

While there was definitely a certain magic to this process, I can imagine it being more exciting if I could make the video content more generative and interactive by having it respond to the specific shape of the fog cloud, or the type of wand movements the operator makes.

Projects I would like to make

  • An interactive video that tracks and analyzes the viewer’s eye movement, and can focus/enhance depending on where they’re looking inside the image
  • Visuals for a live music performance that interact with biometrics from the singer (mouth positions, diaphragm motion, sweat), and with physical input from instruments (bow movement on a string instrument)
  • Use projection to animate a physical book in response to the reader’s touch on the page
  • Research and prototype an AI performer – what would it take for an AI to perform dramatic text in an interesting way? Could it improvise with human performers? Could we take about a machine embodying a character the same way we talk about a method actor doing the same?
  • An algorithm that can evaluate what effect a certain economic action would have on a given set of social, public health, and ecological goals (e.g. how would cancelling all U.S. student debt affect GDP, income inequality and workforce education level).

I guess what I would like to work on in this semester are building-block projects that will lead me to being able to tackle the above projects, hopefully before I graduate. Some of the above ideas seem like a thesis-scale project, but perhaps others aren’t as complicated as I perceive them to be?

Posted in Fall '19 - Introduction to Computational Media |

Comments are closed.