Using a Medium’s Limitations as Narrative Structural Elements

I had an interesting conversation with a developer at the Microsoft Hololens Hackathon last Saturday. This fellow was dismayed that 360 images lost their resolution at the outer edges. Which means that a user sees the objects closest to the 360 camera as crisper than objects that are at the outer edges of the “dome.” When you can navigate within this 3D space in VR, the resolution of your surroundings degrades the closer to the edge you get.

I went and slept on a beanbag in the lobby of Fremont Studios and as I was passing out, I thought of a saying I used to hear a lot at Amazon, “It’s not a bug, it’s a feature.”

When I woke up, I started thinking about old photographs. In recent months as I’ve been writing a memoir, I’ve been digging through boxes of family photos taken from the early seventies through this decade. This means not just traveling back in time to glimpse images of myself and my family when we were younger, but viewing those younger versions of people via older iterations of the medium of photography.

What’s “wrong” with this photograph taken in the early 1980s? Well, besides the fact that I apparently didn’t know how to wear shorts?

And when we look at photographs, we don’t say that there’s anything “wrong” with them if they happen to have been taken in the seventies with a rudimentary, 35 mm point-and-click Pentax. We don’t have the sense that something has been lost or degraded if we’re looking through fuzzy shots within the frames of Polaroids.

If the point of VR film is to capture the physical world with as much verisimilitude as possible, then the resolution issue in a navigable 360 image is going to cause you headaches. But if you accept those limitations, you free yourself and open up some interesting creative opportunities.

First, break down 360 VR film into nonfiction and fiction. If you want viewers to be able to experience the beauty of an alpine meadow, they will learn that the resolution gets grainier the farther one moves from the center of the shot. And this is entirely okay. It’s like knowing that objects in the distance in a 2D photo are going to be lower res than images in the foreground. No one has a problem with this.

But if you’re filming something intended to be a fictional narrative in 360, then you can use the degradation of resolution as a narrative component. For instance, you might create a scene in which the objects and figures at the outer edges of the dome are ghosts. Your production design might even accentuate the loss of resolution, applying monochromatic makeup to your actors, creating props that are purposely “pixelated” looking.

Remember those now crappy-looking music videos from the eighties that were made with the pioneering effects tool the Video Toaster? Nowadays they look really dated, but at the time they seemed cutting edge. There are ways to take advantage of a primitive-looking medium and convert its bugs into features. The best example of this I can think of is Michel Gondry’s video for The Chemical Brothers’ “Let Forever Be.”

Here’s Gondry’s basic trick. He replicates “video effects” with physical props, then edits the footage  to create a sense of disorientation as the video segues between digital and analog. This is just one example of how Gondry manipulates the grammar of film with his “homemade” aesthetic. At no point do we think there’s something “wrong” with the way he uses the medium. He understands its limitations, and the limitations become the tools.

You eliminate the wrong by doing it on purpose. The point is to approach the limitations of the medium not as barriers to telling stories, but as the very elements that allow you to tell a new kind of story.