After seeing the work-in-progress Omni map[gif. 0], Bryan Newbold (added to CC) suggested that I explore the use of space-filling Hilbert Curves to represent paths of video. My impulse is to be immediately suspicious of overly math-y visualizations, but the idea of the Hilbert Curve resonated with the fundamental contradiction of making a video timeline. Where a Hilbert Curve makes space out of path (1D->2D), so then must a timeline attempt to render time out of space. I threw together a simple implementation this afternoon.

Consider the old “griddle” style rendering of all the frames in a video:

<teenage-griddle.jpg>

and how much more “spatial” it appears using hilbert indexing:

<teenage-hilbert.jpg>

Here, to use the vocabulary of Kevin Lynch’s Image of the City, each cut of video becomes a clear spatial “region.” How, exactly, time moves in this map is unclear, but at the same time it is undeniably a map. Here is a gif-animation of the transformation from griddle to hilbert (link for the gif-impaired):

<teenage_morph.gif>

I think this technique is starting to resolve the “they all look the same” critique of video grids. Here are 45,000 frames of John Berger’s Ways of Seeing (ep.1):

<wos_hilbert.jpg>

The crucial thing here is that you can zoom in without any re-flowing and get to a close-up like this, where contiguous regions are still for the most part contiguous sections of time:

<wos_crop.jpg>

(Can you pattern-match to find where this closeup came from? Cool!)

The morphing for Ways of Seeing is mesmerizing, by the way. I love looking at the frames halfway between line-wrap and hilbert grid:

<wos_middle.jpg>

Your correspondent,

R.M.O.

[gif. 0]

Also available as a link for the inline-gif-challenged

<omni-tour.gif>

On Nov 9, 2014, at 4:25 PM, Dave Cerf wrote:

The notion of media—and edited media—as terrain is very interesting to me. Subway and geographical maps tend to maintain a consistent shape over time, and so landmarks can be relied upon. Media is much more dynamic: new material constantly coming in, and the structure of an edit can shift radically. To paraphrase Walter M., the rough cut is largely scaffolding: a prototype of what the film could be. The twist is that the scaffolding and the building materials—media—are the same. Or, in map terms, the underlying land masses and the boundaries around them are interdependent. Removing one removes the other, so that memorable shapes and landmarks can easily disappear. 

I have wondered if timelines could benefit from fixed landmarks that are independent of the edited media clips placed on its surface. In the example below, I can direct you to different parts of the movie using landmarks in the image: Alcatraz: that’s Act 1. The midpoint of the bridge is Act Two. Where the fog is entering on the right, that’s Act 3. See the boat going underneath the bridge? There’s a clip just to the right of it that needs to be replaced with a new take. I was able to convey that to you without the need for timecode nor tedious “go forwards, no backwards… you went too far."

<LandmarkTimeline.jpeg>

How about a BART map?

<il_570xN.329527626.jpg>

Could that be used as a scaffolding on to which media clips can be placed?” Over time, as the movie evolves, the scaffolding and its labels can be replaced with more relevant structural names: Act 1, The Breakfast Scene, Terror in the Woods, and so on. “Daly City is Act 1” would be a sentence that makes sense.

<PastedGraphic-21.png>

Using Google, I found some movie maps, with some

more

interesting

possibilities

Dave

On Nov 9, 2014, at 9:17 AM, Chaim Gingold wrote:

This is exciting to see, and you working out these new mockup muscles. :-) looking forward to discussing this week! I'll bring my copy of Lynch in if I can find it and also I see most of it online as pdf

On Nov 8, 2014, at 6:13 PM, Robert M Ochshorn wrote:

...stepping back a bit.

I don't make (non-dynamic) mock-ups very often, so you'll have to
forgive the clumsy sketches, attached. It's both empowering and
daunting not to have to write code to see how something might
look. Empowering, because I can just make a few boxes and then there
seems to be some crazy functionality. Daunting, because of the
code-pain that will inevitably follow from every rounded corner.

Almost certainly I don't have all the details right yet, but I think
the idea of gaps (suggested in subtly different ways by Chaim and
Glen, and probably everyone who I've shown Interlace[0]) is very
powerful:

1. a gap between sources of material can expand to show more
 information about that source; and

2. a gap between lines can expand to show time-based annotations.

The premise of these timelines, broadly, is that they are more
"terrain" than mere "transitional object" (i.e., towards a "final,
professional, cut") and that (to paraphrase a book Chaim was talking
about but I didn't read yet[1]) the landscape can accommodate
landmarks, wayfinding, &c. Shared and private landscapes connect
personal and political (?).

I guess these would be generally suited for (combinations of) video,
text, sound, and image.

R.M.O.


[0] A lot of design decisions were/are as much reactions *against* as
  proposals *for*. In the case of reifying sources (or not), I
  reject the assumption that files as markers are "naturally" a
  meaningful place to segment. For example:

  * Sometimes the camera runs much longer than than subject
    breaks. For example, a conference is filmed until the breaks,
    but there are multiple speakers in a continuous take;

  * sometimes arbitrary factors like the length of a reel of film,
    or the file-size limit of a filesystem force premature
    truncation of single "units" of time.

  Clearly, though, the correct* approach is to make the segmentation
  user-editable. Start with files, and then allow merge and split
  operations.

  --
  * Does anyone still believe in "the right way"? I think I do.


[1] Do we have [The Image of the City](https://mitpress.mit.edu/books/image-city)
  in CDG?
<corners.png>