Here's a z-order, or Morton, curve that I'm calling Grilton because I mistakenly thought Grilbert came from "grill" and "hilbert":

​

And the video is attached.

I thought this would be a more natural-feeling order because it sort of preserves the left-to-right, top-to-bottom thing we're used to while adding that nice fractal sauce hilbert has.
​

On Sat, Dec 6, 2014 at 2:12 PM, Dave Cerf wrote:

This is officially fantastic. What I’m seeing—regardless of what you’re going for—is something we discussed a while back coming to fruition: a means of overcoming the abstract patterns formed in linear thumbnail grids.

———————————————————————————

1) We know that a weakness of the wrapping thumbnail grid is that abstract visual patterns emerge. In the visual parsing war within our brains, those patterns win out over the actual content of the thumbnails, which defeats some of the purpose of the grid in the first place. Despite its Tufte-like small multiple honesty (“Just plot all the data and people’s eyes will do the rest”), the linear thumbnail grid has failed to deliver (e.g. “Hey Dave, cool art project, but what is it for?”).

———————————————————————————

2) Changing the thumbnail size in a linear grid, which helps us identify the content of individual frames, is massively disorienting in the spatiotemporal dimension. Now we might be able to recognize what this shot is (though my guess is the “pattern abstraction" may still be overwhelming for most of you, not having seen the shot below playing normally), but we are really lost in time because we are so zoomed in.

———————————————————————————

3A) Like the linear grid, Grilbert forms abstract visual patterns, but the uniquely shaped, semi-rectangular land masses provide a “zoom zone” allowing for larger, easier-to-parse thumbnails! In terms of recognizing content while remaining oriented in time, this is a big improvement over the linear grid; the size and shape of the map never changes (maintains orientation) and we can see larger thumbnail images. Excellent!

Small thumbnails

Large thumbnails

————

3B) Grilbert’s current shortcoming, in theory, is its lack of linearly mapped time. Without the yellow maze-like outline RMO showed and hid in the example, it isn’t obvious what the path to follow is for linear time, and a person “skimming” over the thumbnails with a cursor would not be able to reliably trace this shape.

The value of easily navigable time cannot be underestimated. Here is a mundane comparison between QuickTime 7 and QuickTime X, where the former has playback controls the full width of the window and underneath the video frame, while the latter imposes “shy” playback controls on top of the video frame and less than the full width.

Quicktime Player 7

QuickTime Player X

In professional circles, QuickTime Player X is universally hated (I can’t speak for non-professional use). Arguably, the shy controls in QuickTime Player X are the most egregious issue, but the narrower navigation track imposes a frustratingly unnecessary limit.

————

3C) Perhaps Grilbert could use a linear navigation track in addition to a skimming line (see the yellow rectangle around “nine seconds” into the Grilbert space—But where is 9 seconds in Grilbert space? you may ask. Yes, my point exactly). Here, I’ve cheaply pasted the QuickTime Player 7 navigation track underneath the Grilbert.

———————————————————————————

4) Grilbert appears to provide a better overview of a video clip than a linear grid. Navigation-wise, it is also useful up to a point: you can get to “roughly around here” with Grilbert, but you still need a linear mapping for more precise navigation. How can one easily morph in and out of linear time from Grilbert as navigation necessitates? Or is that too much to ask (of the person using the tool)?

———————————————————————————

5) There is definitely a video-game-esque quality of this map. And probably the most important thing to get right in video games is navigation. Sometimes that’s all there is to a video game! Perhaps this may seem naive (or intriguing) to the current CDG team, but I’ve often had vague fantasies about building a video editing system into a playable video game: here is your footage, let the game begin! “Winning” the game might require some kind of MacGuffin (high score, save the princess), since the actual goal is simply to finish your movie.

Here are some video game maps in different dimensions and perspectives that remind me vaguely of the Hilbert grid, or interesting Hilbert variations to explore.

On Dec 6, 2014, at 2:52 AM, Robert M Ochshorn wrote:

Grilbert!

I think I’m starting to figure out the hierarchical possibilities, though this isn’t quite right yet:

<teenage-hilbert-sm.mov>

For some reason I always forget that assembling big frames out of parts of little frames almost never a good idea.

I think I want zoomed-in levels to only show one of the containing frames, but to move along the hilbert track (in time & space). Likely I’ll have to implement this before that makes any sense.

This starts to show that the “land masses” can really start giving new possibilities, in terms of different sorts of zoom. Amazingly, this (sort of) works in all the desktop browsers.

Yr corresp,

r.m.o.

ps - dave, I’ve already had to serialize-and-destroy our “what’s good about griddle” whiteboard to even begin to understand the subtleties of hilbert indexing:

<IMG_0428.jpeg>

pps - chaim, i love your whitespace idea, but i don’t understand hilbert indexing well enough to know exactly how to implement it correctly, yet :-)

On Dec 5, 2014, at 9:12 PM, Bret Victor wrote:

I think we need to explore something that's somewhere between Griddle and Hilbert (possibly Griddle at micro scale but Hilbert at macro scale), mostly just to use the name "Grilbert".

On Dec 5, 2014, at 3:22 PM, Dave Cerf wrote:

I had similar thoughts, wanting a connect-the-dots line drawn from the center point of each frame to show their order in the grid of thumbnails. But then I thought, is this really what I want, or have we taken Hilbert too far in this case? For example, in a traditional linear thumbnail grid (Griddle, per RMO), we already know the order based on our reading skills: left to right, top to bottom. So we don’t need indicators at all—order is “intuitive” in the linear case.

RMO, maybe now is the time to photograph the whiteboard listing the benefits of the thumbnail grid, and then make a second list of benefits for the Hilbert Curve approach. When needed, a person can quickly switch between Hilbert and linear. For instance, in Hilbert, I like the recognizability of the “land masses” we see when zooming in and out, which is something totally absent from a linear grid.

Is one benefit of a dynamic medium the fact that representations can quickly transform from one to the other, smoothly enough to maintain orientation between the two? Maybe we can have multiple representations, of which Hilbert is one, and the new interaction isn’t so much learning about a new representation as it is learning to quickly and comfortably switch between representations as needed.



On Dec 5, 2014, at 12:58 PM, Chaim Gingold wrote:

That’s really nice, and helps explain the curve.

But now I want even more cues. What if it charted out with light outlined empty rectangles, or a path, the near future of where the video will pour into? (Also, by the end, I want the source video to get out of the way again and situate itself below the thumbs.) After the video sits down the order is harder to see. What about something like this, and while I’m sure I’ve got the curve wrong hopefully the idea is clear, where adjacent neighborhoods slightly pull away from one another. As if it was all on paper and you cut along these neighborhood boundaries and pulled a little...

<PastedGraphic-3.tiff>

On Dec 5, 2014, at 12:35 PM, Robert M Ochshorn wrote:

I was trying to keep it simple at first, but the pressures of the CDG prototyping environment escalated me towards the “spiral of death” design. Here’s another take:

<simple-hilbert-sm.mov>

On Dec 5, 2014, at 11:14 AM, Chaim Gingold wrote:

lol

(i think?)

On Dec 5, 2014, at 11:06 AM, Bret Victor wrote:

I hadn't realized there was a Poe's Law of prototyping.


On Dec 5, 2014, at 10:36 AM, Chaim Gingold wrote:

On Dec 5, 2014, at 10:06 AM, Dave Cerf wrote:

It may or may not help, but I am impressed by the real-time Hilbertizing you pulled off. What I struggled to see here was the Hilbert pattern forming in the background—but even when the video is near full-size, is it moving in a Hilbert trajectory? Maybe if the images were more transparent when they were near full-size, I could see the formation of the small cells in the background.

...Or if the foreground video sat still, didn’t go off the page, or otherwise hide itself or what is happening underneath! Its motion distracts from the background activity. It mostly just needs to sit still and let us see the video unfolding while getting out of the way of the background activity.

I am reminded of our other discussion regarding “peeking” underneath something, both in regards to quoted text and the hand-flipping animation technique. Your video made me wish I could peek under the layers obscuring the final Hilbert grid underneath.

I was just thinking about rack focus as a method of changing interface “focus.” Perhaps this is just skeuomorphism and I should be fired along with Scott Forstall. Is the desire to bring familiar and comforting interfaces from the “real world” to screens and keyboards inherently regressive?


On Dec 4, 2014, at 7:07 PM, Robert M Ochshorn wrote:

On Dec 3, 2014, at 9:57 PM, Dave Cerf wrote:

to better track with my eye what is going on

This may not help:

<hilbert-sm.mov>