Date: Thu, 18 Feb 2016 18:03:48 -0800
From: Toby Schachman
Subject: Under the table tracking
Today I played with the Trackmate fiducials, which are from Adam Kumpf's thesis (2009, student of Hiroshi Ishii).

They are not different in concept than any other fiducials, but they are the best designed ones I've come across so far.

Inline image 2
Inline image 1

They are slightly smaller than 1"x1" square, or 1" diameter circle if you're not using the analog values. It tracks position, orientation, and identity (>280 trillion unique IDs!). The tracking algorithm is not scale invariant, so you need to calibrate the camera so that it's unwarped and knows the scale of the image it's looking at. But this makes the algorithm relatively simple (about 400 lines of naively optimized C++, probably about 50 lines of Python).

For testing, I clamped a transparent sheet to the end of a table and hooked up a PS3 Eye underneath. I made tokens by glueing paper to 1"x1" balsa.

Inline image 3

The tracking out of the box worked pretty well, I would say definitely well enough for the kinds of applications we have in mind (laying out tokens to specify programs, etc). The software is wonky (openFrameworks and Processing), but manageable.

Inline image 4

Here is a video of it tracking. (The X's are when it reports an unknown ID.)

https://dl.dropboxusercontent.com/u/1802603/web/snapshots/trackmate-test.m4v

What do you think about these "under the table" fiducial trackers? Here are some instantiations from the paper:

Inline image 5

I hesitate because they seem screen-like, a finite resource, just a bigger rectangle. They are also necessarily semi-transparent which usually means flimsier materials, not a nice wood table. Would it be okay to put your coffee on the table?

But it would be easy to make a bunch of them, and we could make them arbitrarily big. We could use nice glass, like a glass coffee table (see also the shelf in the image above). I imagine we'd have these tables on casters like Paula's moveable table. We could also have vertically-oriented "whiteboards" (is there a material that is transparent that magnets stick to?). For a given task or exhibit, you'd just wheel in as many of these surfaces as you need.

If we wanted them not so deep, we could use wider lenses on the Eyes and use multiple Eyes in an array. For reference, I'm tracking a 8"x9" area with the Eye 11" below the surface on its "zoomed out" setting. We could lasercut/etc a nice base that the Eyes could precisely fit into to make calibration simpler.

We could frost the top surface of the acrylic so you wouldn't see the camera internals. We could light from the bottom with IR if we wanted it to be really discreet. You could also write on such a surface with dry erase markers and we could image that from the bottom for free. We could put projectors underneath.

Like every other big thing in the Room, the surface itself would have calibration LEDs or whatever so that the system would know where in the space it is. So then you could also augment from the top: projection map onto the tokens, track taps and gestures with a Kinect, etc.