Full video here.

You can see that there is some lag. I have not done any optimization (including suppressing copious log statements), so I'm not sure yet how much the lag can be reduced. I also have not corrected for lens distortion, which may be why the bounds of the projection are not quite right. Finally, probably the tracking could be improved to make it less jittery.

I'm using a single camera that is very close to the projector, so I don't need to track in 3D.

I have retroreflective dots on the four corners of the foamcore and also four dots that I use to mark the four corners of the projection boundary.

This is what the camera sees (above) and the detected dots (below),

By calculating the relationship between the inner dots and the outer dots I can warp the image in the projection to project onto the foamcore.

Specifically, I compute the homography (CV term for a perspective transform) from the outer dots to the projector bounds, use this to transform the inner dots, then compute the homography from the image bounds to the transformed inner dots. I apply this homography to warp the image.

This was mostly a test for me to see if I understood how homographies work in OpenCV. I think I have a better understanding now, though it's still confusing and it would be nice to design a diagrammatic language for making the above paragraph clearer.

It was also a test of mounting techniques. I'm using a Manfrotto "Magic Arm" and "Super Clamp" attached to a coat rack.

The Magic Arm can be loosened and tightened with the big knob. It has an elbow and two ball heads on either end. The Super Clamp can hold on to anything that it can attach around. It's not wide enough to clamp to our columns, but it could certainly clamp to the beams on our ceiling.

I kind of jury-rigged the camera to the projector with a metal bar and magnets. Probably a second magic arm would've made things easier. Maybe in a few weeks as we get more stuff Paula and I can consolidate our findings and do a brief presentation on rigging equipment.

Dealing with all the wiring was a pain. I also can't use my big monitor while also driving the projector with my laptop. So getting IMPs working will make doing experiments like this much more pleasant.