Date: Tue, 10 May 2016 10:32:19 -0700
From: Bret Victor
Subject: Re: Euler's Theorem in the world (pirate game)
I worry that implementation of recognition will be a challenge with respect to lighting, etc., but we will find a way.

Yep.  I think a few things that might help are the computer giving very clear feedback about what it's recognized (perhaps even rendering its own map that shows its interpretation of the scene, as well as visualizing the CV data that it's working with), the recognition being a "snapshot" rather than continuous (you set things up, the feedback looks correct, you hit the "capture" button, and then you don't have to worry about occlusion from hands, lighting changes, etc.), and maybe being able to manually specify certain things if the computer really can't seem to recognize it for some reason (e.g., it's not getting this vertex, so I'll laser-point to it and say there's a vertex there).  There would be some symbiosis in the person and computer working together to recognize the scene.

We might come up with standard patterns (and perhaps libraries) for this feedback and intervention, and they might fade as our CV gets better.



On May 10, 2016, at 9:21 AM, Toby Schachman wrote:

On Mon, May 9, 2016 at 6:33 PM, Bret Victor wrote:
 
I wonder if you could play on the clay objects themselves, instead of a rendered or flat map.  Just like this:
  
<image.png>

The unconquered territories would be highlighted by the projector, and they would flash and go dark as you moved the ship over them.

This might be especially interesting if the computer didn't have to have an internal model of the topology.  It would just "recognize" faces, vertices, and edges on the real-world object (probably simply as blobs of some given colors).  So you could build any polyhedron with your hands and just start playing on it, without having to program it into the computer.  You could try polyhedra with holes, or "cheat" and leave off edges, etc.

(You could do a similar thing with drawings on paper, instead of clay.)

I really like this idea.

That the geometry objects had "render methods" was a bad smell, as was the manually typing in the coordinates for the map layout. Indeed I drew out the map on paper and did the labeling/indexing there to make it easier to type it in. Why not just use this paper map?

<2016-05-10 09.16.08.jpg><image.png>

I worry that implementation of recognition will be a challenge with respect to lighting, etc., but we will find a way. Matthias and I had made some progress with our hand-drawn diagram interpretation.

Will do some experiments...


--
************************
************************