I'm going to try to implement our own retroreflective marker tracking system. I had an email exchange with Ken Perlin and he said there shouldn't be anything stopping us from doing it. He went with a commercial solution just because it was quicker. But since we will need more cameras and have tracking needs that aren't covered by the included software, rolling our own makes sense. It's also a good test for "self-hosting".
I soldered a quick ring light with two LEDs:
I did a test with the PCR model from the programming jam. Video attached.
Distance from camera to furthest test spot: about 11 ft
Size of markers: about 1/4" squared
Distance between markers: 4"
If we put more LEDs on the ring light it should give us even brighter markers, though this is already almost good enough. We just need to look for bright dots, like tracking lasers!
This method needs at least two cameras to see a marker in order to localize it in 3D space. Correspondence between markers across camera images should come for free if our cameras are pre-calibrated (know where they are relative to each other), just shoot a ray from each of the camera locations in the direction of its lit pixels and intersect the rays.
We correlate tracked points across time by taking closest point to last frame (like how laser point identity is done). There will be occlusions, so we will need to be clever as points enter and leave the camera view. But the more cameras we have, the more confident we can be about tracking a point.
I'm imagining each camera attached to a raspberry pi, streaming the locations of the bright dots it sees. (These are essentially wiimotes, except wiimotes only stream up to 4 dot locations.) We'd have tens-hundreds of these in the space.
This tracks points, but leaves the identity of those points an open question. I think we will want to identify objects using another system and then use this tracking system to keep track of where the identified objects are. Object identity might be done by traditional CV, RFID, or by humans. That is, you laser an object and say "this is the PCR machine, keep track of it". Laser points would also be tracked in 3D, and the object would be identified for as long as some cameras kept it in view.
You can also identify an object by its marker points if the object is rigid and the placement of points is unique. I'm not sure if this would give us a big enough identity space but it can be used to increase confidence / fill in missing data.