Here's the final chapter of Recognition Kit 2 fully in action: we're recognizing symbols from the alphabet defined by the playing cards on the far right with a brand new one-shot symbol recognizer that Actually Works (and, once I flip back over to the C implementation, matches symbols at something like 130,000 per second). We're then spatially parsing the diagrams on the two "stone tablets" in the middle, defining the axioms of a Hilbert-curve L-system — the parsing starts at the small horizontal or vertical line, and then uses the symbol it sees to determine which direction to parse next, allowing us to recover a linear sequence of drawing instructions from the winding road of arrows on the page (if you look closely you can see the look-here-next boxes). The arrows are turtle graphics instructions (turn left, turn right, draw forward) and the A and B are instructions to substitute the whole drawing of A or B as a subroutine — and the A and B tablets are named by looking for the symbol below the big horizontal line.
The whole program is on the left, including recognition, parsing, L-system iteration, and drawing in 66 lines of code.
(Also visible at the top left is a "stamped drawing", the Recognition/Marks equivalent of snapshots, though it's not in live use here! Oh, and the alphabet cards are created by sticking a Dymo label below a hand-drawn shape (the label is read by Tesseract until we make it to a full recognition kit alphabet), as you can also see at the far right)
Video coming when it's done!
On Feb 2, 2024, at 4:00 PM, Luke Iannini <****************> wrote:(Due to aforementioned scrambling haven't had a chance to reply to this yet but just wanted to to say thanks for the excellent thoughts and I'll respond in detail once unscrambled! "Drawings" are also likely going to take on some new meanings in this new world as we've anticipated long ago with our marks plans!)On Jan 27, 2024, at 5:12 PM, Bret Victor <****************> wrote:Cliffhanger! It all looks great, and looking forward to the big reveal.The "red rectangle" and "to the right of" rules remind me of the "interpreting statements" / shrdlite games from a few years ago -- they clearly want to be written as/x/ is to the right of /y/rather than/x/ is "to the right of" /y/and rules that interpret statements in more flexible ways than mad-libs matching is something we should discuss for the next Realtalk.The next Realtalk will also be a good opportunity to sweep through our lexicon and reconsider what we call things. e.g. a thresholding operation probably shouldn't be called a "recognizer". :) But I think it makes sense that we have to invent the things first and then retroactively figure out the right names after they've settled down.Another retroactive pass would be revisiting how we do spatial relations etc for objects, in light of what you've been doing with marks. That is, marks and objects can probably be largely unified -- objects should also have contours etc, as I'm sure you've thought about. (One change I'm looking forward to is considering the contour of an object to be the bounds of the physical card itself, rather than the dot frame -- treating dot frames as just one kind of mark for inferring the presence of an real object, as suggested in email [composable recognizers][2023-09-10].)One more thing that your video made me think of was sort some of generic "stabilizing" facility or abstraction. There are a number of instances where we there's a jittery source that we filter. For example, we have bothWhen /p/ contains dots /dots/:When /p/ contains stable dots /dots/:and each "stable" variant like this is a one-off. I'm imagining a way of stabilizing anything:When /p/ contains dots /dots/, /stable_dots/ are stabilized /dots/:That might won't work as is because different kinds of values have to be stabilized in different ways... but maybe there are pluggable stabilizers that recognize those different kinds and offer to stabilize them?(A theme: "composable interpreters" for statements, "composable recognizers" for objects, "composable stabilizers" for values.....)On Jan 26, 2024, at 11:55 AM, Luke Iannini <****************> wrote:
Here's an early look at Recognition Kit 2 — this isn't the "big new thing" yet!! This is just set of also-cool things inspired by & setting up for the big thing : ).
[video.mp4]--
************************