I would love to sit down and do a post-mortem on Hypercard in the World.
Bret, I also like your analysis of the "db" object. I think I felt a good amount of friction from using that kind of querying and it would be great to imagine a new one for the next iteration.
I've also been thinking a lot about state both in Flowsheets and in Hypercard in the World. If we take Rich Hickey's dictum seriously that state is the hard part of programming, we should think hard about how it should be represented in our system.
I like that data, behavior, and virtual state are attached to physical objects. It seems worth pushing that idea further. Something I worry about is that in our haste to physicalize everything we'll forget that "disembodied" search is actually incredibly powerful (e.g. searching with cmd+f vs. trying to do a visual scan). I bet RMO probably has more thoughts on this.
I think having a "wall of system state" (Big Board) has been great and should be developed further, but has had clear limitations. I've noticed a lot of awkward shuffling back and forth between the thing you're trying to make in one part of the room and Big Board in the other. Maybe it would be interesting to have a mobile Big Board, or possibly be able to instantiate a new Big Board elsewhere in the room while you're working on something else.
One thing I'd like to be able to try is just blessing a piece of paper with the virtual state of an object, kind of like in Toby's mockup. Even though Laser Socks and Laser Painting were conceptually simple, I think I felt a lot of friction around not being able to see/edit the state was easily. I wonder what it would be like to have that state spatialized around me, kind of like Big Board, but on a per-application basis. If I could somehow use this while developing Flowsheets it would be great! I already tried to make my own version of this using an ipad, but it hasn't been that useful yet:
Using React has made me love its style of declarative programming, where you update state and all the things depending on that state change automatically. It really feels like it makes the system simpler, but I also think it has a lot of benefits in tracing cause and effect. For example, being able to ask "what things can cause this piece of state to change" and "what things depend on this state" seems important, and is not something you can do easily in any system I know of. One of the difficult parts of stateful programming seems to be that something in one part of the system can affect something elsewhere in the system and it's hard to know how that happened or why. This could be especially bewildering in a physical location where something in one part of the room invisibly affects something in another part of the room.
Taking the declarative programming idea further, if most of the virtual state displays (like the projection on Laser Socks' sock-meters) could be expressed as deterministic computations, then I think that would have benefits in being able to explore the history of running programs. We could just save everything non-deterministic and recompute everything else on-the-fly, allowing scrubbing through history as many of us have been dreaming of.
Anyway, I'm not sure how much of this makes sense to anyone (I'm writing this on the plane back to SF), but I'd be happy to talk more about this in-person.
P.S. I don't think I can explain it in this email, but animations in Hypercard in the World seem pretty interesting to me. E.g. lasering a book, and a visual animation flies from the book to the wall above to indicate their relationship, kind of like the animations in iOS. Something to think about going forward.
On Sun, Jan 17, 2016 at 4:33 PM, Bret Victor wrote:
What do you see when you inspect one of the "collection" foamcore objects (like "identities", "attachments", "displays")? Are the rows attachments on these objects, showing up on the left column of the inspector? I'm trying to place where this state lives.A collection is just an attachment. In the same way that the content of a daemon is a string of code, the content of a collection is a set of tuples.Is all state that is accessible to daemons and illuminations (via db.find, get, and set calls) also accessible by inspecting an appropriate object (the object that the state lives on)?Yes. All tuples returned by db.find, db.get, etc. are members of a collection, and all collections are attached to some physical object.The only way to add new state to the system is with db.add(collection, tuple), which explicitly puts your new tuple into a collection (attached to a physical object).On Jan 17, 2016, at 12:49 PM, Toby Schachman wrote:Some clarifying questions:What do you see when you inspect one of the "collection" foamcore objects (like "identities", "attachments", "displays")? Are the rows attachments on these objects, showing up on the left column of the inspector? I'm trying to place where this state lives.Is all state that is accessible to daemons and illuminations (via db.find, get, and set calls) also accessible by inspecting an appropriate object (the object that the state lives on)?On Sun, Jan 17, 2016 at 11:13 AM, Bret Victor wrote:Nagle suggested a "post-mortem" on Hypercard in the World as it currently is, to discuss what worked well and what didn't. I'd like to do that at some point. In the meantime, I thought I would write a brief description of how the system currently works.---Hypercard in the World is a world of physical objects with virtual attachments.In order to participate, an object must be blessed, which provides it with an identity.It can then have attachments added. Attachments can describe:- physical state (e.g., where it is: "location in container"; or the trio "location in camera", "location in display", "physical size")- virtual state ("variable", "collection", "file")- behavior ("daemon", "illumination")---Objects can be inspected through the web-based inspector:<Screen Shot 2015-04-21 at 1.24.34 PM.png>In the left column, you see that the "hit-test root objects" object has a "location in container" attachment and a "daemon" attachment.Selecting an attachment in the left column lets you edit it in the right column. There are different types of editors for different types of attachments. In the screenshot, you see a daemon attachment being edited with a code editor.Attachments are viewed live, so if (for example) an object is changing a variable, you can watch that change simply by inspecting the variable attachment.---Hypercard in the World is "built in itself", in the sense that the identities and attachments of all objects are represented by values which themselves live in collections on physical objects.<IMG_7957.jpg>At the left, you see the "identities" and "attachments" objects. The "identities" object holds a collection whose values look like:{ id, name, family, selection_mode }The "attachments" object holds a collection whose values look like:{ id, object_id, type, ... } (where "..." is specific to the type of attachment)Blessing an object simply means adding a value to the "identities" collection. Unblessing simply means removing this identity, as well as associated attachments. (Blessing and unblessing by hand is normally done through the inspector.)---There are no databases at this level of abstraction.Underneath, all collections are stored in one big Seatbelt database. Sometimes I used to create additional databases for special purposes, but that was cheating and I found I mostly didn't need to do that.In scripts, there is a Javascript API for accessing and querying collections, which looks like:db.find(collection, query)db.get(collection, id)db.set(collection, id, value)etc.The "db" prefix was a bad choice, because it misleadingly blurs the abstraction boundary. Early mockups referred to state with tokens instead:<v2.png><sin.png>I couldn't find a reliable way to implement tokens in the code editor.The API is also awkward because it only deals with values in collections. There is no API for working with identities, variables, etc. Instead, you read a variable by looking it up in the meta-level "attachments" collection. I kept it this way to avoid adding layers of opaque "helper functions" until I better understood what was needed -- the principle was to keep things as transparent as possible, so you stayed in touch with what was really going on.What I should have done was add the helper functions anyway for the game jam etc., recognizing that no raw Javascript API could provide the desired transparency without being awkward, and simply looking ahead to when we could get away from raw Javascript.On Jan 17, 2016, at 3:20 AM, Toby Schachman wrote:Here are a few ideas about organizing state in the new system.In the current Room OS, there are two levels of state. There's the database and then on top of this the "hypercard-y" bits of state on objects (Variables, Attachments, ...).In our next system, it seems like we want all state to live on objects. Whether or not there is a database that makes this work is behind the abstraction boundary. Such a database would not be exposed by the system.In Hypercard, all state (properties) and behavior (scripts) are "attached to" or "live on" an on-screen object (card, button, text field, etc). In Apparatus, all state (attribute values) and behavior (attribute expressions) live on a shape. In both, you can pull up an inspector to see or edit any of this state.<image.png><image.png>I was inspired by Bret mentioning on Friday how he wants to be able to laser a camera and then see what the camera is seeing. What the camera sees is virtual state, so it needs to live on an object. And what better object for it to live on than the physical camera itself.Here is a mockup of inspecting a webcam:<image.png>Here's what inspecting the lion in the Serengeti might show you:<image.png>You see all the virtual state as a list of properties. Values are shown as their visualization (as in previous email). You can use (reference) any of these values within expressions, as in Apparatus/DDV. Or you can reference objects in scripts and change their properties when events happen, as in Hypercard/Smalltalk.I feel like this simple model can take us very far. Just inspect a few objects--use as many inspector boards as you want, we've got all this space!--then hook the data together like in Apparatus or Hypercard.The hooking together is easy. Generally Hypercard scripts and Apparatus expressions are short. You get rich behavior by setting up rich relationships between objects.Virtual state encompasses:- In data: image the camera sees, what keys are pressed down on the keyboard, the accelerometer data on the wiimote.- Out data: brightness level of the LED, image the screen should show, image the printer should print, angle the servo should go to.- Variables: angle the frog's eyes should be looking at, mole game time remaining, laser socks "health level".- Attachments: lion's roar sound, mole image, Whole Earth Catalog's page images. (Attachments are essentially constant variables.)- Behavior definitions: expressions that evaluate, scripts that run.All of this state would be accessible and editable in a uniform manner by inspecting the object it lives on.
