A week or two ago, Bret fixed a big issue in the current system that was a result of the laser painting system. It had something to do with the large number of paintings saved in the database, but since none of that was visible, the bug was never discovered until I mentioned it off-hand to Bret.
I think if the Laser Painting processes showed what they were doing more clearly, this kind of thing wouldn't have happened, and it would be more understandable (and thus more changeable by other people). I began wondering about this in the shower and sketched out one way this could be a little more visible:
In the lower right, I added a display that reflects the textual form of the current strokes on the canvas (maybe it would just be a view of Canvas.strokes, in current programming terms). This display would update in real-time as you paint in an attempt to gain some casual visibility into how the system works. There's also a display that shows a preview of past paintings that have been made, the database.
When someone hits the Clear button, the current painting's strokes move into the database. I wanted a way of illustrating the spatial relationship, and I thought animation might help:
When I mentioned the idea of animation to show message sending earlier today, Bret expressed some discomfort with the idea of animation in this system. I don't fully understand that discomfort, but he said it felt too much like "UI thinking", which I take to mean it feels too much like creating a world in the computer, rather than letting the world represent itself. I think I understand that impulse (we should be wary of our implicit assumptions about how our technological systems should work and not try to recreate what currently exists), but I think animation is a pretty intriguing way to show communication between physical objects.
I began to imagine how one would make animations to show observations and message-sending, and I thought that it would be a shame if someone had to make this one-off for a particular application. I thought it would be cool if there was a tool you could bring around anywhere in the system that would show you all the communication between objects around it:
It doesn't have to be a piece of paper (Toby imagined it as a physical lamp shedding light on it surroundings), but I like the idea of a tool that you can place near a system and see how all the objects are communicating in real-time. If you don't want to see the communication (maybe it's too distracting), you take the thing away.
In addition to seeing what's currently happening, there's also the issue of seeing what happened in the past. I've been thinking about this a fair bit, but am having some trouble articulating it. Basically, I'm worried that as our applications get more complex, their behavior is going to get a lot less understandable. I've been making sketches with simple systems like Laser Socks, but I want to make sure that our understanding scales with the complexity of our systems. I'm worried that the current programming thing will happen where the programmer sees a bug and goes "wait, what just happened?" and can't go back and understand the causal relationships that lead to that bug. I think this is pretty important, but don't know what that impulse will lead to yet.
On Fri, Apr 22, 2016 at 4:40 PM, Glen Chiacchieri wrote:
Not sure why that would happen (can everyone else see them?) but I copied the emails here for documentation purposes: http://glench.com/ProgrammingTheWorld/On Fri, Apr 22, 2016 at 3:11 PM, Yoshiki Ohshima wrote:Regrettably, all .png files presumably in the mail are missing. But
thanks for sending this.
On Fri, Apr 22, 2016 at 12:59 PM, Glen Chiacchieri
--wrote:
> Adding our remote collaborators.
>
> On Tue, Apr 19, 2016 at 8:47 PM, Glen Chiacchieri
> wrote:
>>>
>>> I want this next to a diagram of a sample install, a blueprint overview
>>> with people and parts in it. That would help me ground these things.
>>
>>
>> These diagrams would be next to the physical install of Laser Socks, which
>> would be augmented with inspectors if needed. Maybe if Laser Socks was in
>> storage, it would be good to have a "virtual" version saved in diagram form
>> when looking through old rulebooks.
>>
>> The reason I wanted to abstract the rules is because combining the
>> currently functioning system with the abstract rules felt like too much for
>> one thing, like I mentioned in the email before this.
>>
>>
>> On Tue, Apr 19, 2016 at 8:36 PM, Chaim Gingold wrote:
>>>
>>> I want this next to a diagram of a sample install, a blueprint overview
>>> with people and parts in it. That would help me ground these things.
>>>
>>> On Apr 19, 2016, at 6:41 PM, Glen Chiacchieri
>>> wrote:
>>>
>>> I tried a different thing this time:
>>>
>>> <stage_design_april_19_2016.png>
>>>
>>> Something I found lacking in the schematic designs was a coherent
>>> sequence to follow if you weren't already familiar with how the program
>>> worked. I thought about the way I explain Laser Socks to people in
>>> conversation and tried to illustrate something along those lines. In the
>>> working system, maybe there is some kind of template and to program these
>>> objects you fill out the template? I don't know.
>>>
>>> Anyway, the first section is meant to set the stage, introduce the
>>> "characters" this program will be describing.
>>>
>>> The second section shows the state, attempting to be explicit about which
>>> objects remember which values. This might be analogous to
>>> "characterization".
>>>
>>> The third section is meant to show how all the objects are related, like
>>> in the schematic diagrams. I wanted one interaction per "panel" so they
>>> wouldn't be all jumbled up together. This is something like the "plot", how
>>> all the characters interact. Each panel would probably highlight whenever
>>> that interaction was taking place in the working system. I'm not totally
>>> happy with how this section turned out and I'm not sure why.
>>>
>>> I was also experimenting with color and icons because the thing I made
>>> yesterday was so drab.
>>>
>>> On Wed, Apr 13, 2016 at 12:42 PM, Bret Victor
>>> wrote:
>>>>
>>>> What's the closest Arena object? (distance query)
>>>> Is that object Lasered? (observable query)
>>>> What is the Arena's internal state? (querying internal state, maybe this
>>>> one isn't supposed to be possible?)
>>>> What messages am I sending/receiving? (querying messages)
>>>>
>>>>
>>>> I was imagining that an object's queries could involve its own internals
>>>> (variables, messages) and other objects' observables.
>>>>
>>>> The distance query is a query about observables (my location and other
>>>> objects' locations).
>>>>
>>>> I imagined that an object could observe (and perhaps manipulate) another
>>>> object's internals via the meta-system. That is, anyone can dig around
>>>> inside anyone else, but you have to climb over a gate to do so. This is the
>>>> facility that inspectors etc would use. Self invented mirrors for this
>>>> purpose -- basically, you can't observe the Arena's internal state directly,
>>>> but you can create a "mirror" of the Arena which reflects that internal
>>>> state as observables. Smalltalk and CLOS have metaobject protocols which I
>>>> don't know much about, but similarly let you talk about an object instead of
>>>> talking to it.
>>>>
>>>> (The complexity of prior work here is daunting, but I think we can come
>>>> up with something very simple and adequate, while still maintaining a clear
>>>> boundary between system and meta-system.)
>>>>
>>>>
>>>>
>>>> On Apr 12, 2016, at 5:20 PM, Glen Chiacchieri
>>>> wrote:
>>>>
>>>> Last week at lunch I showed some early sketches I made to understand
>>>> Bret's proposed object model. I was/am hoping that it's also possible to
>>>> work backward from representations of how Laser Socks and MoleTank work
>>>> toward how our object model should work. The first sketch was this quick one
>>>> of MoleTank:
>>>>
>>>> <moletank_first_sketch.png>
>>>>
>>>> It's just a rough outline of how the objects communicate, what states
>>>> they have, and action/contexts cause state to be modified.
>>>>
>>>> Laser Socks Diagram
>>>>
>>>> This week, I've been toying around with some other representations.
>>>>
>>>> I started sketching out a Laser Socks representation much larger on a
>>>> whiteboard and came up with this:
>>>>
>>>> <laser_socks_whiteboard.jpg>
>>>>
>>>> You'll notice I added a box for "systems-level" stuff near the top (in
>>>> case someone wanted to dig into how that stuff worked), and also showed the
>>>> particular instances of the objects with whatever their particular state
>>>> was.
>>>>
>>>> Making this diagram, I felt a tension between the abstract schematic of
>>>> how these objects worked and the status of objects in the world. For
>>>> example, it felt kind of messy to show two versions of each object, each
>>>> annotated with its current state. I also didn't like how flat the diagram
>>>> was and that there was a separation between the actual physical Progress Bar
>>>> object and its image in the diagram.
>>>>
>>>> I hypothesized that maybe the abstract communications of the objects
>>>> should be separate from their particular running instances, which lead me to
>>>> my next sketches.
>>>>
>>>> Physical Inspectorators
>>>>
>>>> Here, I've physically annotated Laser Socks objects with two
>>>> Inspectorators:
>>>>
>>>> <laser_socks_physically_annotated.jpg>
>>>>
>>>> Each Inspectorator shows the current and recent states of the closest
>>>> object. Here's what it might show if it was placed next to the nearest
>>>> Progress Bar:
>>>>
>>>> <progress_bar_inspectorator.jpg>
>>>> (excuse the sloppiness of both presentation and concepts)
>>>>
>>>> From top to bottom:
>>>>
>>>> An arrow points toward the currently inspected object and the title says
>>>> which class of object it is.
>>>> The object's current state variables are shown, along with the recent
>>>> history of the state.
>>>> The object's received and sent messages are shown, along with a recent
>>>> history of those messages.
>>>> The object's current illumination is shown, along with a recent history
>>>> of its illuminations.
>>>> Maybe the object's abstract diagram is shown?
>>>> Maybe a history of an object's observables is shown? (I was thinking
>>>> that "is lasered" is kind of a red herring in the case of a Progress Bar,
>>>> because lasering it shouldn't do anything, but it can still be lasered by
>>>> accident)
>>>>
>>>> Here's what it might look like for an Arena:
>>>>
>>>> <arena_inspectorator.jpg>
>>>> (Much less information because it's a dumber object.)
>>>>
>>>>
>>>> Something I like about these inspectors is that they're physically in
>>>> the world next to other physical objects. If you want to know about a
>>>> particular object's current behavior, you put an inspector next to it. You
>>>> can have a lot of inspectors laid out at once and see all the objects
>>>> operating in real-time. With better visual design, you could see more
>>>> important information from a distance and fine detail by moving closer (like
>>>> in many of Bret's posters).
>>>>
>>>> While thinking about these inspectors, I started to noticing some subtle
>>>> things about the object model. For example, there are several types of
>>>> queries you can do that feel like they overlap a bit:
>>>>
>>>> What's the closest Arena object? (distance query)
>>>> Is that object Lasered? (observable query)
>>>> What is the Arena's internal state? (querying internal state, maybe this
>>>> one isn't supposed to be possible?)
>>>> What messages am I sending/receiving? (querying messages)
>>>>
>>>> I'm not sure if the overlap I'm detecting matters or anything, but it
>>>> does have some implications for how you see the recent history of this
>>>> stuff. (For example, you should be able to inspect a progress bar and see
>>>> when its nearest Arena was lasered, but right now that doesn't seem part of
>>>> the Progress Bar's history.)
>>>>
>>>> So anyway, physical inspectors may be a way to see the history and
>>>> functioning of physical objects, but how do you program them?
>>>>
>>>> Physical Schematic
>>>>
>>>> I liked the idea of having an abstract class diagram, but it seemed like
>>>> it should be more physical, so I made a tiny Laser Socks:
>>>>
>>>> <tiny_laser_socks.jpg>
>>>>
>>>> What is this, a Laser Socks for ants?!
>>>>
>>>> It may be hard to see in this picture, but there is a physical start
>>>> button, progress bar, and black plastic arena on a big piece of paper. I
>>>> imagined this as a live schematic. When you laser the tiny socks on the tiny
>>>> arena, the tiny progress bar would fill up. The objects' state and
>>>> illumination code are also on this diagram.
>>>>
>>>> I tried to draw how the objects were communicating (using red waves to
>>>> show that Progress Bar was detecting their lasered observable). Maybe some
>>>> or all of this is displayed dynamically, so when you move the physical
>>>> objects around the connections change. Or maybe the red waves themselves
>>>> should be physical objects? I dunno. I also tried to annotate the connection
>>>> between objects with the code that runs as a result of that path being
>>>> activated (like score being set to 0 when the start button is pressed).
>>>>
>>>> This brings up some weird philosophical questions about the difference
>>>> between a model and reality (and determining from a model what is inside or
>>>> outside a system's concern), but as always in these matters we can look to
>>>> the wisdom of The Simpsons for the answer to these questions. (this is
>>>> apparently one of Dave's favorite Simpsons jokes)
>>>>
>>>>
>>>
>>
>
-- Yoshiki