On Mar 17, 2015, at 12:33 PM, Bret Victor wrote:

[plan / design spec]

This email describes a new layer to the room system that I'll be trying to implement over the next few weeks.

The immediate impetus was that I was making variations of the page-spread-above-the-bookshelf, and I wanted to be able to switch between the variations by drawing boxes on the whiteboard and pointing to them.  

<IMG_7247.JPG>

Essentially, I wanted real-world HyperCard-like functionality, to be able to easily turn physical objects into "buttons" and define their behavior.

Other immediate motivations included wanting to enable people to add labels to the research gallery, wanting to make the Seeing Spaces poster and Representation Gallery dynamic, etc.  The long-term general motivation was wanting to represent code and data as situated physical objects and get rid of thinking in virtual filesystems and databases.

                                                                                                            

The basic idea

Physical objects can be assigned virtual attachments, which allow the objects to hold data and exhibit behavior.  An object can be any old thing lying around the space.

Instead of thinking of some data as "inside an icon on my desktop", you think of it as "inside this physical object in the space".  Instead of thinking of some code as "running on this computer", you think of it as "running inside this physical object".

<p123.png>

A single web interface at room.local/o allows you to "see inside" any object and edit its data or behavior.

<asd.png>

There is no "filesystem" or "database" or "server".  There are only physical objects you can point to, which serve as the organizing units for the system.

Objects

An object is a physical thing in the world.  For now, our objects will mostly be affixed to surfaces and defined by their location -- e.g., labels, panels of a poster, marks on a whiteboard -- although the basic idea should generalize to movable objects once we can track them.

<pp.png>

Blessing an object

In order for an object participate in the system (i.e., be magical), the object must be blessed.  

To bless an object, go to room.local/o in a web browser.   (There will be several iPad Minis lying around the space whose only function is to run this web page.)

<a.png>

Tap the "Bless an object" button, and laser-point to each corner of the object in turn, tapping the "Define corner" button after each one.  An object can be a polygon with any many corners as you like.

<b.png>

Now that you've indicated the shape of the object, you can go on to define its identity and add attachments.

Opening an object

To view and edit an (already-blessed) object, also go to room.local/o in a web browser, and simply laser-point to the object.  All currently-pointed-at objects are listed; just tap on it to edit it.

<c.png>

Identity

Every object has an identity, which consists of a name and a family.  These are arbitrary strings which can be used to refer to the object.

You can choose a selection behavior -- whether the object is selected when the laser enters it, or when the laser goes up inside it, etc.  Only one object in a family is selected at a time.  This allows you to make things like toggle buttons and radio buttons.  

<badf.png>

Attachments

Some objects, such as labels, might be "dumb", and not need anything more than an identity.  (Other objects will be watching for their selection.)

For not-so-dumb objects, you add attachments.  To add an attachment, view the object and tap "Add Attachment", then tap the type of attachment you want to add.

<f.png>

                                                                                                            

DATA ATTACHMENTS

                                                                                                            

Links

A web link or an email link.

To follow the link, go to room.local/link in a web browser and point at the object.  (There will be iPads and laptops around the space whose home page is room.local/link, and you are encouraged to set it as your home page as well.)

<c2.png>

Values

Immutable key-value data that you'd like to attach to the object.  The value can be any JSON value.

These values are available both to the object's own processes, and to other objects' processes.

Files

Files that you'd like to store in the object, such as images, code libraries, etc.  These files are also available to processes.

                                                                                                            

PROCESS ATTACHMENTS

                                                                                                            

Triggers

If this, then that.  When a condition is met, a program will run.

Some conditions have to do with the laser -- something can happen on enter, leave, up, etc.  

<fr.png>

The condition can also be an expression which refers to the state of the object or other objects.

<sin.png>

As specified here, trigger actions are pretty imperative, and probably usable only for simple things.  It'll be interesting to think about how this could eventually expand into a more declarative, FRP-like system.

Illumination

For an object to have a dynamic appearance, it must be illuminated. 

An illumination is a program that draws on the surface of an object.  (Illuminations were formerly known as "windows".  They are implemented as web pages that are projection-mapped onto objects.)

A object can illuminate itself:

<illum.png>

It can also illuminate other objects, or a region of some surface.  (Below, the active object is a box drawn on the whiteboard, but it's illuminating two regions of the bookshelf wall.)

<strip.png>

Attach an illumination for whatever you want to draw on.  An object's illuminations are active only when the object is selected.

<as2.png>

The code for the illumination, like all code, is edited directly in the object viewer at room.local/o.  You can view, edit, and debug the code for any illumination simply by pointing to the illuminated surface and tapping some button in the viewer.

Illuminations are currently HTML or Javascript files, but the intent is that eventually they can be Apparatus drawings or programs in other homegrown programming systems.

Daemons

A daemon is a program that runs "in the background".  An object's daemons are active only when the object is selected.

<pre.png>

For now, daemons will be Javascript files that run in a node.js instance.  (Unlike illuminations, they are not trapped inside a web browser, so there is more freedom in what they can do.)

The code, again, is edited directly in the object viewer at room.local/o.

                                                                                                            

STATE ATTACHMENTS

                                                                                                            

Variables and Collections

If an object wants to remember anything, it needs state.

A variable is a mutable value attached to the object.  The value can be any JSON value.

A collection is a mutable "database table".

Variables and collections can be read or written by the object's processes, or other objects' processes.  Objects can trigger on changes.

Like all attachments, the state can be viewed and edited in the object viewer.

<mgr.png>

                                                                                                            

Room implications

ProcessHost is gone, "apps" are gone, the "room database" is gone (at least, conceptually).  The Research Gallery is trivial.

There will be a poster board labeled "services", with areas marked as "display manager", "mail manager", etc.  Each area will have a daemon attached, and will probably illuminated with diagnostics.  (At the very least, activity timelines that indicate when triggers fired.)

BigBoard will be a physical big board.  Major database tables, such as the ones currently shown in BigBoard (surfaces, cameras, etc), will each be an object, a chunk of area on the big board, with a collection attached, and will be illuminated with the contents of the collection.  (You can also get a live view of the collection from anywhere simply by opening the object in the viewer.)

Hopes

I am hoping that this architecture will make simple HyperCard-like toys easy (draw a button and add a trigger to make sound).  No need to muck around with files and databases, just point and bless.  Or copy someone else's object and modify it.  I am hoping that we make a lot of these, and it gets us thinking "out in the space".

I am also hoping that it makes complex systems (such as the room system itself) more understandable, by giving every component a physical presence that can be pointed at, looked inside, and illuminated with a representation of its activity.

I am hoping that taking the big step of removing the filesystem and filesystem-based text editors, and editing code directly "on the object", will give us the flexibility to move away from programs-as-text limitations.  Initially, perhaps, with simple things like code containing "tokens" that refer to other objects, inserting the token by directly pointing at the object instead of naming it:

<v2.png>

and eventually moving toward more radical ways of specifying programs.