A few themes came up across the projects as well.
Glen, Toby, Bret, Paula all needed to be able to track the arrangement of tokens/foamcores/coasters on a table/placemat/whiteboard. Bret suggested making a generic PCB-in-a-foamcore that can be tracked on a "smart table", and then producing ~1000 of these. This way we don't each have to reinvent this tracking capability.
"Active pictures" came up as well. For example, Toby wants to be able to "collapse" a set of video processing nodes into a single node. Glen suggested being able to take the whiteboard scale representation and squash it into a picture on a piece of paper. But in the squashed version, you still want this smaller representation to be dynamic. Likewise, Paula's calendar wants to be able to print pages out that still have the dynamic behavior (e.g. get more details of an event) of the original whiteboard calendar.
Many of the projects seem to suggest a feature of "dynamic binding". For example, we brought up different ways of creating sliders and knobs, for example the distance between two tokens, the placement of a token on a drawn line, or the distance between your hands (Paula's air-tape-measure). You would want to be able to hook these up to other objects in the system, for example to use the slider to scrub through a video. Also, for "mashups", e.g. taking one of Glen's video coasters and hooking the sound up to Bret's DSP graph, one would want to be able to take an arbitrary output and plug it in to an arbitrary input. (This dynamic binding is exactly my vision of "physical apparatus".)
Are there any other themes that came up?
On Mon, Jan 25, 2016 at 6:31 PM, Bret Victor wrote:
Notes from today's "imagination jam". The goal was to bring up various projects we'd like to build in an "in-the-world" authoring platform, so we can think about common building blocks and technological needs.Participants: Toby, Matthias, Glen, Paula, Joshua, BretToby described a system where the person on the left is moving objects under a camera, and the person on the right is programming a processing algorithm for the camera feed, in the form of foamcore pieces on a surface, with a library of operations laserable on a large board to the right.Toby also described building a PCR machine out of a fan, temperature sensor, etc, where the input/output signals for those components can be inspected on pieces of paper and pinned to a wall. The behavior of the machine is programmed by specifying a state machine using physical tokens on the desk.Matthias described a "visceral earth primer", where a person could walk into a geology simulation and experience the rain, and what it's like to be rain, and in generally feel and perform the forces shaping the landscape.Glen is stuck calling them "video coasters". He has a collection of objects which represent video clips, and he'd like to freely arrange them on a desk, trigger clips by tapping them with his finger, and smoothly transition into more precise editing, such as specifying frame-accurate in/out points. He doesn't want to build a video editor so much as improvise a video editor as he edits this particular video.Paula described hand-drawing a calendar that could be made dynamic. One could take away a "page" from the calendar that would remain dynamically connected to it. She also described hand-drawing UI elements, such as scrubbers and meters.Paula also described a dynamic workbench which could, for example, display the sources or supply chain of the materials placed on it. She described a hybrid physical-digital CAD workflow, involving scanning physical objects and using them in virtual simulations.Paula also talked about a city simulation with physical toys whose virtual properties could be set through physical interaction, such as a building whose occupancy can be set with physical tokens. She considered this midway between "virtual authoring" and "robot authoring".Joshua described a museum-like exhibit around interacting with Maxwell's thermodynamic surface. He also described a "statistical mechanics sandbox", where one could experiment with macroscopic simulations of microscopic phenomena. He also described a "thermodynamic mosh pit", where one could feel with their body what it's like to be a particle as a container changes volume, pressure, etc.Bret described a technical paper in the form of a museum-like exhibit, where one could experiment with simulations of a signal processing system that an author has presented to make a point.
