It's a beautiful vision.
I love the tactility and physicality of the notecards for "seeing inside", as opposed to the AR approach of ethereal visualizations floating in midair.
Moving the cards around and interacting with a pencil is very suggestive. It makes me think about what a pencil-based "annotation notation" would be for specifying transforms or measurements. E.g., you want to measure the time between peaks, you would draw something like:
and you could then hook up the resulting signal (of that measurement over time) to something else. But you'd want the notation to be general and unambiguous, a kind of graphical programming language for specifying conditions in a signal. It would be interesting in that it would be intended for "marking up" a shown signal -- more like a copy editor's notation than most math notations.
"Binding" a card to a component is interesting. I wonder about a more explicit action for it (since simply touching the card and the component could be ambiguous, whether it's intended as input or output or just accidental -- maybe the card has various "ports" along the edge, like Quartz Composer, which you touch to the component, etc)
And I wonder about how to (visually?) indicate on the card which component it's bound to. And in general, how to show the network of bindings that makes up the structure of the circuit that you've built.
Toby has a line in a talk about the spatialness of the nodes in max/msp etc being unrelated to the spatialness of the what the program is actually drawing. It's interesting to think about how the spatialness and physicality of the cards could be used to reflect the spatialness and physicality of heart pulse monitoring (or whatever the problem is). I'm not sure exactly what I mean. At the least, I think there's something interesting in how this thing
replaces the components with visualizations of the component's behavior, so the layout of the "cards" directly reflects the structure of the circuit. You may not necessarily want such a tight constraint, but it's interesting to think about how *where the cards are placed in relation to each other* could have some sort of semantic meaning.
I really like the idea of having a library or "toolbox" of transforms on hand, like your stack here, that you can peruse and literally grab from as needed:
I wonder what that toolbox would look like, how it would be organized, etc. A couple things it reminds me of is the mathematical modeling TOC:
and the library in Flux Garden:
On Jul 30, 2015, at 11:24 AM, Paula wrote:
Toby and I made a video prototype of what it might look like to tinker/build hardware using dynamic media.
In this scenario, we're creating a heart pulse monitor.
https://drive.google.com/file/d/0B6cFA4xbIfMTcHVILVpuODhxelU/view
What do you think?
---
brief background:
In our discussions, we were wondering what different processes/interactions would look like without the keyboard/screen interface. We brainstormed a few different scenarios (documented) ... and went with the pulse sensor scenario, making a draft storyboard first before doing the stop motion animation. Excerpts of storyboard:
<DSC08819.JPG>
<DSC08829.JPG>