Date: Tue, 18 Aug 2015 18:58:36 -0700
From: Bret Victor
Subject: sigma-delta
[notes on an upcoming project]

I'll be trying to redesign the paper Why 1-Bit Sigma-Delta Conversion is Unsuitable for High-Quality Applications in the form of an interactive exhibit of some sort.  Sort of in the same spirit as the Watts and Strogatz redesign or the Latour poster, etc., but spatial.

I find scientific papers to be more interesting targets for redesign than pedagogical material such as textbooks or "explainer" articles, because the paper author (a) has some new knowledge or idea they've personally come up with (b) that they want to convey to readers who can make immediate use of it.  So both the writer and the reader are directly invested in the knowledge and the communication and understanding of it.  As opposed to pedagogical material, where neither teacher nor student have any immediate reason to care, making it more about entertainment than useable understanding.

This paper is a particularly interesting target because it's an argument.  The basic gist is, "Sony is proposing a new standard format for encoding audio on CDs.  This is a bad format, because it makes audio distortion impossible to avoid.  Here is a technical proof, and examples and analyses of the different types of distortions that are induced.  Here are some alternative formats which can be made distortion-free, and analyses showing how they're better.  Audio engineers, please don't adopt Sony's proposal."

The fact that it's an argument instead of merely a report gives it an urgency which makes both the writer and reader even more invested -- the writer wants to convince, and the reader (the audio engineer deciding whether to adopt this format) wants to avoid making a mistake.  I think it will be an interesting task to design a specific, dense technical argument in the form of a space.

The paper is also good for a design prototype because the examples can be made into interactive simulations without too much difficulty, and while I'm somewhat familiar with the subject matter, I've never "seen and played" with sigma-deltas in the way that a good design should allow.  Also sigma-deltas are sort of mysterious and magical creatures.

---

The redesign probably won't follow the paper directly.  The intention is that the paper's argument will "frame" the design -- you've downloaded this exhibit in order to be convinced about something -- but the actual material in the exhibit may go quite a bit deeper than what's covered in the paper.

The idea is that the spatial layout will allow you to read/play at just the depth you care about for any given concept.  If you are unfamiliar with some concept, you go down as far as you need into background material.  If you already know some area well, you skip over the details and barely see them.  Different readers will take different "paths" depending on what they already know about and care about.

---

Here are some of the design challenges that I'm thinking about:


Familiar (not too worrisome)

- How to visually organize elements of the argument.  ("spread out")  Visual hierarchy, skimming and browsing, getting the gist from a distance, etc.  A number of the posters were good practice here.

- How to visually represent concepts.  ("show and tell")  Like the Strogatz paper, there are a lot of nouns that can be depicted instead of described.  This is always fun.

- How to enable exploration of the examples.  (In the original sense of explorable explanations -- giving readers an interactive model that they can use to investigate or refute the author's claims.)  The techniques from multitrack signal processing and other interactive signal-processing representations will carry over here.  Ideally, the reader should have the freedom to explore any question that the simulation environment can calculate -- like a spatial Mathematica notebook.


Unfamiliar (I have no idea)

- How to spatially organize elements of the argument.  (it's an exhibit, not a poster)  Breaking out from a flat, bounded poster and making use "all the space" is unfamiliar ground.  

- How to spatially represent and tangibly represent concepts.  This is the most important one for me -- the challenge of designing new representations, instead of merely reorganizing old representations.  Non-flat representations, instead of flat representations hung in a non-flat space.

- How to enable rich tactile and kinesthetic interaction. Not just staring at things or sliding fingers on surfaces, but actually thinking with the hands and the body.

- How to offer supporting material ("going deeper into background material") while avoiding peek-a-boo (multiplexed space) to the extent possible.

- How to enable bystander transfer.  Imagine that the exhibit is downloaded by an experienced audio engineer who already knows a lot about sigma deltas.  Ideally, we'd like a nearby colleague, who perhaps knows nothing about signal processing, to be able to pick up a sense for it simply by casually being around the first engineer as they interact with the representations.  This is the effect Hutchins describes.


These "unfamiliar" challenges are quite daunting, and I don't know how successful I'll be with any with them, or even the redesign itself.