Date: Tue, 24 Feb 2015 14:53:30 -0800
From: Bret Victor
Subject: Spectorgraph / Dynamic History [11/16/14]
Work-in-progress (stalled) around a kind of "Dynamic History", or dynamic representation of historical knowledge.  What Wikipedia should be.

This is related to "representing the shape of the whole" for an infinite state space.  In this case, the "space" is the context of what happened (and the various points-of-view on what happened).

The "spectorgraph" came out of wanting to understand the entire context around Warren Spector's work -- what he did, who he worked with, what they went on to do, the paths of influence.  I wanted to learn about this by moving around in a structured space -- the "shape of the whole" -- instead of accreting bits of context pinhole and piecemeal by jumping around "narrative-structured" wikipedia articles.


Turned into thinking about a set of representations for understanding the matrix of:

                    what was it like to be (experience from the inside)      what was it like to have been with (context from the outside)
  a person
  a project
  a place
  a time
  a thing


"A non-narrative view of history"
perspective: Every entity has its own history and way of seeing.
context: Every entity is influenced by and influenced other entities.
motion: In the dynamic medium, reading is movement.


I don't have a write-up or really much to show -- below are my notes and sketches as I thought about this.


------------------------------------------------------------------------------------------------------------------------------------

11/6/2014

Working on a sort of "conceptual cartography" -- stable maps of abstract spaces.  
A few "types" are:
- game/algorithm state space (tic tac toe, etc)
- mathematical situation
- historical situation
Exploring the latter by trying to make an environment for understanding the context of Warren Spector.

Also, peripheral context while reading a book.  RMO had interesting idea of using overlapping pages of a book as "glanceable digest" that could be interacted with.



warren-spector.txt:

(see connections -- e.g., the connections between Warren Spector, Ken Levine, Marc LeBlanc, John Romero, System Shock, Bioshock, Deus Ex, etc etc) 

string theory

view by
    what (project)
    who (biography)
    when (timeline)
    where (pseudo-map)

read narrative (wikipedia) embedded in context


(why?) techniques, ideas, novelties, and how they moved and influenced others through the structure

what (game):
    where
    when
    where
    whats (sequels, inspirations, etc)

who (person):
    whens (timeline)
    whats (games)
    who (this is the interesting part, personal connections)

where (company)

when (as a timeline)
    timeline of whats

interesting is the trajectory of influence
(like the state trajectory in tic-tac-toe or in a dynamical system)
    
narrative is the connective tissue

history is modeling a system, causal influences

http://www.eurogamer.net/articles/2014-04-17-the-true-story-of-bioshock

-------------------------------------

tension between

 - author provides map
    - reading Jared Diamond book in environment which serves as context / structure
 - reader passively creates map as byproduct
    - by exploring
    - clearing out "fog of war" 
 - reader actively designs map
    - adjusts the map itself, not just the position of the turtle in the space
    
connections
 - synthesized in reader's mind
 - seen explicitly in medium
    - statically (lines, arrangement, topology)
    - dynamically (follows attention or position)
    
-------

templates

    director of x
    writer of x
    people involved in x
    
relative motion
    (original contextual dashboard)
    fluidly move toward "director"
        - see all things for whom Warren Spector is the director
        - see all directors for projects that Warren Spector was involved in

more than just fact queries -- "so-and-so worked with so-and-so on such-and-such project at such-and-such time"
  - we are trying to understand vectors and patterns of *influence*


"locus"

how to shape the space?

"thought vectors in concept space"
  - but represent the space itself (and the vectors!), not just the current point in the space!
  
  
contextual dashboard
  ************************
  - expand as you go closer (more detailed information
  - muscle memory, move left for images [can this define a space?]
  
data visualization dimensions
  - plot games by revenue (or whatever)
  
dual-column, category and item


periphery
 - reading about warren spector, and it mentions Ken Levine.
   - if you already know about ken, lots of associations "come to mind" 
     - companies, games, GDC talks, philosophies, awards, friends, etc.
     - these associations are *what it means* to "know about" ken
   - if you don't know about ken, nothing is triggered, and the name is meaningless
     - or a placeholder that will have associations connected to it later
   - we want to be read about warren, and see the reference to ken as that "space of associations"
     - but as "peripheral vision"
     - we're not "going off" to read about ken instead
     - we're importing a ready-made representation of Levine-space
       - and most importantly, seeing where it fits into Spector-space




------------------------------------------------------------------------------------------------------------------------------------

11/12/2014

Continuing to sketch Spectorgraph.  Thinking of it as a new kind of Wikipedia.

----

Person's point of view
(Not just where and when and what they did, but what they think about things and said/wrote about things)
When two people are selected, follow/compare not just their works/actions, but their opinions and  the "view of events and causes from their perspective".  Compare two people's viewpoints/testimony/claims in this way!
(Inspired by Alan's "alan-oriented" view of history.)

------

See people in action (eg giving a talk) instead of static posed photos 

For each thing you could see -- 
 - what "moves" would you want to make
 - what "context" would you need to see to understand or feel grounded?
For now, don't worry about shape/rendering so much as topology

----

vertical timeline
strings for people
horizontal regions for studios
  horizontal regions for projects
see strings go in and out of projects
color strings for current project
mark project as bar along timeline
mark project release as dot on timeline

-----

Different quasimodes (modifier keys or whatever) for different skimming.
Skim as vertical time, show cross-section
Skim for pointing at people
Pointing at projects
Pointing at organizations
(Each gives different supplemental information and allows different moves in and and around)

Keys/actions/gestures for switching to pointed thing, or comparing to current selection, or seeing all the names, or seeing names as faces.  Don't try to overload click/touch.  This is a pro tool.

-----

select person
 . then select project
 . see person's perspective on project (?)
 . unselect person to see neutral POV on project?

Like a timeline map, but --
 . moving in the space, not viewing from above the space (ladder of abstraction -- street-level view with helicopter context)
 . history (events, ideas, causality) relative to a particular perspective (and moving between perspectives), not a "god's view" of events ("The Truth" / NPOV)

----






------------------------------------------------------------------------------------------------------------------------------------

11/13/2014

Redrew mockup in photoshop
Starting to think about the "player-character" object that runs along the track and jumps from track to track
Thinking about movement, and understanding by moving your character (locus-of-attention proxy) around the space, instead of moving your eyes around a static space...

---

First-person "bead" (circle with spector's face that travels along the line)
Jump the bead up and down to other people to switch perspectives.
Select two people to have two parallel beads, and compare their experiences

Fluidity 

Reading in the dynamic medium is always *doing something*.  Not just "floating above" the material, but down in it, moving around, with peripheral context.  Read the timeline by traveling along it, and seeing what pops up along the way.

Want to evoke -- 
 . seeing history through a particular person's eyes
 . walking into a room where a project is going on
 . being alive at a particular time, aware of all the things that are "going on"

See tiny beads for other people traveling along and moving.  
See mosaic of "period" documents and images in background while moving through? Like flipping though whole earth catalog

Beads flying through like running in (my imaginary) big giant circles' "yeah!" video.  Slaloming down tracks, jumping from one perspective to another.  Feel the motion, feel fun.  Canabalt.  

Today, studying is associated with still, motionless contemplation, because the media is static.  In a dynamic medium, we can have understanding through movement.  Canabalt.

Narrative in response to movement (?)  narrative connecting states, describing the flow from one state to another, giving the *why* for the movement (not the who when what where facts).  

Bring up parallel tracks for world events or industry events in whatever seems related (or follow an unrelated person in parallel -- warren Spector and Steve jobs)

Think about other application areas
Science, scientific research (papers, experiments, people?)
Doing research for ... (Don't want to say "writing a book", because this should be the "readers" experience for getting understanding, not for an author to pre-digest (although authors can find "trails" (memes) or "levels" for readers to explore))
Tukey collected works
Priestley diagram of scientists, etc

Look at "structure in human experience" poster..  Each thing on parallel tracks (eg, enactive iconic symbolic)... Following things, activities what? though time (?)

Adjacent people/subjects via google-like pagerank-like discovery. Things that appear together in the Great Corpus of The Entire Web get placed ("suggested") nearby as you are moving through.  Jump your bead to related subject and drive on from there.
Leave "checkpoints" like tabs that you can come back to (or branch off).  Leaves phantom bead behind.  Switch quickly between checkpoint positions. (Animated eased pan from one to the other, like "The Swapper")

3D space -- slight tilt around y axis while moving forward and back.
Rotate around y axis to perpendicular plane showing supplementary documents to read, etc 


----







------------------------------------------------------------------------------------------------------------------------------------

11/16/2014

(on drive down to Kavli conference in Irvine)

see from the perspective of:
   a person
       be in that person's head, experience their life, understand what they understood and when
           but with a higher-level view than their own experience
       two people
           experience two lives in parallel
           compare
           understand their interaction
           understand causality, currents of influence, propagation of memes
   a project
       what was it like in the room
       what was it like to experience the project, and know what they knew in the room
       digital detritus --
           a lot of the work on a project exists as digital documents, records, communication
           assume all the data exists -- skate to where the puck will be
       want to show structure, not just a big pile of documents
           the whole point of researching is for the reader to build a structure in their head
               connections, associations, causality
               that structure should be reflected in the external representation
           structure can come from
               primary material
               reader
                   RMO-style reading is a creative act
                   connections that the reader makes in their head must be reflected externally
                   memex -- seeing and recalling associative trails
               other readers
                   connections that other people have found can be surfaced to the reader
                   be careful of prematurely imposing other people's structure on a new reader
                       balance between find-your-own-way and use-other-people's-ideas-as-scaffolding
               algorithm
   a place
          what is it like to be a place?
   a time
          what is it like to be a time?

(sort of like Ian Bogost's "what is like to be a thing"?)

inward and outward perspectives
   for a person
       inward: how did the person think, feel, do
       outward: what was the person's environment, what influenced the person and who did they influence
   for a project
       inward: what was it like to be in the room, what was the experience of the people involved at the time
       outward: what projects influenced this project, what project were influenced by it
   for a place
       inward: what happened there, who went through
       outward: what led to people coming there, where did they go afterwards and what did they do
   for a time
       inward: what were people thinking and doing at the time, what was going on
       outward: what events at other times caused the events at this time, and vice-versa

thinking about diverse "structures" found on a single bed of "facts"
   facts:  what happened when, who was where when, primary-source documents, etc
       can live in a consensus platform like wikipedia
       facts are the "space"
   structure: why did this happen, what caused this, why did he do that, etc
       different people "draw" different structures (theories) over the facts, people compare them
       structures are "drawn in the space"
       take a look at Google Earth as an example of many people's structures superimposed on a common space

Nowadays, it seems like we use more and more spatial metaphors (infographics, etc) while less and less using space itself (people sit at a desk immobile all day etc).  
When trying to understand a human situation such as a project, real physical space has become uninteresting/irrelevant as a dimension to visualize, because people do all work and communication without leaving their desks.  Visualizing the physical location of the workers gives no useful information about the workings of the project.
I wonder if spatial metaphors will start to fail as people lose acquaintance with actually navigating real space.

Avatar as locus of attention in dynamic information design (to replace gaze in static design)
   static graphics require people to navigate by eye (and multiplexed in time is seen as bad, in space is seen as good)
   dynamic graphics could, in theory, be better navigated via the "player-character", if it is fast and tight enough
       with the actual gaze taking in peripheral context


------------------------------------------------------------------------------------------------------------------------------------

11/24/2014

diorama is static -- model train is dynamic
should feel like driving model train through landscape (and jumping from one track to another)

- perspective
    what is like to be a
        person
        project
        place
        time
    looking
        inward
        outward
        
- motion
    "down in it", looking around, street-level experience
    fluidity, momentum, body-feeling
    avatar as locus of attention
    
- context
    map, higher-level view
    
- hyperlinking
    jumping from one [person/project/place/time]'s perspective to another
    shifting perspective

- diverse structures of a bed of facts
    google earth layers
    has to do more with how the data gets there (or how the structure get there, sense-making)
        contributions, editing, sense-making, note-taking, 

- diorama displays
    model trains, etc


------------------------------------------------------------------------------------------------------------------------------------

11/25/2014

The stuff below is drifting pretty far away from the original "conceptual cartography" concept (maybe).  I should try doing mockups for each of the "representations" below.  (Those were (obviously) off the top of my head, and I might think of better ones.)


person-perspective
------------------
[a person's experience of the world]

goals
- experience what he experienced (On Jan 1, 1861, Maxwell read about M. W. Weber's electromagnetic theory.)
- do what he did (On Sept 7, 1864, Maxwell calculated the velocity of an electromagnetic wave.)
- say what he said (On Dec 8, 1864, Maxwell published "A Dynamical Theory of the Electromagnetic Field".)
- see as he saw (In 1864, Maxwell thought about light as an electromagnetic wave.)
- understand what he understood (In 1864, Maxwell understood the need for the displacement current, but not what it meant.)
- feel what he felt (In 1864, Maxwell was afraid that nobody would understand his theory.)
- see history, current events, and causality as he did (From Maxwell's perspective, Faraday invented the concept of induction.)

representation
- montage of documents
- mosaic of documents
- documents... but structured, or annotated, or "matching highlights" or something
  - see all documents/excerpts that contain a particular phrase (or idea)
- causality map, what influenced what according to him
- view of history as understood by him 
  - fog of war for unknown things -- feel what is not yet known
  - distortion/overlay for non-consensus things
- first-person video game where you play a virtual Maxwell
  - literally do what he did, say what he said, etc
- comic strip
- synthesized documentary
- synthesized diary / notebook
- hear his voice
- understand-o-view highlighting what he understood and didn't understand in a piece of text
- "interpreted view" interstitial-translation of a piece of text into what it meant to him
- first-person view of his office
- "virtual desk" with read papers, etc
- day-in-the-life
- perspective-lens -- look at any event and see what he felt about it
  - what did/would he say about this
- maxwell-simulator -- What Would Maxwell Do, what if
- rose-colored glasses
- take his commute to work
- electric shock when you look at something he didn't like
- wear-his-pants, dynamic pants that make you feel like you're wearing his actual pants

person-context
--------------
[the world's experience of a person]
see what others said about him
see what influenced him (Faraday's work)
see what he influenced (Heaviside's reaction to Maxwell)

motion
------
through time
through evidence?
through relevance axes

shift
-----
jump (person to person)
pivot (person to project)?

---------

inward -> perspective
outward -> context

person looking inward at himself (empathy)
person looking outward at the world (perspective)
world looking at the person (where he fits in, context)

what they did [actions taken]
what they experienced [ambient action]
what they understood [viewing inside]
how they saw the world [viewing outside]

---------

[evidence-based representations]
systems thinking -- emergent effects
particles and fields
external imagination means more raw material for others to get person-perspective

is a "thing" (or an "idea") different than a "project"?
history of
   a mathematical symbol
   ethernet
   the SR-71
   democracy
   the gothic arch
   the alphabet


------------------------------------------------------------------------------------------------------------------------------------

11/26/2014

perspective: Every Thing has its own history and way of seeing.
context: Every Thing is influenced by and influenced other Things.
motion: In the dynamic medium, reading is movement.

---

The Dynamic Representation of Knowledge
A Dynamic Representation of History
A Dynamic Encyclopedia
A Dynamic Representation of Encyclopedic Knowledge
A Dynamic Representation of the Past

---

zinn, people's history
NNPOV (non-neutral points of view)

common knowledge is a myth
don't say "what people at that time knew", but *show* who believe it
   - individually, via perspective-representations
   - in aggregate, via data graphics

kill narrative history, kill generalizations about "what people did back then"
zinn p2, columbus came upon an "unknown, uncharted land"
   - not true!  Some people knew it and some had charted it!
   - represent the whole situation, from everyone's perspective, instead of trying to condense it into a cute narrative
the representation of incoherence
embrace incoherence

this is 100% a tool that uses representations to think powerfully about systems

a name is a bad pointer.  Mentioning a person's name should evoke that person's entire world.

primary materials: plantation records, city directories, census data, speeches, songs, photographs, newspaper articles, diaries, and letters (lies my teacher told me, p 7)

---

documents / primary sources
- interviews
- montage of documents
- mosaic of documents
- documents... but structured, or annotated, or "matching highlights" or something
  - see all documents/excerpts that contain a particular phrase (or idea)

synthesized narrative
- biography / documentary
- comic strip
- synthesized diary / notebook
- day-in-the-life
- synthesized interview / dialog

see the world as he did
- causality map, what influenced what according to him
- view of history as understood by him 
  - fog of war for unknown things -- feel what is not yet known
  - distortion/overlay for non-consensus things
- understand-o-view highlighting what he understood and didn't understand in a piece of text
- "interpreted view" interstitial-translation of a piece of text into what it meant to him
- perspective-lens -- look at any event and see what he felt about it
  - what did/would he say about this
- maxwell-simulator -- What Would Maxwell Do, what if

live in his shoes
- first-person video game where you play a virtual Maxwell
  - literally do what he did, say what he said, etc
- first-person view of his office
- "virtual desk" with read papers, etc
- take his commute to work

experience him as a person
- see his picture
- hear his voice
- see video of him
- talk with him
- shake his hand

silly
- electric shock when you look at something he didn't like
- wear-his-pants, dynamic pants that make you feel like you're wearing his actual pants


------------------------------------------------------------------------------------------------------------------------------------

11/29/2014

how to understand what he understood at the time
his references, his room, his book,
    probably can't see his room
    but can see entire collection (at a glance) of everything he's referenced, and what those say
narrative/story summary? of what "was known" or "he knew"
definitions of words at the time

---

What's it like to be a bat
How does one write in this medium
Meme machine 
Rashoman

------------------------------------------------------------------------------------------------------------------------------------

12/1/2014

perspective (particles) (inner world) (a person's experience in the world)
context (fields) (outer world) (the world's experience of a person)

You want viewpoint to be something that is authored and shared (on a bed of facts).  [viewpoint can come from (1) authors (2) readers (3) algorithms]
Algorithm gives a first shot, and humans clean up?
This includes "proximity", things that are nearby in the context space -- this can be inferred by algorithms (like pagerank) for different viewpoints, but can also be inferred from reader behavior/annotations, or explicitly authored by a "trailblazer"

Show the controversy, grounded in facts (show the provenance of the facts)

------------------------------------------------------------------------------------------------------------------------------------