Click on panels for more information.
Dynamicland is a place for all people to build, study, play, speak, learn, in fundamentally new ways. (Also it's a new kind of computer.) We are a non-profit, long-term research group in the spirit of Doug Engelbart and Xerox PARC. We are inventing a new computational medium where people work together with real objects in physical space, not alone with virtual objects on a screen. We are constructing a community workspace at 9th and Broadway in Oakland. The entire building is the computer.
First, some context. Dynamicland is about TOOLS TO THINK NEW THOUGHTS. Clay tokens (8000 BC) were the precursors of both writing and mathematics. Each shape stood for a different commodity. These physical objects enabled people to compute by arranging, rearranging, and grouping them, to communicate these computations, and to store accounts. The alphabet (800 BC) physicalized speech. Intangible, ephemeral sounds became physical marks that could be seen, studied, played with, and preserved. Alphabetic writing has been credited with the “Greek miracle” of philosophy, art, and literature around 400 BC. The printing press (1450 AD) gave ideas a physical form that was portable and replicable. Print made possible mass literacy and the democratization of knowledge. The invention of journals, citations, and data graphics ushered in the scientific revolution. The personal computer (1970 AD) enabled dynamic media. Doug Engelbart saw it as a tool to augment the intellect of knowledge workers, and enable humanity to solve the world’s complex problems. Seymour Papert saw it as a generator of microworlds, where one could learn math in “Mathland” like learning French in France. Alan Kay saw it as a medium for reading and writing with systems and simulations, as opposed to stories and arguments, giving humans a new lens to understand the world. These inventors of the personal computer envisioned it as a medium to empower humanity and lift up civilization. It has that potential. But as long as the computer is a closed box, it only truly empowers the select few with keys to the box. Like scribes before the printing press. And as long as computation resides in intangible virtual worlds, it leaves untapped the vast range of human capabilities for working with thoughts as things. Things that can be seen with the eyes, manipulated with the hands, and placed in physical space. Dynamicland is a computer with no box. People walk around inside it. Computer-stuff is not made of fleeting, ephemeral images on a screen, but real physical things that enable people to think with their whole selves. (Film editor Walter Murch surrounds himself with reference photos. 'The photos are vinegar and I'm a cucumber - the goal is to get pickled by them.') (Mathematician and Flelds medalist Maryam Mirzakhani works out her ideas as pictures on large sheets of paper.) (Eyes and brains are only found in animals that move. The sea squirt swims around until it settles into a permanent home, then digests its brain. Brains evolved for movement.)
And in particular, Dynamicland is about thinking TOGETHER, NOT IN ISOLATION. Artistic and scientific revolutions are often mistakenly attributed to “lone geniuses”, when in fact they emerged from unusually fertile places — Florence in the Renaissance, the Copenhagen cafés where quantum mechanics was born, Bell Labs... According to Mervin Kelly, director of Bell Labs in its prime, physical proximity and personal contact among workers “was everything”. Feynman diagrams spread throughout the physics community almost entirely through people physically working together. Nobody learned this tool merely by reading about it. Studying navigators on a Navy ship, Edwin Hutchins coined the term “horizon of observation” for the degree to which using a tool teaches the people around it. Navigation involved shouting questions and drawing lines on charts, which revealed and taught the process to everyone. In dance and martial arts communities, group learning is spontaneous. Beginners learn by watching advanced students. People see others who are struggling, and offer help or advice. Working in the same physical space as others, performing activities that others can observe, learn from, and help with, is a core element of being human. Computer screens bias towards isolated personal spaces. Hallucinatory technologies such as VR and AR pull people even more strongly into private worlds. Dynamicland is a real place, a part of our shared physical reality, not an illusion on a screen. Working in Dynamicland means being present with other people, talking with them, seeing how they work, learning from them and helping them. (In Doreen Nelson's 'city-building education', students focus on a central shared project, not a lecturer or a laptop.) (Most traditional games - board games, card games, role playing games, sports - bring people into the same space. Most computer games are played in physical isolation.)
Dynamicland deeply embodies the spirit of TOOLKITS, NOT APPS. Jerome Bruner suggests that we shouldn’t celebrate a child’s first word, but when a child first combines two words to form a new meaning. Animals can learn individual words, but cannot combine words to express new ideas. The power of language lies in recombination. Flexible tools like pen and paper are eminently recombinable. You can draw pictures in text, or write text on pictures. You can fold or tear paper. You can write on a wall, or on anything. In the physical crafts, the very notion of a “tool” is flexible. Tools are improvised as they are needed. A metalworker welds a bar to an adjustable wrench for performing two-handed twists. Tools are materials. A woodworker builds a custom jig from scrap wood for making precise, repeatable cuts. Materials are tools. The original vision of personal computing was people writing their own software for their own unique situations. In Smalltalk, Genera, and HyperCard, every interactive element is an “object” that can be combined with other objects. Every mechanism can be inspected and modified. Every program is a toolkit for building other programs. Even the original World Wide Web program was both browser and editor. Today, apps are black boxes that cannot be combined, inspected, or modified. When you have a need, you go to the app store. With cloud-based oracles, the mechanism is completely hidden. When it works, you don’t know why. When it doesn’t work, you’re stuck. Dynamicland is not an app store, with one-size-fits-all products to use. Dynamicland is an authoring environment. It provides the tools for people to make their own tools, remix the tools of others, and develop fluency in a medium. (An abacus infuses knowledge of arithmetic into its user, such that when the abacus is taken away, the capability remains. A calculator provides an answer, but no insight.) (Marshall McLuhan belleved we shape our tools, and our tools shape us. With any tool, ask - who's holding the handle?)
We are a non-profit long-term research group, chartered with inventing the future of the computer as a humane, inclusive tool for understanding the world and thinking new thoughts. As we developed our vision and created the technology, we came to realize that this new computer wasn't a product. It was a place. The technology we’re building doesn't fit into a box or a pocket. It isn't a gadget that people buy and strap to their wrist or their head. It's more like a library or a workshop or a kitchen — a real place where real people come together to work with their hands on real physical things. After years of research and prototyping, Dynamicland is to be our first full-scale realization of this dream. The scenarios on the following pages may look fantastical, but we’ve got a good start on them, and we believe they are completely achievable in a few years.
DYNAMICLAND IN 2020 In conversation over coffee, a question comes up — do humans have better lives now than they did a hundred years ago? How have standards of living changed over time, from place to place? They start exploring data related to their question — not by looking into a screen, but by arranging papers, cards, and stones right on the lunch table. Technology in the ceiling sees the materials on the table, and brings them to life by projecting information onto them. Visualizations are controlled and changed by moving objects by hand. A new plot is just a new piece of paper. A simulation is sketched in a notebook. Data is projected onto a globe. An impromptu slider shares the table with a cup of coffee. These materials are programmable; they embody computation and display data. But they are not precious. Just scraps of paper.
A team is designing a video game. Each part of the project is a place in the room. The room/project is always seen and experienced as a whole. In their peripheral vision, everyone sees how their work effects everything else. They see how everything fits together. Software is built out of real-world materials that can be seen by anyone passing by. People learn the project by walking through it. They don’t scroll — they stroll.
DYNAMICLAND IN 2020 Two people have modeled a spacecraft’s trajectory by hoisting objects from danglebots. Danglebots are robots that travel around the ceiling, allowing ordinary objects to move and be moved in three dimensions. One person adjusts the spacecraft’s velocity by moving two objects by hand. The objects making up the rest of the trajectory move themselves accordingly. The people stand inside their dynamic model, exploring and understanding with their hands, their eyes, their physical bodies. The components of the spacecraft program are paper cards brought to life. A person finds components by browsing shelves in the library. The program is built by arranging these cards on a table, visibly, out in the open. The activity attracts spontaneous collaborators, brings people together to learn from one another.
A child draws a raindrop, and arranges a program to make it fall from clouds. Another child crafts clouds. The materials are inviting, playful, remixable. Magical yet mundane. Real-world things can be put anywhere and attached to anything. They beg for improvisation. The children invent the activity together, riffing off one another. The entire building is the canvas. The children’s imagination is not constrained by the features provided by some app.
THE ROAD TO DYNAMICLAND: EARLY WORK Dynamicland is the culmination of years of experimentation, prototyping, and invention. For many years, we’ve sought to empower people to see further and understand more deeply, by designing directly-visible and directly-manipulable computing environments. Our group has an extensive history of pursuing these ideals in virtual space. Inventing on Principle: What if a code editor were like an artist's canvas? Programming with immediate feedback and realtime direct manipulation of code. Stop Drawing Dead Fish: Creating animation by performing with behaviors. Programming behaviors by performing geometric constructions. Media for Thinking the Unthinkable: Redesigning how we read and write systems in the interactive medium.
Learnable Progamming: Programming environments designed for people to see and understand what a program is doing. Tufte-approved! Drawing Dynamic Visualizations: Programming data-driven visualizations, like D3, by direct-manipulation drawing, like Illustrator. Apparatus: A hybrid vector graphics editor and spreadsheet-like programming environment for creating interactive diagrams. Refractor: Multitouch transforming and distorting images by directly manipulating mathematical transformations. Shadershop: Create GPU shaders by directly manipulating mathematical functions and composing them like Photoshop layers. Rumpus: A live-coding playground for room-scale VR. Every object's source code is visible and editable in real-time, right down to the source code for Rumpus itself. TADCAD: 3D modeling using tangible pieces and gestural interactions. These sorts of playful, direct-manipulation computing environments have a long lineage, going back to the seminal interactive programming systems of our group’s founder, Alan Kay. Smalltalk: The original live-object computing environment, designed in the 1970s as a dynamic medium for children. The origin of both object-oriented programming and the graphical user interface as we know them today, and a key inspiration for the Apple Macintosh. Etoys: An authoring environment and visual programming system for teaching children powerful ideas in compelling ways. The worthiest successor to Logo, and a key influence on MIT's Scratch.
THE ROAD TO DYNAMICLAND: PROTOTYPES Despite the potential of our designs, we came to feel oppressed by the intangibility, isolation, and claustrophobia of our screen-based creations. We were building worlds that we didn’t want to live in. Was there a humane vision of interactive computing? One in which people could truly work together, using their eyes and hands, their bodies, the space around them? In early 2015, we built Hypercard in the World - a live-programming environment for the real world. It used a network of computers, projectors, and cameras hung from the ceiling throughout our space. Point with a laser pointer to the four corners of any physical thing to “bless” it as a programmable object. You can now write code that runs on that object and interacts with all the other objects in the space. With Hypercard in the World, we could start asking questions like: What is like to read a book while seeing the entire book? In our library, laser a book on the shelf to see a “map” of all its pages. Laser a page on the map to jump there in an ebook. This interactive poster depicts an entire video at once. Laser a frame to watch from that point, while seeing the context. Each of our projects got printed as a sticky label in our research gallery. Laser a label to bring the project up on the screen, to show it to others or revisit the email discussion around it. Years of work, all visible at once, immediately accessible. A diorama of paper animals come to life. Animals make sounds when you laser them. The frog’s eyes follow the laser pointer. And so on. Silly, but so easy to make and so much fun. Laser-draw on the wall. Hypercard in the World is implemented in itself, and Bigboard is the “engine room”. Touch the code, see the data flow in real time, modify anything in place. Work done out in the open invites casual conversation and collaboration. A couple people started making this diorama; everyone else noticed and joined in. This wouldn't have happened if it were a webpage. Systems that seem dauntingly complex scrolling through a screen become simple when seen as a whole. With Bigboard, everyone in the space understood how Hypercard in the World worked, even if they didn't care. Because everyone walked past it every day, it was part of the furniture, it became familiar.
Game jam! Over three days, friends and guests built a dozen real-world games using Hypercard in the World. Every game was multiplayer. This wasn’t agreed upon in advance; it was just a consequence of designing for the real world. Games intended for two players worked even better with four, six, or more people joining in. Most games centered around body movement, and physically interacting with other players. Most games generated very rich experiences with very simple code. In a screen-based video game, the code is very complex because the computer is modeling, simulating, and drawing a virtual world. The real world simulates and draws itself. We wanted to go beyond pointing at things. We moved into designing systems where people manipulated objects by hand. What is it like to craft dynamic media with paper, pens, and scissors? On this table, any row of papers becomes the frames of an animation. Play pinball while friends remix the board. Paper defines the playfield, stones create flippers. Compose music together. Anything dark on the light-colored paper plays a note. Doesn’t matter what it is. Is this a sculpture or a melody? The real world lets you improvise new rules instead of being trapped in an app. We thought we had made a Pong game, but players decided to play it as Breakout instead — without any reprogramming. Less code means more improvisation. In many of these activities, the computer is only responding to dark marks on a light background, which means that people can use any materials they can get their hands on, in any way they can think of. We wanted to go beyond projecting light onto things. On our magnetic table, you can move objects, and objects can move themselves. This ghost was programmed to run away from the monster. A tangible maze game. Move a piece by hand, and the other piece mirrors its movements. By building robotics into the space, dynamic objects aren't trapped inside screens, but can participate with us in the physical world.
WHERE WE’RE AT, WHERE WE’RE GOING Realtalk After learning from all of these prototypes, we were ready to begin Realtalk. Realtalk is the operating system / authoring environment underlying Dynamicland, designed from the ground up around people creating computational media together in the real world with active objects. Much of 2016 was spent prototyping the conceptual and technical foundations for this system, and development began in earnest in 2017. On June 14, 2017, Realtalk took its first thrilling baby-steps, running itself off of papers it saw on a wall, and things have been moving quickly since then. Realtalk is visible. The entire system is literally on the table. It’s designed to be examined, understood, and modified, with no barriers or black boxes. The chart explaining the system is itself the running system. There can’t be anything hidden, because if a person can’t see something, Realtalk can’t see it either! Realtalk is general. Any physical thing can be an “object”, as long as some other object claims to recognize it. Anything can be a “program” (whether it’s a page of text, a diagram, or a pile of stuff on a table) as long as some object claims to have interpreted it. Realtalk is not limited to cameras and projectors, but accommodates any sensors or actuators — they’re all just objects making and noticing claims about the world. In Realtalk, the real world is ground truth. Instead of simulating and imposing a virtual world, Realtalk simply enables real objects to notice and respond to what's happening with other real objects, in real time. Like a physical tool, Realtalk doesn't drive the action, or even have any idea what exactly people are doing with it. People manipulate objects, other objects notice and respond, and “user interfaces” are usually unnecessary. Realtalk is fun, even at such an early stage. Words can’t describe what it’s like to build a software system by crafting a living mural with your hands, with your friends, with code you can see and hold and pass around. Or what it’s like to pull a card out of your pocket and see rainbow hues dance around the room. Every day we discover new ways of improvising with this strange and wonderful mix of computational and physical material. Live text search over all pages, added in a few minutes by simply attaching a short page to the wall. A community space A demo is not enough. We want real people using Realtalk, both for their own benefit and for us to learn from them and continue the research. But Realtalk isn't an app that can be passed around on the web. It’s built into a building. So we’ll bring people to it. The building is at 9th and Broadway in Oakland, in a lively, diverse neighborhood, two blocks from the BART station. Our goal is for this space to eventually become a public institution serving the community, like a library, museum, or makerspace. Later this year, we'd like to start by hosting a few artists-in-residence and scientists-in-residence, as well teaching workshops and a university class in/on/about Realtalk. By the end of 2018, we hope to have a set of community members working out of the space full-time, using Realtalk for their own work. Dynamicland will also be hosting public events, art installations, lectures, and classes — not about Realtalk itself, but using Realtalk as a next-generation creative tool and communication medium. As Dynamicland makes the transition to an open community workshop with public hours, we’ll continue research on the next version of Realtalk. We are thinking of the Oakland Dynamicland itself as the prototype for a new kind of civic institution, which will someday bring a new form of literacy to the people of every city, as did the public library. Perhaps existing libraries will become dynamiclands. In 1978, Carver Mead and Lynn Conway had invented a new way of designing digital systems which would soon give rise to the microchip revolution. But this couldn’t be packaged as a product for people to buy; it was a way of thinking that people had to learn. So they designed a class, and started a fabrication service so students could get hands-on experience in this new medium. It was, in surprisingly large part, the students from these classes that went on to create Silicon Valley. We are developing a new computational medium. We think it needs to spread not with a product, but by teaching people how to make it for themselves, and giving students a slice of the future where they can get hands-on experience and push at the boundaries of the medium. There are two ways to bring new a new technology to the people: individually-owned products or shared infrastructure. Cars or public transit. Personal laptops or public libraries. Cell phones or a payphone on every block. Xboxes or arcades. Given that current funding and market structures have made products the default, it's worth considering the tradeoffs. A product benefits the individual; infrastructure can benefit all. A product locks people into a vendor; infrastructure allows services to be sampled piecemeal without buying in. Products tend to be disposable; infrastructure encourages long-term thinking. Products encourage isolation; infrastructure demands discussion, wherein people decide as a group how they want to shape the society they live in. Discos are more fun than dancing alone. Timelines To the right are timelines of the two most influential research systems in personal computing, NLS and Smalltalk. In research, it can take several years from a vision to the first prototype that hints at the concepts, several more years to a working system in which the ideas have flowered, and several more years of exploring the possibility space now available. Much of the time is experimentation, prototyping, blind alleys, and the general confusion of inventing a way of thinking that is apart from current practices and trends. For deep, non-incremental work, this is simply part of the process. It’s a different pace than industry work. But it’s these research projects which give rise to entire industries in the first place. We aspire to do such work, with such consequences. By this measure, we’re right on schedule. 1959: Engelbart's first funding for 'augmenting human intellect'. 1962: “Conceptual Framework' vision / research agenda / manifesto. 1964: Experimentation with computers, input devices, interfaces. 1968: NLS first used for daily work by researchers; big public demo. 1970: Mail/journal system and other major innovations. 1974: 'peak activity' in research group. 1968: Kay publishes thesis on 'Flex Machine'. 1970: Xerox PARC founded. 1971: Experimentation with bitmap displays. 1972: First prototype Smalltalk system. 1973: First experiments with children. 1976: First Smalltalk with graphical user interface and object-oriented programming recognizable today. 1980: First Smalltalk suitable for public use outside of PARC. 2013: Our lab founded, research vision written. 2014: Prototyping, vision refined. 2015: Hypercard in the World invented, used. 2016: Realtalk concepts developed, prototyped. 2017: First Realtalk system implementation. 2018: Beginnings of Dynamicland. 2019: New forms of media invented in Realtalk. 2020: Dynamicland is real, Realtalk discarded and reinvented based on what we’ve learned, ad infinitum.
In a possible future, just within reach, computing is continuous with human life. We shape ideas in our hands. The world becomes a thing to think with. We walk through each others’ imaginations, conversing by making, telling by showing. Around every concept, we see the context. Around every part, we see the whole. We see the invisible, think the unthinkable, share the inexpressible. Together.
The Dynamic Medium Group: Bret Victor, Luke Iannini, Toby Schachman, Virginia McArthur, Josh Horowitz, Paula Te (Debug output from the first successful run of Realtalk!) Thanks to our friends, colleagues, and patrons: Alan Kay, Chaim Gingold, Dave Cerf, David Hellman, Glen Chiacchieri, Götz Bachmann, Jennifer Jacobs, Matthias Graf, May-Li Khoe, Michael Nagle, Rob Ochshorn, Sam Altman, Vishal Sikka, The HARC community Our group is organized under a 501(c)(3) tax-exempt nonprofit. We're actively seeking founding donors to make Dynamicland a reality. Get in touch at donate@dynamicland.org. 9th and Broadway, Oakland, CA Write to us at: hi@dynamicland.org Art by David Hellman July 2017