An independent nonprofit research lab, whose mission is to enable universal literacy in a humane dynamic medium. This involves inventing a humane form of computing, and developing educational and community-based institutions in which a culture can grow. [more]
A computing environment (operating system, programming languages, philosophy) invented by Dynamicland researchers to enable us to prototype a new medium.
In Realtalk, people work together side-by-side in the real world, using their hands to create and explore computational models made of physical materials. Realtalk is made in Realtalk itself, and we do all of our day-to-day work in it. [more]
A physical place where we can grow a culture around a new medium.
From 2017 to Covid, Dynamicland was a community workspace in Oakland, California. Through ongoing community hours, open houses, workshops, and residencies, a thousand participants created hundreds of projects that could not have existed anywhere else, and helped define what communal computing looks like.
We are currently developing the next iterations of the Dynamicland space.
Documentation of what we’re hoping to achieve, what we’ve accomplished so far, and the steps along the way. We are reporting our progress, not selling or promoting anything.
The entire website is made in Realtalk, which means that everything on it physically exists. (Even this sentence.) It’s not a rendering of a virtual space — it’s a real place. [more]
A small staff of full-time researchers over the last decade, supported by a community of volunteers and collaborators. [more]
Our current major projects are communal science and Realtalk-2024.
You sure can. The Dynamicland Foundation is an independent 501(c)(3) nonprofit, supported entirely by donations and an occasional grant. Donations are fully tax-deductable, and go entirely to the research.
To make an individual gift or set up a monthly donation, see our donate page.
To make a major gift or discuss a sponsorship, email sponsor@dynamicland.org.
We might be able to make good use of your services, equipment, or real estate, and donations of assets are tax-deductable. Please get in touch.
If you represent a hardware vendor — computers, cameras, projectors, printers, CNC, robotics, motion-control systems, scientific instruments, anything that physically exists — we’d love to discuss a sponsorship.
Please write, but keep in mind that we are currently a very small staff, and coordinating volunteer work is itself a large amount of work, especially for something as unusual as Dynamicland. Depending on what we have going on, we may or may not be able to accept help. But thank you regardless!
Maybe! At the moment, we’re too few to take on a large number of collaborations, but if you have something in mind, let us know.
We don’t currently have the means to support additional staff.
Our public community space in Oakland was closed for Covid. We are currently setting up a new space in Berkeley, initially focused on our vision of “communal science”. It is not yet open for visitors, but will be at some point. Other spaces are still in planning.
A way in which people conceive and share thoughts. An idea might be expressed as a speech, a song, a drawing, a video, an essay, an equation, a tweet... These are different media.
Certain media open up new threads of thought that are otherwise inconceivable. Greek drama was made possible by writing; Shakespearean drama was made possible by print; Newtonian physics was made possible by equations.
The deepest effects are realized when a medium is diffused throughout a culture, not in the hands of a select few. A literate society is one in which all people participate in the exchange of written ideas, where the visual organization of words is second nature in the cultural consciousness. Societies with designated scribes do not enjoy the most significant benefits of literacy.
The conceiving and sharing of ideas represented computationally.
Computers can be used for efficiently distributing static media, as when reading an article or watching a video. But by “dynamic medium”, we mean the representation of ideas in which computation is essential, by enabling active exploration of implications and possibilities.
The modern world is shaped by vast complex systems — technical systems, environmental systems, societal systems — which cannot be clearly seen nor deeply understood via non-dynamic media. The dynamic medium may enable humanity to grasp and grapple with this century’s most critical ideas.
A dynamic medium which is communal, gives all people full agency, and is part of the real world. [more]
By “communal”, we mean bringing people together in the same physical space, with a medium that supports and strengthens face-to-face interaction, shared hands-on work, tacit knowledge, mutual context, and generally being present in the same reality.
By “agency”, we mean a person’s ability and confidence to view, change, extend, and remake every aspect of a system that they rely on, especially for fluently exploring new ideas and improvising solutions in unique situations. In the case of computing systems, this implies top-to-bottom programmability and composability, in a form that is accessible and human-scale.
By “real world”, we mean that material in the medium physically exists, and all of our human abilities and human senses can be applied to it. People are free to make use of their whole selves, every feature of their physical body and of the physical world, instead of interacting with a simulation through an interface.
“Real world” also refers to being situated in reality — understanding what’s actually happening and how things actually work instead of just abstractions; awareness of larger contexts — and especially the local reality of local needs and local knowledge rather than top-down centralized mass-produced solutions.
Because we’re trying to accomplish something unusual, and normal words are misleading.
We prefer to talk about “authoring” rather than “programming”, because programming evokes the image of a developer coding a product or service, and in almost every respect, this is not the image we have in mind. Authoring is for all educated citizens, not reserved for specialists; the act involves writing sentences, drawing pictures, and arranging materials, not manipulating “code”; and the intent is to express an idea for people to explore, discuss, and reply to, not to produce a tool for users to “use”. Authoring is not a perfect metaphor, but it’s better than programming. [more]
We talk about a “dynamic medium” rather than “computers” also because of the emphasis on representing ideas, but especially because we aren’t talking about individual devices. In order to read a book indoors, you need room lighting, but you don’t normally discuss literature in terms of light bulbs. Our view of a dynamic medium focuses on the books on the table, and how you read and write them. The supporting infrastructure is just part of the room; there is no thing that is “the computer”.
We haven’t yet published a full explanation of these, but here’s a hopefully somewhat evocative list:
Realtalk is about noticing and responding to what’s happening, here and now, in the real world.
Imagine a group of players sitting down at a board game. There will be a rulebook, or perhaps a rulecard for each player, that gives the players roles: When you see this, do that.
Realtalk is a way for us to also (literally) give rulecards to ordinary physical objects, which (literally) tell them: When you see this, say that. A Realtalk activity is like a board game that people and things all play together.
It’s rulecards all the way down. The entire system is just physical cards playing the game. We use the cards to make more cards, and we can fluidly change the rules at any time.
Realtalk is an attempt to erase both “the computer” and “the interface” to it.
When people talk about interacting with a computer, they are imagining the computer as a box with an inside and an outside. The interface is how people outside the computer navigate the virtual world inside the computer.
Realtalk does not simulate a virtual world, but takes place in the real world. So, instead of thinking of computation as confined to a world of its own, we think of computation as a new property of physical materials. In the same way that a physical object has a color and a shape, it may also have a program. [more]
We do not think of the object as an “interface” to a program running on a “computer”. We think of the object itself as running the program, noticing and communicating with the objects around it.
Of course there must be electrical hardware to make this work. An analogy might be reading a book at night with a lamp. We do not think of the book as an “interface” to words coming from the lamp. (Although the photons might think of it this way.) We think of the lamp as creating an environment in which words can come from the book. Likewise, Realtalk hardware creates an environment in which physical objects exhibit dynamic behavior.
One consequence is an enormous reduction in computational complexity. A conventional computer must model and simulate its virtual world in every detail. In Realtalk, we lightly apply computation only where needed. We accomplish most of our needs with zero lines of code, by using physical tools and just moving stuff around. [more]
We prefer the term “computing environment”, but nobody knows what that means, so we’ll say “operating system” if we need to convey Realtalk’s scope.
Some operating system engineers might not call Realtalk an operating system, because it’s currently bootstrapped on a kernel which is not (yet) in Realtalk.
But most people’s conception of an operating system is the base set of concepts that they work with. For example, the macOS operating system is based around files, apps, clicking, etc. Working in Realtalk means working with a completely different set of base concepts: physical objects, statements, spatial relations. These concepts are used for everything, including building the system itself. In that sense, Realtalk is an operating system.
On the other hand, we wouldn’t consider macOS a proper computing environment, because it’s not transparently programmable. It’s more like a game console. [more]
From our perspective, “operating system” is a flawed concept. As Dan Ingalls said forty years ago, “An operating system is a collection of things that don’t fit into a language. There shouldn’t be one.”
We sometimes refer to the Linda distinction between a “coordination language” and a “process language”. The coordination language is for programming-in-the-large; it determines how programs are structured and how parts interact. The process language is for programming-in-the-small; it determines how you add numbers and form arrays.
Realtalk is a coordination language. It provides a natural-language-like way for objects to make statements and react to the statements of others: When you see this, say that. Many idiomatic Realtalk programs are almost entirely at this level.
A process language fills in gaps in this natural-language structure, and can be whatever is appropriate — currently, Lua, C, C++, Python, JavaScript, Julia, or Haskell. However, all these languages are off-putting for many people that we want to reach. Realtalk-2024 will have its own process language which is both readable as natural language and interpretable by a simple, visibly-understandable compiler.
All programs, anywhere in the system, in any language, are live-editable, recompile instantaneously, live-display their state, can be physically picked up and pointed to, and can react to objects around them or on them.
Language is just one of the ways of making a program in Realtalk. The environment also encourages hand-drawn notations and diagrams, and spatially arranging other programs. Every program, no matter what form it takes, makes and notices statements, so all programs naturally interoperate.
A loose, decentralized group of physical objects, noticing aspects of the physical world and wishing for the world to be different.
Consider a group of people working on a big project spread across a table. All the people can see each other, and everyone can hear what everyone else is saying, although they might only pay attention to smaller discussions that they’re part of. Most of the talk will be about materials physically present on the table. One person might notice something, another person might make a further comment building on that point, and so on. As a result, a person might express a wish that some part of the project be different, other people might make further suggestions on how that change could be accomplished, until someone feels ready to perform the change on the table.
The organization of computation in Realtalk is modeled on this situation. Computational objects can all see each other, and can hear the statements everyone else is making, although they generally only pay attention to statements relevant to them. Objects make statements about what they’ve noticed in the physical world (say, how close two other objects are), or implications they’ve derived from such statements. As a result, an object might wish that the physical world be different in some way (say, that those objects are illuminated with text, or are in a different place), and this wish is noticed by other objects who make further suggestions, until some object (say, a projector or robotic arm) has enough information to physically perform the change.
The correspondence to the human situation is more than an analogy though, because in Realtalk, people and objects are in the same space, playing the same game. Objects can’t see people (Realtalk does not track people), but people see the objects and the statements they’re making, people make statements of their own, and people make changes to the physical world themselves, using their hands to fulfill the wishes of the objects as well as their own. The verbal discussion among people happens in parallel with the statement-based discussion among objects (and people).
Working in Realtalk means adding computation to physical materials so people and materials can all work together in the same space. The programming paradigm doesn’t just mimic social practices; it integrates with them. [more]
At present, Realtalk exists in Dynamicland spaces and in the spaces of our collaborators, where we can carefully grow and tend in-person communities of practice.
In the short term, additional spaces will be started by people who have contributed significantly to an existing space and have internalized the culture and its values.
Long term, we intend to distribute the ideas in the form of kits+games which will guide communities through building their own computing environments that they fully understand and control.
Long long term, computing may be built into all infrastructure as electric light is today. This would also require an extensive network of educational support. [more]
We have not yet published a report about how Realtalk works in detail. We intend to do so after the next major iteration, Realtalk-2024, currently in progress.
A number of people have started projects that are inspired by Realtalk. These projects may have their own merits, but we do not believe that studying them will give insight into how Realtalk works or the Realtalk vision.
A common misconception is that Realtalk is a “projection-mapping system”, in comparison to, for example, an AR “head-tracking system”.
Realtalk is, first and foremost, a “programming system”, for programming real-world materials. Projection mapping, using cameras and projectors, is one technique for realizing programs on real-world materials. But a Realtalk environment incorporates any sensors for recognizing real-world objects, and any actuators for giving these objects physical capabilities.
For example, projectors enable our real-world objects to display text and images. But on a robotic table, real-world objects gain an additional capability — they can move themselves around. Objects with integrated robotics can move around anywhere. These robotic capabilities fit seamlessly into Realtalk, because Realtalk is about programming things in the real world, not about overlaying a virtual world via projection mapping.
Accordingly, our efforts are not directed toward developing the world’s best projection-mapping system, but rather the world’s best way for all people to create and explain their own projection-mapping systems (and everything else).
Paper is an abundant and versatile material, with mature technologies for writing, drawing, printing, cutting, collecting, and mounting. Don’t scoff at pens, paperclips, binders, and corkboards. They do their jobs simply and silently, with zero lines of code, and are effortlessly multiplayer in a communal environment.
But Realtalk is not about paper. It’s about working with whatever physical materials or forms are appropriate for a given activity or domain. In Biomolecular design in Realtalk, protein design and microscopy use the tangible vocabulary of board games, arranging game pieces on game boards. Nanostructure design is modeled on an architecture studio, building up 3D scale models. In the wet lab, Realtalk integrates with the actual test tube racks and plates, and these are used in simulation as well.
Realtalk’s concept of “recognizers” are intended to encourage the use of a wide variety of materials.
Realtalk is an environment in which people make things for themselves, rather than obtaining mass-produced products from vendors. If projects look handmade, it’s because they literally are handmade, in the moment, for the needs of the moment. They are then remade for the needs of the next moment. Craftiness is not an aesthetic, but an effect of accessibility — paper and tape are convenient and flexible materials.
Anyone is free to make their own projects as polished as they like, and some community members have made beautiful works of art. [1] [2] [3] But note that real-world creative workspaces (woodworking benches, electronics benches) tend to look messy, and when you see someone’s Realtalk project, you are often seeing the “workbench” rather than a “finished piece”.
We use computational physical objects, and it’s very difficult for people who haven’t experienced Realtalk to imagine what this is like.
For example, this website is made of physical cards that are collected into binders. Because they are physical, the cards are easily browsed, shuffled by hand, annotated, pinned up, passed around, put away, and so on.
But the cards can also be searched. It’s trivial to do a find-and-replace across the entire collection, or do anything across the collection with a two-line scratch program. We’ll (physically) put two binders side-by-side and (computationally) diff them. If we don’t want to physically pull a card out, we’ll instantiate a (virtual but spatial) proxy representation to point at. If we lose an object, we’ll print another copy. We can apply the full power of computation to our objects, but with the real world as ground truth.
One move that is currently more convenient on screen-based computers is to save a document or workspace that can be reopened at a different time or place. In Realtalk, we can take snapshots of physical arrangements, even preserving hand-drawn features, and we’ve prototyped ways of reconstituting the arrangement back into physical space — regenerating the materials with printers, cutters, and 3D printers, and placing them robotically or by hand with guidance. Robust reconstitution is a goal of Realtalk-2024.
Realtalk aims to keep the computational system small and simple by offloading as much as possible to the physical world, and this offloads many problems to the physical world as well.
For example, if an object is not being recognized, it’s typically because one of its corners is covered up, or the paper is creased, or a sunbeam is too bright, or the camera is out of focus. These faults can be annoying, but they are entirely understandable in physical terms, and can be addressed by uncreasing the paper or putting a shade over the window.
When you encounter faulty behavior in a desktop OS or a web app, it is almost always due to some condition that you will never be able to see or understand, let alone fix with sticky tape.
We occasionally experience computational bugs as well, but the system is small and designed for live real-time visibility into the internals of every program even under normal conditions, so anything unexpected can be noticed.
In the original Dynamicland space, hardware was bolted into the ceiling. Today, we use floor lamps and desk lamps which can be moved wherever needed at the moment. These dynalamps can be built by hand out of inexpensive commodity parts. We have brought Realtalk to conferences in a suitcase, and to classrooms in a backpack.
Realtalk emphasizes in-person collaboration in communal spaces, but all Realtalk spaces are networked, and people at different sites work together in a variety of ways.
In one example, you can not only see, on your desk, an image of what’s on my desk, you can directly interact with the objects on my desk through that image. This way of working is not a feature built into Realtalk, but is something that anyone can make for themselves.
Every site has its own independent physical copy of Realtalk. People share updates with each other by getting together and talking about them. There is no centralized official version of Realtalk.
When we envision a humane dynamic medium at scale, we imagine a decentralized network of local communities, not a database serving millions of customers. [more]
Realtalk aims to be even more open than open source is today. [more]
Agency is the ultimate accessibility. We believe that true accessibility will come from empowering all people to freely adapt their tools and environments for their own needs, not “accessibility features” as manufacturer-provided add-ons to a closed system. [more]
Dynamicland’s community has been drawn from diverse cultures, disciplines, and local groups, with a focus on those who are underserved or alienated by current forms of computing. We aim to enable these groups to realize their own visions of next-generation computing, rather than having one imposed on them by some tech company. [more]
We are not product-solutionist. Our approach is “helping to create a context in which society can gradually transform over the next 100 years into one that can steward itself instead of destroy itself”, not “releasing a product that everyone will use and will fix everything”. We are not making a product. [more]
But it’s impossible to study the historical transition from orality to literacy, or from manuscript to print, without recognizing that a shift in the dominant medium has the greatest effect on humanity’s ability to conceptualize and affect the world. A medium on par with literacy requires four things: a widely-literate culture, a body of literature, a form of education, and a set of technologies which make the medium physically realizable and practical.
Pursuing a humane dynamic medium means pursuing all four in parallel. Dynamicland spaces are for incubating a culture and developing educational programs. The community’s creations and activities are for seeding a literature. Realtalk is the technological substrate. All four will diffuse, gradually, through education and community-building, remade by each community for its own needs.
This process is enabled by new technology, but the technology itself not a “solution”.
We are opposed to mass ignorance. We’re opposed to engendered helplessness, to superficial and superstitious beliefs around black boxes. We’re therefore wary of complexity that is beyond human scale, where mass ignorance is unavoidable.
We pursue technical systems, and models of natural systems, that are simple enough to be constructed by communities for themselves, where people gain full agency and confident understanding. These necessarily would have to spread via education and knowledge-sharing, not as products.
Current trends in conventional computing are to gather large amounts of personal data, and move all interaction into virtual spaces which can be trivially monitored.
In Realtalk, most personal information never gets into the computing system in the first place. Interpersonal interactions take place in physical space. Realtalk has no user accounts — there’s no concept whatsoever of a “user” within the system. There’s just stuff on the table. The stuff doesn’t know whose stuff it is. [more]
Realtalk biases toward the simplest and most minimally-aware computational stuff, because that gives the most flexibility for improvisation and recombination in physical space. Most Realtalk objects respond to the current physical situation, the here and now, and do not remember anything. There’s very little “data”.
Realtalk does not track people.
Realtalk is not “about” having cameras all over the place; it’s about people working with physical material that has computational capabilities. That requires either embedding hardware into the material itself, or having sensors in the environment. Augmenting the environment invites a much wider variety of materials and ways of working, and biases toward communal work instead of personal devices.
The sensors in our current systems include off-the-shelf webcams, because our current systems are research prototypes. But an ideal sensor might, for example, sense physical material with high fidelity while being blind to human beings. Such sensors are more likely to be developed if we demonstrate the need for them.
We are not proposing a future with cameras everywhere.
Realtalk is, first and foremost, an authoring environment. The essence of Realtalk is programming objects in physical space, using objects in physical space, including programming the entire system itself. We know of no other “spatial computing” system that is a complete self-hosted programming environment used for day-to-day work, and we know of no other programming system that uses physicality in this way.
In particular, the entire system physically exists — every running program is a real thing which is organized, inspected, and edited in the real world. Nothing like this has ever been done before. Many other systems track physical objects, but the systems themselves are developed within the screen of a conventional computer.
Realtalk leans on physicality to dramatically reduce computational complexity, so as to make the entire computing system visible, understandable, and buildable by everyone. Other systems do not share this motivation.
Realtalk is designed for groups of people working together face-to-face, in the same physical place, sharing the same reality. Many other systems are oriented around individual users and individual devices.
In Realtalk, people work with physical materials that actually exist and can be directly manipulated by any physical means, and which naturally engage social cues such as pointing, line of sight, and shared attention. In many other systems, people work with intangible ephemeral hallucinations which only exist while the device is on, are only visible to people with the same device and configuration, and are manipulated indirectly via mediating software.
Realtalk is intended for prototyping a new medium in which all people create and explore computational models in order to understand and discuss the real world. Most computing products do not share this motivation.
Head-mounted displays violate our core principles of visibility, agency, physical reality, and in-person collaboration. [more]
In Dynamicland, people mostly use their voices to talk with other people.
People do not generally issue commands in Realtalk, because there is no disembodied state to command. People navigate the physical space with their bodies, and rearrange it with their hands. They point to things. Everyone sees what everyone else is doing, and groups work together on activities whose state is represented by physical arrangements.
People do form queries (which are, like everything, spatially located on physical objects). While voice recognition could apply here, it’s also likely to disrupt conversation.
Instead of using voice recognition to talk to things, it may have potential in augmenting human conversation. Some of our earlier research focused on that. [more]
An essential component of our concept of agency is decentralization. We imagine a diversity of self-sufficient communities building idiosyncratic computing systems for themselves, bottom-to-top. We imagine computational ideas shared as practices rather than as products. This is only possible if the underlying technologies are decentralizable. [more]
From this perspective, the development of the semiconductor industry has not been an unqualified success. In its early years, integrated circuit technology was seen by some pioneers as a blank canvas for all, in which a vast diversity of computational ideas would be realized simply and efficiently in application-specific microchips. (Caltech’s computer science department was in fact founded around the “new computer science” of designing in silicon.) [more]
But while the computational power of these chips proceeded to scale exponentially with Moore’s law, the manufacturing cost per design scaled as well. After a couple of decades, silicon design was out of reach for all but the biggest companies and highest-volume orders. Today, the entirety of the world’s silicon manufacturing is concentrated in a handful of giant companies, and the cost of building a fabrication plant is astronomical.
The truest expression of computational design is design in silicon, and this art form has been made unavailable to the public. The landscape of computational possibilities has been restricted to just those that can be realized in software on one or two obscenely-complex general-purpose architectures. These architectures are accepted like laws of nature, and few people recognize the vast potential diversity which has been lost.
Dynamicland’s prototypes today are built on these architectures out of necessity, but we are very interested in new technologies which are suitable for truly-bottom-up design. In the short term, we wish to extend the Realtalk approach down to FPGAs and other reconfigurable hardware.
But we’re also looking beyond silicon. Our current major collaborator is in the field of DNA nanotechnology, where individuals routinely design full-custom nanoscale components which self-assemble by the trillions out of readily-available materials using inexpensive benchtop equipment. These are not yet computers, and nobody knows when or how biomolecular computing will evolve, but we are encouraged by the vision of a biodegradable computing substrate which can be locally designed, fabricated, and understood from the atoms up.
If we take “AI” to mean the current trend of deep-learning models trained on large datasets, there are a number of ways in which these techniques are incompatible with our values.
Visibility. To empower people to understand and have full agency over the systems they are involved in, we aim for a computing system that is fully visible and understandable top-to-bottom — as simple, transparent, trustable, and non-magical as possible. When it works, you learn how it works. When it doesn’t work, you can see why. Because everyone is familiar with the internals, they can be changed and adapted for immediate needs, on the fly, in group discussion.
Current AI systems do not share these priorities.
Agency. A Realtalk system “knows” as little as possible. One of our most surprising discoveries has been: to maximize agency, minimize what the computer knows. Every time we make the computing system less aware of what’s going on, every time we remove user interfaces in favor of noticing simple spatial relations between objects, every time we remove programmed rules in favor of socially-agreed-upon practices, our systems become more flexible and composable, and new dimensions open up for improvisational modification.
We often take a “human-in-the-loop” approach: Instead of a complex program which accomplishes a task entirely automatically, we design for people and simple programs working cooperatively. Instead of a computer making decisions and taking actions, we prefer people making decisions and taking actions within a rich environment of realtime feedback and analysis.
Most current AI systems rely on enormous datasets to know as much as possible about their domain and make the most autonomous decisions. Failures and biases are addressed by further enlarging the dataset. These systems rarely have the property that reducing their knowledge makes them perform better.
Human connection. Realtalk is about people connecting with people — working side-by-side with other people, learning from other people, developing relationships with other people, trusting other people. Computation takes the form of physical tools and materials that people use in conversation with other people. People do not have conversations with their tools.
Many current AI systems adopt an agent-like posture, where users are encouraged to converse directly with the system as if it were an “assistant” or “servant”. In our view, reliance on agents detracts from opportunities to deepen relationships with real people. We are also uncomfortable with the mindset that accompanies owning servants.
Human growth. Our view of a “humane dynamic medium” extends from what made the medium of print-based literacy so powerful. A book cannot do something for you. Instead, reading a book can change you into someone who can do something for yourself. The role of a great medium is not to help people get things done, but to help people become deeper people — by providing a context in which they grow their skills and knowledge, broaden their context and perspective, and deepen their awareness and discernment.
One of our oppositions to product consumption is that it is a form of “outsourcing understanding”. Instead of learning a practice, one uses a product which insulates them from the underlying knowledge. The “smarter” the product, the less the user needs to understand. While each product may be convenient individually, the cumulative result is an almost universal ignorance, helplessness, dependence, and fragility. [more]
We see many current AI products, particularly so-called generative tools, as “outsourcing understanding” in its most virulent form. [more]
On the other hand, there are some “non-semantic” uses of AI techniques, such as text recognition and speech transcription, which can be used in ways that don’t displace learning and development. We are still opposed to their complexity, opacity, and enormous training sets, but we may be more willing to cautiously admit them for peripheral purposes, and seek ways to enhance their visibility.
Reality. Realtalk is focused on the real world. It is of the utmost importance that our models reflect reality, and that everyone understands how much they reflect reality and why we believe so. Realtalk’s rules form explicit chains of inference, and Realtalk’s statements can be seen as the supporting evidence.
Many current AI systems have a loose regard for reality, and readily produce results that appear real but are not.
The STEPS project from Viewpoints Research Institute pursued order-of-magnitude reductions in the complexity of desktop computing by leveraging specialized languages, metalanguages, and architectures. Realtalk pursues order-of-magnitude reductions in complexity by leveraging the physical world to avoid simulating a desktop in the first place (along with a flexible reactive architecture for the computing that remains).
We see physicality and metalanguages as complementary forms of leverage, and we are eager to combine them as we continue down the path that STEPS laid out. Realtalk is already multilingual, and many non-textual notations (both symbolic and diagrammatic) are more comfortably and communally drawn on paper than on a screen. Future Realtalks will be even more oriented around defining bespoke interpreters — for sentences, for notations, for diagrams, for scenes — and integrating all of these modes to achieve the most compact and understandable representations.
Some of our general inspirations for the dynamic medium are on the shelf. [more]
The technical design of Realtalk was deeply inspired by Sketchpad, Smalltalk, Lisp machines, Self, HyperCard, Etoys, Inform 7, Linda, and Datalog, and we’ve also drawn ideas from Erlang, Clojure, Haskell, Forth, and many others.