In Hypercard in the World, all data about a physical object was virtually "attached" to it.  This included what and where it was ("identify" and "location" attachments), the code it ran ("daemon" and "illumination" attachments), its state ("collection" and "variable" attachments), and other kinds of data ("link", "value", "file").  It was conceptually encouraging to know that every bit of state in the system was attached to some physical object, and could be found in that object's inspector.  But again, HitW didn't go very deep, so there wasn't that much state to attach.

Realtalk tries to be more about recognizing the current state of the world rather than working with lots of virtual state.  But you can't recognize everything, or represent everything physically.  Realtalk '17 kept page texts and patch texts as files on a server, which broadcast them as claims when requested.  Other kinds of data (video, audio, datasets) were placed in a "shared" folder using sshfs.  Other than patching pages or writing to the shared folder, there weren't good ways of persisting state, and those ways weren't very resonant with the Realtalk concepts of statements and spatiality.

In Realtalk 2020, memories are statements pinned to physical objects.  When the object is present, these statements are made.  Memories can be about large things like video and audio files, so instead of mucking around in shared folders, we think of these files as "remembered on" the physical object.  You can remember a video on a page, carry the video to another area, and there it is, still on that page.

A page's text and patched text are now memories on the page itself.  This means, like in HitW, an object is self-sufficient -- everything that needs to be remembered about an object is remembered on the object, instead of floating in the filesystem.  Spatially-localized memories now encompass all statements that can't be derived by looking at the real world.

Objects now remember their own page text.  But every page has text, and it would be distracting to see a blue memory box at the bottom of every page, instead of just on pages with interesting state.  Worse, you don't want to type "Forget." and accidentally wipe out a page's text, or accidentally modify the page's original text when editing memories in the editor.  If we want to tell people that they can't break anything by messing around in Realtalk, we need a way to protect certain memories from accidental clobbering.

Page and patched text are stored as deep memories:

This shows up in the editor as

 - Deep memories remain even when you wish the object does not run.

While normal memories are like "application state", deep memories are more like "metadata".  What other metadata might we want to associate with an object?  We could, for example, remember the data and time the object was created, or the image of the page we printed it from.  In a Marks Kit world, we might even create an object that has no text, but only an image, and that image is the program.  

In this way, moving text into memories is actually a step towards non-textual programs.  A "program" can now be any set of statements you want to associate with any recognizable object.

Objects in rulebooks are peculiar, in that the same object id is "present" simultaneously in many areas.  The memories of such objects normally just pertain to the local area.  For example, "Print previewing" wants to remember the most recent print job in the area, not across the site.  Likewise with projection configuration, sync and networking state, etc.

Additionally, it's very useful to be able to supersede these objects -- to make a variant of "Previewing print jobs" on a Blank, which automatically doesn't run until it's swapped in with "Supersede duplicates".  We'd like to be able to swap these variants in and out while keeping their memories intact, and that's not possible if the memories are associated with the object id.

Instead, you can remember [locally]:

These memories appear on the object like normal memories, but instead of being associated with the object id, they are associated with the area and title.  (It actually uses the "base title" minus any parentheticals, which is also how superseding works.)  Because these memories are local to the area, they are never synced.

I'm not 100% enthusiastic about how local memories work yet.  They do serve their purpose, if you remember to use them.  And they're only useful for rulebook objects.  I wish I could make the editor's undo stack a local memory, but there are a lot of pages titled "Code editor" and they certainly don't supersede each other.

Some memories just shouldn't save.  For example, "Keyboards" remembers the key state of each keyboard.  If you shut down Realtalk with the control key down, it shouldn't assume the key is still down when it restarts.  So:

Like all memories on transient objects, transient memories on persistent objects are discarded when the object stops being "present".

Realtalk can still boot itself from nothing more than the memory database and a short "boot.sh" shell script.  The shell script does include a couple frightening "sed" commands to underhandedly extract page text from the memory files.  But you still can

and Realtalk will build the Lua binary and all our modules automatically.  You'll be up and running in about seven seconds.

Currently, a site is specified simply with:

Whatever machines claim the same site leader are thereby the same site, and will share memories.

Unlike Realtalk '17 with its special server, there's nothing special about the site leader -- it's just an ordinary Realtalk machine that's been given an additional role.  The site leader's post office syncs memories between machines, in exactly the same way that other post offices sync memories between local areas.

It's probably good for the site leader to be on most of the time other machines are on.  Unlike Realtalk '17, every machine is self-sufficient -- you can turn it on, use it, and change memories, even if all other machines are off.  (Although you can't allocate pages without the internet-based page allocator.)  However, while the site leader is off, memories remembered on one machine won't show up other machines.  When the site leader is turned back on, all those pent-up memories will sync and everything will catch up.

It is almost the case that you can change the site leader simply by changing the claim.  (I still need to do a bit more work to make that seamless.)

Syncing happens in realtime, through ordinary directed statements.  All memories are stored on all local machines, and no data leaves the Realtalk network.  I did set up a Github repo as periodic offsite backup, but that's not related to syncing.

When a post office receives a memory with refs while syncing, it holds off on acknowledging the memory until it has the ref'd value.  These values aren't transferred through Realtalk's networking; instead, each post office runs a lightweight web server (!) and the receiving post office simply "curls" it from the sender.

Values like the huge table are always downloaded along with their memories, but files like the video file only sync "upstream" to the site leader.  Downstream machines download files only when they need them. (For example, if you try to play the video.)  This works because all files are represented as URLs (in this case, a realtalk: URL), so the only way to do anything with them is to download them.  (With a bit more work, we should be able to stream the video instead of downloading it.)

This means that the site leader has a full set of files, whereas other machines might only have files that they have created or used.

Garbage refs can accumulate.  (At the moment, the editor's undo stack is usually a "huge table", so it gets ref'd and synced every time an editor goes away.)  Garbage collecting refs is straightforward -- scan all memories for Ref, and delete the unreferenced files in the "refs/" directory.  I currently put down a "Garbage-collect refs" page (33325) by hand occasionally, but I'd like to make it automatic.

When an area shuts down, either because of a signal or a wish

it tries to sync before exiting.  Local areas sync unsaved memories to the post office, and post offices sync to the site leader's post office, if it's connected.  Even if the site leader isn't connected, the memories remain safely stored on the local machine, and will be synced whenever the site leader is available.

Cross-site sharing could be asynchronous (push to GitHub, pull from GitHub , or it could be a realtime act where I bring your area into mine (using "included regions") and we talk about what we're sharing (either in person or through video call).

In any case, I'll probably cook up an interim solution via GitHub, but I haven't yet.