Skip to main content

A Builder's Tour of Automerge

Automerge is a suite of tools for building local-first web applications with real-time synchronization that works on and offline.

In this tutorial, you'll build a local-first multiplayer app with TypeScript, React, Vite, and Automerge. You'll discover how to:

  • Represent data as Automerge Documents
  • Change documents' data and merge changes from different peers
  • Store & synchronize a set of documents in an Automerge Repository
  • Build a multiplayer realtime web app with the Automerge React client
The app in action. Data is stored locally, and Automerge syncs changes between users automatically.

Setup​

info

All the code here can be found at the automerge-repo-quickstart repo.

To get started:

  • clone the tutorial project from automerge-repo-quickstart
  • switch to the without-automerge branch
  • in the automerge-repo-quickstart directory, install the project dependencies
  • start the local Vite development server
$ git clone https://github.com/automerge/automerge-repo-quickstart
# Cloning into 'automerge-repo-quickstart'...
$ cd automerge-repo-quickstart
$ git checkout without-automerge
$ npm install
# ...installing dependencies...
$ npm run dev

Visit localhost:5173/automerge-repo-quickstart/ to see the app in its "starter" state, as a basic React app not yet using Automerge: the task list can be edited, but changes are not synced between users, and all local changes are lost when the page is closed or reloaded.

The (unimpressive) app before you give it superpowers with Automerge

Let's fix all that with Automerge!

In the exercises that follow, you'll modify the source code to:

  1. Configure a Repository to store & sync document changes locally
  2. Create/retrieve a task list Document by its Document URL
  3. Use the Automerge React client to update the Doc's data on user input
  4. Update the Repo to also sync changes over the network (when available)

Architecture of an Automerge App​

Building apps with Automerge requires familiarity with two key concepts: Documents and Repositories.

  • An Automerge Document (Doc) models app data using a specialized data structure that supports conflict-free collaboration via git-like merges.
  • An Automerge Repository (Repo) determines how/where the app stores and synchronizes those documents, locally and/or over the network.

Automerge is built in Rust, but stack-agnostic and useful for building apps on any platform, with client libraries for many popular languages/frameworks.

Diagram of automerge project components, including automerge and automerge-repo
Automerge system diagram from "New algorithms for collaborative text editing" by Martin Kleppmann (Strange Loop 2023)

The foundational Document data structure & related algorithms are defined in the @automerge/automerge core library, which used under the hood by the @automerge/automerge-repo library, which exposes the practical conveniences for managing documents via a Repo.

Manage docs with a Repo​

A Repo keeps track of all the documents you load and makes sure they're properly synchronized and stored. It provides an interface to:

  • create, modify, and manage documents locally
  • send & receive changes to/from others, and
  • merge multiple changes as needed.

Each Repo needs to know:

  • Where its documents should be saved, specified via a StorageAdapter
  • How/Where to send, retrieve, and synchronize doc updates, specified via zero or more NetworkAdapters

The Repo constructor, which comes from @automerge/automerge-repo, lets you create & configure a Repository, specifying the StorageAdapter and NetworkAdapter(s) you need.

Adapters can be imported from their respective @automerge/automerge-repo-storage-* and @automerge/automerge-repo-network-* packages.

For convenience, we're going to use the @automerge/react package to simplify our imports, but all that package does is re-export the most common dependencies that a React web application might want.

Roll your own adapter

If none of the pre-built adapters fit your needs, you can create custom adapter(s) as needed.

Storage & Network Adapters​

Currently, the task list app doesn't persist or sync any changes, even locally.

To prepare to add local multiplayer capabilities to the app, you'll initialize a local-first Repo to:

  • save Docs client-side in the browser's IndexedDB, using the IndexedDBStorageAdapter from @automerge/automerge-repo-storage-indexeddb
  • keep local users (i.e. tabs within the same browser/origin) in sync via a BroadcastChannel, using the BroadcastChannelNetworkAdapter.
Exercise

Create a Repo to hold your documents​

Start by adding @automerge/react to your project.

$ npm install @automerge/react
# ...installing dependencies...

In src/main.tsx, import @automerge/react and create a Repo object configured with networking and storage.

We'll start by storing our data in IndexedDB so we don't lose it when we refresh the browser, and we'll use the inefficient but simple BroadcastChannel networking adapter to keep our browser tabs in sync.

src/main.tsx
import React from "react";
import ReactDOM from "react-dom/client";
import App from "./App.tsx";
import "./index.css";

import {
  Repo,
  BroadcastChannelNetworkAdapter,
  IndexedDBStorageAdapter,
} from "@automerge/react";

const repo = new Repo({
  storage: new IndexedDBStorageAdapter(),
  network: [new BroadcastChannelNetworkAdapter()],
});

ReactDOM.createRoot(document.getElementById("root")!).render(
  <React.StrictMode>
    <App />
  </React.StrictMode>
);

There are lots of other storage and networking adapters for all kinds of different environments. We'll see more of them later.

Repos in React: RepoContext​

The @automerge/react package provides some React-specific conveniences for working with Automerge repositories.

A RepoContext makes your repo and its documents available throughout your React application, via useRepo and useDocument hooks which can be called in any client component.

Exercise

Add a RepoContext to the React app​

In main.tsx, import RepoContext and modify the React.render() call to wrap the App component with a RepoContext.Provider, passing in your fresh new repo to the context's value prop.

src/main.tsx
import React from "react";
import ReactDOM from "react-dom/client";
import App from "./App.tsx";
import "./index.css";
import {
  Repo,
  BroadcastChannelNetworkAdapter,
  IndexedDBStorageAdapter,
  RepoContext,
} from "@automerge/react";

const repo = new Repo({
  storage: new IndexedDBStorageAdapter(),
  network: [new BroadcastChannelNetworkAdapter()],
});

ReactDOM.createRoot(document.getElementById("root")!).render(
  <React.StrictMode>
    <RepoContext.Provider value={repo}>
      <App />
    </RepoContext.Provider>
  </React.StrictMode>
);

Doc Handles & URLs​

A Repo isn't very useful until it has some documents in it! To create a new document, pass its initial value to repo.create(), which accepts a type parameter representing your data:

const listHandle = repo.create<TaskList>({
  tasks: [
    {
      title: "Learn Automerge",
      done: false,
    },
  ],
});

The object returned from repo.create() is a DocHandle, which provides an interface for working with the document.

A DocHandle's .url property provides the document's unique identifier:

listHandle.url; // automerge:37Qr33Ub26dnS2txNCjEJDC37KFT

To retrieve a handle for a document that's already in your repo, you can pass its document URL to repo.find():

const existingDocHandle = repo.find(existingDoc.url);

It's common practice to pass document URLs around as URL hashes. For example:

http://my-automerge-app.com/automerge-repo-quickstart/#automerge:37Qr33Ub26dnS2txNCjEJDC37KFT

The automerge-repo package exports an isValidAutomergeUrl() function that you can use to determine if a given hash is a valid Document URL.

In your task list app, you'll check the page's hash and:

  • retrieve the existing task list document if it exists, or
  • create a new document if we don't have one already.
Exercise

Quick & Dirty URL-based sharing​

Now that we have automerge added to our project and a Repo available, we're going to add a simple URL-based sharing mechanism. You might recognize this approach to link sharing from other projects. Going to the page with an empty path creates a new document and puts the ID of that document into the location's hash fragment. If you open a link that includes a valid document in the hash, we open it.

This is the last bit of our setup, and while this is very convenient for prototyping and testing, it isn't the right approach for a production application. We'll discuss what we'd recommend for a production application below.

src/main.tsx
import React from "react";
import ReactDOM from "react-dom/client";
import App from "./App.tsx";
import "./index.css";
import {
  Repo,
  BroadcastChannelNetworkAdapter,
  WebsocketClientNetworkAdapter,
  IndexedDBStorageAdapter,
  RepoContext,
  isValidAutomergeUrl,
} from "@automerge/react";

import type { TaskList } from "./App.tsx";

const repo = new Repo({
  storage: new IndexedDBStorageAdapter(),
  network: [
    new WebsocketClientNetworkAdapter("wss://sync.automerge.org"),
    new BroadcastChannelNetworkAdapter(),
  ],
});

// Check the URL for a document to load
const locationHash = document.location.hash.substring(1);

// Depending if we have an AutomergeUrl, either find or create the document
let handle;
if (isValidAutomergeUrl(locationHash)) {
  handle = await repo.find(locationHash);
} else {
  handle = repo.create<TaskList>({
    tasks: [],
  });
  // Set the location hash to the new document we just made.
  document.location.hash = handle.url;
}


ReactDOM.createRoot(document.getElementById("root")!).render(
  <React.StrictMode>
    <RepoContext.Provider value={repo}>
      <App docUrl={handle.url} />
    </RepoContext.Provider>
  </React.StrictMode>
);

Working with your new document​

We recommend you keep Automerge documents small and granular: you can load just what you need and share with other users. As a general principle, our motto is "if the data should always travel together, put it in the same document". Remember that each document has its own history, and you can't only share part of a document.

The simplest and idomatic way of linking docs together is to use their AutomergeUrl; we'll look at an example of that later.

Docs in React: useDocument​

Once you have the URL of the document you want to work with, you can access & modify it from your components with the useDocument hook.

Similar to React's useState, useDocument returns a two-item array with a reactive doc value representing the document's current contents and a changeDoc function which can be used to update that value.

The doc object will look and feel just like a Plain Old Javascript Object, because it is one. Just like with useState, changes directly to the value won't behave the way you expect. Use the changeDoc callback to update the document, recording your changes, and both saving and replicating them.

Now let's work with the document we created in the App component. We're going to use the useDocument hook which has a similar interface to React's built-in useState. When you build an application with Automerge, you don't have to worry about where updates come from. No matter whether they're local or remote, your application will update the same way as they arrive.

Exercise

Reading a document​

Let's look at reading the contents of a document. Until the document loads, it's undefined. After that, it will become a POJO.

src/App.tsx
// ...
import { useState } from "react";
import { useDocument, type AutomergeUrl } from "@automerge/react";

// ...

function App({ docUrl }: { docUrl: AutomergeUrl }) {
  const [doc, changeDoc] = useDocument<TaskList>(docUrl);

  // Until the document loads, useDocument returns undefined.
  if (!doc) return <div>Loading...</div>;

  // Now we can get the tasks out of the document and render them below.
  const { tasks } = doc;

  // ...
}

export default App;

Editing a document​

The Automerge equivalent of setState(state => state + 1) is changeDoc(doc => doc.state += 1). changeDoc is the only way to update a document and will record any mutations you make in your callback to the doc object.

There's one important difference between your usual JS style and working with an Automerge document: you will generally want to avoid immutable style.

It's idiomatic in JS to use syntax like spread operators to update a document, but if you do this, you'll make merging with other users ineffective. That's because Automerge doens't second-guess your intention: if you replace the whole array, we'll trust that's what you meant to do! Instead, you'll want to only update the data you actually want to change.

We've got three places we edit the document: creating a new item, toggling completion, and editing the item's text.

Here, we replace the React setState style array spread syntax with an "unshift" call. Remember, Automerge does what you ask, so if you replace the complete array, your changes won't merge well with other users'.

  <button
    type="button"
    onClick={() => {
      changeDoc((d) =>
        d.tasks.unshift({
          title: "",
          done: false,
        })
      );
    }}
  >

Updating the task's state is similar, but we use the index of the item to make sure we target the right item. If we weren't iterating over the array already, we could use .find() to determine the index of the item we need.

  <div className="task" key={index}>
    <input
      type="checkbox"
      checked={done}
      onChange={() =>
        changeDoc((d) => {
          d.tasks[index].done = !d.tasks[index].done;
        })
      }
    />

Updating text​

Finally, we're going to handle text a little differently in this example. Following the same principle we discuss above, if you reassign a text field in an Automerge document, we will replace the whole string. This might be what you want in some cases, but often, you'll want to support collaborative editing. This can be particularly important on large documents.

There are two approaches you can use here. The simplest approach is to use the utility function updateText. It compares the before-and-after values of a string and applies a minimum edit script to combine the two. Typically for a more advanced integration with a text editor, you would use the Automerge.splice() function as part of an event handler, or -- ideally -- you'd just use an existing text-editor plugin like @automerge/codemirror.

First, we'll add updateText to our imports from the library.

import { updateText } from "@automerge/react";

Next, we replace the text updating function with one that uses it instead of just replacing the value completely.

    <input
      type="text"
      placeholder="What needs doing?"
      value={title || ""}
      onChange={(e) =>
        changeDoc((doc) =>
          updateText(
            doc, // the document to update
            ["tasks", 0, "title"], // array representing the path to the text to update
            e.target.value
          )
        )
      }
      style={done ? { textDecoration: "line-through" } : {}}
    />

Collaboration in Automerge​

As the name implies, one of the key powers of Automerge is its ability to merge different changes to a given document, much like git lets you merge multiple edits to a given file.

When merging different changes to the same property, Automerge uses various strategies to avoid conflicts and ensure that the merged document will be identical for every user.

This makes Automerge an ideal tool for building collaborative apps that let multiple users work together on the same documents (with or without a network connection).

Collaborating Locally​

Since the Repo in this app uses a BroadcastChannelNetworkAdapter, any changes made to documents in that Repo sync automatically to all other clients with the same origin (i.e. tabs within the same browser) who know the given document's URL (its unique identifier).

Open a second tab with the same URL and edit the list, and you'll see the first tab's list updated accordingly. If you close all the tabs and reopen them, the document is preserved, as it is stored in your browser's IndexedDB.

Local collaboration via the BroadcastChannelNetworkAdapter

That's right, you've already built a working Automerge-backed React app with live local synchronization! Congrats!

Collaborating over the internet​

Thus far, we've been using the BroadcastChannel NetworkAdapter to move data between tabs in the same browser. Automerge treats all network adapters similarly: they are just peers you may choose to synchronize documents with.

One straightforward way of getting data to other people is to send it to the cloud; then they can come along and fetch the data at their leisure.

When you configure automerge to run on an internet server, listen for connections, and store data on disk, then we call that a "sync server". There's nothing really special about a sync server: it runs the exact same version of Auotmerge as you run locally. With a little configuration work, you could even connect to multiple sync servers and choose what data you want to send.

The Automerge team provides a public community sync server at wss://sync.automerge.org. For production software, you should run your own server, but for prototyping and development you are welcome to use ours on an "as-is" basis.

Exercise

Connect to a sync server via a websocket​

This is as simple as adding WebSocketClientAdapter to your Repo's network subsystem. We'll do this at creation time, but remember you can add and remove adapters later, too.

Solution
src/main.tsx
//...
import { WebSocketClientAdapter } from "@automerge/react";

const repo = new Repo({
  network: [
    new BroadcastChannelNetworkAdapter(),
    new WebSocketClientAdapter("wss://sync.automerge.org"),
  ],
  storage: new IndexedDBStorageAdapter(),
});

Now, when the Repo sees any changes it will sync them not only locally via the BroadcastChannel, but also over a websocket connection to sync.automerge.org, and any other process can connect to that server and use the URL to get the changes we've made.

caution

The Automerge project provides a public sync server for you to experiment with, at sync.automerge.org. This is not a private instance, and as an experimental service has no reliability or data safety guarantees. Feel free to use it for demos and prototyping, but run your own sync server for production apps.

To see this in action, open the same URL (including the document ID) in a different browser, or on a different device. Unlike the local-only version, you'll now see the data updates synced across all open clients.

Network Not Required​

Now that the Repo is syncing changes remotely, what happens when the websocket connection is unavailable?

Since the repo stores documents locally with the IndexedDBStorageAdapter, methods like Repo.find will consult local storage to retrieve/modify documents, so clients can create new documents while disconnected, and any clients who've already loaded a given document will still be able to make changes to it while offline.

Once connectivity has been re-established, the Repo will sync any local changes with those from remote peers, so everyone ultimately sees the same data.

Go ahead and experiment with this by opening your site in two browsers, turning off wifi, making some changes, and turning it back on.

Next Steps​

Congratulations! You've built a local-first, offline-capable app that supports multiplayer collaboration locally and over the network.

If you're hungry for more:

  • Look at the Cookbook section for tips on how to model your app's data in Automerge
  • Dive deeper into how Automerge stores and merges documents in the 'Under the Hood' section
  • Join the Discord to ask questions, show off your Automerge apps, and connect with the Automerge team & community