// highlight-next-line
{doc &&
doc.tasks?.map(({ title, done }, index) => (
))}
);
// highlight-end
};
```
#### Checking it works
At this point we haven't hooked up any way of modifying the document. But we can check that the state of the document is reflected in the UI using `window.handle`.
First, create a new list item. Open the console and type:
```js
window.handle.change(d => d.tasks.push({title: "Milk", done: false}))
```
You should see a new task titled "Milk" appear in the UI.
Now let's mark it as done. Open the console again and type:
```js
window.handle.change(d => d.tasks[d.tasks.length - 1].done = true)
```
You should see the checkbox for the final task in the list become ticked in the UI.
### Editing a document
The Automerge equivalent of `setState(state => state + 1)` is `changeDoc(doc => doc.state += 1)`. `changeDoc` is the only way to update a document and will record any mutations you make in your callback to the `doc` object.
There's one important difference between your usual JS style and working with an Automerge document: you will generally want to avoid immutable style.
It's idiomatic in JS to use syntax like spread operators to update a document, but if you do this, you'll make merging with other users ineffective. That's because Automerge doesn't second-guess your intention: if you replace the whole array, we'll trust that's what you meant to do! Instead, you'll want to only update the data you actually want to change.
We've got three places we edit the document: creating a new item, toggling completion, and editing the item's text.
#### Creating a new Item
```tsx title="src/TaskList.tsx"
// ...
export const TaskList: React.FC<{
docUrl: AutomergeUrl;
}> = ({ docUrl }) => {
const [doc, changeDoc] = useDocument
{doc &&
doc.tasks?.map(({ title, done }, index) => (
))}
);
};
```
Here, we replace the React `setState` style array spread syntax with an "unshift" call. Remember, Automerge does what you ask, so if you replace the complete array, your changes won't merge well with other users'.
### Updating the `done` state
Updating the task's state is similar, but we use the index of the item to make sure we target the right item. If we weren't iterating over the array already, we could use `.find()` to determine the index of the item we need.
```tsx title="src/TaskList.tsx"
// ..
export const TaskList: React.FC<{
docUrl: AutomergeUrl;
}> = ({ docUrl }) => {
const [doc, changeDoc] = useDocument
{doc &&
doc.tasks?.map(({ title, done }, index) => (
);
};
```
### Updating text
Finally, we're going to handle text a little differently in this example. Following the same principle we discuss above, if you reassign a text field in an Automerge document, we will replace the whole string. This might be what you want in some cases, but often, you'll want to support collaborative editing. This can be particularly important on large documents.
There are two approaches you can use here. The simplest approach is to use the utility function `updateText`. It compares the before-and-after values of a string and applies a minimum edit script to combine the two. Typically for a more advanced integration with a text editor, you would use the `Automerge.splice()` function as part of an event handler, or -- ideally -- you'd just use an existing text-editor plugin like `@automerge/codemirror`.
First, we'll add `updateText` to our imports from the library.
```tsx
import { updateText } from "@automerge/react";
```
Next, we replace the text updating function with one that uses it instead of just replacing the value completely.
```tsx title="src/TaskList.tsx"
// ...
export const TaskList: React.FC<{
docUrl: AutomergeUrl;
}> = ({ docUrl }) => {
const [doc, changeDoc] = useDocument
changeDoc((d) => {
d.tasks[index].done = !d.tasks[index].done;
})
}
//highlight-end
/>
))}
{doc &&
doc.tasks?.map(({ title, done }, index) => (
);
};
```
## Checking your work
We've finished wiring up the UI, we've got link sharing via the URL hash and storage and synchronization between tabs. If you open the application in one tab, then copy the URL and open it in another you should be able to create new tasks, toggle their done state, and update the description and see the changes synchronize between tabs.
Next, to sync over the network.
# [Network Sync](https://automerge.org/docs/tutorial/network-sync/)
## Collaborating over the internet
Thus far, we've been using the BroadcastChannel NetworkAdapter to move data between tabs in the same browser. Automerge treats all network adapters similarly: they are just peers you may choose to synchronize documents with.
One straightforward way of getting data to other people is to send it to the cloud; then they can come along and fetch the data at their leisure.
When you configure automerge to run on an internet server, listen for connections, and store data on disk, then we call that a "sync server". There's nothing really special about a sync server: it runs the exact same version of Automerge as you run locally. With a little configuration work, you could even connect to multiple sync servers and choose what data you want to send.
The Automerge team provides a public community sync server at `wss://sync.automerge.org`. For production software, you should run your own server, but for prototyping and development you are welcome to use ours on an "as-is" basis.
### Exercise
#### Connect to a sync server via a websocket
This is as simple as adding `WebSocketClientAdapter` to your Repo's network subsystem. We'll do this at creation time, but remember you can add and remove adapters later, too.
#### Solution
```tsx title="src/main.tsx"
//...
// highlight-next-line
import { WebSocketClientAdapter } from "@automerge/react";
const repo = new Repo({
network: [
new BroadcastChannelNetworkAdapter(),
// highlight-next-line
new WebSocketClientAdapter("wss://sync.automerge.org"),
],
storage: new IndexedDBStorageAdapter(),
});
```
Now, when the Repo sees any changes it will sync them not only locally via the BroadcastChannel, but also over a websocket connection to `sync.automerge.org`, and any other process can connect to that server and use the URL to get the changes we've made.
changeDoc((d) => {
d.tasks[index].done = !d.tasks[index].done;
})
}
/>
changeDoc((d) => {
updateText(d, ["tasks", index, "title"], e.target.value);
})
}
// highlight-end
style={done ? { textDecoration: "line-through" } : {}}
/>
))}
#### Caution
The Automerge project provides a public sync server for you to experiment with, at `sync.automerge.org`. This is not a private instance, and as an experimental service has no reliability or data safety guarantees. Feel free to use it for demos and prototyping, but run your own sync server for production apps.
To see this in action, open the same URL (including the document ID) in a different browser, or on a different device. Unlike the local-only version, you'll now see the data updates synced across _all_ open clients.
{{ figure  Local collaboration via the BroadcastChannelNetworkAdapter }}
## Network Not Required
Now that the Repo is syncing changes remotely, what happens when the websocket connection is unavailable?
Since the repo stores documents locally with the `IndexedDBStorageAdapter`, methods like `Repo.find` will consult local storage to retrieve/modify documents, so clients can create new documents while disconnected, and any clients who've already loaded a given document will still be able to make changes to it while offline.
Once connectivity has been re-established, the Repo will sync any local changes with those from remote peers, so everyone ultimately sees the same data.
Go ahead and experiment with this by opening your site in two browsers, turning off wifi, making some changes, and turning it back on.
# [Multiple Task Lists](https://automerge.org/docs/tutorial/multiple-task-lists/)
You might have noticed that if you lose the URL of a task list, it's gone forever. This is fine for testing and demos, but obviously no good for a real application.
Automerge is built around the document as a building block for your application, and we use `AutomergeUrl` links to connect those documents together. We've already created one kind of document, a task list, and now we're going to build on that foundation by collecting your task lists into something like a folder to keep them organized.
This is going to give us a chance to see a few patterns in action: linking between documents with URLs, working with multiple documents at once, and using a single document as the "entry point" for your application.
You can think of the "entry point" as being akin to a user's account. By synchronizing that document between their devices, a user can get access to their documents from multiple browsers or devices, but be careful: until we implement some kind of security on the sync server, a user's privacy relies on their not sharing that "root" document ID with anyone else.
Here's the plan. We are going to:
1. Create a new "root" document which links to all the task lists the user has opened.
1. Create some UI to handle switching between several different task lists
2. Register task lists we open or create in that root document (if we don't have them already.)
3. Store the root document's ID in localStorage in a well-known key to check during startup.
4. Create a simple UI for copying the root document between browsers, creating rudimentary "accounts".
If this feels different to you from how a traditional database works, that's normal. With Automerge, building an application will feel more like linking together a web of documents than querying a database.
## Create a root document
Our root document is going to track all the task lists the user has created or opened. Let's add a type for it in `src/rootDoc.ts`:
```typescript file="src/rootDoc.ts"
import { type AutomergeUrl } from "@automerge/react";
export type RootDocument = {
taskLists: AutomergeUrl[];
};
```
It's just a list of `AutomergeUrl`s, each of which points to a document containing a task list.
Intially we'll create a new root document on every page load and we'll put the URL of the current task list in that root document. This will allow us to focus on the UI work but then we'll come back and add the logic to persist the root document.
Add this code to `src/main.tsx` to create the root document:
```tsx title="src/main.tsx"
// highlight-next-line
import { RootDocument } from "./rootDoc.ts"
// ..
// Add the repo to the global window object so it can be accessed in the browser console
// This is useful for debugging and testing purposes.
declare global {
interface Window {
repo: Repo;
// We also add the handle to the global window object for debugging
// highlight-next-line
handle: DocHandle
Automerge Task List
// highlight-start
{doc.taskLists.map((docUrl) => (
{title}
;
};
```
Then render the `DocumentList` in `App.tsx`
```tsx title="src/components/App.tsx"
// ..
// highlight-next-line
import { DocumentList } from "./DocumentList";
// ..
function App({ docUrl }: { docUrl: AutomergeUrl }) {
const [doc] = useDocument
Automerge Task List
{doc.taskLists.map((docUrl) => (
onSelectDocument(docUrl)}
// highlight-end
>
))}
{title}
;
};
```
Add the following code to `src/components/App.tsx`:
```tsx title="src/components/App.tsx"
// ..
// highlight-next-line
import { useState } from "react";
function App({ docUrl }: { docUrl: AutomergeUrl }) {
const [doc] = useDocument
Automerge Task List
// highlight-start
// highlight-next-line
{selectedDocUrl ?
{doc.taskLists.map((docUrl) => (
// highlight-next-line
onSelectDocument(docUrl)}
>
))}
{title}
;
};
```
Now if you load up the app you'll see a "+ Task List" button in the sidebar. Clicking this will create a new task list document, register it in the root document, and switch focus to the new task list.
## URL Management
This has all worked very well, but before we finish this section there's one deficiency we should address. The URL in the browser location hash does not update when we switch task list selection. This means that when we create a new task list, there's no way to share it with others. To fix this, we're going to slightly change how we handle the browser location hash.
At the moment, we look up the document URL from the location hash on boot, then we never change it. Now, we are going to manage the location hash as part of the application. To do this we will push responsibility for the URL hash management into the `App` component. This will allow us to update the URL hash whenever we switch task lists, making it easier to share task lists with others.
Here's how we'll do it:
* First, update the initialization logic to create an empty root document if it doesn't exist
* Add hash management to the `App` component using the `useHash` function from [`react-use`](https://www.npmjs.com/package/react-use)
First, let's update our initialization logic. Remove the lines highlighted in red in the following code snippet, and replace them with the single line `window.handle = repo.create({ taskLists: []})` that follows.
```tsx title="src/main.tsx"
// ..
// highlight-red-start
// Depending if we have an AutomergeUrl, either find or create the document
if (isValidAutomergeUrl(locationHash)) {
const taskList = await repo.find(locationHash);
window.handle = repo.create({ taskLists: [taskList.url] });
} else {
const taskList = repo.create
Automerge Task List
{selectedDocUrl ?
{showImportDialog && (
)}
);
};
```
Now let's add it to our footer in `src/components/App.tsx`:
```tsx
// ..
import { useHash } from "react-use";
// highlight-next-line
import { SyncControls } from "./SyncControls";
// ..
```
This component provides buttons to export the current root document ID to the clipboard and to import a new root document ID. When the user clicks "Copy account token", the current document URL is copied to the clipboard. When they click "Import account token", a dialog appears where they can paste a new root document ID. When the root document is pasted we update the stored root document ID and reload the page to reflect the changes.
### Checking it works
Open the application and create a few task lists, then click the "Copy account token" button. This will copy the root document ID to your clipboard. Now, open a new browser, or an incognito window, and paste the copied URL into the address bar. You should see the same task lists and items you created earlier.
# [Next Steps](https://automerge.org/docs/tutorial/next-steps/)
## Congratulations!
You've built a local-first, offline-capable app that supports multiplayer collaboration locally and over the network.
## Production Considerations
While the URL-based sharing mechanism we implemented is great for prototyping, there are some considerations for production applications:
1. **Security**: The public sync server is not secure for production use
2. **Reliability**: You should run your own sync server for production
3. **User Experience**: Consider implementing a more user-friendly sharing mechanism
4. **Performance**: Monitor document size and consider splitting large documents
5. **Error Handling**: Add proper error handling for network issues
## Further Learning
If you're hungry for more:
- Look at the [Cookbook](/docs/cookbook/modeling-data/) section for tips on how to model your app's data in Automerge
- Dive deeper into how Automerge [stores](/docs/reference/under-the-hood/storage/) and [merges](/docs/reference/under-the-hood/merge-rules/) documents in the 'Under the Hood' section
- Explore advanced features like:
- [Rich Text Editing](/docs/cookbook/rich-text-prosemirror-vanilla/)
- [Custom Network Adapters](/docs/reference/repositories/networking/)
- [Custom Storage Adapters](/docs/reference/repositories/storage/)
## Community
Join the [Automerge community](/community) to:
- Ask questions
- Show off your Automerge apps
- Connect with the Automerge team & community
- Get help with implementation
- Share your experiences
# [Migrating from Automerge 2 to Automerge 3](https://automerge.org/docs/guides/migrating-from-automerge-2-to-automerge-3/)
Automerge 3 is a major update to the Automerge library which enormously reduces memory usage and in many cases improves performance. It is however, almost entirely backwards compatible with Automerge 2. The main difference is that we have deprecated the old `Text` based API for collaborative string.
In Automerge 2 we had two APIs for collaborative text. In one API, we represented collaborative text using a class called `Text`, which had methods on it for mutating the string. We eventually deprecated this API and introduced a new API in which collaborative text was represented using a plain `string` and you use methods like `Automerge.splice` and `Automerge.updateText` to modify those strings. Accessing the new API was done by importing the `next` API of Automerge, like so:
```typescript
import { next as Automerge } from "@automerge/automerge"
```
In Automerge 3 the old `Text` API is no longer available. All collaborative strings are represented as `string`s. The `next` module is still available (to avoid breaking codebases which use it) but it just re-exports the `Automerge` module.
## Steps to upgrade
If you are not using the old `Text` API (you import Automerge using `import { next as Automerge } from "@automerge/automerge"`), then you can just change your imports to `import * as Automerge from "@automerge/automerge"` and you are done. You don't even _have_ to do this, but we recommend it as the `next` module is now deprecated.
If you _are_ using the old `Text` API then you will need to make the following changes:
* Anywhere your code expects to work with a `Text` class, update it to use a `string`.
* Anywhere your code expects to work with a `string`, update it to use an `ImmutableString`
### Updating `Text` to `string`
Here's an example. If your old code looks like this:
```typescript
import * as Automerge from "@automerge/automerge"
let doc = Automerge.from({
content: new Automerge.Text()
})
doc = Automerge.change(doc, d => {
d.content.insertAt(0, "Hello ")
})
```
You would change it to this:
```typescript
import * as Automerge from "@automerge/automerge"
let doc = Automerge.from({
content: ""
})
doc = Automerge.change(doc, d => {
Automerge.splice(d, ["content"], 0, 0, "Hello")
})
```
### Updating `string` to `ImmutableString`
If your old code looks like this:
```typescript
import * as Automerge from "@automerge/automerge"
let doc = Automerge.from({
content: "Hello world"
})
doc = Automerge.change(doc, d => {
d.content = "Goodbye world"
})
```
You would change it to this:
```typescript
import * as Automerge from "@automerge/automerge"
let doc = Automerge.from({
content: new Automerge.ImmutableString("Hello world")
})
doc = Automerge.change(doc, d => {
d.content = new Automerge.ImmutableString("Goodbye world")
})
```
# [Using Automerge with LLMs](https://automerge.org/docs/guides/using-automerge-with-llms/)
We support the [llms.txt](https://llmstxt.org/) standard for making documentation available to large language models.
We offer the following pages:
- [llms.txt](/llms.txt) — a listing of the available pages
- [llms-full.txt](/llms-full.txt) — complete documentation
# [Modeling Data](https://automerge.org/docs/cookbook/modeling-data/)
All data in Automerge must be stored in a document. A document can be modeled in a variety of ways, and there are many design patterns that can be used. An application could have many documents, typically identified by a UUID.
In this section, we will discuss how to model data within a particular document, including how to version and manage data with Automerge in production scenarios.
## How many documents?
You can decide which things to group together as one Automerge document (more fine grained or more coarse grained) based on what makes sense in your app. Having hundreds of docs should be fine — we've built prototypes of that scale. One major automerge project, [PushPin](https://github.com/automerge/pushpin), was built around very granular documents. This had a lot of benefits, but the overhead of syncing many thousands of documents was high.
We believe on the whole there's an art to the granularity of data that is universal. When should you have two JSON documents or two SQLite databases or two rows? We suspect that an Automerge document is best suited to being a unit of collaboration between two people or a small group.
## Setting up an initial document structure
When you create a document using `Automerge.init()`, it's just an empty JSON document with no properties. As the first change, most applications will need to initialize some empty collection objects that are expected to be present within the document.
The easiest way of doing this is with a call to `Automerge.change()` that sets up the document schema in the form that you need it, like in the code sample above. You can then sync this initial change to all of your devices; once everybody has the schema, you can have different users updating the document on different devices, and the updates should merge nicely. For example:
```js
// Set up the `cards` array in doc1
let doc1 = Automerge.change(Automerge.init(), (doc) => {
doc.cards = [];
});
// In doc2, don't create `cards` again! Instead, merge
// the schema initialization from doc1
let doc2 = Automerge.merge(Automerge.init(), doc1);
// Now we can update both documents
doc1 = Automerge.change(doc1, (doc) => {
doc.cards.push({ title: "card1" });
});
doc2 = Automerge.change(doc2, (doc) => {
doc.cards.push({ title: "card2" });
});
// The merged document will contain both cards
doc1 = Automerge.merge(doc1, doc2);
doc2 = Automerge.merge(doc2, doc1);
```
However, sometimes it's inconvenient to have to sync the initial change to a device before you can modify the document on that device. If you want two devices to be able to independently set up their own document schema, but still to be able to merge those documents, you have to be careful. Simply doing `Automerge.change()` on each device to initialize the schema **will not work**, because you now have two different documents with no shared ancestry (even if the initial change performs the same operations, each device has a different actorId and so the changes will be different).
If you really must initialize each device's copy of a document independently, one option is to do the initial `Automerge.change()` once to set up your schema, then call `Automerge.save()` on the document (which returns a byte array), and _hard-code that byte array into your application_. Now, on each device that needs to initialize a document, you do this:
```js
// hard-code the initial change here
const initChange = new Uint8Array([133, 111, 74, 131, ...])
let [doc] = Automerge.load(initChange)
```
This will set you up with a document whose initial change is the one you hard-coded. Any documents you set up with the same initial change will be able to merge.
## Versioning
Often, there comes a time in the production lifecycle where you will need to change the schema of a document. Because Automerge uses a JSON document model, it's similar to a NoSQL database, where properties can be arbitrarily removed and added at will.
You can implement your own versioning scheme, for example by embedding a schema version number into the document, and writing a function that can upgrade a document from one schema version to the next. However, doing this in a CRDT like Automerge is more difficult than migrations in a centralized relational database, because it could happen that two users independently perform the same migration. In this case, you need to ensure that the two migrations don't clash with each other, which is difficult.
One way of making migrations safe is by using the tricks from the previous section: in addition to hard-coding the initial change that sets up the document, you can also hard-code migrations that upgrade from one schema version to the next, using the same technique (either hard-coding the change as a byte array, or making a change on the fly with hard-coded actorId and timestamp). Do not modify the initial change; instead, every migration should be a separate hard-coded change that depends only on the preceding change. This way, you can have multiple devices independently applying the same migration, and they will all be compatible because the migration is performed identically on every device.
```js
type DocV1 = {
version: 1,
cards: Card[]
}
type DocV2 = {
version: 2,
title: Automerge.Text,
cards: Card[]
}
// This change creates the `title` property required in V2,
// and updates the `version` property from 1 to 2
const migrateV1toV2 = new Uint8Array([133, 111, 74, 131, ...])
let doc = getDocumentFromNetwork()
if (doc.version === 1) {
[doc] = Automerge.applyChange(doc, [migrateV1toV2])
}
```
Also keep in mind that in your app there might be some users using an old version of the app while other users are using a newer version; you will need to take care with migrations to ensure that they do not break compatibility with older app versions, or force all users to update to the latest version.
Some further ideas on safe schema migrations in CRDT apps are discussed in the [Cambria](https://www.inkandswitch.com/cambria) paper, but these are not yet implemented in Automerge. If you want to work on improving schema migrations in Automerge, please get in touch — contributions are welcome!
## Performance
Automerge documents hold their entire change histories. It is fairly performant, and can handle a significant amount of data in a single document's history. Performance depends very much on your workload, so we strongly suggest you do your own measurements with the type and quantity of data that you will have in your app.
Some developers have proposed “garbage collecting” large documents. If a document gets to a certain size, a central authority could emit a message to each peer that it would like to reduce it in size and only save the history from a specific change (hash). Martin Kleppmann did some experiments with a benchmark document to see how much space would be saved by discarding history, with and without preserving tombstones. See [this video at 55 minutes in](https://youtu.be/x7drE24geUw?t=3289). The savings are not all that great, which is why we haven't prioritised history truncation so far.
Typically, performance improvements can come at the networking level. You can set up a single connection (between peers or client-server) and sync many docs over a single connection. The basic idea is to tag each message with the ID of the document it belongs to. There are possible ways of optimising this if necessary. In general, having fewer documents that a client must load over the network or into memory at any given time will reduce the synchronization and startup time for your application.
# [Prosemirror + React + Automerge](https://automerge.org/docs/cookbook/rich-text-prosemirror-react/)
Automerge supports rich text editing on top of [ProseMirror](https://prosemirror.net/). This guide will show you how to set up a simple collaborative rich text editor in React using Automerge and ProseMirror.
All the code here can be found at https://github.com/automerge/automerge-prosemirror/tree/main/examples/react
First, create a an example vite app using the `@automerge/vite-app` template. This will give you a basic React app with the Automerge dependencies already installed.
```bash
yarn create @automerge/vite-app
```
Then install our prosemirror dependencies
```bash
yarn add @automerge/prosemirror prosemirror-example-setup prosemirror-model prosemirror-state prosemirror-view
```
Now, the app created by `@automerge/vite-app` creates a document which contains a `Counter`, but we want a `string` which will contain the text. Modify `main.tsx` so that the handle initialization logic looks like this:
```tsx title="src/main.tsx"
...
let handle
if (isValidAutomergeUrl(rootDocUrl)) {
handle = await repo.find(rootDocUrl)
} else {
handle = repo.create({text: ""})
}
...
```
First, let's create a basic skeleton component which just loads the document handle. The prosemirror bindings require that the document handle be loaded before we begin, so we'll add a bit of boilerplate to achieve this:
```tsx title="src/App.tsx"
import { AutomergeUrl } from "@automerge/automerge-repo"
import { useHandle } from "@automerge/automerge-repo-react-hooks"
import { useEffect, useState } from "react"
function App({ docUrl }: { docUrl: AutomergeUrl }) {
const handle = useHandle<{text: string}>(docUrl)
const [loaded, setLoaded] = useState(handle && handle.docSync() != null)
useEffect(() => {
if (handle != null) {
handle.whenReady().then(() => {
if (handle.docSync() != null) {
setLoaded(true)
}
})
}
}, [handle])
return
}
export default App
```
Now, we're going to create a ProseMirror editor. Prosemirror manages its own UI and state, it just needs to be attached to the DOM somehow. To achieve this we'll use the `useRef` hook to get hold of a reference to a dom element inside a React component which we can pass to prosemirror.
```tsx title="src/App.tsx"
import { AutomergeUrl } from "@automerge/automerge-repo"
import { useHandle } from "@automerge/automerge-repo-react-hooks"
// highlight-start
import { useEffect, useRef, useState } from "react"
import {EditorState} from "prosemirror-state"
import {EditorView} from "prosemirror-view"
import {exampleSetup} from "prosemirror-example-setup"
import { init } from "@automerge/prosemirror"
import "prosemirror-example-setup/style/style.css"
import "prosemirror-menu/style/menu.css"
import "prosemirror-view/style/prosemirror.css"
import "./App.css"
// highlight-end
function App({ docUrl }: { docUrl: AutomergeUrl }) {
const editorRoot = useRef
This documentation is mostly focused on the javascript implementation of automerge. Some things will translate to other languages but some things — in particular the "repository" concept and `automerge-repo` library — will not.
## Core concepts
Using automerge means storing your data in automerge [documents](#documents). Documents have [URL](#document-urls)s which you can use to share or request documents with/from other peers using a [repository](#repositories). Repositories give you [`DocHandle`](#dochandles)s which you use to make changes to the document and listen for changes from other peers.
Automerge as used in javascript applications is actually a composition of two libraries. [`automerge-repo`](https://www.npmjs.com/package/@automerge/automerge-repo) which provides the networking and storage plumbing, and [`automerge`](https://www.npmjs.com/package/@automerge/automerge) which provides the CRDT implementation, a transport agnostic [sync protocol](#sync-protocol), and a compressed [storage format](#storage-format) which `automerge-repo` uses to implement various networking and storage plugins.
### Documents
A document is the "unit of change" in automerge. It's like a combination of a JSON object and a git repository. What does that mean?
Like a JSON object, an automerge document is a map from strings to values, where the values can be maps, arrays, or simple types like strings or numbers. See the [data model](/docs/reference/documents/) section for more details.
Like a git repository, an automerge document has a history made up of commits. Every time you make a change to a document you are adding to the history of the document. The combination of this history and some rules about how to handle conflicts means that any two automerge documents can always be merged. See [merging](/docs/reference/under-the-hood/merge-rules) for the gory details.
### Repositories
A repository manages connections to remote peers and access to some kind of local storage. Typically you create a repository at application startup and then inject it into the parts of your application which need it. The repository gives out `DocHandle`s, which allow you to access the current state of a document and make changes to it without thinking about how to store those changes, transmit them to others, or fetch changes from others.
Networking and storage for a repository are pluggable. There are various ready-made network transports and storage implementations but it is also easy to build your own.
### DocHandles
A `DocHandle` is an object returned from the various methods on a repository which create or request a document. The `DocHandle` has methods on it to access the underlying automerge document and to create new changes which are stored locally and transmitted to connected peers.
### Document URLs
Documents in a repository have a URL. An automerge URL looks like this:
```
automerge:2akvofn6L1o4RMUEMQi7qzwRjKWZ
```
That is, a string of the form `automerge:
#### A Word of Warning
If you see obscure looking rust stack traces complaining about being unable to create random bytes while constructing a UUID then this is because you are trying to create a document (either a new one, or loading or forking one) outside of a handler. If you run the problematic code in a handler you should be fine.
### Deno
If your Deno instance allows access to the filesystem (the default for local development) then you can import Automerge from an npm specifier like so:
```typescript
import * as Am from "npm:@automerge/automerge";
```
However, if your Deno process doesn't have filesystem permission then you'll need to manually initialize the WebAssembly module. One way of doing that is:
```typescript
import { automergeWasmBase64 } from "npm:@automerge/automerge";
import * as Am from "npm:@automerge/automerge";
await Am.initializeBase64Wasm(automergeWasmBase64);
```
### Val.town
Val.town is a cloud-based Deno execution platform. Here's the text of a simple "val" which returns the contents of the documentId passed via the path.
```typescript
import { BrowserWebSocketClientAdapter } from "npm:@automerge/automerge-repo-network-websocket";
import { isValidAutomergeUrl, Repo } from "npm:@automerge/automerge-repo/slim";
/* set up Automerge's internal wasm guts manually */
import { automergeWasmBase64 } from "npm:@automerge/automerge/automerge.wasm.base64.js";
import * as automerge from "npm:@automerge/automerge/slim";
await automerge.initializeBase64Wasm(automergeWasmBase64);
/* This example will return the contents of a documentID passed in as the path as JSON. */
export default async function (req: Request): Promise
Automerge's npm module uses the [package exports](https://nodejs.org/api/packages.html#exports) feature, which means your environment will need to support that.
For example, React Native requires [configuring](https://reactnative.dev/blog/2023/06/21/package-exports-support) a `metro.config.js` to support package exports:
```js
const { getDefaultConfig } = require("expo/metro-config");
const config = getDefaultConfig(__dirname);
config.resolver && (config.resolver.unstable_enablePackageExports = true);
module.exports = config;
```
Once you've obtained the WebAssembly file you initialize it by passing it to either `initializeWasm` - which expects a WebAssembly module or a URL to fetch - or to `initializeBase64Wasm` which expects a base64 encoded string.
### Using the raw WebAssembly
Here's an example of using the raw WebAssembly in a Vite application. Here we can use the `?url` suffix on an import to obtain the URL to an asset.
```javascript
// Note the ?url suffix
import wasmUrl from "@automerge/automerge/automerge.wasm?url";
// Note the `/slim` suffixes
import * as Automerge from "@automerge/automerge/slim";
import { Repo } from "@automerge/automerge-repo/slim";
await Automerge.initializeWasm(wasmUrl)
// Now we can get on with our lives
const repo = new Repo({..})
```
### Using the base64 encoded WebAssembly
Here's an example of using the raw WebAssembly in an application where we can load JavaScript files but nothing else.
```javascript
import { automergeWasmBase64 } from "@automerge/automerge/automerge.wasm.base64.js";
// Note the `/slim` suffixes
import * as Automerge from "@automerge/automerge/slim";
import { Repo } from `@automerge/automerge-repo/slim`;
await Automerge.initializeBase64Wasm(automergeWasmBase64)
// Now we can get on with our lives
const repo = new Repo({..})
```
# [The JavaScript packages](https://automerge.org/docs/reference/the-js-packages/)
The javascript API has been through several iterations and is currently split over a few libraries. In greenfield applications, here's how the library is intended to be used:
Install both the `@automerge/automerge` and `@automerge/automerge-repo` packages. Then install the networking and storage plugins you need (typically `@automerge/automerge-repo-network-*` and `@automerge/automerge-repo-storage-*`) packages. Take a look at the cookbook for examples of different ways of using these.
## `@automerge/react` and `@automerge/vanillajs`
For React and vanilla JS applications we offer the [`@automerge/react`](https://www.npmjs.com/package/@automerge/react) and [`@automerge/vanillajs`](https://www.npmjs.com/package/@automerge/vanillajs) packages respectively. These packages re-export the public items from `automerge` and `automerge-repo` and several common network and storage adapters for your convenience.
## Relationship between `@automerge/automerge` and `@automerge/automerge-repo`
The core automerge libraries (both the original classic library and the WASM implementation) offer a compact storage format and a network agnostic sync protocol, but they don't actually do the work of wiring these things up to real storage engines (such as filesystems) or transports (such as websockets). `automerge-repo` implements all of this plumbing and is how we recommend using automerge going forward.
# [Document Data Model](https://automerge.org/docs/reference/documents/)
Automerge documents are quite similar to JSON objects. A document always consists of a root map which is a map from strings to other automerge values, which can themselves be composite types.
The types in automerge are:
- Composite types
- Maps
- [List](lists)
- [Text](text)
- Scalar (non-composite) types:
- IEEE 754 64 bit floating point numbers
- Unsigned integers
- Signed integers
- Booleans
- Strings
- Timestamps
- Counters
- Byte arrays
See [below](#javascript-language-mapping) for how these types map to JavaScript types.
## Maps
Maps have string keys and any automerge type as a value. "string" here means a unicode string. The underlying representation in automerge is as UTF-8 byte sequences but they are exposed as utf-16 strings in javascript.
## Lists
A list is an ordered sequence of automerge values. The underlying data structure is an RGA sequence, which means that concurrent insertions and deletions can be merged in a manner which attempts to preserve user intent.
## Text
Text is an implementation of the [peritext](https://www.inkandswitch.com/peritext/) CRDT. This is conceptually similar to a [list](#lists) where each element is a single unicode scalar value representing a single character. In addition to the characters `Text` also supports "marks". Marks are tuples of the form `(start, end, name, value)` which have the following meanings:
- `start` - the index of the beginning of the mark
- `end` - the index of the end of the mark
- `name` - the name of the mark
- `value` - any scalar (as in automerge scalar) value
For example, a bold mark from characters 1 to 5 might be represented as `(1, 5, "bold", true)`.
Note that the restriction to scalar values for the value of a mark will be lifted in future, although mark values will never be mutable - instead you should always create a new mark when updating a value. For now, if you need complex values in a mark you should serialize the value to a string.
## Timestamps
Timestamps are the integer number of milliseconds since the unix epoch (midnight 1970, UTC).
## Counter
Counters are a simple CRDT which just merges by adding all concurrent operations. They can be incremented and decremented.
## Javascript language mapping
The mapping to javascript is accomplished with the use of proxies. This means that in the javascript library maps appear as `object`s and lists appear as `Array`s. There is only one numeric type in javascript - `number` - so the javascript library guesses a bit. If you insert a javascript `number` for which [`Number.isInteger`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Number/isInteger) returns `true` then the number will be inserted as an integer, otherwise it will be a floating point value.
There are two representations for strings. Plain old javascript `string`s represent collaborative text. This means that you should modify these strings using `Automerge.splice` or `Automerge.updateText`, this will ensure that your changes merge well with concurrent changes. On the other hand, non-collaborative text is represented using `ImmutableString`, which you create using `new Automerge.ImmutableString`.
Timestamps are represented as javascript `Date`s.
Counters are represented as instances of the `Counter` class.
Putting it all together, here's an example of an automerge document containing all the value types:
```typescript
import * as A from "@automerge/automerge";
let doc = A.from({
map: {
key: "value",
nested_map: { key: "value" },
nested_list: [1],
},
list: ["a", "b", "c", { nested: "map" }, ["nested list"]],
text: "world",
raw_string: new A.ImmutableString("immutablestring"),
integer: 1,
float: 2.3,
boolean: true,
bytes: new Uint8Array([1, 2, 3]),
date: new Date(),
counter: new A.Counter(1),
none: null,
});
doc = A.change(doc, (d) => {
// Insert 'Hello' at the beginning of the string
A.splice(d, ["text"], 0, 0, "Hello ");
d.counter.increment(20);
d.map.key = "new value";
d.map.nested_map.key = "new nested value";
d.list[0] = "A";
d.list.insertAt(0, "Z");
d.list[4].nested = "MAP";
d.list[5][0] = "NESTED LIST";
});
console.log(doc);
// Prints
// {
// map: {
// key: 'new value',
// nested_map: { key: 'new nested value' },
// nested_list: [ 1 ]
// },
// list: [ 'Z', 'A', 'b', 'c', { nested: 'MAP' }, [ 'NESTED LIST' ] ],
// text: 'Hello world',
// raw_string: ImmutableString { val: 'ImmutableString' },
// integer: 1,
// float: 2.3,
// boolean: true,
// bytes: Uint8Array(3) [ 1, 2, 3 ],
// date: 2023-09-11T13:35:12.229Z,
// counter: Counter { value: 21 },
// none: null
// }
```
# [Simple Values](https://automerge.org/docs/reference/documents/values/)
All JSON primitive datatypes are supported in an Automerge document. In addition, JavaScript [Date objects](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date) are supported.
_Remember, never modify `currentDoc` directly, only ever change `doc` inside the callback to `Automerge.change`!_
```js
newDoc = Automerge.change(currentDoc, (doc) => {
doc.property = "value"; // assigns a string value to a property
doc["property"] = "value"; // equivalent to the previous line
delete doc["property"]; // removes a property
doc.stringValue = "value";
doc.numberValue = 1;
doc.boolValue = true;
doc.nullValue = null;
doc.nestedObject = {}; // creates a nested object
doc.nestedObject.property = "value";
// you can also assign an object that already has some properties
doc.otherObject = { key: "value", number: 42 };
// By default, strings are collaborative sequences of characters. There are
// cases where you want a string which is not collaborative - URLs for example
// should generally be updated in one go. In this case you can use `ImmutableString`,
// which does not allow concurrent updates.
doc.atomicStringValue = new Automerge.ImmutableString("")
});
```
# [Counters](https://automerge.org/docs/reference/documents/counters/)
If you have a numeric value that is only ever changed by adding or subtracting (e.g. counting how
many times the user has done a particular thing), you should use the `Automerge.Counter` datatype
instead of a plain number, because it deals with concurrent changes correctly.
> **Note:** Using the `Automerge.Counter` datatype is safer than changing a number value yourself
> using the `++` or `+= 1` operators. For example, suppose the value is currently **3**:
>
> - If two users increment it concurrently, they will both register **4** as the new value, whereas
> the two increments should result in a value of **5**.
> - If one user increments twice and the other user increments three times before the documents are
> merged, we will now have conflicting changes (**5** vs. **6**), rather > than the desired value
> of **8** (3 + 2 + 3).
To set up a `Counter`:
```js
state = Automerge.change(state, (doc) => {
// The counter is initialized to 0 by default. You can pass a number to the
// Automerge.Counter constructor if you want a different initial value.
doc.buttonClicks = new Automerge.Counter();
});
```
To get the current counter value, use `doc.buttonClicks.value`. Whenever you want to increase or
decrease the counter value, you can use the `.increment()` or `.decrement()` method:
```js
state = Automerge.change(state, (doc) => {
doc.buttonClicks.increment(); // Add 1 to counter value
doc.buttonClicks.increment(4); // Add 4 to counter value
doc.buttonClicks.decrement(3); // Subtract 3 from counter value
});
```
> **Note:** In relational databases it is common to use an auto-incrementing counter to generate
> primary keys for rows in a table, but this is not safe in Automerge, since several users may end
> up generating the same counter value! Instead it is best to use UUIDs to identify entities.
# [Lists](https://automerge.org/docs/reference/documents/lists/)
JavaScript Arrays are fully supported in Automerge. You can use `push`, `unshift`, `insertAt`, `deleteAt`, `splice`, loops, and nested objects.
```js
newDoc = Automerge.change(currentDoc, (doc) => {
doc.list = []; // creates an empty list object
doc.list.push(2, 3);
doc.list.unshift(0, 1); // unshift() adds elements at the beginning
doc.list[3] = Math.PI; // overwriting list element by index
// now doc.list is [0, 1, 2, 3.141592653589793]
// Looping over lists works as you'd expect:
for (let i = 0; i < doc.list.length; i++) doc.list[i] *= 2;
// now doc.list is [0, 2, 4, 6.283185307179586]
doc.list.splice(2, 2, "automerge");
// now doc.list is [0, 'hello', 'automerge', 4]
doc.list[4] = { key: "value" }; // objects can be nested inside lists as well
// Arrays in Automerge offer the convenience functions `insertAt` and `deleteAt`
doc.list.insertAt(1, "hello", "world"); // inserts elements at given index
doc.list.deleteAt(5); // deletes element at given index
// now doc.list is [0, 'hello', 'world', 2, 4]
});
```
If you have previously worked with immutable state in JavaScript, you might be in the habit of
using [idioms like these](https://redux.js.org/recipes/structuring-reducers/updating-normalized-data):
```js
state = Automerge.change(state, "Add card", (doc) => {
const newItem = { id: 123, title: "Rewrite everything in Rust", done: false };
doc.cards = {
ids: [...doc.cards.ids, newItem.id],
entities: { ...doc.cards.entities, [newItem.id]: newItem },
};
});
```
While this pattern works fine outside of Automerge, please **don't do this in Automerge**! Please
use mutable idioms to update the state instead, like this:
```js
state = Automerge.change(state, "Add card", (doc) => {
const newItem = { id: 123, title: "Rewrite everything in Rust", done: false };
doc.cards.ids.push(newItem.id);
doc.cards.entities[newItem.id] = newItem;
});
```
Even though you are using mutating APIs, Automerge ensures that the code above does not actually
mutate `state`, but returns a new copy of `state` in which the changes are reflected. The problem
with the first example is that from Automerge's point of view, you are replacing the entire
`doc.cards` object (and everything inside it) with a brand new object. Thus, if two users
concurrently update the document, Automerge will not be able to merge those changes (instead, you
will just get a conflict on the `doc.cards` property).
You can avoid this problem by making the changes at a fine-grained level: adding one
item to the array of IDs with `ids.push(newItem.id)`, and adding one item to the map of entities
with `entities[newItem.id] = newItem`. This code works much better, since it tells Automerge
exactly which changes you are making to the state, and this information allows Automerge to deal
much better with concurrent updates by different users.
As a general principle with Automerge, you should make state updates at the most fine-grained
level possible. Don't replace an entire object if you're only modifying one property of that
object; just assign that one property instead.
# [Text](https://automerge.org/docs/reference/documents/text/)
Automerge provides support for collaborative text editing. Under the hood, whenever you create a `string` in Automerge you are creating a collaborative text object which supports merging concurrent changes to the `string`.
If you want changes to a `string` to be collaborative, you should use `Automerge.splice` to modify the string.
```js
import * as Automerge from "@automerge/automerge"
let doc = Automerge.from({text: "hello world"})
// Fork the doc and make a change
let forked = Automerge.clone(doc)
forked = Automerge.change(forked, d => {
// Insert ' wonderful' at index 5, don't delete anything
Automerge.splice(d, ["text"], 5, 0, " wonderful")
})
// Make a concurrent change on the original document
doc = Automerge.change(doc, d => {
// Insert at the start, delete 5 characters (the "hello")
Automerge.splice(d, ["text"], 0, 5, "Greetings")
})
// Merge the changes
doc = Automerge.merge(doc, forked)
console.log(doc.text) // "Greetings wonderful world"
```
## Using `updateText` when you can't use `splice`
`splice` works in terms of low level input events, sometimes it's hard to get hold of these. For example, in a simple web form the `input` event is fired every time an `input` element changes, but the value of the event is the whole content of the text box. In this case you can use `Automerge.updateText`, which will figure out what has changed for you and convert the changes into `splice` operations internally.
Imagine you have a simple text box:
```html
```
Then with this HTML you can use `updateText` to make the text box collaborative:
```typescript
const handle: DocHandle<{text: string}> = ... // some how get a DocHandle
const input = document.getElementById("input")!
input.value = handle.docSync()!.text!
// On every keystroke use `updateText` to update the value of the text field
input.oninput = (e) => {
handle.change((doc) => {
// @ts-ignore
const newValue: string = e.target.value
console.log("newValue", newValue)
Automerge.updateText(doc, ["text"], newValue)
})
}
// Any time the document changes, update the value of the text field
handle.on("change", () => {
// @ts-ignore
input.value = handle.docSync()!.text!
})
```
`updateText` works best when you call it as frequently as possible. If the text has changed a lot between calls to `updateText` (for example if you were calling it in `onchange`) the diff will not merge well with concurrent changes. The best case is to call it after every keystroke.
# [Rich Text](https://automerge.org/docs/reference/documents/rich-text/)
As well as [supporting](../text) plain text Automerge supports rich text editing. The rich text APIs are extensions of the plain text API. In addition to using `splice` and `updateText` to modify a string, we also provide functions to manipulate two extra data types which are associated with a string:
* Marks: formatting spans which apply to a range of characters and can overlap
* Block markers which divide the text into blocks
## Marks
Marks represent things like bold or italic text, or inline elements such as hyperlinks. Every mark has a name - such as "bold" - and a value, which must be a primitive value such as a boolean or string.
When you create a mark you must decide how that mark will behave when characters are inserted at its boundaries. For example, bold marks typically expand when characters are inserted at the boundaries whilst a hyperlink normally wouldn't.
To create a mark, call `Automerge.mark` with the start and end of the range, the name of the mark, the value of the mark, and an `expand` option. You can obtain the set of active marks on a string by calling `Automerge.marks`.
```typescript
import * as Automerge from "@automerge/automerge"
let doc = Automerge.from({text: "hello world"})
Automerge.change(doc, d => {
Automerge.mark(d, ["text"], {start: 0, end: 5, expand: "both"}, "bold", true)
})
console.log(Automerge.marks(doc, ["text"]))
>> [ { name: 'bold', value: true, start: 0, end: 5 } ]
```
Here we can see that the bold span applies to the "hello".
It is up to your application to decide what different mark names mean, but if you are interested in interoperability consider adopting our [rich text schema](../../under-the-hood/rich-text-schema).
## Block Markers
Block markers are maps which are inserted inline in the text. They are used to divide text into structural roles such as paragraphs, headings, or code blocks. The underlying primitive of a block marker is very flexible, so specific editor integrations can use it however they like. The `automerge-prosemirror` bindings use the [rich text schema](../../under-the-hood/rich-text-schema).
Block markers can be created using `Automerge.splitBlock` and updated using `Automerge.updateBlock` and you can find the active block at a given index using `Automerge.block`.
## The Spans API
### Reading spans
Frequently working directly with block markers and spans is tedious. You can use `Automerge.spans` to retrieve a sequence of text spans grouped by their marks and interspersed with block markers. For example
```typescript
import * as Automerge from "@automerge/automerge"
let doc = Automerge.from({text: ""})
doc = Automerge.change(doc, d => {
// Insert an opening paragraph block
Automerge.splitBlock(d, ["text"], 0, {type: "paragraph", parents: []})
// Note that the block markers appear inline in the text and so to insert
// _after_ the block marker we need to insert at position 1
Automerge.splice(d, ["text"], 1, 0, "Hello")
// Insert another paragraph
Automerge.splitBlock(d, ["text"], 6, {type: "paragraph", parents: []})
Automerge.splice(d, ["text"], 7, 0, "world")
// Add a mark which covers the end of "hello" and the start of "world"
Automerge.mark(d, ["text"], {start: 4, end: 8, expand: "both"}, "bold", true)
})
console.log(Automerge.spans(doc, ["text"]))
```
And this outputs:
```javascript
[
{ type: 'block', value: { parents: [], type: 'paragraph' } },
{ type: 'text', value: 'Hel' },
{ type: 'text', value: 'lo', marks: { bold: true } },
{ type: 'block', value: { type: 'paragraph', parents: [] } },
{ type: 'text', value: 'w', marks: { bold: true } },
{ type: 'text', value: 'orld' }
]
```
Here you can see that the text has been broken up into sections with distinct spans and separated by block markers.
### Updating spans
When writing an editor integration it's often difficult to capture exactly what change has been made by the underlying editor you are integrating with. In these cases you can use `Automerge.updateSpans` to update the block structure of the text. This function takes a sequence of spans and block markers - just like that output by `Automege.spans` - and attempts to perform a minimal diff to update the text to the new structure.
One important note: `Automerge.updateSpans` does not yet update the formatting spans of the text, just the block structure. You will need to separately reconcile the formatting span changes.
For example, let's say we want to add a new paragraph marker in the string "hello world".
```javascript
import * as Automerge from "@automerge/automerge"
let doc = Automerge.from({text: "hello world"})
doc = Automerge.change(doc, d => {
Automerge.updateSpans(d, ["text"], [
{ type: "text", value: "hello" },
{ type: "block", value: { type: "paragraph", parents: [] } },
{ type: "text", value: "world" }
])
})
console.log(Automerge.spans(doc, ["text"]))
```
This will output:
```javascript
[
{ type: 'text', value: 'hello' },
{ type: 'block', value: { type: 'paragraph', parents: [] } },
{ type: 'text', value: 'world' }
]
```
`updateSpans` will try and perform minimal updates to block markers and text.
# [Conflicts](https://automerge.org/docs/reference/documents/conflicts/)
Automerge allows different nodes to independently make arbitrary changes to their respective copies
of a document. In most cases, those changes can be combined without any trouble. For example, if
users modify two different objects, or two different properties in the same object, then it is
straightforward to combine those changes.
## What is a Conflict?
If users concurrently insert or delete items in a list (or characters in a text document), Automerge
preserves all the insertions and deletions. If two users concurrently insert at the same position,
Automerge will ensure that on all nodes the inserted items are placed in the same order.
The only case Automerge cannot handle automatically, because there is no well-defined resolution, is
**when users concurrently update the same property in the same object** (or, similarly, the same
index in the same list). In this case, Automerge picks one of the concurrently written
values as the "winner", and it ensures that this winner is the same on all nodes:
```js
// Create two different documents
let doc1 = Automerge.change(Automerge.init(), (doc) => {
doc.x = 1;
});
let doc2 = Automerge.change(Automerge.init(), (doc) => {
doc.x = 2;
});
let doc1_merged_with_doc2 = Automerge.merge(doc1, doc2);
let doc2_merged_with_doc1 = Automerge.merge(doc2, doc1);
// Now, we can't tell which value x will assume in the new, merged docs --
// the choice is arbitrary, but deterministic and equal across both documents.
assert.deepEqual(doc1_merged_with_doc2, doc2_merged_with_doc1);
```
Although only one of the concurrently written values shows up in the object, the other values are
not lost. They are merely relegated to a conflicts object. Suppose `doc.x = 2` is chosen as the
"winning" value:
```js
doc1; // {x: 1}
doc2; // {x: 2}
doc1_merged_with_doc2; // {x: 2}
doc2_merged_with_doc1; // {x: 2}
Automerge.getConflicts(doc1_merged_with_doc2, "x"); // {'1@01234567': 1, '1@89abcdef': 2}
Automerge.getConflicts(doc2_merged_with_doc1, "x"); // {'1@01234567': 1, '1@89abcdef': 2}
```
Here, we've recorded a conflict on property `x`. The object returned by `getConflicts` contains the
conflicting values, both the "winner" and any "losers". You might use the information in the
conflicts object to show the conflict in the user interface. The keys in the conflicts object are
the internal IDs of the operations that updated the property `x`.
The next time you assign to a conflicting property, the conflict is automatically considered to be
resolved, and the conflict disappears from the object returned by `Automerge.getConflicts()`.
Automerge uses a combination of LWW (last writer wins) and multi-value register. By default, if you read from `doc.foo` you will get the LWW semantics, but you can also see the conflicts by calling `Automerge.getConflicts(doc, 'foo')` which has multi-value semantics.
Note that "last writer wins" here is based on the internal ID of the operation, not a wall clock time. The internal ID is a unique operation ID that is the combination of a counter and the actorId that generated it. Conflicts are ordered based on the counter first (using the actorId only to break ties when operations have the same counter value).
# [Repositories](https://automerge.org/docs/reference/repositories/)
`@automerge/automerge` provides a JSON-like CRDT and a sync protocol, but this still leaves a lot of plumbing to do to use it in an application. [`@automerge/automerge-repo`](https://www.npmjs.com/package/@automerge/automerge-repo) is that plumbing.
The entry point for an `automerge-repo` based application is to create a [`Repo`](https://automerge.org/automerge-repo/classes/_automerge_automerge-repo.Repo.html), passing it some form of [`StorageAdapter`](https://automerge.org/automerge-repo/classes/_automerge_automerge-repo.StorageAdapter.html) - which knows how to save data locally - and zero or more [`NetworkAdapter`](https://automerge.org/automerge-repo/classes/_automerge_automerge-repo.NetworkAdapter.html)s, which know how to talk to other peers running `automerge-repo`.
For example, this snippet creates a `Repo` which listens for websocket connections and stores data in the local file system:
```typescript
import { Repo } from "@automerge/automerge-repo";
import { WebSocketServer } from "ws";
import { NodeWSServerAdapter } from "@automerge/automerge-repo-network-websocket";
import { NodeFSStorageAdapter } from "@automerge/automerge-repo-storage-nodefs";
const wss = new WebSocketServer({ noServer: true });
const repo = new Repo({
network: [new NodeWSServerAdapter(wss)],
storage: new NodeFSStorageAdapter(dir),
});
```
A `Repo` is a little like a database. It allows you to create and request [`DocHandle`](https://automerge.org/automerge-repo/classes/_automerge_automerge-repo.DocHandle.html)s. Once you have a `DocHandle` you can make changes to it and listen for changes received from other peers.
```typescript
let doc = repo.create();
// Make a change ourselves and send that to everyone else
doc.change((d) => (d.text = "hello world"));
// Listen for changes from other peers
doc.on("change", ({ doc }) => {
console.log("new text is ", doc.text);
});
```
Any changes you make - or which are received from the network - will be stored in the attached storage adapter and distributed to other peers
# [Storage](https://automerge.org/docs/reference/repositories/storage/)
In `automerge-repo` "storage" refers to any implementation of [`StorageAdapter`](https://automerge.org/automerge-repo/classes/_automerge_automerge-repo.StorageAdapter.html). You _can_ run a `Repo` without a `StorageAdapter` but it will be entirely transient and will have to load all its data from remote peers on every restart.
`StorageAdapter` is designed to be safe to use concurrently, that is to say it is safe to have multiple `Repo`s talking to the same storage.
There are two built in storage adapters:
## IndexedDB
[`@automerge/automerge-repo-storage-indexeddb`](https://www.npmjs.com/package/@automerge/automerge-repo-storage-indexeddb) stores data in an [`IndexedDB`](https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API) in the browser.
```typescript
import { IndexedDBStorageAdapter } from "@automerge/automerge-repo-storage-indexeddb"
const storage = new IndexedDBStorageAdapter()
```
You can customize the object database and object store the storage uses, see [the docs](https://automerge.org/automerge-repo/classes/_automerge_automerge-repo-storage-indexeddb.IndexedDBStorageAdapter.html#constructor)
As noted above, this is safe for concurrent use so you can have multiple tabs pointing at the same storage. Note that they will not live update (you may want to use a [`MessageChannel`](https://automerge.org/automerge-repo/modules/_automerge_automerge-repo-network-messagechannel.html) or [`BroadcastChannel`](https://automerge.org/automerge-repo/modules/_automerge_automerge-repo-network-broadcastchannel.html) based `NetworkAdapter` for that) but on refresh the concurrent changes will be merged as per the normal merge rules.
## File system
[`@automerge/automerge-repo-storage-nodefs`](https://www.npmjs.com/package/@automerge/automerge-repo-storage-nodefs) is a `StorageAdapter` which stores its data in a directory on the local filesystem. The location can be customized as per [the docs](https://automerge.org/automerge-repo/classes/_automerge_automerge-repo-storage-nodefs.NodeFSStorageAdapter.html#constructor)
```typescript
import { NodeFSStorageAdapter } from "@automerge/automerge-repo-storage-nodefs";
const storage = new NodeFSStorageAdapter();
```
As with the `IndexedDB` adapter this adapter is safe for multiple processes to use the same data directory.
## Roll your own
`StorageAdapter` is designed to be easy to implement. It should be straightforward to build on top of any key/value store which supports range queries.
# [Networking](https://automerge.org/docs/reference/repositories/networking/)
There are many ways to talk to other peers. In `automerge-repo` this is captured by the [`NetworkAdapter`](https://automerge.org/automerge-repo/classes/_automerge_automerge-repo.NetworkAdapter.html) interface. Unlike `StorageAdapter`s a repository can have many (or zero) `NetworkAdapter`s.
"network" is quite a broad term in `automerge-repo`. It really means "any other instance of `Repo` which I am communicating with by message passing". This means that as well as network adapters for obvious things like websockets, we also implement network adapters for less traditional channels such as [`MessageChannel`](https://developer.mozilla.org/en-US/docs/Web/API/MessageChannel) or [`BroadcastChannel`](https://developer.mozilla.org/en-US/docs/Web/API/BroadcastChannel).
## Websockets
The websocket `NetworkAdapter` has two parts. This is because the websocket protocol requires a server and a client. The parts are named `WebSocketServerAdapter` and `WebSocketClientAdapter`, but don't take these names too seriously, they will both work in a browser or in Node.
### Server
The server side of the adapter is [`WebSocketServerAdapter`](https://automerge.org/automerge-repo/classes/_automerge_automerge-repo-network-websocket.WebSocketServerAdapter.html), which should be used in combination with the [`ws`](https://www.npmjs.com/package/ws) library.
```typescript
import { WebSocketServer } from "ws";
import { WebSocketServerAdapter } from "@automerge/automerge-repo-network-websocket";
const wss = new WebSocketServer({ port: 8080 });
const adapter = new WebSocketServerAdapter(wss);
```
#### Usage with `express`
Often you aren't running the websocket server as a standalone thing but instead as part of an existing HTTP server. Here's an example of such a situation in an `express` app.
```typescript
import { WebSocketServer } from "ws";
import { WebSocketServerAdapter } from "@automerge/automerge-repo-network-websocket";
import express from "express";
const wss = new WebSocketServer({ noServer: true });
const server = express();
server.on("upgrade", (request, socket, head) => {
wss.handleUpgrade(request, socket, head, (socket) => {
wss.emit("connection", socket, request);
});
});
const adapter = new WebSocketServerAdapter(wss);
server.listen(8080);
```
### Client
The client side of the connection is [`WebSocketClientAdapter`](https://automerge.org/automerge-repo/classes/_automerge_automerge-repo-network-websocket.WebSocketClientAdapter.html).
```typescript
import { WebSocketClientAdapter } from "@automerge/automerge-repo-network-websocket";
const network = new WebSocketClientAdapter("ws://localhost:3030");
```
## MessageChannel
[`@automerge/automerge-repo-network-messagechannel`](https://automerge.org/automerge-repo/modules/_automerge_automerge-repo-network-messagechannel.html) is a `NetworkAdapter` for communicating between processes within the same browser using a [`MessageChannel`](https://developer.mozilla.org/en-US/docs/Web/API/MessageChannel).
```typescript
import { MessageChannelNetworkAdapter } from "@automerge/automerge-repo-network-messagechannel";
import { Repo } from "@automerge/automerge-repo";
const { port1: leftToRight, port2: rightToLeft } = new MessageChannel();
const rightToLeft = new MessageChannelNetworkAdapter(rightToLeft);
const leftToRight = new MessageChannelNetworkAdapter(leftToRight);
const left = new Repo({
network: [leftToRight],
});
const right = new Repo({
network: [rightToLeft],
});
```
## BroadcastChannel
[`@automerge/automerge-repo-network-broadcastchannel`](https://automerge.org/automerge-repo/modules/_automerge_automerge-repo-network-broadcastchannel.html) is a `NetworkAdapter` for communicating between processes in the same browser using a [`BroadcastChannel`](https://developer.mozilla.org/en-US/docs/Web/API/BroadcastChannel). This will in general be quite inefficient because the sync protocol is point-to-point so even though `BroadcastChannel` is a _broadcast_ channel, we still have to duplicate each message for every peer in the channel. It's better to use `MessageChannel` if you can, but `BroadcastChannel` is good in a pinch.
```typescript
import { BroadcastChannelNetworkAdapter } from "@automerge/automerge-repo-network-broadcastchannel";
const network = new BroadcastChannelNetworkAdapter();
```
# [DocHandles](https://automerge.org/docs/reference/repositories/dochandles/)
Once you have a `Repo` with a `NetworkAdapter` and a `StorageAdapter` you can get down to the business of creating and working with [`DocHandle`](https://automerge.org/automerge-repo/classes/_automerge_automerge-repo.DocHandle.html)s.
It's useful to understand a little about why we need a `DocHandle`. `@automerge/automerge` documents are fairly inert data structures. You can create a document, you can mutate it, you can generate sync messages to send elsewhere and you can receive sync messages from elsewhere. None of this is very "live" though. Because the document has no concept of a network, or of storage, you can't say "every time I change a document, tell everyone else about it and save the change to storage". This "live document" is what a `DocHandle` is. A `DocHandle` is a wrapper around a document managed by a `Repo`. It provides the following kinds of "liveness":
- Whenever you change the document using [`DocHandle.change`](https://automerge.org/automerge-repo/classes/_automerge_automerge-repo.DocHandle.html#change) or [`DocHandle.changeAt`](https://automerge.org/automerge-repo/classes/_automerge_automerge-repo.DocHandle.html#changeat) the changes will be saved to the attached `StorageAdapter` and sent to any connected `NetworkAdapter`s
- Whenever a change is received from a connected peer the `DocHandle` will fire a "change" event
- There is a concept of an ephemeral message, which you can send using `DocHandle.broadcast`. Whenever a `DocHandle` receives an ephemeral message it will fire a `"ephemeral-message"` event
- You can wait for a `DocHandle` to be loaded, or to be retrieved from another peer
- `DocHandle`s have a `URL`, which can be used to uniquely refer to the document it wraps when requesting it from another peer
`DocHandle`s are very useful, how do you obtain one?
## Creating a `DocHandle`
This is the easy one, just call [`Repo.create`](https://automerge.org/automerge-repo/classes/_automerge_automerge-repo.Repo.html#create). This creates a new document, stores it, and then enqueues messages to all connected peers informing them of the new document.
## Waiting for a `DocHandle`
Typically you are _not_ creating a new document, but working with an existing one. Maybe the document URL was stored in `localStorage`, maybe the URL was in the hash fragment of the browser, etc. In this case you use [`Repo.find`](https://automerge.org/automerge-repo/classes/_automerge_automerge-repo.Repo.html#find) to lookup the document. This means the `DocHandle` can be in several different states, to understand this we'll first look at the states in detail, then some convenience methods `DocHandle` exposes for waiting for different states.
### `DocHandle` states
`Repo.find` will do two things simultaneously:
- Look in the attached `StorageAdapter` to see if we have any data for the document
- Send a request to any connected peers to ask if they have the document
These actions are asynchronous, as they complete the state of the document changes. This state is represented most explicitly in the [`HandleState`](https://automerge.org/automerge-repo/enums/_automerge_automerge_repo.HandleState.html) enum, which has the following states:
- `IDLE` - This is really just a start state, every dochandle immediately transitions to another state
- `AWAITING_NETWORK` - in this state we are waiting for the `NetworkAdapter`s to be ready to process messages. This typically occurs at application startup. Most `NetworkAdapter`s have an asynchronous startup period. The `Repo` waits until every `NetworkAdapter` has emitted a `ready` event before beginning to request documents
- `LOADING` - we are waiting for storage to finish trying to load this document
- `REQUESTING` - we are waiting to hear back from other peers about this document
- `READY` - The document is available, either we created it, found it in storage, or someone sent it to us
- `DELETED` - The document was removed from the repo
- `UNAVAILABLE` - We don't have the document in storage and none of our peers have the document either
The transitions between these states look like this:
{{# Note — I tried to add Mermaid support to the build system, but it was a rabbit hole, so for now here's the original markup for reference, followed by a pre-compiled SVG. Sorry!
stateDiagram-V2
direction LR
[*] --> LOADING: Repo.find
[*] --> READY: Repo.create
LOADING --> AWAITING_NETWORK
AWAITING_NETWORK --> REQUESTING: network ready
AWAITING_NETWORK --> DELETED
LOADING --> READY: found in storage
LOADING --> REQUESTING: not found in storage
LOADING --> UNAVAILABLE: no peers had the doc
LOADING --> DELETED
UNAVAILABLE --> READY: Received sync for this doc
UNAVAILABLE --> DELETED
REQUESTING --> READY: Received sync for this doc
READY --> DELETED
}}
Note that every state can transition to `DELETED`, either via `DocHandle.delete` or `Repo.delete`.
One other point to note is that a `DocHandle` can be unavailable because we didn't have it in storage and no peers responded to our request for it, but then another peer comes online and sends us sync messages for the document and so it transitions to `READY`.
You can check what state a handle is in using [`DocHandle.inState`](https://automerge.org/automerge-repo/classes/_automerge_automerge-repo.DocHandle.html#instate).
### Waiting for a handle to be ready
If all we care about is whether the document is ready then we can use a few different methods.
- `DocHandle.isReady()` is a synchronous method which will return `true` if the document is ready
- `DocHandle.whenReady()` is an asynchronous method that will return when the handle is ready
- `DocHandle.doc()` is an asynchronous method which will return the value of the document when it is ready
- `DocHandle.docSync()` is a synchronous method which returns the value of the document if it is ready. This method _will throw if the handle is not ready_. Therefore you should guard calls to `docSync` with calls to `isReady`
Once the document is ready the value of the document (either `DocHandle.doc()` or `DocHandle.docSync()`) will be `undefined` if the document was unavailable, but otherwise will be an automerge document.
# [Ephemeral Data](https://automerge.org/docs/reference/repositories/ephemeral/)
Automerge encourages you to persist most of your application state. Sometimes however there is state which it doesn't make any sense to persist. Good reasons to not persist state are if it changes extremely fast, or is only useful to the user in the context of a live "session" of some kind. One example of such data is cursor positions in collaboratively edited text. We refer to this kind of data as "ephemeral data".
Ephemeral data is associated with a particular document, which means you need to obtain a `DocHandle` for the document in question in order to send and receive ephemeral data. The rationale for this is that most of the time ephemeral data is related to a particular document. However, if you need to exchange ephemeral data which has no associated document you can always create a blank document and use that.
## Sending
```typescript
const handle = await repo.find("
It isn't important to understand this section to use automerge. You can just let automerge handle merging for you. But it may be interesting to understand.
How does automerge merge concurent changes? Well, let's think about what kinds of concurrent changes are possible. Automerge documents always carry their history with them, so the way to think about two concurrent versions of a document is as the set of changes since some common ancestor.
{{# Note — I tried to add Mermaid support to the build system, but it was a rabbit hole, so for now here's the original markup for reference, followed by a pre-compiled SVG. Sorry!
graph LR
A --> B
B --> C
C --> D
D --> E
C --> F
F --> G
}}
Here the common ancestor is `C` and the concurrent changes are `(D,E)` and `(F,G)`.
Automerge documents are composed of nested maps and lists or simple values or text sequences. We can describe the merge rules by describing the rules for maps, lists, text, and counters independently. In each case we describe how to merge two sets of concurrent changes we refer to as `A` and `B`.
## Map merge rules
- If `A` sets key $x$ to a value and `B` sets key $y$ to a value and $x \neq y$ then add both $x$ and $y$ to the merged map
- If `A` deletes key $x$ and `B` makes no change to $x$ then remove $x$ from the merged map
- If `A` deletes key $x$ and `B` sets $x$ to a new value then set the value of $x$ to the new value `B` set in the merged map
- If both `A` and `B` delete key $x$ then delete $x$ from the merged map
- If both `A` and `B` set the key $x$ to some value then randomly choose one value
Note that "randomly choose" means "choose one arbitrarily, but in such a way that all nodes agree on the chosen value".
## List merge rules
To understand the way lists merge you need to know a little about how the operations on lists are expressed. Every element in a list has an ID and operations on the list reference these IDs. When you update an index in a list (using `list[
Sometimes a block will reference a parent that doesn't exist in the list. When that happens - that parent is implicitly created with the defaults expected for it's block type. This ensures you can't accidentally remove a block's container by deleting the containing block or one of the block's siblings, since each block contains a minimal copy of the hierarchy needed to properly place it.
For example, the following sequence of block marks:
```ts
{ parents: ["blockquote"], type: "paragraph" }
{ parents: ["blockquote", "ordered-list-item"], type: "paragraph" }
{ parents: [], type: "paragraph" }
```
Will result in the following hierarchy:
```
blockquote:
- paragraph
- ordered-list-item:
- paragraph
paragraph
```
Note that the "blockquote" and "ordered-list-item" blocks are generated because they are parent's of the first two paragraphs, even though they aren't explicitly listed.
### Embeds
Blocks with `isEmbed: true` are blocks which are not part of the flow of text and represent some non-textual content such as an image. Embed block markers should _not_ break up the flow of text. I.e. the text following an `isEmbed: true` block marker belongs to the first non embed block preceding the embed block marker.
If an application encounters an unknown embed block it should render the block using some sort of generic UI and round trip the block through the editor. The editor **SHOULD** allow the user to delete the embedded block marker in some manner.
## Putting It All Together
When retrieving the current value of a rich text document via the [Spans API](../../documents/rich-text#the-spans-api), you will get an array of Spans with the following structure:
```typescript
{
type: "block",
value: {
type: string,
parents: string[],
attrs: Record