← Blog

How Sharing Works: A Log You Write and a Shadow Log They Write

Recently, we took another look at how sharing could work in LastDB. As we’ve been reworking multi-device sync, we realized that some of what we learned applies to sharing as well.

In multi-device sync, there is one mutation log for all the devices to write to, and then a snapshot. The snapshot lets us reclaim space and avoid keeping deletes in the mutation log. The primary device produces the snapshot.

We realized the same model can be applied to sharing. Say you want to share photos with other people. You publish a photo, and you don’t necessarily want other people to change the photo, but you might allow them to comment on it. What you have is a mutation log that you write to, and a snapshot, and everyone else writes to a shadow log.

The shadow log lets you read the mutations that they want to apply. Your code, running locally, then decides which ones are appropriate, applies them to your database, and publishes them as the publisher’s log. That way everyone can see them, or they can go into the snapshot when it’s snapshotting time.

This way you get to share your data and also restrict how other people can write to it.

What we like about this approach is that it can scale up really well, because the code and the rules can be arbitrarily defined by the code that’s running in your own node. You can make it as complex or as simple as you want.

There’s no cloud configuration. It’s just your local app reading the shadow log and deciding what to apply to your database.

The rest of this post is how that works in the code on main, and where it goes from here. One note on names: in the code the shadow log is called the inbox, and the log you publish is called the publisher log. This post says shadow log and publisher log. The Social app, which sets up sharing for you, gets its own post.

The two logs in the code

A personal database has one log. A shared database has exactly two logs in Exemem, under the same database ID, and their positions are independent.

  • The publisher log is the database. Only the publisher appends to it, and it holds only the publisher’s entries. Every active receiver reads it, and every receiver sees the same entries in the same order.
  • The shadow log is where receivers propose changes. Any active receiver can append to it. Only the publisher can read it.
PUBLISHER NODE signed member entries write scopes record rules DECIDES EXEMEM · CIPHERTEXT PUBLISHER LOG publisher appends everyone reads SHADOW LOG receivers append only the publisher reads (the inbox, in the code) RECEIVER NODES read the whole database propose changes, never write 3 4 1 2 1 a receiver proposes, encrypted for the publisher · 2 the publisher reads and decides 3 the publisher appends accepted changes · 4 every receiver reads the publisher log
Fig. 1 — One shared database. Receivers append proposals to the shadow log, and only the publisher reads them. Accepted changes go into the publisher log, which everyone reads.

There is no mixed log. A receiver’s proposal never lands in the publisher log. A contribution reaches readers only after the publisher commits it and appends that committed change to its own log. Someone who is not on the member list can do none of these things.

A shared database has exactly one publisher. If two people want to share something with the same group, each creates and shares their own database, and a member’s node holds both as separate received databases. A node can hold at most 32 received shares. LastDB itself has no idea what a group is. It knows a database and a list of node keys, and the Social app decides which list of people is linked to which databases. Data is encrypted on the node before it leaves, and Exemem stores ciphertext.

What a receiver does when it writes

A received database is read only on the receiver’s own node, and a normal write to it is refused. To change something, the receiver’s app posts a proposal to its local node. The node answers that the proposal was submitted and writes nothing to the receiver’s database. It encrypts the proposal for the publisher, signs the request with its node key, and appends it to the shadow log. Exemem checks that the sender is an active member and stores the encrypted proposal. It cannot read it. The operation in a proposal is at most 96 KiB, and a receiver cannot propose a purge or a file attachment.

The receiver’s app can ask for the status of its own proposal and show it as pending. The publisher supplies the outcome, published or discarded, when it finishes with the proposal. A receiver sees only the outcome of its own proposals.

What the publisher’s node decides

The publisher’s node downloads the shadow log, decrypts each proposal, and decides. The decision itself is a pure function of the sender’s grant, the proposal, and the current local version of the record. For each proposal the node:

  1. Reads the sender’s signed member entry and requires it to be active, with the contributor role and a write scope. A reader’s proposal, a revoked member’s proposal, and a proposal from a key with no entry are discarded.
  2. Checks that the schema the proposal writes is inside that member’s write scope. A scope is a list of schema names, a list of exact schema definitions, or all schemas. The definition named in the proposal must match the definition installed in the database.
  3. Applies the record rules, described below.
  4. Commits the change to its own copy of the database, then appends it to the publisher log under the publisher’s key, then marks the shadow log entry processed. A change that fails any check is discarded and never reaches the publisher log.

An accepted change is then an ordinary entry in the publisher log, and every receiver gets it when it syncs, including the one that proposed it. Because the check is on the publisher’s node, the receiver’s app does not enforce anything. Apps read the grants to decide what to offer, and a modified copy of an app that sends a write it was not granted gets it discarded.

The decision is made by code on the publisher’s node, not in the cloud. Today the node ships with a schema write scope for each person and a built-in rule for comments, described next. Those are the first rules, not the limit of the design.

Example: comments on shared photos

Say the publisher shares an album. Each photo is a record in a Photo schema, and each comment is a record in the standard social/Comment schema. A friend’s grant names only social/Comment as writable.

  • The friend proposes a new comment. It is inside the scope, and anyone with a comment grant may create one. It is accepted. The node fills in the comment’s author from the verified sender, so the sender cannot choose it.
  • The friend proposes a change to the photo. Photo is not in the scope, so it is discarded.
  • The friend proposes to edit their own comment, naming the version they are editing. The node keeps the original creator of each comment in a small file beside the download journal, not in the record, and it matches the sender. It is accepted. An edit that does not name a version is discarded.
  • A second friend proposes to overwrite the first friend’s comment. The creator does not match, so it is discarded. Deletes of comments from receivers are not accepted either.
  • The publisher can edit or delete any comment, because the publisher writes to its own database directly.

The rule that you can only replace your own comment is the first built-in record rule, and today it covers the one standard schema, social/Comment. For every other schema, the control that ships today is the write scope: whether a member may write that schema at all. A conditional replace of an existing record is also discarded if the record has changed since the receiver last saw it. On any create, LastDB records the verified sender as the author. Richer rules, such as ones an app writes for its own schemas, are the direction this goes, and they are not shipped yet.

The member list is one signed entry per member

The publisher signs one small entry for each member: the database, the member’s node key, a role (read or submit), a state (active or revoked), the schemas the member may write, a sequence number for that member, the publisher’s key, the time, and the signature. Exemem verifies the signature and stores the entry as that member’s own row. A new entry for a member is accepted only as the next sequence number for that member. A request from a member is checked against that one row, which is a single read however many members the database has, and the publisher’s node also reads one entry when it checks a proposal.

This replaced a single signed document that listed every member. That document was one stored item and one request body, it capped a database at about 1,500 members, and every change re-sent all of it. The target for the entry design is 10,000 members per database. We have not published a load test for that number.

  • A revoked member keeps its row with the state set to revoked. A revoked key comes back only when the publisher explicitly regrants it. The publisher’s node enforces that rule before it signs.
  • A row with no signed entry gives no access. Shared databases made before this change were not converted: the publisher adds each member again once.
  • Entries are signed, not encrypted. Exemem can see which node keys are in a database, their roles, and the schema names they may write. It cannot read records or proposals. The cloud holds who is a member. It does not decide which changes are applied.

What is not done

  • Revoking a member stops new reads and new proposals. It does not take back records the member already downloaded, and keys are not rotated in this version.
  • The publisher’s node has to be running for proposals to be decided. While it is off, proposals wait in the shadow log and no proposed change is published.
  • A whole-database share has one active publisher device. If a second device with the same publisher key publishes, the first reports that it cannot continue shared sync. Replicas and a handoff between devices need a design we have not shipped.
  • Rules beyond the schema scope and the comment rule are not built yet. Today there is no per-record ownership rule for schemas other than the standard comment schema.
  • A node can publish a snapshot of a shared database, and receivers can install one. The automatic log cutoff that reclaims space after a snapshot is only wired up for personal databases so far.
  • The Social app, which turns circles of people into these databases and grants, is not finished. It gets its own post.

Where this stands

The per-member entries are merged on main in lastdb (#1749 and #1751) and in the Exemem service, and the service change was deployed to DEV and PROD after it merged. The two-log shape, with no mixed-log path, is merged as lastdb #1768. The design is written up in the lastdb repository at docs/designs/shared-database-mutation-logs.md.

Related: We Deleted the Molecules · The LastDB App Registry · Blog index

← Blog