shifathosn
/
figcraft
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

M9: collaboration: per-user undo, rejections, reconnect (#105) 

* M9: collaboration: per-user undo, rejections, reconnect

Step 3 of docs/collab-protocol.md, the parts that don't need the transport:

- Per-user undo and redo (figcraft_collab::Client::{undo, redo}): each undoable batch is a step
  (values before/after by target, nodes created/deleted). Others' batches ordered after the step
  take its targets for good; creates invert to deletes (not when others built inside), deletes
  re-create the subtree. An undo or redo waits for every earlier batch and is remade from the
  confirmed state when it goes out; derived data follows as a follow-up batch.
- Rejections: refused batches leave the history (with an undo waiting on them) and queue a
  Notice; the engine shows it through Session::message (the UI's toast).
- Offline queue and reconnect: Hello.resume {session, client_id, seq}; the server keeps a
  bounded log and answers with the missed batches or a snapshot, plus what it did with the
  client's batches (processed {last_batch, refused}); queued batches then go out.
  Server::state / restore persist the session (client-id counter, log, session id).
- Presence carries editing_text {node, caret, selection}.
- Engine: documents in a live session (LiveDoc, Session::collab_*): commands, gestures and
  groups sync as one step each; edit.undo / edit.redo route to the per-user history; remote
  edits show under a gesture in progress. Outside a session snapshot undo is unchanged.
- Sim: dropped connections, undo/redo, and a checker that no undo reverts a change by someone
  else its author had seen. New tests in undo_tests, reconnect_tests, collab_live_tests,
  collab_undo_tests.
- CLAUDE.md / AGENTS.md: collab in the L2 layer list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* M9: collaboration wording in the progress docs after per-user undo and reconnect

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* M9: protocol doc: what the engine's live-session tests cover

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
BFlat 2026-10-11 03:19:24 +07:00 committed by GitHub
parent 4d7a9187ff
commit 94ff3d4df6
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
38 changed files with 2890 additions and 273 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
AGENTS.md
View File

@ -75,7 +75,7 @@ code.
incumbent's menu catalog in `crates/engine/src/catalog.rs` (the parity metric: `cargo xtask parity`).
- **Programmatic calls never open dialogs.** Commands only *request* UI (`Session::request_ui`); the
control channel drops dialog requests.
- **Layering** is enforced by `cargo xtask layers`: geom/color (L0) → doc (L1) → layout/text/pathops/plugin
- **Layering** is enforced by `cargo xtask layers`: geom/color (L0) → doc (L1) → layout/text/pathops/plugin/collab
(L2) → render/format (L3) → svg/pdf (L4) → engine (L5) → ui-egui/mcp (L6) → apps. Nothing below
L6 depends on egui/eframe/rfd/winit. **The UI crate is swappable.**
- **The UI is thin**: panels read engine state and act through `app.run(id, params)` /

2
CLAUDE.md
View File

@ -75,7 +75,7 @@ code.
incumbent's menu catalog in `crates/engine/src/catalog.rs` (the parity metric: `cargo xtask parity`).
- **Programmatic calls never open dialogs.** Commands only *request* UI (`Session::request_ui`); the
control channel drops dialog requests.
- **Layering** is enforced by `cargo xtask layers`: geom/color (L0) → doc (L1) → layout/text/pathops/plugin
- **Layering** is enforced by `cargo xtask layers`: geom/color (L0) → doc (L1) → layout/text/pathops/plugin/collab
(L2) → render/format (L3) → svg/pdf (L4) → engine (L5) → ui-egui/mcp (L6) → apps. Nothing below
L6 depends on egui/eframe/rfd/winit. **The UI crate is swappable.**
- **The UI is thin**: panels read engine state and act through `app.run(id, params)` /

2
README.md
View File

@ -166,7 +166,7 @@ New contributors and AI agents: start with [`AGENTS.md`](AGENTS.md), then [`docs
Installers for macOS (universal, signed and notarized), Windows (x64, x86 and arm64), Linux (AppImage, `.deb`, `.rpm`, Flatpak and tarball), FreeBSD and the web are attached to each [GitHub release](https://github.com/storytold/figcraft/releases). Maintainers: [`docs/releasing.md`](docs/releasing.md) explains how releases are built, signed and published.
> [!IMPORTANT]
> **Status:** FigCraft is in **alpha**: about 47% of the way to replacing Figma for real work, by our own estimate. Nearly all of Figma's main menu is wired ([`docs/parity-checklist.md`](docs/parity-checklist.md)) and the core editor works end to end, but FigCraft can't open `.fig` files yet, has only a first preview of multiplayer (live sessions hosted from the app on a local network; no per-person undo, sharing or permissions yet), runs its own WebAssembly plugins rather than Figma's, and hasn't been checked against real Figma documents. The honest, area-by-area picture is in the [roadmap](ROADMAP.md) and [`docs/target-app-parity.md`](docs/target-app-parity.md). Expect rough edges and please file issues, or tell us what broke on [Discord](https://discord.gg/artcraft).
> **Status:** FigCraft is in **alpha**: about 47% of the way to replacing Figma for real work, by our own estimate. Nearly all of Figma's main menu is wired ([`docs/parity-checklist.md`](docs/parity-checklist.md)) and the core editor works end to end, but FigCraft can't open `.fig` files yet, has only a first preview of multiplayer (live sessions hosted from the app on a local network, with per-person undo; no sharing or permissions yet), runs its own WebAssembly plugins rather than Figma's, and hasn't been checked against real Figma documents. The honest, area-by-area picture is in the [roadmap](ROADMAP.md) and [`docs/target-app-parity.md`](docs/target-app-parity.md). Expect rough edges and please file issues, or tell us what broke on [Discord](https://discord.gg/artcraft).
## The Crafting Apps

2
ROADMAP.md
View File

@ -123,7 +123,7 @@ Ranked; detail and milestones in [`docs/roadmap.md`](docs/roadmap.md).
2. Real-file corpus and measured fidelity: 15–30 h.
3. Tool-by-tool interaction pass against Figma: 40–60 h.
4. Localization layer, then ja, ko, es, pt-BR, fr, de: 35–55 h.
5. Multiplayer (M9): per-user undo, reconnect with offline edits, the browser client and sharing on top of the live sessions (transport, hosting, invite links, document sync and presence landed): 45–90 h.
5. Multiplayer (M9): the browser client and sharing on top of the live sessions (transport, hosting, invite links, document sync, presence, per-user undo and reconnect with offline edits landed): 45–90 h.
## Progress log

46
crates/collab-net/src/guest.rs
View File

@ -8,11 +8,12 @@
use std::io;
use std::net::{SocketAddr, TcpStream, ToSocketAddrs};
use std::sync::mpsc::{Receiver, RecvTimeoutError, Sender, channel};
use std::sync::{Arc, Mutex};
use std::thread::JoinHandle;
use std::time::{Duration, Instant};
use figcraft_collab::Client;
use figcraft_collab::message::{self, ClientMsg, Peer, Presence, ServerMsg};
use figcraft_collab::message::{self, ClientMsg, Peer, Presence, Resume, ServerMsg};
use figcraft_engine::live::{Event, Role, SessionInfo, Status};
use tungstenite::protocol::frame::coding::CloseCode;
use tungstenite::protocol::{CloseFrame, Role as WsRole, WebSocket};
@ -160,7 +161,13 @@ fn backoff(cfg: &GuestConfig, attempt: u32) -> Duration {
cfg.backoff.saturating_mul(1u32 << attempt.saturating_sub(1).min(16)).min(cfg.max_backoff)
}
/// Who this guest is in the session and the last sequence number it got (from the welcome and
/// the batches since): what a reconnect resumes with. The engine applies every message it got,
/// in order, before the next welcome, so its sync client is at that sequence number by then.
type Resuming = Arc<Mutex<Option<Resume>>>;
fn supervise(cfg: &GuestConfig, inbox: &Inbox, rx: &Receiver<Cmd>, me: &Sender<Cmd>) {
let resume: Resuming = Arc::new(Mutex::new(None));
let mut attempt = 0u32;
let mut was_live = false;
let mut pending: Option<Presence> = None;
@ -169,7 +176,7 @@ fn supervise(cfg: &GuestConfig, inbox: &Inbox, rx: &Receiver<Cmd>, me: &Sender<C
loop {
epoch += 1;
let outcome = match connect(cfg) {
Ok(c) => session(c, cfg, inbox, rx, me, epoch, &mut pending, &mut was_live, &mut attempt),
Ok(c) => session(c, cfg, inbox, rx, me, epoch, &resume, &mut pending, &mut was_live, &mut attempt),
Err(e) => Outcome::Retry(e),
};
match outcome {
@ -209,13 +216,14 @@ fn session(
rx: &Receiver<Cmd>,
me: &Sender<Cmd>,
epoch: u64,
resume: &Resuming,
pending: &mut Option<Presence>,
was_live: &mut bool,
attempt: &mut u32,
) -> Outcome {
let Conn { reader, mut writer, sock } = c;
let reader = {
let ctx = ReaderCtx { inbox: inbox.clone(), tx: me.clone(), epoch, cfg: cfg.clone() };
let ctx = ReaderCtx { inbox: inbox.clone(), tx: me.clone(), epoch, cfg: cfg.clone(), resume: resume.clone() };
match std::thread::Builder::new().name("live-guest-read".into()).spawn(move || read_loop(reader, &ctx)) {
Ok(t) => t,
Err(e) => {
@ -225,9 +233,12 @@ fn session(
}
};
let mut last_sent = Instant::now();
let hello = Client::hello(&cfg.invite.file, &cfg.invite.token, &cfg.name, "");
let mut hello = Client::hello(&cfg.invite.file, &cfg.invite.token, &cfg.name, "");
// Back after a drop: the same client, catching up from where it was.
if let (ClientMsg::Hello(h), Some(r)) = (&mut hello, resume.lock().ok().and_then(|r| *r)) {
h.resume = Some(r);
}
let mut welcomed = false;
let mut queued: Vec<ClientMsg> = Vec::new();
let mut throttle = Throttle::new(PRESENCE_INTERVAL);
let outcome = if send(&mut writer, &hello, &mut last_sent) {
loop {
@ -237,7 +248,9 @@ fn session(
let mut now_send = Vec::new();
match rx.recv_timeout(wake.saturating_duration_since(Instant::now())) {
Ok(Cmd::Send(m)) if welcomed => now_send.push(m),
Ok(Cmd::Send(m)) => queued.push(m),
// Made for the previous connection (the sync client sends again what the server
// didn't get once it has caught up with this connection's welcome).
Ok(Cmd::Send(_)) => {}
Ok(Cmd::Presence(p)) if welcomed => out = throttle.offer(p, Instant::now()),
Ok(Cmd::Presence(p)) => *pending = Some(p),
Ok(Cmd::Welcomed(e)) if e == epoch => {
@ -245,7 +258,6 @@ fn session(
*was_live = true;
*attempt = 0;
out = pending.take().and_then(|p| throttle.offer(p, Instant::now()));
now_send.append(&mut queued);
}
Ok(Cmd::Lost(e, o)) if e == epoch => break o,
Ok(Cmd::Leave) | Err(RecvTimeoutError::Disconnected) => {
@ -289,6 +301,7 @@ struct ReaderCtx {
tx: Sender<Cmd>,
epoch: u64,
cfg: GuestConfig,
resume: Resuming,
}
/// What the reader knows of the session (to tell the UI who's here).
@ -330,7 +343,10 @@ fn read_loop(mut ws: WebSocket<Duplex>, ctx: &ReaderCtx) {
/// Take in one message from the host; `Some` when the connection should end.
fn take(m: ServerMsg, ctx: &ReaderCtx, seen: &mut Seen) -> Option<Outcome> {
match m {
ServerMsg::Welcome { client_id, ref peers, .. } => {
ServerMsg::Welcome { session, client_id, seq, ref peers, .. } => {
if let Ok(mut r) = ctx.resume.lock() {
*r = Some(Resume { session, client_id, seq });
}
// The sync client opens the document (and fills in its name and pages).
let peers = peers.clone();
let inv = &ctx.cfg.invite;
@ -349,9 +365,11 @@ fn take(m: ServerMsg, ctx: &ReaderCtx, seen: &mut Seen) -> Option<Outcome> {
};
seen.peers = peers.clone();
seen.info = Some(info.clone());
// The writer goes live first: what the sync client sends in answer to this welcome
// must go out on this connection.
let _ = ctx.tx.send(Cmd::Welcomed(ctx.epoch));
ctx.inbox.push(Event::Welcome { info, peers });
ctx.inbox.push(Event::Sync(m));
let _ = ctx.tx.send(Cmd::Welcomed(ctx.epoch));
}
ServerMsg::PeerJoined(p) => match seen.info.as_mut() {
// The host tells you how you are shown.
@ -376,7 +394,15 @@ fn take(m: ServerMsg, ctx: &ReaderCtx, seen: &mut Seen) -> Option<Outcome> {
}
}
ServerMsg::Refused { reason } => return Some(Outcome::Refused(reason)),
ServerMsg::Changes { .. } | ServerMsg::Reject { .. } => ctx.inbox.push(Event::Sync(m)),
ServerMsg::Changes(ref acc) => {
if let Ok(mut r) = ctx.resume.lock()
&& let Some(r) = r.as_mut()
{
r.seq = acc.seq;
}
ctx.inbox.push(Event::Sync(m));
}
ServerMsg::Reject { .. } => ctx.inbox.push(Event::Sync(m)),
ServerMsg::Pong => {}
}
None

8
crates/collab-net/src/host.rs
View File

@ -124,6 +124,7 @@ impl Host {
name,
color: PEOPLE[0].to_string(),
client_version: env!("CARGO_PKG_VERSION").to_string(),
resume: None,
};
let (me, outs) = server.join(hello).map_err(|r| io::Error::other(format!("{r:?}")))?;
// The host's window syncs through its own client too: its welcome carries the document.
@ -524,6 +525,13 @@ impl Hub {
};
match self.server.join(hello) {
Ok((id, outs)) => {
// A guest back after a drop keeps its id: its old connection, if the hub hasn't
// noticed it's gone yet, is closed without the guest leaving.
if let Some(old) = self.ids.get(&id).copied().filter(|old| *old != conn)
&& let Some(c) = self.conns.remove(&old)
{
shutdown(&c.sock);
}
self.conns.insert(conn, Conn { out, sock, id });
self.ids.insert(id, conn);
self.deliver(outs);

14
crates/collab-net/src/loopback_tests.rs
View File

@ -187,7 +187,9 @@ fn guests_join_and_see_each_other() {
let Some(Event::Welcome { info, peers }) = ap.events.iter().rev().find(|e| matches!(e, Event::Welcome { .. })) else { panic!("no welcome") };
assert_eq!(info.name, "Ada");
assert!(
ap.events.iter().any(|e| matches!(e, Event::Sync(ServerMsg::Welcome { snapshot, .. }) if !snapshot.is_empty())),
ap.events.iter().any(
|e| matches!(e, Event::Sync(ServerMsg::Welcome { catch_up: figcraft_collab::CatchUp::Snapshot(snapshot), .. }) if !snapshot.is_empty())
),
"the welcome carries the document for the sync client"
);
assert_eq!(peers.iter().map(|p| p.name.as_str()).collect::<Vec<_>>(), ["Hana"], "the host is already here");
@ -496,10 +498,14 @@ fn guests_reconnect_after_a_network_failure() {
gp.wait("reconnecting", |e| e.iter().any(|e| matches!(e, Event::Status(Status::Reconnecting { attempt: 1 }))));
gp.wait("welcome again", |e| e.iter().filter(|e| matches!(e, Event::Welcome { .. })).count() >= 3);
let second = gp.me();
assert_ne!(first, second, "a new connection is a new client id");
hp.wait("the old one left and the new one joined", |e| {
assert_eq!(first, second, "a reconnect resumes as the same client");
assert!(
gp.events.iter().any(|e| matches!(e, Event::Sync(ServerMsg::Welcome { catch_up: figcraft_collab::CatchUp::Delta(_), .. }))),
"it catches up with what it missed, not the whole file"
);
hp.wait("it left and came back", |e| {
e.iter().any(|e| matches!(e, Event::PeerLeft(id) if *id == first))
&& e.iter().any(|e| matches!(e, Event::PeerJoined(p) if p.client_id == second))
&& e.iter().filter(|e| matches!(e, Event::PeerJoined(p) if p.client_id == second)).count() >= 2
});
// Presence flows again on the new connection.
g.presence(presence(9.0, 9.0));

2
crates/collab/Cargo.toml
View File

@ -3,7 +3,7 @@ name = "figcraft-collab"
version.workspace = true
edition.workspace = true
license.workspace = true
description = "FigCraft real-time collaboration core: the synced object/property model, change sets, fractional positions, server ordering and validation, client reconciliation and the wire messages (no networking)"
description = "FigCraft real-time collaboration core: the synced object/property model, change sets, fractional positions, server ordering and validation, client reconciliation, per-user undo, offline queueing and reconnect, and the wire messages (no networking)"
[lints]
workspace = true

72
crates/collab/src/change.rs
View File

@ -234,3 +234,75 @@ pub enum Entry {
/// An image by content hash (value: `{width, height, bytes}` with the encoded bytes in base64).
Image(String),
}
/// One synced value a change writes: a node property, a document-level property or a
/// document-level entry. Per-user undo tracks what a batch wrote by target.
#[derive(Clone, Debug, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub enum Target {
Node(NodeId, Prop),
Doc(DocProp),
Entry(Entry),
}
impl Target {
/// What `change` writes (`None` for creations and deletions, which change whole nodes).
pub fn of(change: &Change) -> Option<Target> {
match change {
Change::Set { id, prop, .. } => Some(Target::Node(*id, *prop)),
Change::SetDoc { prop, .. } => Some(Target::Doc(*prop)),
Change::Put { entry, .. } | Change::Remove { entry } => Some(Target::Entry(entry.clone())),
Change::Create { .. } | Change::Delete { .. } => None,
}
}
/// The change that writes `value` here (`None`: an entry that isn't there, or no value for a
/// property, which writes nothing).
pub fn write(&self, value: Option<&Json>) -> Option<Change> {
match (self, value) {
(Target::Node(id, prop), Some(v)) => Some(Change::Set { id: *id, prop: *prop, value: v.clone() }),
(Target::Doc(prop), Some(v)) => Some(Change::SetDoc { prop: *prop, value: v.clone() }),
(Target::Entry(entry), Some(v)) => Some(Change::Put { entry: entry.clone(), value: v.clone() }),
(Target::Entry(entry), None) => Some(Change::Remove { entry: entry.clone() }),
(Target::Node(..) | Target::Doc(_), None) => None,
}
}
}
/// A document-level list that syncs per entry and as a whole (see [`DocProp`]).
#[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord)]
pub(crate) enum Family {
Styles,
Comments,
Collections,
Variables,
}
impl DocProp {
/// The list this property is the whole of (`Ok`) or the order of (`Err`).
pub(crate) fn list(self) -> Option<Result<Family, Family>> {
Some(match self {
DocProp::Styles => Ok(Family::Styles),
DocProp::StyleOrder => Err(Family::Styles),
DocProp::Comments => Ok(Family::Comments),
DocProp::CommentOrder => Err(Family::Comments),
DocProp::Collections => Ok(Family::Collections),
DocProp::CollectionOrder => Err(Family::Collections),
DocProp::Variables => Ok(Family::Variables),
DocProp::VariableOrder => Err(Family::Variables),
_ => return None,
})
}
}
impl Entry {
/// The list this entry belongs to (dev snapshots and images aren't ordered lists).
pub(crate) fn list(&self) -> Option<Family> {
match self {
Entry::Style(_) => Some(Family::Styles),
Entry::Comment(_) => Some(Family::Comments),
Entry::Collection(_) => Some(Family::Collections),
Entry::Variable(_) => Some(Family::Variables),
Entry::DevSnapshot(_) | Entry::Image(_) => None,
}
}
}

542
crates/collab/src/client.rs
View File

@ -1,27 +1,45 @@
//! The client side: the confirmed server state, this client's unconfirmed batches on top of it,
//! and the optimistic view the user edits.
//! the optimistic view the user edits, the user's own undo history, and the queue of batches made
//! while the connection is down.
use std::collections::{BTreeMap, BTreeSet, VecDeque};
use figcraft_doc::{Document, NodeId};
use crate::change::{Change, Prop};
use crate::message::{ClientMsg, Hello, PROTOCOL, Peer, Presence, ServerMsg};
use crate::history::{BACK, BatchKind, FWD, Notice, Step, Touches};
use crate::message::{Accepted, CatchUp, ClientMsg, Hello, PROTOCOL, Peer, Presence, Processed, Resume, ServerMsg};
use crate::replica::{Mode, Replica};
use crate::{Error, Recompute, diff};
use crate::{Diff, Error, NoRecompute, Recompute, diff};
/// A batch sent and not yet confirmed.
/// The most undo steps a client keeps by default.
pub const HISTORY_LIMIT: usize = 1000;
/// A batch not yet confirmed or rejected: sent, or waiting to be sent.
#[derive(Clone, Debug)]
struct Pending {
batch_id: u64,
/// The server state it was made on (kept when it is sent again after a reconnect).
base_seq: u64,
changes: Vec<Change>,
kind: BatchKind,
/// The undo step this batch applies (its edit, undo or redo).
step: Option<u64>,
/// Someone else's batch was ordered before this one: its derived data may be stale.
rebased: bool,
/// Sent on the current connection. Batches go out in order; one waits (with every batch
/// after it) while the client is offline, and an undo or redo waits until every earlier batch
/// is confirmed or refused: it is then made again from the confirmed state (what it shows
/// meanwhile is a preview), so it never rests on an edit the server refused or on stale
/// values.
sent: bool,
}
/// One participant's sync state.
#[derive(Clone, Debug)]
pub struct Client {
/// The session joined (see [`ServerMsg::Welcome`]).
session: u64,
id: u32,
/// The server's state as of `seq`.
confirmed: Replica,
@ -34,6 +52,18 @@ pub struct Client {
next_id: NodeId,
peers: BTreeMap<u32, Peer>,
presence: BTreeMap<u32, Presence>,
/// Connected (batches go out); otherwise they wait for [`reconnect`](Self::reconnect).
online: bool,
undo: Vec<Step>,
redo: Vec<Step>,
next_step: u64,
history_limit: usize,
notices: Vec<Notice>,
}
/// The step with id `id` in either stack.
fn find_step<'a>(undo: &'a mut [Step], redo: &'a mut [Step], id: u64) -> Option<&'a mut Step> {
undo.iter_mut().chain(redo.iter_mut()).find(|s| s.id == id)
}
impl Client {
@ -46,18 +76,20 @@ impl Client {
name: name.to_string(),
color: color.to_string(),
client_version: env!("CARGO_PKG_VERSION").to_string(),
resume: None,
})
}
/// Start from the server's `Welcome`.
/// Start from the server's `Welcome` (which must carry a snapshot).
pub fn join(welcome: ServerMsg) -> Result<Client, Error> {
let ServerMsg::Welcome { client_id, seq, snapshot, peers } = welcome else {
return Err(Error::Protocol("expected a welcome".into()));
let ServerMsg::Welcome { session, client_id, seq, catch_up: CatchUp::Snapshot(snapshot), peers, .. } = welcome else {
return Err(Error::Protocol("expected a welcome with the document".into()));
};
let mut confirmed = Replica::from_snapshot(&snapshot)?;
confirmed.document_mut().set_client_id(client_id);
let next_id = confirmed.document().next_id();
Ok(Client {
session,
id: client_id,
view: confirmed.clone(),
confirmed,
@ -67,6 +99,12 @@ impl Client {
next_id,
peers: peers.into_iter().map(|p| (p.client_id, p)).collect(),
presence: BTreeMap::new(),
online: true,
undo: Vec::new(),
redo: Vec::new(),
next_step: 1,
history_limit: HISTORY_LIMIT,
notices: Vec::new(),
})
}
@ -96,11 +134,21 @@ impl Client {
self.seq
}
/// Batches sent but not yet confirmed or rejected.
/// Batches not yet confirmed or rejected (sent or waiting).
pub fn pending(&self) -> usize {
self.pending.len()
}
/// What this client's unconfirmed batch `batch_id` is for.
pub fn batch_kind(&self, batch_id: u64) -> Option<BatchKind> {
self.pending.iter().find(|p| p.batch_id == batch_id).map(|p| p.kind)
}
/// Connected: batches go out as they are made.
pub fn is_online(&self) -> bool {
self.online
}
/// The other people in the session.
pub fn peers(&self) -> impl Iterator<Item = &Peer> {
self.peers.values()
@ -111,24 +159,223 @@ impl Client {
self.presence.get(&id)
}
/// What happened that the user should hear about since the last call (rejected edits…).
pub fn take_notices(&mut self) -> Vec<Notice> {
std::mem::take(&mut self.notices)
}
/// The most undo steps kept (at least one).
pub fn set_history_limit(&mut self, limit: usize) {
self.history_limit = limit.max(1);
self.trim_history();
}
/// Undo steps, oldest first, by label.
pub fn undo_labels(&self) -> impl DoubleEndedIterator<Item = &str> {
self.undo.iter().map(|s| s.label.as_str())
}
/// Redo steps, the next one last, by label.
pub fn redo_labels(&self) -> impl DoubleEndedIterator<Item = &str> {
self.redo.iter().map(|s| s.label.as_str())
}
/// The user edited the document (`doc` is [`document`](Self::document) after a command): show
/// it at once and return the batch to send, if anything synced changed.
/// it at once and return the batch to send, if anything synced changed and it can go now
/// (otherwise it is queued: see [`Pending::sent`]). The edit is one undo step.
pub fn edit(&mut self, doc: &Document) -> Result<Option<ClientMsg>, Error> {
self.edit_with(doc, Some(""))
}
/// [`edit`](Self::edit) with the undo step's label (`None`: not undoable, e.g. a change no
/// command made; the redo steps stay).
pub fn edit_with(&mut self, doc: &Document, undo_label: Option<&str>) -> Result<Option<ClientMsg>, Error> {
let d = diff(&self.view, doc)?;
if d.changes.is_empty() {
return Ok(None);
}
let step = match undo_label {
Some(label) => {
let id = self.next_step;
self.next_step += 1;
Some(Step::new(id, label, self.next_batch, &self.view, &d.replica, &d.changes)?)
}
None => None,
};
self.view = d.replica;
self.keep_ids();
Ok(Some(self.queue(d.changes)))
let step_id = step.as_ref().map(|s| s.id);
if let Some(s) = step {
self.undo.push(s);
self.redo.clear();
self.trim_history();
}
self.queue(d.changes, BatchKind::Edit, step_id);
// Everything queued before this batch is waiting (or it would have gone already), so at
// most this edit goes now, and no undo becomes ready to go.
Ok(self.flush(&NoRecompute).pop())
}
/// Queue `changes` as a new batch (already shown in the view) and make its message.
fn queue(&mut self, changes: Vec<Change>) -> ClientMsg {
/// Undo the user's last step that still changes something: write back the values it replaced
/// where nobody else wrote since (their later writes win), delete what it created (unless
/// others put layers inside) and bring back what it deleted. Returns the batches to send now
/// (the undo, then any derived data `recompute` brings up to date as a follow-up); none when
/// there is nothing to undo, or when they have to wait (offline, or the step's own edit is
/// still unconfirmed).
pub fn undo(&mut self, recompute: &dyn Recompute) -> Result<Vec<ClientMsg>, Error> {
self.travel(BACK, recompute)
}
/// Redo the last undone step (the inverse of [`undo`](Self::undo)).
pub fn redo(&mut self, recompute: &dyn Recompute) -> Result<Vec<ClientMsg>, Error> {
self.travel(FWD, recompute)
}
/// How many undo and redo steps there are.
pub fn history_len(&self) -> (usize, usize) {
(self.undo.len(), self.redo.len())
}
/// Whether [`undo`](Self::undo) has a step to try.
pub fn can_undo(&self) -> bool {
!self.undo.is_empty()
}
/// Whether [`redo`](Self::redo) has a step to try.
pub fn can_redo(&self) -> bool {
!self.redo.is_empty()
}
fn travel(&mut self, side: usize, recompute: &dyn Recompute) -> Result<Vec<ClientMsg>, Error> {
loop {
let popped = if side == BACK { self.undo.pop() } else { self.redo.pop() };
let Some(mut step) = popped else { return Ok(Vec::new()) };
let changes = match step.toward(side, &self.view) {
Ok(c) => c,
Err(e) => {
// Put it back: nothing was done.
if side == BACK {
self.undo.push(step)
} else {
self.redo.push(step)
}
return Err(e);
}
};
if changes.is_empty() {
// Everything it changed was changed again by others (or is gone): skip it.
continue;
}
let before = self.view.clone();
let applied = self.view.apply(&changes, Mode::Lenient)?;
self.keep_ids();
step.batch = self.next_batch;
let kind = if side == BACK { BatchKind::Undo } else { BatchKind::Redo };
self.queue(changes, kind, Some(step.id));
// The derived data that follows, as its own batch (possibly empty: it is made again
// with the undo when that goes out).
let follow = self.recomputed(&self.view, before.document(), &applied.touched, recompute)?;
if !follow.changes.is_empty() {
self.view = follow.replica;
self.keep_ids();
}
self.queue(follow.changes, BatchKind::FollowUp, Some(step.id));
if side == BACK {
self.redo.push(step)
} else {
self.undo.push(step)
}
return Ok(self.flush(recompute));
}
}
fn trim_history(&mut self) {
let over = self.undo.len().saturating_sub(self.history_limit);
self.undo.drain(..over);
}
/// Queue `changes` as a new batch (already shown in the view).
fn queue(&mut self, changes: Vec<Change>, kind: BatchKind, step: Option<u64>) {
let batch_id = self.next_batch;
self.next_batch += 1;
self.pending.push_back(Pending { batch_id, changes: changes.clone(), rebased: false });
ClientMsg::Changes { base_seq: self.seq, batch_id, changes }
self.pending.push_back(Pending { batch_id, base_seq: self.seq, changes, kind, step, rebased: false, sent: false });
}
/// The waiting batches that can go now, in order (see [`Pending::sent`]).
fn flush(&mut self, recompute: &dyn Recompute) -> Vec<ClientMsg> {
let mut out = Vec::new();
if !self.online {
return out;
}
let mut i = 0;
while let Some(p) = self.pending.get(i) {
if p.sent {
i += 1;
continue;
}
if matches!(p.kind, BatchKind::Undo | BatchKind::Redo) {
if i > 0 {
break;
}
if !self.remake_front(recompute) {
// Nothing of it is left to do: it's gone; look at what came after it.
continue;
}
}
let Some(p) = self.pending.get_mut(i) else { break };
if p.changes.is_empty() {
// A follow-up with nothing to bring up to date.
self.pending.remove(i);
continue;
}
p.sent = true;
out.push(ClientMsg::Changes { base_seq: p.base_seq, batch_id: p.batch_id, changes: p.changes.clone() });
i += 1;
}
out
}
/// The undo or redo at the front of the queue goes out now: make it again from the confirmed
/// state (others' writes since it was previewed are left alone, nodes that no longer fit are
/// skipped), with its follow-up. False when nothing of it is left (it and its step are
/// dropped).
fn remake_front(&mut self, recompute: &dyn Recompute) -> bool {
let Some((batch, step_id, side)) = self.pending.front().map(|p| (p.batch_id, p.step, if p.kind == BatchKind::Undo { BACK } else { FWD }))
else {
return true;
};
let Some(step) = step_id.and_then(|id| find_step(&mut self.undo, &mut self.redo, id)).filter(|s| s.batch == batch) else {
// Its step left the history (the size limit): send it as previewed.
return true;
};
let Ok(changes) = step.toward(side, &self.confirmed) else { return true };
if changes.is_empty() {
let id = step.id;
self.pending.pop_front();
self.undo.retain(|s| s.id != id);
self.redo.retain(|s| s.id != id);
self.pending.retain(|q| q.step != Some(id) || q.sent);
self.rebuild();
return false;
}
step.send(&changes);
let mut after = self.confirmed.clone();
let follow = after
.apply(&changes, Mode::Lenient)
.and_then(|a| self.recomputed(&after, self.confirmed.document(), &a.touched, recompute))
.map(|d| d.changes)
.unwrap_or_default();
let seq = self.seq;
if let Some(p) = self.pending.front_mut() {
p.changes = changes;
p.base_seq = seq;
}
if let Some(f) = self.pending.get_mut(1).filter(|q| q.kind == BatchKind::FollowUp && q.step == step_id && !q.sent) {
f.changes = follow;
f.base_seq = seq;
}
self.rebuild();
true
}
/// Keep allocating in this client's space, after every id already handed out.
@ -143,46 +390,43 @@ impl Client {
self.next_id = doc.next_id();
}
/// Handle a message from the server. Returns messages to send: a follow-up batch when this
/// client's confirmed batch was ordered after someone else's and `recompute` changes derived
/// data over it. Errors mean the connection is unusable (rejoin).
/// The connection dropped: keep editing; batches wait for [`reconnect`](Self::reconnect).
pub fn disconnect(&mut self) {
self.online = false;
}
/// The first message of a new connection after [`disconnect`](Self::disconnect): `hello`
/// (as for the first join) with this client's session, id and last sequence number. The
/// server's `Welcome` then goes to [`receive`](Self::receive), which returns the batches to
/// send.
pub fn reconnect(&mut self, hello: Hello) -> ClientMsg {
self.online = false;
ClientMsg::Hello(Hello { resume: Some(self.resume()), ..hello })
}
/// What a reconnect resumes with: this client's session, id and last sequence number.
pub fn resume(&self) -> Resume {
Resume { session: self.session, client_id: self.id, seq: self.seq }
}
/// Handle a message from the server. Returns messages to send: batches that were waiting for
/// this one, a follow-up batch when this client's confirmed batch was ordered after someone
/// else's and `recompute` changes derived data over it, or after a reconnect's `Welcome`, the
/// queued batches. Errors mean the connection is unusable (rejoin).
pub fn receive(&mut self, msg: ServerMsg, recompute: &dyn Recompute) -> Result<Vec<ClientMsg>, Error> {
match msg {
ServerMsg::Changes { seq, author, batch_id, changes } => {
if seq != self.seq + 1 {
return Err(Error::Protocol(format!("sequence {seq} after {}", self.seq)));
}
if author == self.id {
let mine = self.pending.pop_front().ok_or_else(|| Error::Protocol("confirmation of an unknown batch".into()))?;
if mine.batch_id != batch_id {
return Err(Error::Protocol(format!("batch {batch_id} confirmed, expected {}", mine.batch_id)));
}
let before = self.confirmed.clone();
let applied = self.confirmed.apply(&changes, Mode::Strict)?;
self.seq = seq;
if mine.rebased {
return self.repair(&before, &applied.touched, recompute);
}
Ok(Vec::new())
} else {
self.confirmed.apply(&changes, Mode::Strict)?;
self.seq = seq;
for p in &mut self.pending {
p.rebased = true;
}
self.rebuild();
Ok(Vec::new())
}
ServerMsg::Changes(acc) => {
self.accepted(acc, recompute)?;
Ok(self.flush(recompute))
}
ServerMsg::Reject { batch_id, .. } => {
// The confirmed state is what the server has: drop the batch and replay the rest
// (the fixes are for clients that keep no confirmed copy).
let mine = self.pending.pop_front().ok_or_else(|| Error::Protocol("rejection of an unknown batch".into()))?;
if mine.batch_id != batch_id {
return Err(Error::Protocol(format!("batch {batch_id} rejected, expected {}", mine.batch_id)));
}
ServerMsg::Reject { batch_id, reason, fixes } => {
let mine = self.pop_sent(batch_id, "rejection")?;
// The confirmed copy already is the server's state (every batch ordered before
// this rejection arrived first); the fixes change nothing there unless it drifted.
let _ = self.confirmed.apply(&fixes, Mode::Lenient);
self.refused(&mine, reason);
self.rebuild();
Ok(Vec::new())
Ok(self.flush(recompute))
}
ServerMsg::Presence { client_id, presence } => {
self.presence.insert(client_id, presence);
@ -198,31 +442,206 @@ impl Client {
Ok(Vec::new())
}
ServerMsg::Pong => Ok(Vec::new()),
ServerMsg::Welcome { .. } => Err(Error::Protocol("already joined".into())),
ServerMsg::Welcome { .. } if self.online => Err(Error::Protocol("already joined".into())),
ServerMsg::Welcome { session, client_id, seq, catch_up, processed, peers } => {
if session != self.session || client_id != self.id {
return self.restart(ServerMsg::Welcome { session, client_id, seq, catch_up, processed, peers });
}
self.rejoin(seq, catch_up, &processed, peers, recompute)
}
ServerMsg::Refused { reason } => Err(Error::Protocol(reason)),
}
}
/// The oldest pending batch, which the server just confirmed or rejected as `batch_id`.
fn pop_sent(&mut self, batch_id: u64, what: &str) -> Result<Pending, Error> {
match self.pending.front() {
Some(p) if p.sent && p.batch_id == batch_id => {
self.pending.pop_front().ok_or_else(|| Error::Protocol(format!("{what} of an unknown batch")))
}
Some(p) => Err(Error::Protocol(format!("{what} of batch {batch_id}, expected {}", p.batch_id))),
None => Err(Error::Protocol(format!("{what} of an unknown batch"))),
}
}
/// A batch the server accepted (in order). Follow-ups it causes are queued.
fn accepted(&mut self, acc: Accepted, recompute: &dyn Recompute) -> Result<(), Error> {
if acc.seq != self.seq + 1 {
return Err(Error::Protocol(format!("sequence {} after {}", acc.seq, self.seq)));
}
if acc.author != self.id {
self.confirmed.apply(&acc.changes, Mode::Strict)?;
self.seq = acc.seq;
// Ordered after the steps the server confirmed already: those leave its targets alone.
let touches = Touches::of(&acc.changes);
for s in self.undo.iter_mut().chain(self.redo.iter_mut()) {
s.others_wrote(&touches);
}
for p in &mut self.pending {
p.rebased = true;
}
self.rebuild();
return Ok(());
}
let mine = self.pop_sent(acc.batch_id, "confirmation")?;
let before = self.confirmed.clone();
let applied = self.confirmed.apply(&acc.changes, Mode::Strict)?;
self.seq = acc.seq;
if mine.kind != BatchKind::FollowUp
&& let Some(s) = mine.step.and_then(|id| find_step(&mut self.undo, &mut self.redo, id))
{
// Later batches of the step haven't gone out yet (an undo waits for what came before).
s.settle(mine.kind);
s.refresh(&before, &self.confirmed, &acc.changes, mine.kind)?;
}
if mine.rebased {
self.repair(&before, &applied.touched, recompute)?;
}
Ok(())
}
/// Batch `p` was refused: its step leaves the undo history, with the batches of that step
/// waiting behind it (an undo of the refused edit), and the user hears why.
fn refused(&mut self, p: &Pending, reason: String) {
if p.kind == BatchKind::FollowUp {
return;
}
if let Some(id) = p.step {
self.undo.retain(|s| s.id != id);
self.redo.retain(|s| s.id != id);
// Those never went out: an undo waits for its edit.
self.pending.retain(|q| q.step != Some(id) || q.batch_id < p.batch_id || q.sent);
}
self.notices.push(Notice::Rejected { batch_id: p.batch_id, kind: p.kind, reason });
}
/// Back after a dropped connection: catch up (missed batches, or a snapshot when the server
/// no longer has them), settle the batches the server processed meanwhile, and return the
/// ones to send.
fn rejoin(
&mut self,
seq: u64,
catch_up: CatchUp,
processed: &Processed,
peers: Vec<Peer>,
recompute: &dyn Recompute,
) -> Result<Vec<ClientMsg>, Error> {
let last_batch = processed.last_batch;
// An undo or redo lost in flight is made again before it goes out, knowing what others did
// meanwhile: its step conflicts with what they wrote while it was in flight, and with what
// comes now, as if it had never been sent.
for p in self.pending.iter().filter(|p| p.sent && p.batch_id > last_batch && matches!(p.kind, BatchKind::Undo | BatchKind::Redo)) {
if let Some(s) = p.step.and_then(|id| find_step(&mut self.undo, &mut self.redo, id)) {
s.lost();
}
}
self.peers = peers.into_iter().map(|p| (p.client_id, p)).collect();
self.presence.clear();
let offline = || "it no longer applied when the connection came back".to_string();
match catch_up {
CatchUp::Delta(batches) => {
for acc in batches {
if acc.author == self.id {
// Batches the server processed before this one and didn't log were refused.
while let Some(p) = self.pending.front().filter(|p| p.batch_id < acc.batch_id).cloned() {
self.pending.pop_front();
self.refused(&p, offline());
}
}
self.accepted(acc, recompute)?;
}
}
CatchUp::Snapshot(snapshot) => {
let mut confirmed = Replica::from_snapshot(&snapshot)?;
confirmed.document_mut().set_client_id(self.id);
self.confirmed = confirmed;
self.seq = seq;
// What the server accepted meanwhile is in the snapshot. Which of the batches since
// were others' isn't known, so the steps check every value they wrote below.
while let Some(p) = self.pending.front().filter(|p| p.batch_id <= last_batch).cloned() {
self.pending.pop_front();
if processed.refused.contains(&p.batch_id) {
self.refused(&p, offline());
} else if p.kind != BatchKind::FollowUp
&& let Some(s) = p.step.and_then(|id| find_step(&mut self.undo, &mut self.redo, id))
{
s.settle(p.kind);
}
}
for s in self.undo.iter_mut().chain(self.redo.iter_mut()).filter(|s| s.confirmed) {
s.verify(&self.confirmed)?;
}
for p in &mut self.pending {
p.rebased = true;
}
}
}
if self.seq != seq {
return Err(Error::Protocol(format!("caught up to sequence {}, the server is at {seq}", self.seq)));
}
// Processed by the server but neither logged nor confirmed: refused.
while let Some(p) = self.pending.front().filter(|p| p.batch_id <= last_batch).cloned() {
self.pending.pop_front();
self.refused(&p, offline());
}
// The server never saw the rest: they go again, in order.
for p in &mut self.pending {
p.sent = false;
}
self.online = true;
self.rebuild();
Ok(self.flush(recompute))
}
/// A session that doesn't know this client (it restarted without its saved state) welcomed
/// it as new: start over from its document; unconfirmed work and the undo history are gone.
fn restart(&mut self, welcome: ServerMsg) -> Result<Vec<ClientMsg>, Error> {
let lost = self.pending.iter().filter(|p| p.kind != BatchKind::FollowUp).count();
let notices = std::mem::take(&mut self.notices);
let limit = self.history_limit;
*self = Client::join(welcome)?;
self.notices = notices;
self.history_limit = limit;
if lost > 0 {
self.notices.push(Notice::Lost { batches: lost });
}
Ok(Vec::new())
}
/// The view is the confirmed state with the pending batches replayed on top (changes that no
/// longer apply are skipped; the server will reject their batch).
/// longer apply are skipped; the server will reject their batch). A step whose batch is
/// replayed learns the values that batch now replaces (not from an undo or redo that waits
/// to go out: that is made again from the step when it goes).
fn rebuild(&mut self) {
let mut v = self.confirmed.clone();
for p in &self.pending {
let learns = p.kind == BatchKind::Edit || (p.sent && p.kind != BatchKind::FollowUp);
let step = p.step.filter(|_| learns).and_then(|id| find_step(&mut self.undo, &mut self.redo, id));
let pre = step.as_ref().map(|_| v.clone());
// Lenient application never fails.
let _ = v.apply(&p.changes, Mode::Lenient);
if let (Some(s), Some(pre)) = (step, pre) {
let _ = s.refresh(&pre, &v, &p.changes, p.kind);
}
}
self.view = v;
self.keep_ids();
}
/// This client's batch landed after someone else's: recompute derived data over what it
/// touched and send what changes (minus properties a later pending batch writes anyway).
fn repair(&mut self, before: &Replica, touched: &BTreeSet<NodeId>, recompute: &dyn Recompute) -> Result<Vec<ClientMsg>, Error> {
let mut work = self.confirmed.document().clone();
// Ids the recompute makes must not collide with those of later pending batches.
/// `base`'s document with derived data recomputed over `touched` (`before`: the state before
/// the edit that touched them), as a diff from `base`.
fn recomputed(&self, base: &Replica, before: &Document, touched: &BTreeSet<NodeId>, recompute: &dyn Recompute) -> Result<Diff, Error> {
let mut work = base.document().clone();
// Ids the recompute makes must not collide with those of pending batches.
work.reserve_ids(NodeId(self.next_id.0.saturating_sub(1)));
recompute.recompute(&mut work, before.document(), touched);
let fix = diff(&self.confirmed, &work)?;
recompute.recompute(&mut work, before, touched);
diff(base, &work)
}
/// This client's batch landed after someone else's: recompute derived data over what it
/// touched and queue what changes (minus properties a later pending batch writes anyway).
fn repair(&mut self, before: &Replica, touched: &BTreeSet<NodeId>, recompute: &dyn Recompute) -> Result<(), Error> {
let fix = self.recomputed(&self.confirmed, before.document(), touched, recompute)?;
let later: BTreeSet<(NodeId, Prop)> = self
.pending
.iter()
@ -235,10 +654,11 @@ impl Client {
let changes: Vec<Change> =
fix.changes.into_iter().filter(|c| !matches!(c, Change::Set { id, prop, .. } if later.contains(&(*id, *prop)))).collect();
if changes.is_empty() {
return Ok(Vec::new());
return Ok(());
}
let _ = self.view.apply(&changes, Mode::Lenient);
self.keep_ids();
Ok(vec![self.queue(changes)])
self.queue(changes, BatchKind::FollowUp, None);
Ok(())
}
}

27
crates/collab/src/codec_tests.rs
View File

@ -3,7 +3,10 @@
use figcraft_doc::{Document, NodeId, NodeKind};
use crate::change::{Change, Json, Prop};
use crate::message::{ClientMsg, Hello, PROTOCOL, Peer, Presence, ServerMsg, WIRE_VERSION, decode, decode_json, encode, encode_json};
use crate::message::{
Accepted, CatchUp, ClientMsg, EditingText, Hello, PROTOCOL, Peer, Presence, Processed, Resume, ServerMsg, WIRE_VERSION, decode, decode_json,
encode, encode_json,
};
use crate::sim::Rng;
use crate::{Error, Replica, diff};
@ -34,6 +37,7 @@ fn samples() -> (Vec<ClientMsg>, Vec<ServerMsg>) {
name: "Ada".into(),
color: "#FF8800".into(),
client_version: "0.2.0".into(),
resume: Some(Resume { session: 77, client_id: 4, seq: 10 }),
}),
moved_layer(),
cursor(),
@ -41,15 +45,30 @@ fn samples() -> (Vec<ClientMsg>, Vec<ServerMsg>) {
cursor: None,
selection: Some(vec![NodeId(2), NodeId(3)]),
viewport: Some((NodeId(1), 0.0, 0.0, 1200.0, 800.0)),
editing_text: Some(NodeId(7)),
editing_text: Some(Box::new(EditingText { node: NodeId(7), caret: 3, selection: Some((1, 3)) })),
}),
ClientMsg::Ack { seq: 9 },
ClientMsg::Ping,
];
let snapshot = Replica::new(Document::new("Snap")).snapshot().unwrap();
let server = vec![
ServerMsg::Welcome { client_id: 4, seq: 10, snapshot, peers: vec![peer.clone()] },
ServerMsg::Changes { seq: 11, author: 4, batch_id: 2, changes: vec![set.clone(), Change::Delete { id: NodeId(5) }] },
ServerMsg::Welcome {
session: 77,
client_id: 4,
seq: 10,
catch_up: CatchUp::Snapshot(snapshot),
processed: Processed::default(),
peers: vec![peer.clone()],
},
ServerMsg::Welcome {
session: 77,
client_id: 4,
seq: 11,
catch_up: CatchUp::Delta(vec![Accepted { seq: 11, author: 5, batch_id: 1, changes: vec![set.clone()] }]),
processed: Processed { last_batch: 7, refused: vec![3, 6] },
peers: vec![],
},
ServerMsg::Changes(Accepted { seq: 11, author: 4, batch_id: 2, changes: vec![set.clone(), Change::Delete { id: NodeId(5) }] }),
ServerMsg::Reject { batch_id: 3, reason: "no".into(), fixes: vec![set] },
ServerMsg::Presence { client_id: 4, presence: Presence::default() },
ServerMsg::PeerJoined(peer),

73
crates/collab/src/docprops.rs
View File

@ -10,7 +10,7 @@ use serde::de::DeserializeOwned;
use serde::{Deserialize, Serialize};
use crate::Error;
use crate::change::{Change, DocProp, Entry, Json};
use crate::change::{Change, DocProp, Entry, Family, Json};
/// An entry of a document-level list that has an id.
trait Keyed: Clone + PartialEq + Serialize + DeserializeOwned {
@ -306,25 +306,29 @@ pub(crate) fn apply(doc: &mut Document, change: &Change) -> Result<bool, Error>
Ok(false)
}
/// The current value of what `change` writes, as a change (for a rejected batch's fixes): the
/// property or entry as it is now, or the entry's removal. List properties revert whole.
pub(crate) fn current(doc: &Document, change: &Change) -> Result<Option<Change>, Error> {
let entry = match change {
Change::SetDoc { prop, .. } => {
if let Some(value) = get_scalar(doc, *prop)? {
return Ok(Some(Change::SetDoc { prop: *prop, value }));
}
let (prop, value) = match prop {
DocProp::Styles | DocProp::StyleOrder => (DocProp::Styles, Json::of(&doc.styles)?),
DocProp::Comments | DocProp::CommentOrder => (DocProp::Comments, Json::of(&doc.comments)?),
DocProp::Collections | DocProp::CollectionOrder => (DocProp::Collections, Json::of(&doc.variables.collections)?),
_ => (DocProp::Variables, Json::of(&doc.variables.variables)?),
};
return Ok(Some(Change::SetDoc { prop, value }));
}
Change::Put { entry, .. } | Change::Remove { entry } => entry,
_ => return Ok(None),
};
/// Document-level property `prop` of `doc` (lists whole; orders as the ids in order).
pub(crate) fn get(doc: &Document, prop: DocProp) -> Result<Json, Error> {
if let Some(v) = get_scalar(doc, prop)? {
return Ok(v);
}
fn order<T: Keyed>(list: &[T]) -> Result<Json, Error> {
Json::of(&list.iter().map(T::key).collect::<Vec<_>>())
}
match prop {
DocProp::Styles => Json::of(&doc.styles),
DocProp::StyleOrder => order(&doc.styles),
DocProp::Comments => Json::of(&doc.comments),
DocProp::CommentOrder => order(&doc.comments),
DocProp::Collections => Json::of(&doc.variables.collections),
DocProp::CollectionOrder => order(&doc.variables.collections),
DocProp::Variables => Json::of(&doc.variables.variables),
DocProp::VariableOrder => order(&doc.variables.variables),
_ => Err(Error::Invalid(format!("{prop:?} has no value"))),
}
}
/// Document-level entry `entry` of `doc` (`None`: there is none).
pub(crate) fn entry_value(doc: &Document, entry: &Entry) -> Result<Option<Json>, Error> {
let value = match entry {
Entry::Style(k) => doc.styles.iter().find(|s| s.id == *k).map(Json::of),
Entry::Comment(k) => doc.comments.iter().find(|c| c.id == *k).map(Json::of),
@ -333,8 +337,29 @@ pub(crate) fn current(doc: &Document, change: &Change) -> Result<Option<Change>,
Entry::DevSnapshot(k) => doc.dev_snapshots.get(k).map(|s| Json::of(s.as_ref())),
Entry::Image(k) => doc.images.get(k).map(image_json),
};
Ok(Some(match value {
Some(v) => Change::Put { entry: entry.clone(), value: v? },
None => Change::Remove { entry: entry.clone() },
}))
value.transpose()
}
/// The current value of what `change` writes, as a change (for a rejected batch's fixes): the
/// property or entry as it is now, or the entry's removal. List properties revert whole.
pub(crate) fn current(doc: &Document, change: &Change) -> Result<Option<Change>, Error> {
match change {
Change::SetDoc { prop, .. } => {
// An order reverts as the whole list (the entries may have changed too).
let prop = match prop.list() {
Some(Ok(_)) | None => *prop,
Some(Err(l)) => match l {
Family::Styles => DocProp::Styles,
Family::Comments => DocProp::Comments,
Family::Collections => DocProp::Collections,
Family::Variables => DocProp::Variables,
},
};
Ok(Some(Change::SetDoc { prop, value: get(doc, prop)? }))
}
Change::Put { entry, .. } | Change::Remove { entry } => {
Ok(crate::change::Target::Entry(entry.clone()).write(entry_value(doc, entry)?.as_ref()))
}
_ => Ok(None),
}
}

428
crates/collab/src/history.rs Normal file
View File

@ -0,0 +1,428 @@
//! Per-user undo and redo in a live session. Every undoable batch of this client becomes a
//! [`Step`]: the values it wrote and the values they replaced, by [`Target`], plus the nodes it
//! created or deleted. Undoing a step writes the replaced values back, but only where nobody else
//! wrote after the step (the server ordered someone else's batch after it); redo goes the other
//! way. Batches from others never become steps.
use std::collections::{BTreeMap, BTreeSet};
use figcraft_doc::NodeId;
use crate::Error;
use crate::change::{Change, DocProp, Entry, Family, Json, Prop, Target};
use crate::replica::{Mode, Replica, create_change};
/// Index of the state before a step in its two-sided records.
pub(crate) const BACK: usize = 0;
/// Index of the state after a step.
pub(crate) const FWD: usize = 1;
/// What a batch of this client is for.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum BatchKind {
/// An edit (one command or gesture).
Edit,
/// Undoing an edit.
Undo,
/// Redoing an undone edit.
Redo,
/// Derived data brought up to date after other batches (never in the undo history).
FollowUp,
}
/// Something to tell the user (a short toast).
#[derive(Clone, Debug, PartialEq)]
pub enum Notice {
/// The server refused one of this client's batches: what it changed is rolled back and it
/// leaves the undo history.
Rejected { batch_id: u64, kind: BatchKind, reason: String },
/// The session restarted while this client was away: its unconfirmed batches are gone.
Lost { batches: usize },
}
impl Notice {
/// The text to show.
pub fn message(&self) -> String {
match self {
Notice::Rejected { kind: BatchKind::Undo, reason, .. } => format!("Couldn't undo: {reason}"),
Notice::Rejected { kind: BatchKind::Redo, reason, .. } => format!("Couldn't redo: {reason}"),
Notice::Rejected { reason, .. } => format!("Your change couldn't be applied: {reason}"),
Notice::Lost { batches } => {
format!("The live session restarted: {batches} unsent change{} couldn't be kept", if *batches == 1 { "" } else { "s" })
}
}
}
}
/// The two sides of a value: before and after the step.
type Sides<T> = [T; 2];
/// What a batch wrote, by target.
type Writes = BTreeMap<Target, Sides<Option<Json>>>;
/// Nodes that exist on one side only: the change creating them as they are there.
type Nodes = BTreeMap<NodeId, Sides<Option<Change>>>;
/// One entry of the undo history.
#[derive(Clone, Debug)]
pub(crate) struct Step {
/// Unique within the client (batches name their step by it).
pub id: u64,
pub label: String,
/// The batch that last applied this step: the edit, or its latest undo or redo (which may
/// still wait to be sent).
pub batch: u64,
/// The server confirmed the step's last batch that was sent, and nothing of it is in flight:
/// batches from others ordered after it conflict with it.
pub confirmed: bool,
/// That confirmed batch went towards the after side (the edit or a redo; false: an undo).
pub settled_forward: bool,
/// While an undo or redo of the step is in flight: what it writes, and what others wrote
/// meanwhile (ordered before it, unless the connection drops and it is made again).
in_flight: Option<(BTreeSet<Target>, Touches)>,
writes: Writes,
nodes: Nodes,
/// Targets others wrote after this step: it never writes them again.
taken: BTreeSet<Target>,
/// Nodes others created again after this step (by their undo): it never deletes them.
taken_nodes: BTreeSet<NodeId>,
}
/// What `changes` (a batch that turned `pre` into `post`) wrote, oriented `[pre, post]`; with
/// `same`, also the targets it wrote that kept their value.
fn capture(pre: &Replica, post: &Replica, changes: &[Change], same: bool) -> Result<(Writes, Nodes), Error> {
let (a, b) = (pre.document(), post.document());
let mut writes = Writes::new();
let mut nodes = Nodes::new();
for c in changes {
match c {
Change::Create { id, .. } => {
if let Some(n) = b.node(*id)
&& !a.contains(*id)
{
nodes.insert(*id, [None, Some(create_change(n, post.key(*id))?)]);
}
}
Change::Delete { id } => {
for d in a.subtree(*id) {
if let Some(n) = a.node(d)
&& !b.contains(d)
{
nodes.insert(d, [Some(create_change(n, pre.key(d))?), None]);
}
}
}
other => {
let Some(t) = Target::of(other) else { continue };
if writes.contains_key(&t) {
continue;
}
let (x, y) = (pre.value(&t)?, post.value(&t)?);
// A node created or deleted by the batch is recorded whole above.
let missing = matches!(t, Target::Node(..)) && (x.is_none() || y.is_none());
if !missing && (same || x != y) {
writes.insert(t, [x, y]);
}
}
}
}
Ok((writes, nodes))
}
impl Step {
/// The step for batch `batch` that turned `pre` into `post` with `changes`.
pub fn new(id: u64, label: &str, batch: u64, pre: &Replica, post: &Replica, changes: &[Change]) -> Result<Step, Error> {
let (writes, nodes) = capture(pre, post, changes, false)?;
Ok(Step {
id,
label: label.to_string(),
batch,
confirmed: false,
settled_forward: true,
writes,
nodes,
taken: BTreeSet::new(),
taken_nodes: BTreeSet::new(),
in_flight: None,
})
}
/// A batch of the step (its edit, an undo or a redo: `kind`) was applied again, now on top of
/// `pre` (giving `post`): what it replaced is now taken from `pre` (others' batches were
/// ordered before it, or an earlier batch of this client was refused) and what it wrote from
/// `post`. The edit's nodes become those it now creates and deletes (a layer someone put
/// inside a layer it deletes comes back with an undo); an undo or redo only updates the nodes
/// of the step (what it takes with it beyond them was someone else's).
pub fn refresh(&mut self, pre: &Replica, post: &Replica, changes: &[Change], kind: BatchKind) -> Result<(), Error> {
let (writes, nodes) = capture(pre, post, changes, true)?;
let (p, q) = if kind == BatchKind::Undo { (FWD, BACK) } else { (BACK, FWD) };
for (t, [x, y]) in writes {
if !self.taken.contains(&t)
&& let Some(s) = self.writes.get_mut(&t)
{
s[p] = x;
s[q] = y;
}
}
let oriented = nodes.into_iter().map(|(id, [x, y])| {
let mut s = [None, None];
s[p] = x;
s[q] = y;
(id, s)
});
if kind == BatchKind::Edit {
self.nodes = oriented.collect();
} else {
for (id, s) in oriented {
if let Some(e) = self.nodes.get_mut(&id) {
*e = s;
}
}
}
Ok(())
}
/// An undo or redo of the step (`changes`) goes out: until the server confirms it, others'
/// batches are ordered before it.
pub fn send(&mut self, changes: &[Change]) {
self.confirmed = false;
self.in_flight = Some((changes.iter().filter_map(Target::of).collect(), Touches::default()));
}
/// The server confirmed the step's batch of `kind`.
pub fn settle(&mut self, kind: BatchKind) {
self.confirmed = true;
self.in_flight = None;
self.settled_forward = kind != BatchKind::Undo;
}
/// Its undo or redo in flight was lost with the connection: it will be made again, so what
/// others wrote meanwhile came after the step's last confirmed batch.
pub fn lost(&mut self) {
self.confirmed = true;
if let Some((_, seen)) = self.in_flight.take() {
self.taint(&seen, &BTreeSet::new());
}
}
/// Someone else's batch (`touches`) was ordered after the step's last batch, or before its
/// batch in flight: the targets it writes are theirs from now on, except those the batch in
/// flight writes again after them.
pub fn others_wrote(&mut self, touches: &Touches) {
if self.confirmed {
self.taint(touches, &BTreeSet::new());
} else if let Some((mine, seen)) = self.in_flight.take() {
self.taint(touches, &mine);
let mut seen = seen;
seen.absorb(touches);
self.in_flight = Some((mine, seen));
}
}
/// Leave the targets `touches` writes (but those in `except`) alone from now on.
fn taint(&mut self, touches: &Touches, except: &BTreeSet<Target>) {
for t in self.writes.keys() {
if touches.hits(t) && !except.contains(t) {
self.taken.insert(t.clone());
}
}
for id in self.nodes.keys() {
if touches.nodes.contains(id) {
self.taken_nodes.insert(*id);
}
}
}
/// The batches since this step are unknown (a reconnect brought a snapshot): every target
/// whose value in `state` isn't the one the step's confirmed batch wrote was changed by others.
pub fn verify(&mut self, state: &Replica) -> Result<(), Error> {
let side = if self.settled_forward { FWD } else { BACK };
for (t, v) in &self.writes {
if !self.taken.contains(t) && state.value(t)? != v[side] {
self.taken.insert(t.clone());
}
}
Ok(())
}
/// The batch that takes `view` to this step's `side` (BACK: undo, FWD: redo): nodes of that
/// side created again (parents first), the values of that side written back, then nodes of
/// the other side deleted. Leaves out targets others wrote since, nodes that no longer fit
/// (a parent gone) and created nodes others have put layers into. Every change of the result
/// applies (validated in order).
pub fn toward(&self, side: usize, view: &Replica) -> Result<Vec<Change>, Error> {
let other = 1 - side;
let mut work = view.clone();
let mut out = Vec::new();
let doc = view.document();
// Nodes to bring back, shallowest first (a parent before its children).
let back: BTreeMap<NodeId, &Change> = self
.nodes
.iter()
.filter(|(id, s)| s[other].is_none() && !self.taken_nodes.contains(*id) && !doc.contains(**id))
.filter_map(|(id, s)| s[side].as_ref().map(|c| (*id, c)))
.collect();
let depth = |mut id: NodeId| {
let mut d = 0usize;
while let Some(Change::Create { parent: Some(p), .. }) = back.get(&id).copied() {
d += 1;
id = *p;
if d > back.len() {
break;
}
}
d
};
let mut todo: Vec<(usize, Change)> = back.iter().map(|(id, c)| (depth(*id), (*c).clone())).collect();
todo.sort_by_key(|(d, _)| *d);
let mut todo: Vec<Change> = todo.into_iter().map(|(_, c)| c).collect();
for (t, v) in &self.writes {
let moved_away = matches!(t, Target::Node(id, Prop::Position) if self.taken.contains(&Target::Node(*id, Prop::Parent)));
if self.taken.contains(t) || moved_away {
continue;
}
let now = work.value(t)?;
if matches!(t, Target::Node(..)) && now.is_none() {
continue;
}
let want = v[side].as_ref();
if now.as_ref() == want {
continue;
}
todo.extend(t.write(want));
}
apply_all(&mut work, &mut out, todo);
// Nodes to remove: those that exist only on the other side, when nothing else is inside.
let gone: BTreeSet<NodeId> = self
.nodes
.iter()
.filter(|(id, s)| s[side].is_none() && s[other].is_some() && !self.taken_nodes.contains(*id))
.map(|(id, _)| *id)
.collect();
let wdoc = work.document();
let mut stack: Vec<NodeId> =
gone.iter().copied().filter(|id| wdoc.node(*id).is_some_and(|n| n.parent.is_none_or(|p| !gone.contains(&p)))).collect();
let mut deletes = Vec::new();
while let Some(id) = stack.pop() {
if wdoc.subtree(id).iter().all(|d| gone.contains(d)) {
deletes.push(Change::Delete { id });
} else if let Some(n) = wdoc.node(id) {
stack.extend(n.children.iter().copied().filter(|c| gone.contains(c)));
}
}
apply_all(&mut work, &mut out, deletes);
// Changes that apply one by one can still fail the batch's checks together (rare): keep
// only those that pass them in order.
if view.clone().apply(&out, Mode::Strict).is_err() {
let mut w = view.clone();
out.retain(|c| {
let mut t = w.clone();
let ok = t.apply(std::slice::from_ref(c), Mode::Strict).is_ok();
if ok {
w = t;
}
ok
});
}
Ok(out)
}
}
/// Apply `todo` to `work` change by change, keeping in `out` those that apply; changes that don't
/// are retried while others still go in (a move back can depend on another move back).
fn apply_all(work: &mut Replica, out: &mut Vec<Change>, mut todo: Vec<Change>) {
loop {
let before = todo.len();
let mut left = Vec::new();
for c in todo {
match work.apply(std::slice::from_ref(&c), Mode::Lenient) {
Ok(a) if a.skipped == 0 => out.push(c),
_ => left.push(c),
}
}
if left.is_empty() || left.len() == before {
return;
}
todo = left;
}
}
/// What a batch from someone else wrote, for marking the steps it conflicts with.
#[derive(Clone, Debug, Default)]
pub(crate) struct Touches {
props: BTreeSet<(NodeId, Prop)>,
/// Nodes created (or created again): all their properties are written.
nodes: BTreeSet<NodeId>,
docs: BTreeSet<DocProp>,
entries: BTreeSet<Entry>,
/// Lists written whole.
whole: BTreeSet<Family>,
/// Lists written in any way (whole, order or an entry).
lists: BTreeSet<Family>,
}
impl Touches {
/// Add what `other` writes.
pub fn absorb(&mut self, other: &Touches) {
self.props.extend(other.props.iter().copied());
self.nodes.extend(other.nodes.iter().copied());
self.docs.extend(other.docs.iter().copied());
self.entries.extend(other.entries.iter().cloned());
self.whole.extend(other.whole.iter().copied());
self.lists.extend(other.lists.iter().copied());
}
pub fn of(changes: &[Change]) -> Touches {
let mut t = Touches::default();
for c in changes {
match c {
Change::Create { id, .. } => {
t.nodes.insert(*id);
}
Change::Set { id, prop, .. } => {
t.props.insert((*id, *prop));
}
Change::Delete { .. } => {}
Change::SetDoc { prop, .. } => {
t.docs.insert(*prop);
match prop.list() {
Some(Ok(l)) => {
t.whole.insert(l);
t.lists.insert(l);
}
Some(Err(l)) => {
t.lists.insert(l);
}
None => {}
}
}
Change::Put { entry, .. } | Change::Remove { entry } => {
t.entries.insert(entry.clone());
if let Some(l) = entry.list() {
t.lists.insert(l);
}
}
}
}
t
}
/// Whether these writes change `target`.
pub fn hits(&self, target: &Target) -> bool {
match target {
Target::Node(id, p) => self.props.contains(&(*id, *p)) || self.nodes.contains(id),
Target::Doc(p) => {
self.docs.contains(p)
|| match p.list() {
Some(Ok(l)) => self.lists.contains(&l),
Some(Err(l)) => self.whole.contains(&l),
None => false,
}
}
Target::Entry(e) => self.entries.contains(e) || e.list().is_some_and(|l| self.whole.contains(&l)),
}
}
/// Whether these writes create node `id`.
pub fn creates(&self, id: NodeId) -> bool {
self.nodes.contains(&id)
}
}

35
crates/collab/src/lib.rs
View File

@ -11,7 +11,12 @@
//! a [`Client`] applies its own edits at once and reconciles when the server confirms, rejects or
//! orders someone else's batch first. Conflicting writes to one property resolve last-writer-wins
//! in server order, so every replica converges to the server's state.
//! - [`sim`] runs a server and several clients over in-memory links with random delays (tests).
//! - The client keeps the user's own undo history ([`Client::undo`], [`Client::redo`]): an undo
//! writes back what the user's step replaced where nobody else wrote since. Refused batches
//! leave the history with a [`Notice`]. Offline, batches queue; on reconnect the server sends
//! what the client missed (or a snapshot) and the queue goes out ([`Client::reconnect`]).
//! - [`sim`] runs a server and several clients over in-memory links with random delays, dropped
//! connections and an undo checker (tests).
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
pub mod change;
@ -19,6 +24,7 @@ pub mod client;
mod diff;
mod docprops;
pub mod frac;
mod history;
pub mod message;
pub mod replica;
pub mod server;
@ -26,13 +32,15 @@ pub mod sim;
use std::collections::BTreeSet;
pub use change::Target;
pub use change::{Change, DocProp, Entry, Json, Prop};
pub use client::Client;
pub use client::{Client, HISTORY_LIMIT};
pub use diff::{Diff, diff};
pub use figcraft_doc::{Document, NodeId};
pub use message::{ClientMsg, Hello, Peer, Presence, ServerMsg};
pub use history::{BatchKind, Notice};
pub use message::{Accepted, CatchUp, ClientMsg, EditingText, Hello, Peer, Presence, Processed, Resume, ServerMsg};
pub use replica::{Applied, Mode, Replica};
pub use server::{Outgoing, Server};
pub use server::{Outgoing, Server, SessionState};
/// What can go wrong syncing.
#[derive(Clone, Debug, PartialEq, thiserror::Error)]
@ -65,11 +73,24 @@ pub struct Limits {
pub max_peers: usize,
/// Longest display name or colour string (chars).
pub max_name: usize,
/// Most accepted batches the server keeps for clients that reconnect (older ones catch up
/// with a snapshot).
pub max_log_batches: usize,
/// Most changes in those batches together.
pub max_log_changes: usize,
}
impl Default for Limits {
fn default() -> Self {
Limits { max_message_bytes: 64 << 20, max_changes: 500_000, max_nodes: 5_000_000, max_peers: 256, max_name: 64 }
Limits {
max_message_bytes: 64 << 20,
max_changes: 500_000,
max_nodes: 5_000_000,
max_peers: 256,
max_name: 64,
max_log_batches: 10_000,
max_log_changes: 1_000_000,
}
}
}
@ -96,6 +117,10 @@ mod diff_tests;
#[cfg(test)]
mod frac_tests;
#[cfg(test)]
mod reconnect_tests;
#[cfg(test)]
mod server_tests;
#[cfg(test)]
mod sim_tests;
#[cfg(test)]
mod undo_tests;

87
crates/collab/src/message.rs
View File

@ -27,6 +27,20 @@ pub struct Hello {
/// Their colour (`#RRGGBB`).
pub color: String,
pub client_version: String,
/// Coming back after a dropped connection: who this client was and the last sequence number
/// it applied (`None`: a new participant).
pub resume: Option<Resume>,
}
/// What a reconnecting client tells the server in its [`Hello`].
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
pub struct Resume {
/// The session it was in ([`ServerMsg::Welcome`]'s `session`).
pub session: u64,
/// The client id the server gave this client before.
pub client_id: u32,
/// The last sequence number it applied.
pub seq: u64,
}
/// Someone in the session.
@ -45,8 +59,60 @@ pub struct Presence {
pub selection: Option<Vec<NodeId>>,
/// The page shown and the visible rectangle `(x, y, width, height)` in page coordinates.
pub viewport: Option<(NodeId, f64, f64, f64, f64)>,
/// The text layer being typed in.
pub editing_text: Option<NodeId>,
/// The text layer being typed in, with the caret and selection.
pub editing_text: Option<Box<EditingText>>,
}
/// Someone typing in a text layer: where their caret and selection are (in characters).
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
pub struct EditingText {
pub node: NodeId,
pub caret: usize,
/// The selected characters `(start, end)`, if any.
pub selection: Option<(usize, usize)>,
}
/// A batch the server accepted: its sequence number, who sent it and their batch id.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
pub struct Accepted {
pub seq: u64,
pub author: u32,
pub batch_id: u64,
pub changes: Vec<Change>,
}
/// What the server did with one client's batches: the last batch id it processed (accepted or
/// refused; 0: none) and the ids of the latest refused ones (at most [`MAX_REFUSED`]).
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize)]
pub struct Processed {
pub last_batch: u64,
pub refused: Vec<u64>,
}
/// The most refused batch ids [`Processed`] remembers per client.
pub const MAX_REFUSED: usize = 1024;
impl Processed {
/// Note batch `batch_id` as processed (`refused`: not accepted).
pub fn note(&mut self, batch_id: u64, refused: bool) {
self.last_batch = self.last_batch.max(batch_id);
if refused {
self.refused.push(batch_id);
let over = self.refused.len().saturating_sub(MAX_REFUSED);
self.refused.drain(..over);
}
}
}
/// How a [`ServerMsg::Welcome`] brings a client up to date.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub enum CatchUp {
/// The whole document ([`Replica::snapshot`](crate::Replica::snapshot)): a new participant,
/// or one whose missed batches are no longer in the server's log.
Snapshot(Vec<Change>),
/// The batches a reconnecting client missed, in order.
Delta(Vec<Accepted>),
}
/// Client → server.
@ -73,21 +139,20 @@ pub enum ClientMsg {
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub enum ServerMsg {
/// Joined: this connection's client id, the current sequence number, the document as a
/// snapshot ([`Replica::snapshot`](crate::Replica::snapshot)) and who else is here.
/// Joined: the session's identity (a server that restarted without its saved state is a new
/// session), this connection's client id, the current sequence number, the document (or the
/// batches missed since a reconnecting client's last sequence number), what the server did
/// with this client's batches so far and who else is here.
Welcome {
session: u64,
client_id: u32,
seq: u64,
snapshot: Vec<Change>,
catch_up: CatchUp,
processed: Processed,
peers: Vec<Peer>,
},
/// Accepted changes, in order, to everyone (the author's own come back as confirmation).
Changes {
seq: u64,
author: u32,
batch_id: u64,
changes: Vec<Change>,
},
Changes(Accepted),
/// A batch was refused (nothing of it applies): why, and the current values of what it
/// touched.
Reject {

255
crates/collab/src/reconnect_tests.rs Normal file
View File

@ -0,0 +1,255 @@
//! Dropped connections: the client keeps editing offline, reconnects with its id and last
//! sequence number, catches up with the batches it missed (or a snapshot once they're no longer
//! logged), settles what the server did with its batches meanwhile, and sends the rest.
use figcraft_doc::{Document, NodeKind};
use crate::message::{self, ClientMsg, Hello, ServerMsg};
use crate::sim::Sim;
use crate::sim_tests::{Width, base};
use crate::undo_tests::run_with_undo;
use crate::{BatchKind, CatchUp, Client, Limits, NoRecompute, Notice, Outgoing, Server};
fn hello() -> Hello {
let ClientMsg::Hello(h) = Client::hello("f", "tok", "Ada", "#FF8800") else { panic!("not a hello") };
h
}
/// Join `s` and return the client and the welcome's other messages.
fn join(s: &mut Server) -> Client {
let (_, out) = s.join(hello()).unwrap();
let Some(Outgoing::To(_, welcome)) = out.into_iter().next() else { panic!("no welcome") };
Client::join(welcome).unwrap()
}
/// Deliver everything in `out` meant for `c`.
fn deliver(c: &mut Client, out: &[Outgoing]) -> Vec<ClientMsg> {
let id = c.id();
let mut replies = Vec::new();
for o in out {
let m = match o {
Outgoing::To(t, m) if *t == id => m,
Outgoing::All(m) => m,
Outgoing::Others(not, m) if *not != id => m,
_ => continue,
};
replies.extend(c.receive(m.clone(), &NoRecompute).unwrap());
}
replies
}
/// `c` comes back: it says hello again and gets its welcome. Returns what it sends then.
fn come_back(s: &mut Server, c: &mut Client) -> Vec<ClientMsg> {
let ClientMsg::Hello(h) = c.reconnect(hello()) else { panic!() };
let (id, out) = s.join(h).unwrap();
assert_eq!(id, c.id(), "a reconnecting client keeps its id");
deliver(c, &out)
}
fn edit(c: &mut Client, f: impl FnOnce(&mut Document)) -> Option<ClientMsg> {
let mut d = c.document().clone();
f(&mut d);
c.edit(&d).unwrap()
}
#[test]
fn a_batch_whose_confirmation_was_lost_is_confirmed_by_the_catch_up() {
let mut s = Server::new(base(), "tok", Limits::default());
let mut a = join(&mut s);
let page = s.document().first_page().unwrap();
let msg = edit(&mut a, |d| {
d.create(page, NodeKind::Rectangle, "R", 0.0, 0.0, 5.0, 5.0).unwrap();
})
.unwrap();
// The server applies it, but the connection drops before the echo arrives.
let _lost = s.receive(a.id(), msg);
a.disconnect();
s.leave(a.id());
let resend = come_back(&mut s, &mut a);
assert!(resend.is_empty(), "nothing is sent twice: {resend:?}");
assert_eq!(a.pending(), 0);
assert_eq!(a.view(), s.replica());
assert!(a.can_undo(), "the confirmed edit stays undoable");
}
#[test]
fn a_batch_refused_while_offline_is_dropped_with_a_reason() {
let mut s = Server::new(base(), "tok", Limits::default());
let mut a = join(&mut s);
let mut b = join(&mut s);
let page = s.document().first_page().unwrap();
let f = s.document().node(page).unwrap().children[0];
let del = edit(&mut b, |d| d.remove(f).unwrap()).unwrap();
let rename = edit(&mut a, |d| d.node_mut(f).unwrap().name = "Mine".into()).unwrap();
let out = s.receive(b.id(), del);
deliver(&mut b, &out);
// a's rename is refused, but a never hears it.
let _lost = s.receive(a.id(), rename);
a.disconnect();
s.leave(a.id());
// a keeps working offline: queued, nothing goes out.
assert!(edit(&mut a, |d| d.node_mut(page).unwrap().name = "Page A".into()).is_none());
assert_eq!(a.pending(), 2);
let resend = come_back(&mut s, &mut a);
assert_eq!(resend.len(), 1, "the offline edit goes out");
let notices = a.take_notices();
assert!(matches!(notices.as_slice(), [Notice::Rejected { kind: BatchKind::Edit, .. }]), "{notices:?}");
let out = s.receive(a.id(), resend.into_iter().next().unwrap());
deliver(&mut a, &out);
deliver(&mut b, &out);
assert_eq!(a.view(), s.replica());
assert_eq!(b.view(), s.replica());
assert_eq!(s.document().node(page).unwrap().name, "Page A");
}
#[test]
fn a_long_absence_catches_up_with_a_snapshot() {
let mut s = Server::new(base(), "tok", Limits { max_log_batches: 3, ..Limits::default() });
let mut a = join(&mut s);
let mut b = join(&mut s);
let page = s.document().first_page().unwrap();
a.disconnect();
s.leave(a.id());
for i in 0..5 {
let m = edit(&mut b, |d| d.node_mut(page).unwrap().name = format!("Page {i}")).unwrap();
let out = s.receive(b.id(), m);
deliver(&mut b, &out);
}
let ClientMsg::Hello(h) = a.reconnect(hello()) else { panic!() };
let (_, out) = s.join(h).unwrap();
assert!(matches!(&out[0], Outgoing::To(_, ServerMsg::Welcome { catch_up: CatchUp::Snapshot(_), .. })));
deliver(&mut a, &out);
assert_eq!(a.view(), s.replica());
// A short absence gets the missed batches only.
a.disconnect();
s.leave(a.id());
let m = edit(&mut b, |d| d.node_mut(page).unwrap().name = "Later".into()).unwrap();
let out = s.receive(b.id(), m);
deliver(&mut b, &out);
let ClientMsg::Hello(h) = a.reconnect(hello()) else { panic!() };
let (_, out) = s.join(h).unwrap();
assert!(matches!(&out[0], Outgoing::To(_, ServerMsg::Welcome { catch_up: CatchUp::Delta(d), .. }) if d.len() == 1));
deliver(&mut a, &out);
assert_eq!(a.view(), s.replica());
}
#[test]
fn after_a_snapshot_undo_leaves_what_others_changed_meanwhile() {
let mut s = Server::new(base(), "tok", Limits { max_log_batches: 2, ..Limits::default() });
let mut a = join(&mut s);
let mut b = join(&mut s);
let page = s.document().first_page().unwrap();
let f = s.document().node(page).unwrap().children[0];
// a renames the page and the frame (one step), confirmed; then goes away for a while.
let m = edit(&mut a, |d| {
d.node_mut(page).unwrap().name = "A page".into();
d.node_mut(f).unwrap().name = "A frame".into();
})
.unwrap();
let out = s.receive(a.id(), m);
deliver(&mut a, &out);
deliver(&mut b, &out);
a.disconnect();
s.leave(a.id());
for i in 0..4 {
let m = edit(&mut b, |d| d.node_mut(f).unwrap().name = format!("B frame {i}")).unwrap();
let out = s.receive(b.id(), m);
deliver(&mut b, &out);
}
let ClientMsg::Hello(h) = a.reconnect(hello()) else { panic!() };
let (_, out) = s.join(h).unwrap();
assert!(matches!(&out[0], Outgoing::To(_, ServerMsg::Welcome { catch_up: CatchUp::Snapshot(_), .. })));
deliver(&mut a, &out);
// Undo: the page name goes back, the frame keeps b's later name.
let msgs = a.undo(&NoRecompute).unwrap();
assert_eq!(msgs.len(), 1);
let out = s.receive(a.id(), msgs.into_iter().next().unwrap());
deliver(&mut a, &out);
assert_eq!(s.document().node(page).unwrap().name, "Page 1");
assert_eq!(s.document().node(f).unwrap().name, "B frame 3");
assert_eq!(a.view(), s.replica());
}
#[test]
fn a_copy_sent_again_is_ignored_and_a_stale_connection_is_taken_over() {
let mut s = Server::new(base(), "tok", Limits::default());
let mut a = join(&mut s);
let page = s.document().first_page().unwrap();
let m = edit(&mut a, |d| d.node_mut(page).unwrap().name = "Once".into()).unwrap();
let out = s.receive(a.id(), m.clone());
assert_eq!(s.seq(), 1);
// The same batch again (a copy sent around a reconnect): nothing happens.
assert!(s.receive(a.id(), m).is_empty());
assert_eq!(s.seq(), 1);
deliver(&mut a, &out);
// a comes back before the server noticed its old connection is gone: same id, no "full".
let mut tight = Server::new(base(), "tok", Limits { max_peers: 1, ..Limits::default() });
let mut b = join(&mut tight);
let resend = come_back(&mut tight, &mut b);
assert!(resend.is_empty());
assert_eq!(tight.peers().count(), 1);
assert_eq!(b.view(), tight.replica());
}
#[test]
fn an_unknown_client_joins_as_new_and_its_queue_is_dropped() {
let mut s = Server::new(base(), "tok", Limits::default());
let mut a = join(&mut s);
let page = s.document().first_page().unwrap();
a.disconnect();
edit(&mut a, |d| d.node_mut(page).unwrap().name = "Offline".into());
// A different session (it never gave out a's id).
let mut other = Server::new(base(), "tok", Limits::default());
join(&mut other);
let ClientMsg::Hello(h) = a.reconnect(hello()) else { panic!() };
let (id, out) = other.join(h).unwrap();
let Some(Outgoing::To(_, welcome)) = out.into_iter().next() else { panic!("no welcome") };
assert!(a.receive(welcome, &NoRecompute).unwrap().is_empty());
assert_eq!(a.id(), id);
assert_eq!(a.pending(), 0);
assert_eq!(a.view(), other.replica());
assert!(matches!(a.take_notices().as_slice(), [Notice::Lost { batches: 1 }]));
}
#[test]
fn the_session_state_survives_a_restart() {
let mut s = Server::new(base(), "tok", Limits::default());
let mut a = join(&mut s);
let page = s.document().first_page().unwrap();
let m = edit(&mut a, |d| d.node_mut(page).unwrap().name = "Saved".into()).unwrap();
let out = s.receive(a.id(), m);
deliver(&mut a, &out);
// Saved in both encodings and read back.
let state = s.state().unwrap();
let bytes = message::encode(&state).unwrap();
assert_eq!(message::decode::<crate::SessionState>(&bytes, usize::MAX).unwrap(), state);
let json = message::encode_json(&state).unwrap();
let state = message::decode_json::<crate::SessionState>(&json, usize::MAX).unwrap();
let mut restarted = Server::restore(state, "tok", Limits::default()).unwrap();
assert_eq!(restarted.replica(), s.replica());
assert_eq!(restarted.seq(), s.seq());
// The client-id counter carries on: a new participant never gets an id handed out before.
let b = join(&mut restarted);
assert!(b.id() > a.id());
// a reconnects to the restarted server and keeps going.
a.disconnect();
let resend = come_back(&mut restarted, &mut a);
assert!(resend.is_empty());
assert_eq!(a.view(), restarted.replica());
}
#[test]
fn random_drops_and_reconnects_converge() {
for seed in 0..10 {
run_with_undo(seed, 3, 90, &NoRecompute, true, None);
}
// A short log: long absences fall back to snapshots.
let mut snapshots = 0;
for seed in 20..30 {
let limits = Limits { max_log_batches: 4, ..Limits::default() };
let sim = Sim::with_limits(base(), 3, seed, 12, limits).unwrap();
let sim = run_with_undo(seed, 3, 90, &Width, true, Some(sim));
snapshots += sim.snapshots;
}
assert!(snapshots > 0, "no reconnect needed a snapshot");
}

19
crates/collab/src/replica.rs
View File

@ -5,7 +5,7 @@ use std::collections::BTreeSet;
use figcraft_doc::{Document, Node, NodeId, NodeKind};
use crate::change::{Change, Json, Prop};
use crate::change::{Change, Json, Prop, Target};
use crate::frac::{self, FIRST_KEY};
use crate::{Error, docprops};
@ -118,6 +118,23 @@ impl Replica {
self.positions.get(&id).map_or("", String::as_str)
}
/// The current value of `target` (`None`: its node or entry doesn't exist).
pub fn value(&self, target: &Target) -> Result<Option<Json>, Error> {
match target {
Target::Node(id, prop) => {
let Some(n) = self.doc.node(*id) else { return Ok(None) };
match prop {
Prop::Parent => Json::of(&n.parent),
Prop::Position => Json::of(self.key(*id)),
p => p.get(n),
}
.map(Some)
}
Target::Doc(prop) => docprops::get(&self.doc, *prop).map(Some),
Target::Entry(entry) => docprops::entry_value(&self.doc, entry),
}
}
pub(crate) fn from_parts(doc: Document, positions: Positions) -> Replica {
Replica { doc, positions }
}

175
crates/collab/src/server.rs
View File

@ -1,12 +1,13 @@
//! The session server for one file: the single authority that orders, validates and rebroadcasts
//! batches. A pure state machine: the transport feeds it messages and delivers what it returns.
use std::collections::{BTreeMap, BTreeSet};
use std::collections::{BTreeMap, BTreeSet, VecDeque};
use figcraft_doc::{Document, NodeId};
use serde::{Deserialize, Serialize};
use crate::change::{Change, Prop};
use crate::message::{ClientMsg, Hello, PROTOCOL, Peer, Presence, ServerMsg};
use crate::change::{Change, Prop, Target};
use crate::message::{Accepted, CatchUp, ClientMsg, Hello, PROTOCOL, Peer, Presence, Processed, ServerMsg};
use crate::replica::{Mode, Replica, create_change};
use crate::{Error, Limits, docprops};
@ -24,6 +25,8 @@ pub enum Outgoing {
/// A session server for one file.
#[derive(Clone, Debug)]
pub struct Server {
/// This session's identity (random): clients resume only into the session they were in.
session: u64,
state: Replica,
seq: u64,
token: String,
@ -35,6 +38,31 @@ pub struct Server {
/// Deleted node ids: only these may be created again (by undo); new ids must come from the
/// creating client's id space.
deleted: BTreeSet<NodeId>,
/// Every client id handed out (a reconnecting client keeps its id) and what was done with
/// its batches.
issued: BTreeMap<u32, Processed>,
/// The latest accepted batches, oldest first (a reconnecting client gets what it missed).
log: VecDeque<Accepted>,
/// Changes in `log`.
log_changes: usize,
}
/// The most client ids a session remembers for reconnecting (the oldest disconnected ones go
/// first; a forgotten client joins as new).
const MAX_REMEMBERED: usize = 65_536;
/// What a session server needs to carry on after a restart: the document, the sequence number,
/// the client-id counter (so ids are never handed out twice), the clients that can reconnect,
/// the deleted ids that may be created again and the log of recent batches.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
pub struct SessionState {
pub session: u64,
pub snapshot: Vec<Change>,
pub seq: u64,
pub next_client: u32,
pub issued: BTreeMap<u32, Processed>,
pub deleted: BTreeSet<NodeId>,
pub log: Vec<Accepted>,
}
impl Server {
@ -45,6 +73,7 @@ impl Server {
let used = doc.ids().map(|id| id.0 >> 32).max().unwrap_or(0);
let next_client = u32::try_from(used.saturating_add(1)).unwrap_or(u32::MAX).max(1);
Server {
session: new_session_id(),
state: Replica::new(doc),
seq: 0,
token: token.into(),
@ -54,9 +83,46 @@ impl Server {
presence: BTreeMap::new(),
acked: BTreeMap::new(),
deleted: BTreeSet::new(),
issued: BTreeMap::new(),
log: VecDeque::new(),
log_changes: 0,
}
}
/// The session's state, to save (see [`restore`](Self::restore)).
pub fn state(&self) -> Result<SessionState, Error> {
Ok(SessionState {
session: self.session,
snapshot: self.state.snapshot()?,
seq: self.seq,
next_client: self.next_client,
issued: self.issued.clone(),
deleted: self.deleted.clone(),
log: self.log.iter().cloned().collect(),
})
}
/// A session carrying on from a saved [`state`](Self::state) (nobody is connected yet; clients
/// reconnect with their ids). A log that doesn't lead up to the sequence number is dropped
/// (clients then catch up with a snapshot).
pub fn restore(state: SessionState, token: impl Into<String>, limits: Limits) -> Result<Server, Error> {
let replica = Replica::from_snapshot(&state.snapshot)?;
let mut server = Server::new(replica.document().clone(), token, limits);
server.state = replica;
server.session = state.session;
server.seq = state.seq;
server.next_client = server.next_client.max(state.next_client);
server.issued = state.issued.into_iter().filter(|(id, _)| *id < server.next_client).collect();
server.deleted = state.deleted;
let contiguous = state.log.windows(2).all(|w| matches!(w, [a, b] if a.seq + 1 == b.seq));
if contiguous && state.log.last().is_none_or(|a| a.seq == state.seq) {
for a in state.log {
server.remember(a);
}
}
Ok(server)
}
/// The authoritative document.
pub fn document(&self) -> &Document {
self.state.document()
@ -83,7 +149,11 @@ impl Server {
}
/// A new connection says hello: on success its client id and the messages to send (its
/// `Welcome`, everyone else's `PeerJoined`); otherwise the `Refused` to send back.
/// `Welcome`, everyone else's `PeerJoined`); otherwise the `Refused` to send back. A client
/// coming back ([`Hello::resume`]) keeps its id (taking over from its old connection if that
/// is still open: the transport closes the old one) and gets the batches it missed, or a
/// snapshot when they are no longer in the log; an id this session never gave out joins as
/// new.
pub fn join(&mut self, hello: Hello) -> Result<(u32, Vec<Outgoing>), ServerMsg> {
let refuse = |reason: &str| ServerMsg::Refused { reason: reason.to_string() };
if hello.protocol != PROTOCOL {
@ -92,20 +162,41 @@ impl Server {
if !same_secret(&hello.token, &self.token) {
return Err(refuse("wrong session token"));
}
if self.peers.len() >= self.limits.max_peers {
let resume = hello.resume.filter(|r| r.session == self.session && self.issued.contains_key(&r.client_id));
if self.peers.len() >= self.limits.max_peers && resume.is_none_or(|r| !self.peers.contains_key(&r.client_id)) {
return Err(refuse("the session is full"));
}
if hello.name.chars().count() > self.limits.max_name || hello.color.chars().count() > self.limits.max_name {
return Err(refuse("name or colour too long"));
}
if self.next_client == u32::MAX {
if resume.is_none() && self.next_client == u32::MAX {
return Err(refuse("no client ids left in this session"));
}
let snapshot = self.state.snapshot().map_err(|e| refuse(&e.to_string()))?;
let id = self.next_client;
self.next_client += 1;
// The log holds every batch after `floor`.
let floor = self.log.front().map_or(self.seq, |a| a.seq.saturating_sub(1));
let catch_up = match resume {
Some(r) if r.seq <= self.seq && r.seq >= floor => CatchUp::Delta(self.log.iter().filter(|a| a.seq > r.seq).cloned().collect()),
_ => CatchUp::Snapshot(self.state.snapshot().map_err(|e| refuse(&e.to_string()))?),
};
let id = match resume {
Some(r) => r.client_id,
None => {
let id = self.next_client;
self.next_client += 1;
self.issued.insert(id, Processed::default());
if self.issued.len() > MAX_REMEMBERED
&& let Some(old) = self.issued.keys().copied().find(|k| !self.peers.contains_key(k))
{
self.issued.remove(&old);
}
id
}
};
let processed = self.issued.get(&id).cloned().unwrap_or_default();
self.presence.remove(&id);
let peer = Peer { client_id: id, name: hello.name, color: hello.color };
let welcome = ServerMsg::Welcome { client_id: id, seq: self.seq, snapshot, peers: self.peers.values().cloned().collect() };
let others = self.peers.values().filter(|p| p.client_id != id).cloned().collect();
let welcome = ServerMsg::Welcome { session: self.session, client_id: id, seq: self.seq, catch_up, processed, peers: others };
self.peers.insert(id, peer.clone());
Ok((id, vec![Outgoing::To(id, welcome), Outgoing::Others(id, ServerMsg::PeerJoined(peer))]))
}
@ -127,13 +218,26 @@ impl Server {
}
match msg {
ClientMsg::Hello(_) => vec![Outgoing::To(from, ServerMsg::Refused { reason: "already joined".into() })],
ClientMsg::Changes { base_seq, batch_id, changes } => match self.accept(from, base_seq, &changes) {
Ok(()) => vec![Outgoing::All(ServerMsg::Changes { seq: self.seq, author: from, batch_id, changes })],
Err(e) => {
let fixes = self.fixes(&changes).unwrap_or_default();
vec![Outgoing::To(from, ServerMsg::Reject { batch_id, reason: e.to_string(), fixes })]
// A batch already processed (a copy sent again around a reconnect) is ignored: the
// client learned what happened to it from the catch-up.
ClientMsg::Changes { batch_id, .. } if self.issued.get(&from).is_some_and(|p| batch_id <= p.last_batch) => Vec::new(),
ClientMsg::Changes { base_seq, batch_id, changes } => {
let result = self.accept(from, base_seq, &changes);
if let Some(p) = self.issued.get_mut(&from) {
p.note(batch_id, result.is_err());
}
},
match result {
Ok(()) => {
let acc = Accepted { seq: self.seq, author: from, batch_id, changes };
self.remember(acc.clone());
vec![Outgoing::All(ServerMsg::Changes(acc))]
}
Err(e) => {
let fixes = self.fixes(&changes).unwrap_or_default();
vec![Outgoing::To(from, ServerMsg::Reject { batch_id, reason: reason(&e), fixes })]
}
}
}
ClientMsg::Presence(p) => {
self.presence.insert(from, p.clone());
vec![Outgoing::Others(from, ServerMsg::Presence { client_id: from, presence: p })]
@ -146,6 +250,17 @@ impl Server {
}
}
/// Keep an accepted batch in the log (dropping the oldest beyond the limits).
fn remember(&mut self, acc: Accepted) {
self.log_changes += acc.changes.len();
self.log.push_back(acc);
while self.log.len() > self.limits.max_log_batches || (self.log_changes > self.limits.max_log_changes && !self.log.is_empty()) {
if let Some(old) = self.log.pop_front() {
self.log_changes = self.log_changes.saturating_sub(old.changes.len());
}
}
}
/// Validate and apply a batch; on success it has the next sequence number.
fn accept(&mut self, from: u32, base_seq: u64, changes: &[Change]) -> Result<(), Error> {
if changes.len() > self.limits.max_changes {
@ -209,17 +324,13 @@ impl Server {
}
let mut nodes = Vec::new();
for (id, ps) in props {
let Some(n) = doc.node(id) else {
if !doc.contains(id) {
nodes.push(Change::Delete { id });
continue;
};
}
for p in ps {
let value = match p {
Prop::Parent => crate::Json::of(&n.parent)?,
Prop::Position => crate::Json::of(self.state.key(id))?,
p => p.get(n)?,
};
nodes.push(Change::Set { id, prop: p, value });
let t = Target::Node(id, p);
nodes.extend(t.write(self.state.value(&t)?.as_ref()));
}
}
for id in gone {
@ -239,6 +350,22 @@ impl Server {
}
}
/// A fresh session identity: random per server (std's hasher keys are seeded from the OS).
fn new_session_id() -> u64 {
use std::hash::{BuildHasher, Hasher};
let mut h = std::collections::hash_map::RandomState::new().build_hasher();
h.write_u8(0);
h.finish()
}
/// Why a batch was refused, for the person who made it.
fn reason(e: &Error) -> String {
match e {
Error::Unknown(_) => "a layer it changes was deleted by someone else".into(),
e => e.to_string(),
}
}
/// Compare secrets without stopping at the first difference.
fn same_secret(a: &str, b: &str) -> bool {
let (a, b) = (a.as_bytes(), b.as_bytes());

27
crates/collab/src/server_tests.rs
View File

@ -115,12 +115,19 @@ fn batches_are_ordered_echoed_and_rebroadcast() {
assert!(matches!(s.receive(999, ClientMsg::Ping).as_slice(), [Outgoing::To(999, ServerMsg::Refused { .. })]));
}
/// A new batch id (the server ignores a batch id it already processed).
fn next_batch() -> u64 {
static NEXT: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(1000);
NEXT.fetch_add(1, std::sync::atomic::Ordering::Relaxed)
}
fn rejected(s: &mut Server, from: u32, changes: Vec<Change>) -> (String, Vec<Change>) {
let batch = next_batch();
let before = s.replica().clone();
let out = s.receive(from, ClientMsg::Changes { base_seq: s.seq(), batch_id: 1, changes });
let out = s.receive(from, ClientMsg::Changes { base_seq: s.seq(), batch_id: batch, changes });
assert_eq!(s.replica(), &before, "nothing applied");
match out.as_slice() {
[Outgoing::To(to, ServerMsg::Reject { batch_id: 1, reason, fixes })] if *to == from => (reason.clone(), fixes.clone()),
[Outgoing::To(to, ServerMsg::Reject { batch_id, reason, fixes })] if *to == from && *batch_id == batch => (reason.clone(), fixes.clone()),
other => panic!("not rejected: {other:?}"),
}
}
@ -144,12 +151,12 @@ fn invalid_batches_are_rejected_whole() {
rejected(&mut s, a.id(), vec![create(77, frame)]);
// A valid change with an invalid one: neither applies.
let (reason, _) = rejected(&mut s, a.id(), vec![set(frame, Prop::Name, "ok".into()), set(NodeId(4242), Prop::Name, "x".into())]);
assert!(reason.contains("4242"), "{reason}");
assert!(reason.contains("deleted by someone else"), "{reason}");
// A cycle through a new frame.
rejected(&mut s, a.id(), vec![create((me << 32) | 1, frame), set(frame, Prop::Parent, serde_json::json!((me << 32) | 1))]);
// Too many changes; based on the future.
rejected(&mut s, a.id(), vec![set(frame, Prop::Name, "a".into()); 4]);
let out = s.receive(a.id(), ClientMsg::Changes { base_seq: 5, batch_id: 1, changes: vec![] });
let out = s.receive(a.id(), ClientMsg::Changes { base_seq: 5, batch_id: next_batch(), changes: vec![] });
assert!(matches!(out.as_slice(), [Outgoing::To(_, ServerMsg::Reject { .. })]));
// Deleting the last page.
rejected(&mut s, a.id(), vec![Change::Delete { id: page }]);
@ -166,12 +173,12 @@ fn deleted_nodes_can_come_back_but_ids_are_never_handed_out_twice() {
let snap = crate::replica::create_change(gone.document().node(frame).unwrap(), gone.key(frame)).unwrap();
// b deletes the frame (an old id); a brings it back, as undo would.
let out = s.receive(b.id(), ClientMsg::Changes { base_seq: 0, batch_id: 1, changes: vec![Change::Delete { id: frame }] });
assert!(matches!(out.as_slice(), [Outgoing::All(ServerMsg::Changes { .. })]));
assert!(matches!(out.as_slice(), [Outgoing::All(ServerMsg::Changes(..))]));
for m in for_client(&out, a.id()) {
a.receive(m, &NoRecompute).unwrap();
}
let out = s.receive(a.id(), ClientMsg::Changes { base_seq: 1, batch_id: 1, changes: vec![snap.clone()] });
assert!(matches!(out.as_slice(), [Outgoing::All(ServerMsg::Changes { .. })]), "{out:?}");
assert!(matches!(out.as_slice(), [Outgoing::All(ServerMsg::Changes(..))]), "{out:?}");
assert_eq!(s.document(), gone.document());
}
@ -194,7 +201,7 @@ fn fixes_roll_a_rejected_client_back_to_the_server_state() {
// A client without a confirmed copy can roll back with the fixes: they delete what the server
// doesn't have (b's delete arrives first on a's connection).
let mut rolled = optimistic.clone();
let ServerMsg::Changes { changes, .. } = &for_client(&out_b, a.id())[0] else { panic!() };
let ServerMsg::Changes(crate::Accepted { changes, .. }) = &for_client(&out_b, a.id())[0] else { panic!() };
rolled.apply(changes, Mode::Lenient).unwrap();
rolled.apply(fixes, Mode::Lenient).unwrap();
assert_eq!(rolled.document(), s.document());
@ -237,7 +244,7 @@ fn remote_batches_apply_under_pending_ones() {
assert_eq!(a.view(), s.replica());
assert_eq!(b.view(), s.replica());
// Messages out of order are protocol errors.
let stale = ServerMsg::Changes { seq: 1, author: b.id(), batch_id: 1, changes: vec![] };
let stale = ServerMsg::Changes(crate::Accepted { seq: 1, author: b.id(), batch_id: 1, changes: vec![] });
assert!(a.receive(stale, &NoRecompute).is_err());
assert!(a.receive(ServerMsg::Reject { batch_id: 9, reason: String::new(), fixes: vec![] }, &NoRecompute).is_err());
assert!(Client::join(ServerMsg::Pong).is_err());
@ -265,7 +272,7 @@ fn junk_batches_never_panic_and_never_corrupt_the_document() {
serde_json::json!([1.0, 0.0, 0.0, 1.0, 5.0, 5.0]),
serde_json::json!(true),
];
for _ in 0..3000 {
for round in 0..3000u64 {
let ids: Vec<NodeId> = s.document().ids().collect();
let id = |rng: &mut Rng| match rng.below(3) {
0 => NodeId((me << 32) | rng.below(50) as u64),
@ -296,7 +303,7 @@ fn junk_batches_never_panic_and_never_corrupt_the_document() {
_ => Change::Put { entry: Entry::Comment(rng.below(3) as u64), value: v(&mut rng) },
});
}
let _ = s.receive(a.id(), ClientMsg::Changes { base_seq: s.seq(), batch_id: 1, changes });
let _ = s.receive(a.id(), ClientMsg::Changes { base_seq: s.seq(), batch_id: round + 1, changes });
assert!(s.document().validate().is_empty(), "{:?}", s.document().validate());
}
}

262
crates/collab/src/sim.rs
View File

@ -1,14 +1,17 @@
//! An in-memory session for tests: one [`Server`], several [`Client`]s and a link each way per
//! client that delivers in order (like a WebSocket) after a random delay, so messages on different
//! links interleave differently for every seed. Every message goes through the binary or the JSON
//! encoding on its way.
//! encoding on its way. Clients can drop their connection and come back, and every undo or redo
//! the server accepts is checked against what others wrote before it ([`Sim::violations`]).
use std::collections::VecDeque;
use figcraft_doc::Document;
use crate::message::{self, ClientMsg, ServerMsg};
use crate::{Client, Error, Limits, Outgoing, Recompute, Replica, Server};
use crate::change::{Change, Target};
use crate::history::Touches;
use crate::message::{self, Accepted, ClientMsg, Hello, ServerMsg};
use crate::{BatchKind, Client, Error, Limits, Outgoing, Recompute, Replica, Server};
/// A small deterministic random number generator (SplitMix64) for simulations and fuzzing.
#[derive(Clone, Debug)]
@ -57,46 +60,82 @@ struct Link {
last_due: u64,
}
/// One accepted batch, as the undo checker remembers it.
struct Logged {
author: u32,
touches: Touches,
}
/// A simulated session.
pub struct Sim {
pub server: Server,
clients: Vec<Client>,
/// Each client's id at the server and its first hello (to reconnect with).
ids: Vec<u32>,
hellos: Vec<Hello>,
online: Vec<bool>,
up: Vec<Link>,
down: Vec<Link>,
rng: Rng,
now: u64,
max_delay: u64,
limits: Limits,
/// The server's state after each sequence number (index: the sequence number).
states: Vec<Replica>,
/// What each accepted batch wrote (index: sequence number − 1).
log: Vec<Logged>,
/// Per client: the batches it only got as part of a snapshot (`(after, upto]`).
gaps: Vec<Vec<(u64, u64)>>,
/// Messages delivered so far.
pub delivered: usize,
/// Batches the server rejected.
pub rejected: usize,
/// Follow-up batches clients sent to repair derived data.
pub repairs: usize,
/// Undo and redo batches the server accepted.
pub undos: usize,
/// Reconnects that caught up with a snapshot (the missed batches were no longer logged).
pub snapshots: usize,
/// Undo or redo batches that reverted a write by someone else the client had already seen.
pub violations: Vec<String>,
}
impl Sim {
/// A session for `doc` with `n` clients joined; message delays are random in `1..=max_delay`.
pub fn new(doc: Document, n: usize, seed: u64, max_delay: u64) -> Result<Sim, Error> {
let limits = Limits::default();
Sim::with_limits(doc, n, seed, max_delay, Limits::default())
}
/// [`new`](Self::new) with the server's limits (e.g. a short log of batches).
pub fn with_limits(doc: Document, n: usize, seed: u64, max_delay: u64, limits: Limits) -> Result<Sim, Error> {
let server = Server::new(doc, "secret", limits);
let mut sim = Sim {
server: Server::new(doc, "secret", limits),
states: vec![server.replica().clone()],
server,
clients: Vec::new(),
ids: Vec::new(),
hellos: Vec::new(),
online: Vec::new(),
up: Vec::new(),
down: Vec::new(),
rng: Rng::new(seed),
now: 0,
max_delay: max_delay.max(1),
limits,
log: Vec::new(),
gaps: Vec::new(),
delivered: 0,
rejected: 0,
repairs: 0,
undos: 0,
snapshots: 0,
violations: Vec::new(),
};
for i in 0..n {
let ClientMsg::Hello(hello) = Client::hello("file", "secret", &format!("Person {}", i + 1), "#3366FF") else {
return Err(Error::Protocol("hello".into()));
};
let (id, out) = sim.server.join(hello).map_err(|m| Error::Protocol(format!("{m:?}")))?;
let (id, out) = sim.server.join(hello.clone()).map_err(|m| Error::Protocol(format!("{m:?}")))?;
let mut welcome = None;
let mut rest = Vec::new();
for o in out {
@ -109,6 +148,10 @@ impl Sim {
// The welcome travels encoded like everything else.
let welcome: ServerMsg = message::decode(&message::encode(&welcome)?, limits.max_message_bytes)?;
sim.clients.push(Client::join(welcome)?);
sim.ids.push(id);
sim.hellos.push(hello);
sim.online.push(true);
sim.gaps.push(Vec::new());
sim.up.push(Link::default());
sim.down.push(Link::default());
sim.route(rest)?;
@ -121,6 +164,11 @@ impl Sim {
&self.clients
}
/// The clients, to read their notices or change settings.
pub fn clients_mut(&mut self) -> &mut [Client] {
&mut self.clients
}
/// Client `i`'s document (to edit a copy of).
pub fn document(&self, i: usize) -> Option<&Document> {
self.clients.get(i).map(Client::document)
@ -131,8 +179,13 @@ impl Sim {
&mut self.rng
}
fn index_of(&self, id: u32) -> Option<usize> {
self.clients.iter().position(|c| c.id() == id)
/// Whether client `i` is connected.
pub fn is_online(&self, i: usize) -> bool {
self.online.get(i).copied().unwrap_or(false)
}
fn client(&mut self, i: usize) -> Result<&mut Client, Error> {
self.clients.get_mut(i).ok_or_else(|| Error::Invalid(format!("no client {i}")))
}
fn wire<T: serde::Serialize>(&mut self, msg: &T) -> Result<Wire, Error> {
@ -158,13 +211,23 @@ impl Sim {
}
}
/// Send client `i`'s messages to the server.
fn send_up(&mut self, i: usize, msgs: Vec<ClientMsg>) -> Result<(), Error> {
for m in msgs {
let w = self.wire(&m)?;
self.send(true, i, w);
}
Ok(())
}
fn route(&mut self, out: Vec<Outgoing>) -> Result<(), Error> {
for o in out {
let (msg, targets): (ServerMsg, Vec<usize>) = match o {
Outgoing::To(id, m) => (m, self.index_of(id).into_iter().collect()),
Outgoing::To(id, m) => (m, self.ids.iter().position(|c| *c == id).into_iter().collect()),
Outgoing::All(m) => (m, (0..self.clients.len()).collect()),
Outgoing::Others(id, m) => (m, (0..self.clients.len()).filter(|i| self.clients.get(*i).is_some_and(|c| c.id() != id)).collect()),
Outgoing::Others(id, m) => (m, (0..self.clients.len()).filter(|i| self.ids.get(*i) != Some(&id)).collect()),
};
let targets: Vec<usize> = targets.into_iter().filter(|t| self.is_online(*t)).collect();
for t in targets {
let w = self.wire(&msg)?;
self.send(false, t, w);
@ -173,18 +236,74 @@ impl Sim {
Ok(())
}
/// Client `i` edited its document into `doc`: it applies it at once and sends the batch.
/// Returns whether anything synced changed.
/// Client `i` edited its document into `doc`: it applies it at once and sends the batch
/// (offline: queues it). Returns whether a batch went out.
pub fn edit(&mut self, i: usize, doc: &Document) -> Result<bool, Error> {
let c = self.clients.get_mut(i).ok_or_else(|| Error::Invalid(format!("no client {i}")))?;
match c.edit(doc)? {
Some(msg) => {
let w = self.wire(&msg)?;
self.send(true, i, w);
Ok(true)
}
None => Ok(false),
let msg = self.client(i)?.edit(doc)?;
let sent = msg.is_some();
self.send_up(i, msg.into_iter().collect())?;
Ok(sent)
}
/// Client `i` undoes its last step. Returns whether batches went out.
pub fn undo(&mut self, i: usize, recompute: &dyn Recompute) -> Result<bool, Error> {
let msgs = self.client(i)?.undo(recompute)?;
let sent = !msgs.is_empty();
self.send_up(i, msgs)?;
Ok(sent)
}
/// Client `i` redoes its last undone step. Returns whether batches went out.
pub fn redo(&mut self, i: usize, recompute: &dyn Recompute) -> Result<bool, Error> {
let msgs = self.client(i)?.redo(recompute)?;
let sent = !msgs.is_empty();
self.send_up(i, msgs)?;
Ok(sent)
}
/// Client `i`'s connection drops: whatever is in flight either way is lost.
pub fn disconnect(&mut self, i: usize) -> Result<(), Error> {
if !self.is_online(i) {
return Ok(());
}
for link in [self.up.get_mut(i), self.down.get_mut(i)].into_iter().flatten() {
link.queue.clear();
}
if let Some(o) = self.online.get_mut(i) {
*o = false;
}
self.client(i)?.disconnect();
let id = self.ids.get(i).copied().unwrap_or(0);
let out = self.server.leave(id);
self.route(out)
}
/// Client `i` connects again: it says hello with its id and last sequence number; the
/// server's welcome reaches it over the new link.
pub fn reconnect(&mut self, i: usize) -> Result<(), Error> {
if self.is_online(i) {
return Ok(());
}
let hello = self.hellos.get(i).cloned().ok_or_else(|| Error::Invalid(format!("no client {i}")))?;
let ClientMsg::Hello(hello) = self.client(i)?.reconnect(hello) else { return Err(Error::Protocol("hello".into())) };
// The hello travels encoded too.
let hello: ClientMsg = message::decode(&message::encode(&ClientMsg::Hello(hello))?, self.limits.max_message_bytes)?;
let ClientMsg::Hello(hello) = hello else { return Err(Error::Protocol("hello".into())) };
let seen = self.clients.get(i).map_or(0, Client::seq);
let (id, out) = self.server.join(hello).map_err(|m| Error::Protocol(format!("{m:?}")))?;
if out.iter().any(|o| matches!(o, Outgoing::To(to, ServerMsg::Welcome { catch_up: crate::CatchUp::Snapshot(_), .. }) if *to == id)) {
self.snapshots += 1;
if let Some(g) = self.gaps.get_mut(i) {
g.push((seen, self.server.seq()));
}
}
if let Some(c) = self.ids.get_mut(i) {
*c = id;
}
if let Some(o) = self.online.get_mut(i) {
*o = true;
}
self.route(out)
}
/// Deliver the next due message (the earliest; ties go to the lowest link). False when none.
@ -202,8 +321,22 @@ impl Sim {
if dir == 0 {
let Some((_, w)) = self.up.get_mut(i).and_then(|l| l.queue.pop_front()) else { return Ok(false) };
let msg: ClientMsg = self.unwire(&w)?;
let id = self.clients.get(i).map(Client::id).unwrap_or(0);
let travel = match &msg {
ClientMsg::Changes { base_seq, batch_id, .. } => self
.clients
.get(i)
.and_then(|c| c.batch_kind(*batch_id))
.filter(|k| matches!(k, BatchKind::Undo | BatchKind::Redo))
.map(|_| *base_seq),
_ => None,
};
let id = self.ids.get(i).copied().unwrap_or(0);
let out = self.server.receive(id, msg);
for o in &out {
if let Outgoing::All(ServerMsg::Changes(acc)) = o {
self.record(acc, travel);
}
}
self.route(out)?;
} else {
let Some((_, w)) = self.down.get_mut(i).and_then(|l| l.queue.pop_front()) else { return Ok(false) };
@ -214,14 +347,84 @@ impl Sim {
None => Vec::new(),
};
self.repairs += replies.len();
for r in replies {
let w = self.wire(&r)?;
self.send(true, i, w);
}
self.send_up(i, replies)?;
}
Ok(true)
}
/// The server accepted `acc`; `travel`: it is an undo or redo made on top of that sequence
/// number, to check first.
fn record(&mut self, acc: &Accepted, travel: Option<u64>) {
if let Some(base) = travel {
self.undos += 1;
self.check(acc, base);
}
self.log.push(Logged { author: acc.author, touches: Touches::of(&acc.changes) });
self.states.push(self.server.replica().clone());
}
/// The sequence number and author of the last batch for which `hit` holds.
fn last_writer(&self, hit: impl Fn(&Touches) -> bool) -> Option<(u64, u32)> {
self.log.iter().enumerate().rev().find(|(_, l)| hit(&l.touches)).map(|(i, l)| (i as u64 + 1, l.author))
}
/// An undo or redo by `acc.author`, made when the client had seen the server up to `base`,
/// may change a value (or delete a node) only if its author wrote (created) it last, or if
/// the last writer's batch came after `base` (the client couldn't know: concurrent edits are
/// ordered last-writer-wins like any other). Batches the client only saw inside a snapshot
/// are exempt: it compares values then, and can't tell an equal value someone else wrote.
fn check(&mut self, acc: &Accepted, base: u64) {
let Some(state) = self.states.last() else { return };
let gaps = self.ids.iter().position(|id| *id == acc.author).and_then(|i| self.gaps.get(i)).cloned().unwrap_or_default();
let allowed = |w: Option<(u64, u32)>| {
w.is_some_and(|(seq, author)| author == acc.author || seq > base || gaps.iter().any(|(a, b)| *a < seq && seq <= *b))
};
let mut found = Vec::new();
// Deletions are checked where they happen in the batch (moves before them take layers out).
let mut work = state.clone();
for c in &acc.changes {
match c {
Change::Delete { id } => {
let doc = work.document();
for d in doc.subtree(*id) {
// Fine if the author created it, or it (or the branch holding it inside the
// deleted node) was created or moved in after `base`.
let mut path = vec![d];
let mut cur = d;
while let Some(p) = doc.node(cur).and_then(|x| x.parent).filter(|p| cur != *id && *p != *id) {
path.push(p);
cur = p;
}
let ok = path.iter().any(|n| {
let moved = self.last_writer(|t| t.hits(&Target::Node(*n, crate::Prop::Parent)));
allowed(self.last_writer(|t| t.creates(*n))) || moved.is_some_and(|(seq, _)| seq > base)
});
if !ok {
found.push(format!("seq {}: client {} deleted {d} (created by someone else, before the undo)", acc.seq, acc.author));
}
}
}
Change::Create { .. } => {}
other => {
let written = match other {
Change::Set { value, .. } | Change::SetDoc { value, .. } | Change::Put { value, .. } => Some(value),
_ => None,
};
if let Some(t) = Target::of(other)
&& state.value(&t).ok().flatten().as_ref() != written
{
let writer = self.last_writer(|w| w.hits(&t));
if !allowed(writer) {
found.push(format!("seq {}: client {} wrote {t:?}, written last by {writer:?}", acc.seq, acc.author));
}
}
}
}
let _ = work.apply(std::slice::from_ref(c), crate::Mode::Lenient);
}
self.violations.extend(found);
}
/// Deliver until no message is in flight (at most `max_steps`). Returns the steps taken.
pub fn settle(&mut self, recompute: &dyn Recompute, max_steps: usize) -> Result<usize, Error> {
let mut n = 0;
@ -234,11 +437,14 @@ impl Sim {
Ok(n)
}
/// Whether every client caught up with the server and holds exactly its state; the first
/// difference otherwise.
/// Whether every client is connected, caught up with the server and holds exactly its state;
/// the first difference otherwise.
pub fn converged(&self) -> Result<(), String> {
let server: &Replica = self.server.replica();
for c in &self.clients {
for (i, c) in self.clients.iter().enumerate() {
if !self.is_online(i) {
return Err(format!("client {} is offline", c.id()));
}
if c.pending() > 0 || c.seq() != self.server.seq() {
return Err(format!("client {} at seq {} with {} pending; server at {}", c.id(), c.seq(), c.pending(), self.server.seq()));
}

6
crates/collab/src/sim_tests.rs
View File

@ -10,7 +10,7 @@ use crate::sim::{Rng, Sim};
use crate::{NoRecompute, Recompute};
/// A toy derived value: a frame's width is ten times its number of children.
struct Width;
pub(crate) struct Width;
impl Recompute for Width {
fn recompute(&self, doc: &mut Document, _: &Document, touched: &BTreeSet<NodeId>) {
@ -26,7 +26,7 @@ impl Recompute for Width {
}
}
fn base() -> Document {
pub(crate) fn base() -> Document {
let mut d = Document::new("Shared");
let page = d.first_page().unwrap();
for i in 0..3 {
@ -41,7 +41,7 @@ fn base() -> Document {
}
/// One random edit of `doc`, the way a command would make it (derived data recomputed after).
fn random_edit(doc: &mut Document, rng: &mut Rng, derive: &dyn Recompute) {
pub(crate) fn random_edit(doc: &mut Document, rng: &mut Rng, derive: &dyn Recompute) {
let before = doc.clone();
doc.take_dirty();
let ids: Vec<NodeId> = doc.ids().collect();

220
crates/collab/src/undo_tests.rs Normal file
View File

@ -0,0 +1,220 @@
//! Per-user undo and redo in a live session: an undo reverts only the user's own writes, creates
//! and deletes invert, rejected work leaves the history, and random sessions with undo converge
//! without ever reverting someone else's change.
use figcraft_doc::{Document, NodeId, NodeKind};
use figcraft_geom::Affine;
use crate::sim::{Rng, Sim};
use crate::sim_tests::{Width, base, random_edit};
use crate::{BatchKind, NoRecompute, Notice, Recompute};
/// Client `i` edits its document with `f`.
fn edit(sim: &mut Sim, i: usize, f: impl FnOnce(&mut Document)) {
let mut d = sim.document(i).unwrap().clone();
f(&mut d);
sim.edit(i, &d).unwrap();
}
fn settle(sim: &mut Sim) {
sim.settle(&NoRecompute, 10_000).unwrap();
sim.converged().unwrap();
}
/// The first frame of the shared file and its first child.
fn frame(sim: &Sim) -> (NodeId, NodeId) {
let doc = sim.server.document();
let page = doc.first_page().unwrap();
let f = doc.node(page).unwrap().children[0];
(f, doc.node(f).unwrap().children[0])
}
#[test]
fn undo_reverts_only_your_own_writes() {
let mut sim = Sim::new(base(), 2, 1, 5).unwrap();
let (f, _) = frame(&sim);
let old = sim.server.document().node(f).unwrap().clone();
edit(&mut sim, 0, |d| {
let n = d.node_mut(f).unwrap();
n.name = "Mine".into();
n.transform = Affine::translate((10.0, 10.0));
});
settle(&mut sim);
// Someone else moves the frame after that.
edit(&mut sim, 1, |d| d.node_mut(f).unwrap().transform = Affine::translate((99.0, 0.0)));
settle(&mut sim);
assert!(sim.undo(0, &NoRecompute).unwrap());
settle(&mut sim);
let n = sim.server.document().node(f).unwrap();
assert_eq!(n.name, old.name, "my rename is undone");
assert_eq!(n.transform, Affine::translate((99.0, 0.0)), "their later move stays");
// Redo puts the rename back, still without touching their move.
assert!(sim.redo(0, &NoRecompute).unwrap());
settle(&mut sim);
let n = sim.server.document().node(f).unwrap();
assert_eq!((n.name.as_str(), n.transform), ("Mine", Affine::translate((99.0, 0.0))));
// Remote batches never enter the history: b has its own step, a has nothing left to redo.
assert!(!sim.clients()[0].can_redo());
assert!(sim.clients()[1].can_undo());
assert!(sim.violations.is_empty(), "{:?}", sim.violations);
}
#[test]
fn creates_and_deletes_invert() {
let mut sim = Sim::new(base(), 2, 2, 5).unwrap();
let (f, _) = frame(&sim);
let start = sim.server.replica().clone();
edit(&mut sim, 0, |d| {
d.create(f, NodeKind::Rectangle, "New", 1.0, 1.0, 5.0, 5.0).unwrap();
});
settle(&mut sim);
let created = sim.server.replica().clone();
sim.undo(0, &NoRecompute).unwrap();
settle(&mut sim);
assert_eq!(sim.server.document(), start.document(), "the creation is undone");
sim.redo(0, &NoRecompute).unwrap();
settle(&mut sim);
assert_eq!(sim.server.replica(), &created, "redo brings it back with its id and position");
// Deleting a frame with layers: undo brings the subtree back whole, in place.
edit(&mut sim, 1, |d| d.remove(f).unwrap());
settle(&mut sim);
assert!(!sim.server.document().contains(f));
sim.undo(1, &NoRecompute).unwrap();
settle(&mut sim);
assert_eq!(sim.server.replica(), &created);
sim.redo(1, &NoRecompute).unwrap();
settle(&mut sim);
assert!(!sim.server.document().contains(f));
assert!(sim.violations.is_empty(), "{:?}", sim.violations);
}
#[test]
fn a_created_frame_someone_built_in_stays() {
let mut sim = Sim::new(base(), 2, 3, 5).unwrap();
let page = sim.server.document().first_page().unwrap();
edit(&mut sim, 0, |d| {
d.create(page, NodeKind::Frame, "Mine", 500.0, 0.0, 100.0, 100.0).unwrap();
});
settle(&mut sim);
let mine = *sim.server.document().node(page).unwrap().children.last().unwrap();
edit(&mut sim, 1, |d| {
d.create(mine, NodeKind::Rectangle, "Theirs", 0.0, 0.0, 5.0, 5.0).unwrap();
});
settle(&mut sim);
// Nothing of a's step can be undone without deleting b's rectangle: the step is skipped.
assert!(!sim.undo(0, &NoRecompute).unwrap());
settle(&mut sim);
assert_eq!(sim.server.document().node(mine).unwrap().children.len(), 1);
}
#[test]
fn rejected_edits_leave_the_history_with_a_reason() {
let mut sim = Sim::new(base(), 2, 4, 5).unwrap();
let (f, r) = frame(&sim);
// At the same time: b deletes the frame, a renames a layer inside it (b's lands first).
edit(&mut sim, 1, |d| d.remove(f).unwrap());
while sim.server.seq() == 0 {
sim.step(&NoRecompute).unwrap();
}
edit(&mut sim, 0, |d| d.node_mut(r).unwrap().name = "Too late".into());
assert!(sim.clients()[0].can_undo());
settle(&mut sim);
assert_eq!(sim.rejected, 1);
let c = &mut sim.clients_mut()[0];
assert!(!c.can_undo(), "no undo step for rejected work");
let notices = c.take_notices();
assert!(matches!(notices.as_slice(), [Notice::Rejected { kind: BatchKind::Edit, .. }]), "{notices:?}");
assert!(notices[0].message().contains("deleted by someone else"), "{}", notices[0].message());
assert!(c.take_notices().is_empty());
}
#[test]
fn undo_right_after_an_edit_that_gets_rejected_is_reverted() {
for seed in 0..20 {
let mut sim = Sim::new(base(), 2, seed, 8).unwrap();
let (f, r) = frame(&sim);
let other = sim.server.document().node(f).unwrap().children[1];
edit(&mut sim, 1, |d| d.remove(r).unwrap());
while sim.server.seq() == 0 {
sim.step(&NoRecompute).unwrap();
}
// a renames r (refused: b deleted it) and another layer, and undoes before hearing back.
edit(&mut sim, 0, |d| {
d.node_mut(r).unwrap().name = "Gone".into();
d.node_mut(other).unwrap().name = "Kept?".into();
});
sim.undo(0, &NoRecompute).unwrap();
settle(&mut sim);
let name = &sim.server.document().node(other).unwrap().name;
assert_ne!(name, "Kept?", "seed {seed}");
assert!(!sim.clients()[0].can_undo() && !sim.clients()[0].can_redo(), "seed {seed}");
}
}
/// Random edits, undos and redos by several clients, with or without dropped connections.
pub(crate) fn run_with_undo(seed: u64, clients: usize, rounds: usize, derive: &dyn Recompute, drops: bool, sim: Option<Sim>) -> Sim {
let mut sim = sim.unwrap_or_else(|| Sim::new(base(), clients, seed, 12).unwrap());
for _ in 0..rounds {
let i = sim.rng().below(clients);
match sim.rng().below(10) {
0 | 1 => {
sim.undo(i, derive).unwrap();
}
2 => {
sim.redo(i, derive).unwrap();
}
3 if drops => {
if sim.is_online(i) {
sim.disconnect(i).unwrap();
} else {
sim.reconnect(i).unwrap();
}
}
_ => {
let mut doc = sim.document(i).unwrap().clone();
let mut rng = Rng::new(sim.rng().next_u64());
random_edit(&mut doc, &mut rng, derive);
sim.edit(i, &doc).unwrap();
}
}
for _ in 0..sim.rng().below(4) {
sim.step(derive).unwrap();
}
}
for i in 0..clients {
sim.reconnect(i).unwrap();
}
sim.settle(derive, 1_000_000).unwrap();
if let Err(e) = sim.converged() {
panic!("seed {seed}: {e}");
}
assert!(sim.violations.is_empty(), "seed {seed}: {:?}", sim.violations);
sim
}
#[test]
fn random_sessions_with_undo_converge_and_never_revert_others() {
let mut undos = 0;
for seed in 0..12 {
let sim = run_with_undo(seed, 3, 80, &NoRecompute, false, None);
undos += sim.undos;
assert!(sim.server.document().validate().is_empty());
}
for seed in 50..56 {
let sim = run_with_undo(seed, 3, 60, &Width, false, None);
undos += sim.undos;
}
assert!(undos > 100, "only {undos} undos and redos accepted");
}
#[test]
#[ignore = "longer run for development: cargo test -p figcraft-collab -- --ignored"]
fn many_seeds_with_undo_converge() {
for seed in 2000..2600 {
// 2 to 5 people, short and long delays, drops, and logs short enough to need snapshots.
let n = 2 + (seed % 4) as usize;
let limits = crate::Limits { max_log_batches: if seed % 3 == 0 { 1 + (seed % 7) as usize } else { 10_000 }, ..crate::Limits::default() };
let sim = Sim::with_limits(base(), n, seed, 1 + seed % 20, limits).unwrap();
run_with_undo(seed, n, 200, &Width, seed % 2 == 0, Some(sim));
}
}

3
crates/engine/src/collab.rs
View File

@ -36,6 +36,9 @@ pub fn specs() -> Vec<CommandSpec> {
fn diff_cmd(s: &mut Session, p: &Value) -> Result<Value> {
let ds = s.ds()?;
let from: Arc<Document> = match string(p, "against").as_deref().unwrap_or("undo") {
"undo" if s.live_doc(ds.uid) => {
return Err(EngineError::Disabled("collab.diff".into(), "in a live session undo is per user: compare against \"saved\"".into()));
}
"undo" => {
ds.history.undo.last().map(|e| e.doc.clone()).ok_or_else(|| EngineError::Disabled("collab.diff".into(), "nothing to undo".into()))?
}

365
crates/engine/src/collab_live_tests.rs Normal file
View File

@ -0,0 +1,365 @@
//! Documents in a live session through the engine's live state, with a scripted link and the core
//! server in the test: commands, gestures and groups sync as one step each, `edit.undo` /
//! `edit.redo` take back only the user's own steps, a refused edit comes back with a toast, a
//! dropped connection resumes and sends what was made offline, and outside a session undo is
//! unchanged.
use std::collections::VecDeque;
use std::sync::{Arc, Mutex};
use figcraft_collab::sim::Rng;
use figcraft_collab::{Client, ClientMsg, Limits, Outgoing, Server, ServerMsg};
use figcraft_doc::{Document, NodeId};
use figcraft_geom::Affine;
use serde_json::json;
use crate::Session;
use crate::collab_tests::{base, random_command};
use crate::live::{Event, Role, Status};
use crate::live_tests::{Mock, Script, info};
const LINK: &str = "figcraft://join?host=127.0.0.1:47900&file=f&token=t";
/// One person: an engine session with a scripted link, what it sent that the server hasn't had
/// yet, and what the server sent that it hasn't taken in yet.
struct Person {
s: Session,
script: Arc<Mutex<Script>>,
id: u32,
up: VecDeque<ClientMsg>,
down: VecDeque<Event>,
online: bool,
}
/// A session server (the host's, minus the sockets) and several people joined as guests.
struct Room {
server: Server,
people: Vec<Person>,
rng: Rng,
}
fn welcome_of(out: &[Outgoing], id: u32) -> ServerMsg {
out.iter().find_map(|o| if let Outgoing::To(to, w @ ServerMsg::Welcome { .. }) = o { (*to == id).then(|| w.clone()) } else { None }).unwrap()
}
impl Room {
fn new(doc: Document, n: usize, seed: u64) -> Room {
let mut room = Room { server: Server::new(doc, "t", Limits::default()), people: Vec::new(), rng: Rng::new(seed) };
for _ in 0..n {
let mut s = Session::new();
let script = Arc::new(Mutex::new(Script::default()));
s.live.install(Box::new(Mock(script.clone())));
s.execute("collab.join", &json!({"link": LINK})).unwrap();
let ClientMsg::Hello(h) = Client::hello("f", "t", "Person", "") else { panic!("not a hello") };
let (id, out) = room.server.join(h).unwrap();
let w = welcome_of(&out, id);
script.lock().unwrap().events.extend([Event::Welcome { info: info(Role::Guest, id), peers: vec![] }, Event::Sync(w)]);
s.live_poll();
assert!(s.live.syncing() && s.live_doc(s.ds().unwrap().uid), "joined: the shared file is the active tab");
room.people.push(Person { s, script, id, up: VecDeque::new(), down: VecDeque::new(), online: true });
room.route(out.into_iter().filter(|o| !matches!(o, Outgoing::To(_, ServerMsg::Welcome { .. }))).collect());
}
room
}
fn s(&mut self, i: usize) -> &mut Session {
&mut self.people[i].s
}
fn route(&mut self, out: Vec<Outgoing>) {
for o in out {
let (msg, to): (ServerMsg, Vec<usize>) = match o {
Outgoing::To(id, m) => (m, (0..self.people.len()).filter(|i| self.people[*i].id == id).collect()),
Outgoing::All(m) => (m, (0..self.people.len()).collect()),
Outgoing::Others(id, m) => (m, (0..self.people.len()).filter(|i| self.people[*i].id != id).collect()),
};
if !matches!(msg, ServerMsg::Changes(_) | ServerMsg::Reject { .. } | ServerMsg::Welcome { .. }) {
continue;
}
let to: Vec<usize> = to.into_iter().filter(|i| self.people[*i].online).collect();
for i in to {
// Through the binary encoding, like the network.
let m = figcraft_collab::message::decode(&figcraft_collab::message::encode(&msg).unwrap(), usize::MAX).unwrap();
self.people[i].down.push_back(Event::Sync(m));
}
}
}
/// What everyone sent (the UI calls `live_sync` every frame).
fn pump(&mut self) {
for p in &mut self.people {
p.s.live_sync();
let msgs = std::mem::take(&mut p.script.lock().unwrap().msgs);
if p.online {
p.up.extend(msgs);
}
}
}
/// Deliver one message on a random busy link. False when nothing is in flight.
fn step(&mut self) -> bool {
self.pump();
let busy: Vec<(bool, usize)> = (0..self.people.len())
.flat_map(|i| [(true, i), (false, i)])
.filter(|(up, i)| if *up { !self.people[*i].up.is_empty() } else { !self.people[*i].down.is_empty() })
.collect();
let Some(&(up, i)) = self.rng.pick(&busy) else { return false };
if up {
let msg = self.people[i].up.pop_front().unwrap();
let out = self.server.receive(self.people[i].id, msg);
self.route(out);
} else {
let p = &mut self.people[i];
let e = p.down.pop_front().unwrap();
p.script.lock().unwrap().events.push(e);
p.s.live_poll();
}
self.pump();
true
}
fn settle(&mut self) {
let mut n = 0;
while self.step() {
n += 1;
assert!(n < 1_000_000, "still busy");
}
}
/// Person `i`'s connection drops: what is in flight is lost; the link says it's reconnecting.
fn disconnect(&mut self, i: usize) {
let p = &mut self.people[i];
p.up.clear();
p.down.clear();
p.online = false;
p.script.lock().unwrap().events.push(Event::Status(Status::Reconnecting { attempt: 1 }));
p.s.live_poll();
let out = self.server.leave(self.people[i].id);
self.route(out);
}
/// Person `i` is back: the link says hello with the client's id and last sequence number.
fn reconnect(&mut self, i: usize) {
let p = &mut self.people[i];
let ClientMsg::Hello(mut h) = Client::hello("f", "t", "Person", "") else { panic!("not a hello") };
h.resume = p.s.live.client().map(Client::resume);
let (id, out) = self.server.join(h).unwrap();
assert_eq!(id, p.id, "the same client");
p.online = true;
p.down.push_back(Event::Welcome { info: info(Role::Guest, id), peers: vec![] });
self.route(out);
}
/// Everyone holds the server's document, in the session and on the canvas.
fn check(&self) {
for (i, p) in self.people.iter().enumerate() {
let c = p.s.live.client().unwrap();
assert_eq!(c.pending(), 0, "person {i}");
assert!(c.view() == self.server.replica(), "person {i}: {}", figcraft_collab::sim::describe(self.server.replica(), c.view()));
assert!(*p.s.doc().unwrap() == *self.server.document(), "person {i}: the canvas shows the session's document");
}
}
}
/// The shared file's first frame and one of its rectangles.
fn frame_and_rect(doc: &Document) -> (NodeId, NodeId) {
let page = doc.first_page().unwrap();
let f = doc.node(page).unwrap().children[0];
(f, doc.node(f).unwrap().children[0])
}
#[test]
fn edits_sync_and_undo_takes_back_only_your_own() {
let mut room = Room::new(base(), 2, 1);
let (f, r) = frame_and_rect(room.server.document());
let old = room.server.document().node(r).unwrap().name.clone();
room.s(0).execute("node.set", &json!({"id": r, "props": {"name": "Mine"}})).unwrap();
room.settle();
room.s(1).execute("transform.move", &json!({"ids": [r], "dx": 25, "dy": 0})).unwrap();
room.settle();
room.check();
let moved = room.server.document().node(r).unwrap().transform;
// Person 0 undoes: their rename goes, person 1's later move stays.
assert_eq!(room.s(0).execute("edit.undo", &json!({})).unwrap()["undone"], true);
room.settle();
room.check();
let n = room.server.document().node(r).unwrap();
assert_eq!((&n.name, n.transform), (&old, moved));
assert_eq!(room.s(0).status()["redo"], 1);
assert_eq!(room.s(0).execute("edit.undo", &json!({})).unwrap()["undone"], false, "nothing else of theirs");
assert_eq!(room.s(1).status()["undo"], 1, "remote edits never enter someone else's history");
room.s(0).execute("edit.redo", &json!({})).unwrap();
room.settle();
room.check();
assert_eq!(room.server.document().node(r).unwrap().name, "Mine");
// Deleting a frame with layers and undoing it brings it back whole.
let before = room.server.replica().clone();
room.s(1).execute("edit.delete", &json!({"ids": [f]})).unwrap();
room.settle();
assert!(!room.server.document().contains(f));
room.s(1).execute("edit.undo", &json!({})).unwrap();
room.settle();
room.check();
assert!(room.server.replica() == &before);
// In a live session the snapshot comparison of `collab.diff` doesn't apply.
assert!(room.s(0).execute("collab.diff", &json!({})).is_err());
assert!(room.s(0).execute("collab.diff", &json!({"against": "saved"})).is_ok());
}
#[test]
fn a_gesture_is_one_step() {
let mut room = Room::new(base(), 2, 2);
let (_, r) = frame_and_rect(room.server.document());
let start = room.server.document().node(r).unwrap().transform;
let s = room.s(0);
s.begin_interaction("Move").unwrap();
for dx in [5.0, 10.0, 15.0] {
s.preview(|doc, _| {
if let Some(n) = doc.node_mut(r) {
n.transform = Affine::translate((dx, 0.0)) * start;
}
Ok(())
})
.unwrap();
}
// Nothing goes out during the drag; someone else renames the layer meanwhile.
room.pump();
assert!(room.people[0].up.is_empty());
room.s(1).execute("node.set", &json!({"id": r, "props": {"name": "Theirs"}})).unwrap();
room.settle();
room.s(0).commit_interaction().unwrap();
room.settle();
room.check();
let n = room.server.document().node(r).unwrap();
assert_eq!((n.name.as_str(), n.transform), ("Theirs", Affine::translate((15.0, 0.0)) * start));
assert_eq!(room.s(0).status()["undo"], 1, "one step for the whole drag");
room.s(0).execute("edit.undo", &json!({})).unwrap();
room.settle();
room.check();
let n = room.server.document().node(r).unwrap();
assert_eq!((n.name.as_str(), n.transform), ("Theirs", start));
}
#[test]
fn a_group_of_commands_syncs_as_one_step() {
let mut room = Room::new(base(), 2, 3);
let (f, r) = frame_and_rect(room.server.document());
let start = room.server.replica().clone();
let cmds = vec![
("node.set".to_string(), json!({"id": r, "props": {"name": "A"}})),
("node.set".to_string(), json!({"id": f, "props": {"name": "B"}})),
("shape.rectangle".to_string(), json!({"x": 0, "y": 0, "width": 5, "height": 5})),
];
room.s(0).batch("Three", &cmds).unwrap();
room.pump();
assert_eq!(room.people[0].up.len(), 1, "one batch");
room.settle();
room.check();
assert_eq!(room.s(0).status()["undo"], 1);
room.s(0).execute("edit.undo", &json!({})).unwrap();
room.settle();
room.check();
assert!(room.server.replica() == &start, "all three are taken back");
}
#[test]
fn an_undo_waits_for_its_edit_and_shows_at_once() {
let mut room = Room::new(base(), 2, 6);
let (_, r) = frame_and_rect(room.server.document());
let old = room.server.document().node(r).unwrap().name.clone();
let start = room.server.replica().clone();
room.s(0).execute("node.set", &json!({"id": r, "props": {"name": "Quick"}})).unwrap();
room.s(0).execute("edit.undo", &json!({})).unwrap();
assert_eq!(room.s(0).doc().unwrap().node(r).unwrap().name, old, "shown at once");
room.pump();
assert_eq!(room.people[0].up.len(), 1, "only the edit goes out until it is confirmed");
room.settle();
room.check();
assert!(room.server.replica() == &start);
}
#[test]
fn a_refused_edit_is_rolled_back_with_a_toast() {
let mut room = Room::new(base(), 2, 4);
let (f, r) = frame_and_rect(room.server.document());
// Person 1 deletes the frame; person 0 renames a layer inside it before hearing of it.
room.s(1).execute("edit.delete", &json!({"ids": [f]})).unwrap();
room.pump();
let msg = room.people[1].up.pop_front().unwrap();
let out = room.server.receive(room.people[1].id, msg);
room.route(out);
room.s(0).execute("node.set", &json!({"id": r, "props": {"name": "Too late"}})).unwrap();
room.settle();
room.check();
let s = room.s(0);
assert!(s.message.as_deref().unwrap_or_default().contains("deleted by someone else"), "{:?}", s.message);
assert_eq!(s.status()["undo"], 0, "no undo step for refused work");
}
#[test]
fn a_dropped_connection_keeps_editing_and_catches_up() {
let mut room = Room::new(base(), 2, 7);
let (f, r) = frame_and_rect(room.server.document());
room.s(0).execute("node.set", &json!({"id": r, "props": {"name": "Sent"}})).unwrap();
room.pump();
room.disconnect(0);
// Offline: edits and undo keep working; nothing goes out.
room.s(0).execute("shape.rectangle", &json!({"x": 0, "y": 0, "width": 5, "height": 5})).unwrap();
room.s(0).execute("edit.undo", &json!({})).unwrap();
room.s(0).execute("node.set", &json!({"id": f, "props": {"name": "Offline"}})).unwrap();
room.pump();
assert!(room.people[0].up.is_empty());
room.s(1).execute("transform.move", &json!({"ids": [r], "dx": 5, "dy": 5})).unwrap();
room.settle();
room.reconnect(0);
room.settle();
room.check();
let doc = room.server.document();
assert_eq!((doc.node(r).unwrap().name.as_str(), doc.node(f).unwrap().name.as_str()), ("Sent", "Offline"));
}
#[test]
fn random_commands_with_undo_and_drops_converge_through_the_engine() {
for seed in 0..6 {
let mut room = Room::new(base(), 3, seed);
let mut rng = Rng::new(seed + 100);
for _ in 0..80 {
let i = rng.below(3);
if rng.chance(0.06) {
if room.people[i].online {
room.disconnect(i);
} else {
room.reconnect(i);
}
continue;
}
let (cmd, params) = random_command(room.people[i].s.doc().unwrap(), &mut rng, true);
let _ = room.s(i).execute(cmd, &params);
for _ in 0..rng.below(4) {
room.step();
}
}
for i in 0..3 {
if !room.people[i].online {
room.reconnect(i);
}
}
room.settle();
room.check();
}
}
#[test]
fn leaving_the_session_goes_back_to_snapshot_undo() {
let mut room = Room::new(base(), 1, 5);
let (_, r) = frame_and_rect(room.server.document());
room.s(0).execute("node.set", &json!({"id": r, "props": {"name": "Live"}})).unwrap();
room.settle();
let s = room.s(0);
assert_eq!(s.execute("collab.leave", &json!({})).unwrap()["ended"], true);
assert_eq!(s.status()["undo"], 0, "the live history doesn't carry over");
s.execute("node.set", &json!({"id": r, "props": {"name": "Solo"}})).unwrap();
assert_eq!(s.status()["undo"], 1);
s.execute("edit.undo", &json!({})).unwrap();
assert_eq!(s.doc().unwrap().node(r).unwrap().name, "Live");
}

4
crates/engine/src/collab_tests.rs
View File

@ -10,7 +10,7 @@ use crate::Session;
use crate::collab::EngineRecompute;
/// A starting file: a frame with a few layers and some text.
fn base() -> Document {
pub(crate) fn base() -> Document {
let mut s = Session::new();
let f = s.execute("frame.create", &json!({"x": 0, "y": 0, "width": 400, "height": 300})).unwrap()["id"].clone();
for i in 0..3 {
@ -32,7 +32,7 @@ fn base() -> Document {
/// A random command with parameters that make sense for `doc` (often valid, sometimes not:
/// cycles, deleted targets…).
fn random_command(doc: &Document, rng: &mut Rng, with_undo: bool) -> (&'static str, Value) {
pub(crate) fn random_command(doc: &Document, rng: &mut Rng, with_undo: bool) -> (&'static str, Value) {
let store = doc.libraries.store;
let all: Vec<&figcraft_doc::Node> = doc.nodes().filter(|n| store.is_none_or(|st| doc.page_of(n.id) != Some(st))).collect();
let ids = |f: &dyn Fn(&figcraft_doc::Node) -> bool| -> Vec<NodeId> { all.iter().filter(|n| f(n)).map(|n| n.id).collect() };

75
crates/engine/src/collab_undo_tests.rs Normal file
View File

@ -0,0 +1,75 @@
//! Per-user undo with real commands: several people running random commands, undoing and
//! redoing, some dropping their connection now and then, all end up with the server's document,
//! and no undo or redo ever reverts a change by someone else that its author had seen.
use figcraft_collab::sim::{Rng, Sim};
use figcraft_collab::{Limits, Recompute};
use crate::Session;
use crate::collab::EngineRecompute;
use crate::collab_tests::{base, random_command};
fn session(seed: u64, clients: usize, rounds: usize, drops: bool, limits: Limits, recompute: &dyn Recompute) -> Sim {
let mut sim = Sim::with_limits(base(), clients, seed, 10, limits).unwrap();
for _ in 0..rounds {
let i = sim.rng().below(clients);
match sim.rng().below(12) {
0 | 1 => {
sim.undo(i, recompute).unwrap();
}
2 => {
sim.redo(i, recompute).unwrap();
}
3 if drops => {
if sim.is_online(i) {
sim.disconnect(i).unwrap();
} else {
sim.reconnect(i).unwrap();
}
}
_ => {
let mut s = Session::with_document(sim.document(i).unwrap().clone());
let mut rng = Rng::new(sim.rng().next_u64());
let (cmd, params) = random_command(s.doc().unwrap(), &mut rng, false);
if s.execute(cmd, &params).is_ok() {
sim.edit(i, s.doc().unwrap()).unwrap_or_else(|e| panic!("seed {seed}: {cmd} {params}: {e}"));
}
}
}
for _ in 0..sim.rng().below(4) {
sim.step(recompute).unwrap();
}
}
for i in 0..clients {
sim.reconnect(i).unwrap();
}
sim.settle(recompute, 1_000_000).unwrap();
if let Err(e) = sim.converged() {
panic!("seed {seed}: {e}");
}
assert!(sim.violations.is_empty(), "seed {seed}: {:?}", sim.violations);
assert!(sim.server.document().validate().is_empty(), "seed {seed}: {:?}", sim.server.document().validate());
sim
}
#[test]
fn people_undoing_real_commands_converge_without_reverting_others() {
let mut undos = 0;
for seed in 0..4 {
undos += session(seed, 3, 60, false, Limits::default(), &EngineRecompute).undos;
}
// Dropped connections, and a log short enough that long absences catch up with snapshots.
for seed in 10..13 {
undos += session(seed, 3, 60, true, Limits { max_log_batches: 4, ..Limits::default() }, &EngineRecompute).undos;
}
assert!(undos > 20, "only {undos} undos and redos accepted");
}
#[test]
#[ignore = "longer run for development: cargo test -p figcraft-engine collab -- --ignored"]
fn many_people_undoing_real_commands_converge() {
for seed in 100..160 {
let limits = Limits { max_log_batches: if seed % 2 == 0 { 5 } else { 10_000 }, ..Limits::default() };
session(seed, 4, 150, seed % 3 == 0, limits, &EngineRecompute);
}
}

2
crates/engine/src/inspect.rs
View File

@ -263,7 +263,7 @@ pub fn document(s: &Session, depth: usize, all_pages: bool) -> Value {
"images": doc.images.len(),
"colorProfile": doc.color_profile.id(),
"textEdit": ds.text_edit,
"history": {"undo": ds.history.undo.iter().rev().take(20).map(|h| h.label.clone()).collect::<Vec<_>>(), "redo": ds.history.redo.len()},
"history": {"undo": s.undo_labels(20), "redo": s.undo_depth().1},
})
}

95
crates/engine/src/lib.rs
View File

@ -334,6 +334,15 @@ impl DocState {
self.checkpoint = self.doc.clone();
}
/// Push a snapshot undo step: the document was `before` (shown with `selection` on `page`).
pub(crate) fn push_undo(&mut self, label: &str, before: Arc<Document>, selection: Vec<NodeId>, page: NodeId) {
self.history.undo.push(HistoryEntry { label: label.to_string(), doc: before, selection, page });
if self.history.undo.len() > self.history.limit.max(1) {
self.history.undo.remove(0);
}
self.history.redo.clear();
}
pub fn is_dirty(&self) -> bool {
let doc = self.preview.as_ref().map(|p| &p.doc).or(self.text_preview.as_ref()).unwrap_or(&self.doc);
!Arc::ptr_eq(doc, &self.saved_doc) || self.versions_changed
@ -673,16 +682,45 @@ impl Session {
let changed = !Arc::ptr_eq(&ds.doc, &bdoc);
if changed {
normalize(ds.doc_mut(), &bdoc);
if spec.history && ds.interaction.is_none() {
ds.history.undo.push(HistoryEntry { label: spec.label.to_string(), doc: bdoc, selection: bsel, page: bpage });
if ds.history.undo.len() > ds.history.limit.max(1) {
ds.history.undo.remove(0);
}
ds.history.redo.clear();
}
}
let edit = changed && ds.interaction.is_none();
ds.clean_selection();
ds.revision += 1;
if edit {
self.record(uid, spec.history.then_some(spec.label), bdoc, bsel, bpage);
}
}
/// Record an edit of document `uid` (it was `before`, shown with `selection` on `page`) as one
/// undo step called `label` (`None`: not undoable). On a live session's document the edit is
/// synced and the step is the user's own ([`live`]); otherwise it is a snapshot of `before`.
pub(crate) fn record(&mut self, uid: u64, label: Option<&str>, before: Arc<Document>, selection: Vec<NodeId>, page: NodeId) {
if self.live_doc(uid) {
return self.live_edit(label);
}
if let (Some(label), Some(ds)) = (label, self.docs.iter_mut().find(|d| d.uid == uid)) {
ds.push_undo(label, before, selection, page);
}
}
/// How many undo and redo steps the active document has (the user's own in a live session).
pub fn undo_depth(&self) -> (usize, usize) {
match self.ds() {
Ok(ds) if self.live_doc(ds.uid) => self.live.client().map_or((0, 0), figcraft_collab::Client::history_len),
Ok(ds) => (ds.history.undo.len(), ds.history.redo.len()),
Err(_) => (0, 0),
}
}
/// The labels of the active document's last `n` undo steps, the next one first.
pub fn undo_labels(&self, n: usize) -> Vec<String> {
match self.ds() {
Ok(ds) if self.live_doc(ds.uid) => {
self.live.client().map(|c| c.undo_labels().rev().take(n).map(str::to_string).collect()).unwrap_or_default()
}
Ok(ds) => ds.history.undo.iter().rev().take(n).map(|h| h.label.clone()).collect(),
Err(_) => Vec::new(),
}
}
/// Start a gesture: later [`preview`](Self::preview) calls re-apply edits to this snapshot.
@ -724,13 +762,13 @@ impl Session {
pub fn commit_interaction(&mut self) -> Result<()> {
let ds = self.ds_mut()?;
let Some(it) = ds.interaction.take() else { return Ok(()) };
let (uid, page) = (ds.uid, ds.page);
ds.revision += 1;
if !Arc::ptr_eq(&it.doc, &ds.doc) && *it.doc != *ds.doc {
ds.history.undo.push(HistoryEntry { label: it.label, doc: it.doc, selection: it.selection, page: ds.page });
ds.history.redo.clear();
self.record(uid, Some(&it.label), it.doc, it.selection, page);
} else {
ds.doc = it.doc;
}
ds.revision += 1;
Ok(())
}
@ -747,12 +785,21 @@ impl Session {
pub fn undo(&mut self) -> Result<bool> {
textpreview::end(self);
let live = self.ds().is_ok_and(|d| self.live_doc(d.uid));
let ds = self.ds_mut()?;
if ds.preview.is_some() {
return Err(EngineError::Disabled("edit.undo".into(), PREVIEW_READ_ONLY.into()));
}
ds.interaction = None;
if let Some(it) = ds.interaction.take()
&& live
{
// The gesture in progress is dropped (it was never synced).
ds.doc = it.doc;
}
ds.text_edit = None;
if live {
return Ok(self.live_travel(true));
}
let Some(e) = ds.history.undo.pop() else { return Ok(false) };
let redo = HistoryEntry { label: e.label.clone(), doc: ds.doc.clone(), selection: ds.selection.clone(), page: ds.page };
ds.history.redo.push(redo);
@ -768,12 +815,20 @@ impl Session {
pub fn redo(&mut self) -> Result<bool> {
textpreview::end(self);
let live = self.ds().is_ok_and(|d| self.live_doc(d.uid));
let ds = self.ds_mut()?;
if ds.preview.is_some() {
return Err(EngineError::Disabled("edit.redo".into(), PREVIEW_READ_ONLY.into()));
}
ds.interaction = None;
if let Some(it) = ds.interaction.take()
&& live
{
ds.doc = it.doc;
}
ds.text_edit = None;
if live {
return Ok(self.live_travel(false));
}
let Some(e) = ds.history.redo.pop() else { return Ok(false) };
let undo = HistoryEntry { label: e.label.clone(), doc: ds.doc.clone(), selection: ds.selection.clone(), page: ds.page };
ds.history.undo.push(undo);
@ -804,7 +859,9 @@ impl Session {
let (before, sel, page, saved_len, uid) = (ds.doc.clone(), ds.selection.clone(), ds.page, ds.history.undo.len(), ds.uid);
// The steps inside are dropped again below, so they mustn't push the oldest ones out.
let limit = std::mem::replace(&mut ds.history.limit, usize::MAX);
self.live_hold(uid, true);
let result = f(self);
self.live_hold(uid, false);
let Some(ds) = self.docs.iter_mut().find(|d| d.uid == uid) else { return result };
ds.history.limit = limit;
ds.history.undo.truncate(saved_len);
@ -813,11 +870,7 @@ impl Session {
ds.selection = sel;
ds.page = page;
} else if !Arc::ptr_eq(&before, &ds.doc) {
ds.history.undo.push(HistoryEntry { label: label.to_string(), doc: before, selection: sel, page });
if ds.history.undo.len() > limit.max(1) {
ds.history.undo.remove(0);
}
ds.history.redo.clear();
self.record(uid, Some(label), before, sel, page);
}
result
}
@ -833,8 +886,8 @@ impl Session {
"selection": ds.selection,
"tool": self.tool.tool.id(),
"zoom": ds.view.zoom,
"undo": ds.history.undo.len(),
"redo": ds.history.redo.len(),
"undo": self.undo_depth().0,
"redo": self.undo_depth().1,
"documents": self.docs.len(),
}),
Err(_) => json!({"documents": 0}),
@ -1071,8 +1124,12 @@ mod autosave_tests;
#[cfg(test)]
mod binding_lists_tests;
#[cfg(test)]
mod collab_live_tests;
#[cfg(test)]
mod collab_tests;
#[cfg(test)]
mod collab_undo_tests;
#[cfg(test)]
mod comments_polish_tests;
#[cfg(test)]
mod component_codegen_tests;

99
crates/engine/src/live.rs
View File

@ -10,16 +10,19 @@
//! Peers, presence and document changes are the wire types of the collaboration core
//! ([`figcraft_collab::message`]). Each participant runs a core [`Client`]: the host's session
//! document is the one it shared, a guest gets the host's file as a new tab from the welcome.
//! After every local edit (once a gesture ends) the session document is diffed into a batch and
//! sent; batches from the host's server are applied through the client and installed with
//! [`Session::apply_remote`], which never touches the undo history.
//! Every command, gesture or group on the session document is synced as one batch and is one step
//! of the user's own undo history ([`Session::record`]); undo and redo there take back only the
//! user's own steps ([`figcraft_collab::Client::undo`]). Batches from the host's server are applied
//! through the client and installed with [`Session::apply_remote`], which never touches the undo
//! history. After a dropped connection the client resumes with its id and catches up; what it
//! made meanwhile goes out then.
use std::collections::BTreeMap;
use std::net::IpAddr;
use std::sync::Arc;
use figcraft_collab::Client;
use figcraft_collab::message::{ClientMsg, ServerMsg};
use figcraft_collab::message::{ClientMsg, EditingText, ServerMsg};
use figcraft_doc::{Document, NodeId};
use serde::Serialize;
use serde_json::{Value, json};
@ -226,6 +229,8 @@ pub struct Live {
synced: Option<Arc<Document>>,
/// Server batches waiting for a gesture or preview on the session document to end.
deferred: Vec<ServerMsg>,
/// Nesting of [`Session::grouped`] on the session document: its edits sync once, at the end.
hold: u32,
}
impl Live {
@ -256,6 +261,13 @@ impl Live {
self.doc_uid = None;
self.synced = None;
self.deferred.clear();
self.hold = 0;
}
/// The document sync client (once the welcome arrived): peers, pending batches, the user's
/// undo history.
pub fn client(&self) -> Option<&Client> {
self.client.as_ref()
}
/// Document changes are syncing (the welcome arrived).
@ -351,7 +363,10 @@ impl Session {
cursor: self.live.cursor.map(|(x, y)| (ds.page, x, y)),
selection: Some(ds.selection.clone()),
viewport: Some((ds.page, x0, y0, ds.view.vw / z, ds.view.vh / z)),
editing_text: ds.text_edit.as_ref().map(|t| t.id),
editing_text: ds.text_edit.as_ref().map(|t| {
let (a, b) = (t.anchor.min(t.caret), t.anchor.max(t.caret));
Box::new(EditingText { node: t.id, caret: t.caret, selection: (a != b).then_some((a, b)) })
}),
})
}
@ -381,11 +396,14 @@ impl Session {
live.reset();
}
Status::Off => live.reset(),
// Who's here comes again with the next welcome.
// Who's here comes again with the next welcome; edits wait for it.
Status::Reconnecting { .. } => {
live.peers.clear();
live.presence.clear();
live.sent = None;
if let Some(c) = live.client.as_mut() {
c.disconnect();
}
}
Status::Connecting | Status::Live => {}
}
@ -427,6 +445,8 @@ impl Session {
live.presence.insert(id, p);
}
}
// Back after a dropped connection: the client catches up in order with the rest.
Event::Sync(msg @ ServerMsg::Welcome { .. }) if live.client.is_some() => live.deferred.push(msg),
Event::Sync(ServerMsg::Welcome { .. }) => {
if let Event::Sync(w) = e {
self.live_welcome(w);
@ -455,6 +475,10 @@ impl Session {
(Some(Role::Host), Some(uid)) => {
if let Some(ds) = self.docs.iter_mut().find(|d| d.uid == uid) {
let was_dirty = ds.is_dirty();
// Undo is per user from now on: earlier whole-document steps would revert
// others' edits.
ds.history.undo.clear();
ds.history.redo.clear();
ds.doc_mut().set_client_id(client.id());
if !was_dirty {
ds.mark_saved();
@ -498,16 +522,30 @@ impl Session {
self.message = Some(reason);
}
/// Send the session document's local edits, unless a gesture or preview is still showing.
/// Send the session document's edits not synced yet (none, unless something changed it
/// outside a command), unless a gesture or preview is still showing.
fn live_flush(&mut self) {
self.live_edit(None);
}
/// Whether document `uid` is the live session's synced document: its edits sync and its undo
/// history is the user's own.
pub fn live_doc(&self, uid: u64) -> bool {
self.live.client.is_some() && self.live.doc_uid == Some(uid)
}
/// Sync the session document's edits as one batch: one step of the user's undo history called
/// `label`, or not undoable (`None`). Nothing while a gesture or preview shows, or inside a
/// group (it syncs when the group ends).
pub(crate) fn live_edit(&mut self, label: Option<&str>) {
let Some(uid) = self.live.doc_uid else { return };
let Some(ds) = self.docs.iter().find(|d| d.uid == uid) else { return };
if busy(ds) || self.live.synced.as_ref().is_some_and(|s| Arc::ptr_eq(s, &ds.doc)) {
if self.live.hold > 0 || busy(ds) || self.live.synced.as_ref().is_some_and(|s| Arc::ptr_eq(s, &ds.doc)) {
return;
}
let doc = ds.doc.clone();
let Some(client) = self.live.client.as_mut() else { return };
match client.edit(&doc) {
match client.edit_with(&doc, label) {
Ok(Some(msg)) => {
if let Some(l) = self.live.link.as_mut() {
l.send(msg);
@ -519,6 +557,44 @@ impl Session {
self.live.synced = Some(doc);
}
/// A group of commands on document `uid` starts (`on`) or ends: in between its edits don't
/// sync on their own.
pub(crate) fn live_hold(&mut self, uid: u64, on: bool) {
if self.live_doc(uid) {
self.live.hold = if on { self.live.hold.saturating_add(1) } else { self.live.hold.saturating_sub(1) };
}
}
/// Undo (`back`) or redo the user's own last step on the session document: what it wrote goes
/// back where nobody else wrote since. Returns whether there was a step to take back.
pub(crate) fn live_travel(&mut self, back: bool) -> bool {
let Some(uid) = self.live.doc_uid else { return false };
// Edits not synced yet go first.
self.live_flush();
let Some(client) = self.live.client.as_mut() else { return false };
let (undo, redo) = client.history_len();
let msgs = match if back { client.undo(&EngineRecompute) } else { client.redo(&EngineRecompute) } {
Ok(m) => m,
Err(e) => {
log::warn!("live session: {} failed: {e}", if back { "undo" } else { "redo" });
return false;
}
};
let (u, r) = client.history_len();
let did = if back { r > redo } else { u > undo };
let doc = Arc::new(client.document().clone());
if let Some(l) = self.live.link.as_mut() {
for m in msgs {
l.send(m);
}
}
if did {
self.apply_remote(uid, doc.clone());
self.live.synced = Some(doc);
}
did
}
/// Apply the server's batches that waited (none while a gesture or preview shows).
fn live_deferred(&mut self) {
if self.live.deferred.is_empty() || self.live.client.is_none() {
@ -540,6 +616,11 @@ impl Session {
}
}
let doc = Arc::new(client.document().clone());
// Edits the server refused were rolled back: say why (the UI's toast).
let notices: Vec<String> = client.take_notices().iter().map(figcraft_collab::Notice::message).collect();
if !notices.is_empty() {
self.message = Some(notices.join("; "));
}
if let Some(l) = self.live.link.as_mut() {
for m in out {
l.send(m);

10
crates/engine/src/live_tests.rs
View File

@ -13,16 +13,16 @@ use crate::{EngineError, Session};
/// What the scripted link was asked to do, and the events it hands out next.
#[derive(Default)]
struct Script {
events: Vec<Event>,
pub(crate) struct Script {
pub(crate) events: Vec<Event>,
sent: Vec<Presence>,
hosted: Vec<HostOptions>,
joined: Vec<String>,
left: usize,
msgs: Vec<ClientMsg>,
pub(crate) msgs: Vec<ClientMsg>,
}
struct Mock(Arc<Mutex<Script>>);
pub(crate) struct Mock(pub(crate) Arc<Mutex<Script>>);
impl LiveLink for Mock {
fn host(&mut self, opts: HostOptions) -> Result<SessionInfo, String> {
@ -50,7 +50,7 @@ impl LiveLink for Mock {
}
}
fn info(role: Role, me: u32) -> SessionInfo {
pub(crate) fn info(role: Role, me: u32) -> SessionInfo {
SessionInfo {
role,
me,

8
crates/engine/src/vectoredit.rs
View File

@ -332,14 +332,14 @@ pub fn begin(s: &mut Session, id: NodeId) -> Result<()> {
ensure_vector(doc, id)?;
}
let ds = s.ds_mut()?;
if !std::sync::Arc::ptr_eq(&before, &ds.doc) {
ds.history.undo.push(crate::HistoryEntry { label: "Edit vector".into(), doc: before, selection: ds.selection.clone(), page: ds.page });
ds.history.redo.clear();
}
let edited = (!std::sync::Arc::ptr_eq(&before, &ds.doc)).then(|| (ds.uid, ds.selection.clone(), ds.page));
ds.vector_edit = Some(VectorEdit::new(id));
ds.selection = vec![id];
ds.text_edit = None;
ds.revision += 1;
if let Some((uid, selection, page)) = edited {
s.record(uid, Some("Edit vector"), before, selection, page);
}
Ok(())
}

4
crates/ui-egui/src/live.rs
View File

@ -18,8 +18,8 @@ use crate::right::{menu_item, menu_item_with, menu_popup};
use crate::theme::{ACCENT, Tokens, regular, semibold};
use crate::widgets;
/// Until undo is per person, it restores whole earlier states.
const UNDO_NOTE: &str = "Undo isn't per person yet: it can also revert others' recent edits.";
/// How undo works in a session (per person).
const UNDO_NOTE: &str = "Undo takes back only your own changes; what others changed since stays.";
/// How fast a remote cursor closes in on where it was last reported (per second).
const GLIDE: f64 = 20.0;

152
docs/collab-protocol.md
View File

@ -5,8 +5,10 @@ identifiers, the object/property model, change sets, ordering and validation, cl
reconciliation, derived data and the message encoding. **Step 2 is implemented** in
`crates/collab-net` and the engine: the WebSocket transport, the session host the desktop app
embeds, invite links, document sync end to end (several apps edit one file), live cursors,
selections, avatars and follow. Undo is not per person yet (step 3). The rest is the plan the next
implementation PRs follow and update; sections say what is built.
selections, avatars and follow. **Step 3 is implemented**: per-user undo and redo, rejection
handling and offline queueing with reconnect (`crates/collab`, wired into the engine's live
session and the transport). The rest is the plan the next implementation PRs follow and update;
sections say what is built.
Goal: several people edit the same `.figcraft` file at the same time and see each other's edits,
cursors and selections live, on the desktop app and in the browser, without a central cloud
@ -54,7 +56,9 @@ fit FigCraft's architecture:
- **Client id**: a 32-bit id the server assigns to each connection on join. The server starts
above every client id space already used by node ids in the file, so ids from earlier sessions
are never handed out again (a persistent server will also store its counter with the file).
are never handed out again. The counter is part of the session state a server saves
(`Server::state` / `Server::restore`), so a restarted server doesn't hand ids out twice either;
a client that reconnects keeps its id.
- **Node ids** are unique across clients without coordination: with a client id set
(`Document::set_client_id`), `Document::alloc_id` returns `client_id << 32 | counter`,
continuing after the highest id of that space already in the document. Ids in existing files
@ -65,7 +69,8 @@ fit FigCraft's architecture:
created again whole (what undo does): the server accepts creating an id that doesn't exist only
if it was deleted before or lies in the sender's own id space.
- Styles, variables, collections and comments keep their existing string or numeric ids (made
unique by the engine); a client-scoped scheme for them comes with per-user undo.
unique by the engine); a client-scoped scheme for them is still to come (until then two people
creating one at the same moment can get the same id, and the later write wins).
## The document as objects and properties
@ -140,9 +145,9 @@ Client → server:
| Message | Meaning |
|---|---|
| `Hello {protocol, file, token, name, color, client_version}` | Join a file's session. |
| `Hello {protocol, file, token, name, color, client_version, resume?}` | Join a file's session; `resume {session, client_id, seq}` when coming back after a dropped connection. |
| `Changes {base_seq, batch_id, changes: [Change]}` | One user action's changes (one undo step). |
| `Presence {cursor?, selection?, viewport?, editing_text?}` | Ephemeral; throttled to ~30 Hz, latest wins. |
| `Presence {cursor?, selection?, viewport?, editing_text? {node, caret, selection?}}` | Ephemeral; throttled to ~30 Hz, latest wins. |
| `Ack {seq}` | Highest sequence applied (lets the server trim per-client buffers). |
| `Ping` | Keep-alive. |
@ -150,7 +155,7 @@ Server → client:
| Message | Meaning |
|---|---|
| `Welcome {client_id, seq, snapshot or delta, peers}` | Join accepted: the full document (or changes since a cached `seq`). |
| `Welcome {session, client_id, seq, catch_up: Snapshot or Delta, processed, peers}` | Join accepted: the full document, or for a reconnecting client the batches it missed; what the server did with that client's batches (`processed {last_batch, refused}`). |
| `Changes {seq, author, batch_id, changes}` | Accepted changes, in order, to everyone (the author gets its own back as confirmation). |
| `Reject {batch_id, reason, fixes: [Change]}` | A batch (or part of it) was invalid: the current values to roll back to. |
| `Presence {client_id, …}` / `PeerJoined` / `PeerLeft` | Other people's cursors, selections and arrivals. |
@ -160,8 +165,8 @@ Server → client:
`Change` is one of: `Create {id, type, props}`, `Set {id, prop, value}`, `Delete {id}` (deleting a
node deletes its subtree; ids are never allocated twice), plus the document-level `SetDoc`, `Put`
and `Remove` (see above). A snapshot is the list of changes that builds the document from nothing;
`Welcome` carries one (deltas since a cached `seq` come with reconnect, step 3). A whole batch is
accepted or rejected.
`Welcome` carries one, or the missed batches when a client reconnects (see Offline and
reconnect). A whole batch is accepted or rejected.
## Ordering, optimism and convergence
@ -190,7 +195,19 @@ or the JSON encoding. Tests drive it with synthetic edits and, in the engine, wi
commands (creating, moving and reparenting layers including cycle attempts, deleting, grouping,
auto layout, components and instances, styles, variables, text, pages, comments): after quiescence
every client's state equals the server's, for every seed (more seeds run as an ignored test).
Undo is not part of these runs (step 3).
The same runs with undo, redo and dropped connections are described below.
### Rejections
*Built.* When the server refuses a batch (a layer it changes was deleted by someone else, a move
would make a cycle, a value is out of range…), the `Reject` carries a short reason (a deleted
layer reads "a layer it changes was deleted by someone else") and the current values of what the
batch touched. The client drops the batch, applies the fixes to its confirmed copy (a no-op there:
every batch ordered before the rejection arrived first; they matter to clients that keep no
confirmed copy), rebuilds its view from the confirmed state and its remaining batches, and takes
the batch's step out of the undo history, together with any undo or redo of it still waiting to be
sent. It then queues a notice (`Client::take_notices`, `Notice::message`); the engine shows it as
the UI's toast (`Session::message`).
### Derived data
@ -215,22 +232,61 @@ Undo is not part of these runs (step 3).
## Undo and redo
*Step 3.* Until then the engine's snapshot undo must not be used in a live session: it restores a
whole earlier document and would revert other people's changes made in between. The server already
accepts re-creating deleted nodes with their ids, which undoing a delete needs.
*Built* (`Client::{undo, redo}`; in the engine, `edit.undo` / `edit.redo` and the UI's undo route
there while the document is in a live session). Outside a live session the engine's snapshot undo
works exactly as before; a document entering or leaving a session starts with an empty history.
Undo stays per user and undoes only **your own** actions: an undo turns your last batch into a new
batch that writes back the values your batch replaced, for the properties nobody has changed since
(a property someone else changed afterwards is left alone). Redo is the inverse. Remote changes
never enter your undo history.
Decisions, as built:
- **A step per batch.** Every undoable batch of the user (one command, one gesture, one
`command.batch` / `Session::grouped` group) becomes a step: per target (a node property, a
document-level property or entry) the value before and after, plus the nodes it created and
deleted, each as the change that creates it as it was. Batches that aren't commands' edits
(derived-data follow-ups) never become steps.
- **"Nobody has changed it since"** is decided by server order: a batch from someone else that the
server ordered after the step's last confirmed batch takes the targets it writes away from the
step for good (a write to a list as a whole takes every entry of that list; creating a node again
takes all its properties; a taken parent takes the position too). Batches ordered before it are
what the step replaced: the step learns their values when its batch is confirmed or replayed, so
an undo restores the value that was there in server order, not the one the user happened to see.
- **Creates become deletes, deletes create the subtree again** with the same ids, positions and
values (the server accepts re-creating deleted ids). A created layer someone else has since put
layers into is not deleted (and if nothing else of the step is left, the step is skipped); a
layer whose parent is gone is not brought back. Changes are checked in order against the current
state, so an undo is never refused for what the client can see.
- **An undo or redo waits for what came before it.** It shows at once, but goes out only when
every earlier batch of the client is confirmed or refused, and is then made again from the
confirmed state. So it never rests on an edit the server refused (the step and its undo are
dropped with the edit) or on values the client hadn't seen yet. Undoing right after an edit
costs one round trip before it reaches the others; edits made meanwhile queue behind it.
- **Derived data** that the restored values imply (auto layout, groups…) is recomputed with the
recompute hook and sent as a follow-up batch, like the repairs after concurrent edits.
- **A step that no longer changes anything is skipped** (the next undo goes one step further).
- Concurrent edits the undoing client couldn't have seen (made while its undo was on the way, or
while it was offline) are ordered last-writer-wins like any other edits.
Tests: the in-memory simulation checks every undo and redo the server accepts against the log of
accepted batches: a value it changes was last written by its author (or after the client's last
seen sequence number), and every layer it deletes was created by its author or moved into the
deleted layer after that. Random sessions of synthetic edits and of real engine commands, with
undo, redo and dropped connections, converge with no violation (thousands of seeds were run while
building it; an ignored test runs more). The engine's tests (its live session with a scripted
link) also cover a gesture staying one step while others edit, a group of commands syncing as one
step, an undo right after an edit waiting for it, a refused edit's toast, a dropped connection
resuming with its offline edits, and leaving the session going back to snapshot undo.
## Presence
*Built* (`figcraft_engine::live`, `crates/collab-net`, `crates/ui-egui/src/live.rs`).
- Cursor position in page coordinates plus the page id; selection ids; viewport rectangle; whether
the user is typing in a text layer (to show their caret and selection inside it). The message is
the core's `Presence`; the engine builds yours from the active document (page, selection, view,
the user is typing in a text layer, with their caret and selection inside it
(`editing_text {node, caret, selection}`, in characters). The message is the core's `Presence`; the engine builds yours from the active document (page, selection, view,
edited text) and the pointer over the canvas, and sends it only when it changed.
- Shown as named, coloured cursors on the canvas (a pointer drawn in code with a name tag in the
person's colour, gliding to each reported position; reduced motion makes it jump), selection
@ -284,8 +340,9 @@ never enter your undo history.
registered with the operating system yet: links are pasted into the Join dialog.
- Reconnect: a connection that drops after it was live reconnects with growing pauses (0.5 s,
doubling up to 10 s, 12 attempts); a refusal (wrong token, full) or the host ending the session
is final; a first join that can't reach the host fails at once. A reconnect is a new client id
(resuming with the last confirmed `seq` is step 3).
is final; a first join that can't reach the host fails at once. A reconnect resumes as the same
client: the guest's connection keeps the session id, client id and last sequence number from
the messages it read and sends them in its next `Hello` (see Offline and reconnect).
- The browser build has the commands but says live sessions need the desktop app (its WebSocket
client is step 4).
@ -296,26 +353,64 @@ never enter your undo history.
its own edits go through the same ordering. A guest's welcome carries the whole document
(pages, layers, styles, variables, images): it opens as a new tab, the guest's own files stay
open, and leaving keeps it as a local copy.
- The engine syncs one open document per session (`Live::doc_uid`). After every local action it
diffs that document against the client's view into one batch (`ClientMsg::Changes`); a gesture
- The engine syncs one open document per session (`Live::doc_uid`). Every command, group of
commands or gesture on it is diffed against the client's view into one batch
(`ClientMsg::Changes`) that is one step of the user's undo history (`Session::record`); a gesture
(a drag, a resize) is sent once, when it ends, and nothing is sent while a version or a live
text preview shows. Batches the server orders (`Changes`, `Reject`) go through
text preview shows (anything changed outside a command goes out at the next frame, not
undoable). Batches the server orders (`Changes`, `Reject`) go through
`Client::receive` with the engine's recompute hook, and the resulting document is installed with
`Session::apply_remote`, which keeps your selection, page and view and never adds an undo step.
Batches that arrive during a gesture wait for it to end. New layers take ids in the client's
space (`client_id << 32 | counter`). Closing the session's document leaves the session.
- Each applied batch wakes the window (the inbox calls `request_repaint`); nothing runs while
nobody edits or moves.
- Limits of this step: undo restores whole earlier states, so it can also revert others' recent
edits until per-user undo (step 3), and the Share popover says so; a reconnect is a new client
whose welcome replaces the document, so edits made while disconnected are lost (resuming from
the last confirmed `seq` with queued batches is step 3).
## Offline and reconnect
On a dropped connection the client keeps editing locally, queues batches, and on reconnect sends
`Hello` with its last confirmed `seq`; the server replies with the missed changes, then the queued
batches are sent (they may be partly rejected if their objects were deleted meanwhile).
*Built* (`Client::{disconnect, reconnect}`, the server's log, the guest's connection in
`crates/collab-net`, the engine's live session). On a dropped connection the client keeps
editing locally, queues batches, and on reconnect sends `Hello` with its last confirmed `seq`; the
server replies with the missed changes, then the queued batches are sent (they may be partly
rejected if their objects were deleted meanwhile).
Decisions, as built:
- **Same identity.** `Hello.resume` carries the session id (random per server, kept in its saved
state), the client id and the last sequence number applied. A client this session gave its id to
keeps it (a stale connection with that id is taken over); anyone else joins as new. A client
welcomed as new after all (the server restarted without its state) starts over from the
snapshot and is told how many unsent changes couldn't be kept.
- **Catch-up.** The server keeps a log of accepted batches (`Limits::max_log_batches`, default
10 000, and `max_log_changes`, default 1 000 000). If every batch after the client's sequence
number is still in it, the welcome carries them (`CatchUp::Delta`); otherwise the whole document
(`CatchUp::Snapshot`).
- **What happened to the batches in flight.** The server remembers per client the last batch id it
processed and the latest refused ids (at most 1 024). Batches up to that id that the catch-up
doesn't confirm were refused (the user is told); the others are sent again, in order, keeping
the sequence number they were made on. Batches the server confirmed but whose echo was lost are
confirmed by the catch-up and never sent twice.
- **Undo across a gap.** With a delta, the missed batches mark the steps they conflict with exactly
as they would have live. With a snapshot, which batches came from others isn't known: a step
keeps a value only if it is still the one the step wrote (someone else setting exactly the same
value can't be told apart). An undo or redo lost in flight is made again from the caught-up
state, treating what others wrote meanwhile as later than the step.
- **Over the network.** The guest's connection keeps what it needs to resume from the messages it
reads (the welcome's session and client id, the last sequence number) and sends it in the next
`Hello`; the engine takes the connection's "reconnecting" status as the client going offline and
the next welcome as its catch-up, in order with the batches before it. What the engine sent for
a dropped connection is not carried over (the client sends again what the server didn't get),
the server ignores a batch id it already processed (a copy sent twice around a reconnect), and
the host's hub hands a resumed client id from its stale connection to the new one.
- **Session state.** `Server::state` returns what a server needs to carry on after a restart
(`SessionState`: snapshot, sequence number, session id, client-id counter, the clients that can
reconnect and what was done with their batches, deleted ids, the log), as one serde value; the
standalone server will store it.
Tests: random sessions where clients drop and come back mid-session converge, with the default log
and with logs short enough that long absences fall back to snapshots; targeted tests cover a lost
echo, a batch refused while offline, a long absence, an unknown client and a restart from the
saved state.
## Agents, MCP and the CLI
@ -376,6 +471,7 @@ chooses to share on the network; the default is the invite link with a random to
a writer thread per connection, no async runtime; the engine holds the live state and the sync
client and reaches the network only through the `LiveLink` the app installs).
3. **Per-user undo**, reconnect and offline queueing, rejection handling, presence for text
editing.
editing. **Done** (the core's client and server, the engine's live session, the guest's
resuming connection).
4. **Browser client** (WebSocket through `web-sys`).
5. **Standalone `figcraft-server`** with persistence, compaction and accounts.

2
docs/collaboration-parity.md
View File

@ -10,7 +10,7 @@ nobody can co-edit yet ([`collab-protocol.md`](collab-protocol.md)). Remaining:
| Feature | Figma | FigCraft | Hours |
|---|---|---|---|
| Real-time co-editing | yes | **core built, no networking** (#103): client-scoped ids, per-property change sets with fractional positions, server ordering and validation, optimistic clients, binary messages; randomized multi-client convergence tests over real commands. Live sessions over the network landed (`crates/collab-net`): the WebSocket transport (keep-alive, size and rate limits, reconnect with backoff), hosting from the app (this machine or the local network) with invite links, live cursors with name tags, selections, avatars and follow, `collab.*` commands. Document sync between apps works over them (the host's server orders every batch; guests open the host's file as a new tab). Left: per-user undo, resuming after a reconnect, browser client, standalone server | 25–55 |
| Real-time co-editing | yes | **core built, no networking** (#103): client-scoped ids, per-property change sets with fractional positions, server ordering and validation, optimistic clients, binary messages; randomized multi-client convergence tests over real commands. Live sessions over the network landed (`crates/collab-net`): the WebSocket transport (keep-alive, size and rate limits, reconnect with backoff), hosting from the app (this machine or the local network) with invite links, live cursors with name tags, selections, avatars and follow, `collab.*` commands. Document sync between apps works over them (the host's server orders every batch; guests open the host's file as a new tab). Undo is per person (only your own steps, others' later writes left alone), refused edits roll back with a toast, and a reconnect resumes as the same client with what it missed (or a snapshot) and sends what was made offline. Left: browser client, standalone server | 25–55 |
| Live cursors, selections, viewports; follow / observation mode; spotlight | yes | no (presence messages encoded; no UI or transport) | 8–15 |
| Cursor chat, audio, high-fives | yes | no | 4–8 |
| Share links, permissions (view / edit), teams and projects | yes | no | 8–17 |

11
docs/gaps.md
View File

@ -60,13 +60,14 @@ No alpha blockers: the alpha gate in [`roadmap.md`](roadmap.md#alpha-gate) passe
- **Evidence:** the core landed in #103 (`crates/collab`: client-scoped ids, change sets, server
ordering and validation, client reconciliation, binary messages, randomized convergence tests over
real commands), and live sessions run over the network (`crates/collab-net`: hosting from the
app, invite links, document sync between apps, live cursors, selections, avatars and follow),
but undo isn't per person, a reconnect loses offline edits, the browser can't join, and there
are no share links or permissions (`docs/collab-protocol.md`).
app, invite links, document sync between apps, live cursors, selections, avatars and follow)
with per-person undo, refused edits rolled back with a reason, and reconnects that resume and
send what was made offline; but the browser can't join, and there are no share links or
permissions (`docs/collab-protocol.md`).
- **Impact:** Figma's defining feature; most teams use it every day. Solo designers can work
without it.
- **Estimate:** 45–90 h: per-user undo, resuming after a reconnect, then sharing and
permissions and a standalone server; the browser build needs its own WebSocket client.
- **Estimate:** 45–90 h: sharing and permissions and a standalone server; the browser build
needs its own WebSocket client.
### 3. Nothing checked against real Figma files
- **Missing:** a corpus of real documents (design systems, app screens, prototypes) imported and

11
docs/mcp.md
View File

@ -482,7 +482,9 @@ and a file keeps the newest 20 autosaves (user versions are never dropped for th
`collab.diff {against? (undo: the last undoable action, the default | saved: since the last
save)}` returns `{changes, count, bytes}`: the change batch a live session would send for that edit
(per-property `create` / `set` / `delete` and document-level changes, see
`docs/collab-protocol.md`) and its binary size. It changes nothing.
`docs/collab-protocol.md`) and its binary size. It changes nothing. In a live session, where undo
is per user (`edit.undo` / `edit.redo` take back only your own steps, see
`docs/collab-protocol.md`), only `saved` applies.
### Live sessions
@ -502,11 +504,16 @@ what happened since the last one and sends your edits (a joined file opens as a
- `collab.peers {}` → `{available, status (off|connecting|live|reconnecting|failed), reason?,
attempt?, role (host|guest), me, name, color, fileName, pages, link, url, address, network,
follow, syncing (the document arrived), document (the index of the synced open document), peers: [{id, name, color, presence: {cursor: [page, x, y], selection: [ids], viewport:
[page, x, y, w, h], editing_text}}]}`.
[page, x, y, w, h], editing_text: {node, caret, selection?}}}]}`.
- `collab.cursor {x, y}` puts your cursor at a page position (omit both to hide it): an agent's
pointer. Your selection, page and view go out by themselves when they change.
- `collab.follow {client}` follows that person's viewport; `collab.follow {}` stops (so does any
pan or zoom of your own, or Escape).
- On the session's document every command (or `batch`, or gesture) is one step of your own undo
history: `edit.undo` / `edit.redo` take back only your steps and leave what others changed since.
An edit the host refuses (someone deleted the layer) is rolled back and the reason is the
session's message. While the connection is reconnecting, editing goes on; the edits go out when
it is back (the same client, catching up with what it missed).
### Plugins

4
docs/roadmap.md
View File

@ -17,7 +17,7 @@ are one Opus 5.5 agent; most milestones split across 4–6 agents on disjoint cr
40–60 h, parallel by tool.
4. **Localization layer**, then Figma's own languages first (ja, ko, es, pt-BR, fr, de): 15–25 h
for the layer, 3–5 h per language plus native review.
5. **Multiplayer M9**: per-user undo, reconnect with offline edits, the browser client, a standalone server and sharing on top of the live sessions (the WebSocket transport, hosting from the app, invite links, document sync, live cursors, selections, avatars and follow landed), per [`collab-protocol.md`](collab-protocol.md): 45–90 h.
5. **Multiplayer M9**: the browser client, a standalone server and sharing on top of the live sessions (the WebSocket transport, hosting from the app, invite links, document sync, live cursors, selections, avatars and follow, per-user undo and reconnect with offline edits landed), per [`collab-protocol.md`](collab-protocol.md): 45–90 h.
## Alpha gate
@ -55,7 +55,7 @@ blockers.
| M6 Interop & Dev Mode | ~55% | `.fig` import, video/HEIC/TIFF, CMYK intent, Code Connect | 40–80 |
| M7 Release & fidelity (signed builds, look & feel, shortcuts) | builds done; fidelity pass started | tool-by-tool pass | 40–60 |
| M8 Hardening & performance (bench, retained canvas, accessibility, idle) | ~60% | real-file corpus, platform QA, GPU canvas, screen readers | 60–110 |
| M9 Real-time collaboration | core done (#103: ids, change sets, server ordering, convergence); live sessions (WebSocket transport with limits and reconnect, hosting from the app, invite links, document sync between apps, cursors, selections, avatars, follow, `collab.*` commands) | per-user undo, resuming after a reconnect, `figcraft://` link handler, browser client, standalone server, sharing | 45–90 |
| M9 Real-time collaboration | core done (#103: ids, change sets, server ordering, convergence); live sessions (WebSocket transport with limits and reconnect, hosting from the app, invite links, document sync between apps, cursors, selections, avatars, follow, `collab.*` commands); per-user undo, rejection handling, offline queue and resuming after a reconnect | `figcraft://` link handler, browser client, standalone server, sharing | 45–90 |
| M10 Localization (new) | not started | layer + 11 languages | 60–100 |
| M11 Plugins | runtime, API, menu, manager and SDK done (#101) | run-time plugin UI, settings storage, distribution, widgets; Figma plugin compatibility is an owner decision | 25–55 |
| M12 AI features (new, owner decision) | not started | local models | 30–60 |

12
docs/target-app-parity.md
View File

@ -441,7 +441,14 @@ Hours in this narrative are the builders' estimates; the tables above supersede
and a client that shows its own edits at once and replays them over others'; derived data
(auto layout, instances, bindings) is repaired by the author after concurrent edits; binary
messages (a moved layer is 43 bytes, a cursor 23) with a JSON form; a randomized multi-client
test over real commands converges.
test over real commands converges. Live sessions run over the network (hosting from the app,
invite links, document sync, live cursors, selections, avatars, follow). Undo is per person in
a session (`edit.undo` takes back only your own steps, leaving what others changed since;
creations and deletions invert; a gesture or command group stays one step); refused edits roll
back with a toast; a dropped connection keeps editing and resumes as the same client with what
it missed (or a snapshot), then sends what was made offline. Randomized sessions with undo,
redo and dropped connections converge, and a checker confirms no undo reverts someone else's
change.
Plugins landed: sandboxed WebAssembly plugins (a metered wasmi
interpreter with a memory cap and no WASI, also in the browser) whose API is the command
registry; each run is one undo step and is rolled back when it fails (error, trap, endless loop,
@ -453,8 +460,7 @@ Hours in this narrative are the builders' estimates; the tables above supersede
plugins, and `plugin.*` commands for scripts, the CLI and MCP; a Rust SDK and two example
plugins (Tidy names, Swatch grid) in `plugins/`, built by `cargo xtask plugins`. *Left:*
plugin-defined UI at run time (forms or panels a plugin returns), plugin settings storage,
distribution; for collaboration, the WebSocket transport, hosting and joining from the app,
live cursors, per-user undo, reconnect, the browser client and a standalone server. 18–45 h.
distribution; for collaboration, the browser client and a standalone server. 18–45 h.
9. **Hardening and performance**: *landed:* `cargo xtask bench` (synthetic 1k/10k/50k-layer
pages: engine, render and headless UI-frame timings). The canvas keeps its last image and
re-renders only what a pan uncovers or an edit touches: moving one layer at 10k layers re-renders