Sync Protocol (Deep Dive)

ArcFlow instances run locally (browser WASM, Node.js, edge device) and optionally sync to a persistent cloud graph via HTTPS. Local operations are instant. Sync is eventual, conflict-resolved, and resumable.

Overview#

Local instance              Cloud / Primary
┌──────────────────┐  HTTPS  ┌──────────────────┐
│ Browser (WASM)   │────────→│ ArcFlow Cloud    │
│ Node.js process  │←────────│                  │
│ Edge device      │  sync   │ Persistent graph │
│                  │         │ Shared state     │
│ Local WAL        │         │ Multi-client     │
└──────────────────┘         └──────────────────┘

All operations happen locally first. Sync pushes WAL deltas to the cloud and pulls remote changes back. Conflicts are resolved algebraically.

Activation#

Browser#

https://oz.com/engine?sync=af_xxxxxxxxxxxx

SDK#

import { open, openInMemory } from '@ozinc/arcflow'
 
// In-memory with cloud sync
const db = openInMemory({
  sync: 'https://api.oz.com/v1/graphs/my-graph',
  token: 'af_xxxxxxxxxxxx'
})
 
// Persistent with cloud sync
const db = open('./local-graph', {
  sync: 'https://api.oz.com/v1/graphs/my-graph',
  token: 'af_xxxxxxxxxxxx'
})
 
// Self-hosted sync target
const db = openInMemory({
  sync: 'https://my-company.com/arcflow/sync',
  token: process.env.ARCFLOW_TOKEN
})

MCP server#

npx arcflow-mcp --sync https://api.oz.com/v1/graphs/my-graph --token af_xxx

Sync protocol#

Data flow#

1. Local mutation
   ↓
2. Append to WAL (MutationOp with sequence number)
   ↓
3. Batch into sync payload { peer_id, seq_start, seq_end, ops[], fingerprint }
   ↓
4. POST to sync endpoint
   ↓
5. Cloud applies ops with conflict resolution
   ↓
6. Cloud responds with remote changes since client's high-water mark
   ↓
7. Client applies remote changes locally
   ↓
8. Fingerprint comparison — both sides should match

Sync batch format#

interface SyncBatch {
  peer_id: string           // Client identifier
  seq_start: number         // First sequence number in batch
  seq_end: number           // Last sequence number in batch
  ops: MutationOp[]              // Mutations (CreateNode, DeleteNode, SetProperty, etc.)
  fingerprint: number       // Graph state hash after applying ops
}

MutationOp types#

type MutationOp =
  | { type: 'CreateNode', id: string, labels: string[], properties: Record<string, string> }
  | { type: 'CreateRelationship', id: string, start: string, end: string, rel_type: string, properties: Record<string, string> }
  | { type: 'UpdateNodeProperties', id: string, properties: Record<string, string> }
  | { type: 'SetNodeProperties', id: string, properties: Record<string, string> }
  | { type: 'RemoveNodeProperty', id: string, key: string }
  | { type: 'DeleteNode', id: string }
  | { type: 'DeleteRelationship', id: string }

Resume protocol#

Each client tracks a high_water_mark — the highest sequence number received from the cloud. On reconnect, the client sends its HWM and receives only the changes since then:

POST /sync
{
  "peer_id": "browser-abc123",
  "high_water_mark": 4207,
  "delta": { ... }         // local changes since last sync
}

Response:
{
  "remote_delta": { ... }, // changes from other clients since HWM
  "new_high_water_mark": 4215,
  "fingerprint": 9823471   // expected state after applying all
}

Conflict resolution#

When two clients mutate the same entity concurrently, ArcFlow resolves conflicts using vector clocks and Z-set algebra.

Vector clock ordering#

Client A: { A: 5, B: 3 }
Client B: { A: 4, B: 6 }

Neither dominates → concurrent → conflict detected

Resolution strategies#

Evidence-aware merge#

For nodes with confidence scores, the merge preserves the highest-confidence observation:

Client A writes: { name: 'Alice', confidence: 0.87, observation: 'inferred' }
Client B writes: { name: 'Alice', confidence: 0.95, observation: 'observed' }

Merge result: { name: 'Alice', confidence: 0.95, observation: 'observed' }

The engine's confidence-aware merge algebra picks the highest-confidence, highest-observation-class value across concurrent writes: add(a, b) = (max_confidence, highest_observation).

Fingerprint verification#

After applying all changes, both client and cloud compute a fingerprint (hash of graph state). If they match, sync is consistent. If not, a full snapshot reconciliation is triggered.

Implementation waves#

Wave 1: Push-only sync (cloud backup)#

Local → cloud. No conflict resolution needed. Client pushes WAL deltas, cloud applies them.

const db = openInMemory({ sync: url, token, mode: 'push' })
// All local mutations automatically batched and pushed

Wave 2: Bidirectional sync (multi-client)#

Cloud → local + local → cloud. Vector clock conflict resolution. Resume via high-water marks.

const db = openInMemory({ sync: url, token, mode: 'sync' })
// Mutations sync both ways, conflicts auto-resolved

Wave 3: Replicated live views#

Standing queries (CREATE LIVE VIEW) replicated across clients. A live view defined on one client is automatically maintained on others.

// Client A defines a view
db.mutate("CREATE LIVE VIEW top_scores AS MATCH (n) WHERE n.score > 0.9 RETURN n")
 
// Client B reads the same view (synced, incrementally maintained)
db.query("MATCH (row) FROM VIEW top_scores RETURN row")

Wave 4: Edge sync (offline-first)#

Edge devices (IoT, mobile) that go offline and reconcile when back online. Full conflict log with manual resolution option.

Sync capabilities#

The sync engine provides all the primitives needed for local-first architectures:

HTTP API endpoints (planned)#

POST   /v1/graphs/:id/sync     — push delta, receive remote changes
GET    /v1/graphs/:id/snapshot  — full graph snapshot (for initial sync)
GET    /v1/graphs/:id/changes   — changes since sequence N
POST   /v1/graphs/:id/resolve   — manual conflict resolution
GET    /v1/graphs/:id/status    — sync status, peer count, lag

Security#

• All sync over HTTPS (TLS 1.3)
• Bearer token authentication per graph
• Optional: per-peer write permissions (read-only replicas)
• Fingerprint verification prevents tampering
• WAL ops are signed with peer_id for provenance

See Also#

• Sync — capability overview: WAL delta, conflict resolution, and db.changesSince()
• Cloud Architecture — edge mesh, WAL segment shipping, and CDC delta sync
• Architecture — in-process engine architecture and storage model