DEV Community

Cover image for Google Docs System Design: Real-Time Editing, OT, CRDT & Scale
MANGESH MANDLIK
MANGESH MANDLIK

Posted on Originally published at seeitflow.com

Google Docs System Design: Real-Time Editing, OT, CRDT & Scale

google docs final architecture

Two people open the same document. Alice inserts a sentence near the top. Bob deletes a word in the same paragraph, before Alice's change reaches his laptop. Both see their own edits immediately. A few moments later, both should see the same document, without silently losing either edit.

That sounds like a text editor with WebSockets. It is actually a distributed synchronization problem.

In this guide, we'll design a Google Docs-style real-time collaborative editor from a single-user autosave application to a production-oriented architecture. We'll examine concurrent edits, Operational Transform (OT), CRDTs, authoritative document ownership, durability, offline recovery, presence, version history, and scaling. This is an illustrative system design—not a description of Google's private implementation.

1. What are we designing?

Users should be able to create and open documents, edit simultaneously, see other collaborators' changes and cursors, reconnect after losing connectivity, review earlier versions, and share documents with appropriate permissions.

The essential engineering requirements are distinct:

  • Local responsiveness: typing should appear immediately, even when the network is slow.
  • Convergence: healthy clients that have processed the same accepted history should arrive at the same document state.
  • Durability: an edit reported as saved must survive failures covered by the stated durability policy.
  • Authorization: a client must not continue reading or writing after access is revoked.
  • Scalability: millions of open connections and occasional extremely popular documents must not overload one global server.

Convergence, durability, and preserving a user's intended meaning are not interchangeable guarantees. A synchronization algorithm may converge perfectly while combining two semantically conflicting edits into awkward text.

2. Start simple: one user, one document

For a single-user editor, the client can maintain a local buffer and periodically save the latest body to storage:

single user

Editor → Autosave API → Document database
Enter fullscreen mode Exit fullscreen mode

This breaks under concurrent writers. Suppose both Alice and Bob read version 10. Alice saves a modified body as version 11. Bob then saves his independently modified version 10 body. If the server simply overwrites the entire document, Bob's save can erase Alice's change.

A conditional write that rejects Bob's stale version prevents silent overwrite, but still does not provide a good collaborative experience: Bob needs to reconcile his edits with Alice's. We therefore need operation-based synchronization, not blind whole-document replacement.

3. Evolve to real-time collaboration

The client applies edits optimistically, then sends operations to a backend over a persistent WebSocket connection. A document owner reconciles concurrent operations and assigns authoritative versions.

                         ┌───────────────────────┐
Editor A ── WebSocket ────┤                       │
                         │   WebSocket Gateways  │
Editor B ── WebSocket ────┤  sessions, auth, I/O  │
                         └──────────┬────────────┘
                                    │
                              Shard Router
                                    │
                         ┌──────────▼────────────┐
                         │ Document Owner        │
                         │ validate + dedupe     │
                         │ OT / merge + version  │
                         └──────────┬────────────┘
                                    │
                         Durable Operation Log
                                    │
                         Commit acknowledgement
                                    │
                            Broadcast Layer
                                    │
                           Other subscribers

Separate supporting services:
  Presence / cursors     Snapshots / history
  Offline resync         Permission changes
  Ownership coordination Monitoring
Enter fullscreen mode Exit fullscreen mode

websocket architecture

Why WebSockets? Edits, acknowledgements, cursors, and presence updates are small and bidirectional. A persistent connection avoids repeated polling overhead and supports low-latency delivery. HTTP remains useful for document creation, initial snapshot retrieval, sharing, and history browsing. WebSockets provide transport—not correctness, ordering, or durable storage by themselves.

The gateway authenticates sessions, applies message-size and rate limits, and routes traffic. It must not be the source of document truth: a gateway can fail while the document's committed history remains intact.

4. Define the operation protocol before discussing OT

Instead of sending the entire document after every keystroke, a client sends a logical operation. A simplified message might be:

{
  "type": "edit_op",
  "docId": "doc_842",
  "sessionId": "session_19",
  "clientSeq": 391,
  "opId": "op_9012",
  "baseVersion": 1842,
  "operation": {
    "kind": "insert",
    "position": 128,
    "text": " collaborative"
  }
}
Enter fullscreen mode Exit fullscreen mode

Each field solves a different problem:

  • docId routes the operation to the correct document.
  • opId identifies a logical edit across retries, allowing idempotent deduplication.
  • sessionId and clientSeq help identify gaps and ordering within a client session.
  • baseVersion tells the server which authoritative document version the operation was prepared against.
  • operation describes the intended mutation, using a schema understood by the synchronization engine.

The server obtains its authoritative ownership epoch from trusted routing/coordination state; it must not trust an arbitrary client-supplied epoch as proof of ownership. The durable writer checks the owner's fencing token.

For a rich-text editor, the operation model also needs structural semantics: paragraphs, formatting spans, lists, tables, and embedded objects. A plain string index is useful for explaining concurrency, but is not a complete rich-document model. Text-offset conventions (for example, Unicode code points versus UTF-16 units) must be consistent across clients and server.

A committed response might look like:

{
  "type": "op_ack",
  "opId": "op_9012",
  "clientSeq": 391,
  "appliedVersion": 1845,
  "status": "committed"
}
Enter fullscreen mode Exit fullscreen mode

Here committed means durably recorded under our chosen durability policy, not merely received by a gateway. A separate received acknowledgement could be sent earlier, but it must not be displayed as saved.

5. The central challenge: concurrent edits

Imagine the document is:

Hello world
Enter fullscreen mode Exit fullscreen mode

Alice inserts beautiful immediately before world. Bob, editing from the same base version, inserts wide at that same position. Applying both original positional inserts without coordination can yield different results depending on arrival order.

A correct collaborative protocol needs deterministic conflict rules. For simultaneous inserts at the same position, a tie-break rule (for example, an agreed ordering of operation identifiers) can produce one consistent outcome. That outcome may be Hello beautiful wide world or Hello wide beautiful world; the algorithm must make replicas agree, not guess the writers' intended sentence.

Naive last-write-wins whole-document saves are unsuitable because one complete document can overwrite the other. Naive application of stale character offsets is also unsafe because earlier edits shift later positions.

6. Operational Transform (OT): reconcile edits against newer history

In a centralized OT-style architecture, the document owner serializes accepted operations and maintains an authoritative version sequence.

Suppose Alice and Bob both start at version 42:

  1. Alice submits an insertion based on version 42.
  2. The owner validates it, commits it as version 43, and distributes the accepted operation.
  3. Bob submits an operation also based on version 42.
  4. The owner transforms Bob's operation against the change committed at version 43, using the editor's defined transformation rules.
  5. The transformed operation is committed as version 44 and distributed.
  6. Clients reconcile their optimistic local edits with the authoritative sequence.

For a simple positional example, an insertion earlier in the text may shift another insertion's position. For overlapping deletes, replacements, and structured content, the rules are substantially more involved. OT is not simply “increment the index whenever someone inserts text.” Correct implementations must handle operation types, ordering, tie-breaking, outstanding local edits, and transformation properties together.

The server needs enough accepted history since baseVersion to transform a stale operation. If that history has been compacted away, it cannot pretend the transform is still possible. It needs a supported snapshot-based rebase or a conflict-resolution path that may require user assistance.

OT is a sensible teaching choice when one authoritative owner assigns a per-document order. It is not the only viable approach.

7. OT vs CRDT: which should we choose?

A Conflict-free Replicated Data Type (CRDT) uses data structures and merge rules designed so replicas can converge despite concurrent updates. For collaborative text, CRDT approaches often associate content with stable element identifiers rather than relying solely on changing character offsets.

Consideration Operational Transform CRDT
Core technique Transform operations against concurrent accepted history Merge operations/state under defined CRDT rules
Natural fit Centralized, versioned collaboration Replica-local and offline-heavy editing
Main complexity Transformation correctness, outstanding edits, retained history Identifiers, causal context, metadata growth and safe garbage collection
Offline support Possible, but requires explicit reconciliation/rebase Often a strong fit, subject to implementation details
Backend required? Yes for this architecture Still needed for permissions, persistence, sync, and other platform services

Neither strategy automatically guarantees perfect semantic intent preservation. Neither removes the need for security, persistence, or reconnect logic. Nor does the word CRDT mean every arbitrary message order and metadata-compaction scheme is safe; the particular algorithm's assumptions matter.

Our design choice: use an authoritative, OT-style per-document owner for the main architecture. A CRDT-based editor is a valid alternative, especially if offline-first behavior is a primary product requirement. This is an architectural choice, not a claim about Google's internal synchronization algorithm.

8. One authoritative owner per document—not one server for the whole platform

We distribute documents across a fleet of document servers. A shard router resolves docId to the current owner. Each owner serializes authoritative edits for its documents and assigns monotonically increasing versions within each document. Unrelated documents do not need a shared global operation order.

Document A ──► Owner 1 ──► versions A: 1, 2, 3...
Document B ──► Owner 2 ──► versions B: 1, 2, 3...
Hot Doc C  ──► Dedicated owner capacity
Enter fullscreen mode Exit fullscreen mode

This makes concurrent-edit reasoning local to one document while allowing the overall fleet to scale horizontally. The trade-off is that one very hot document can still overwhelm its owner, so we must distinguish many writers from many passive viewers. Those are different bottlenecks.

Owner failover and fencing

A server being designated as the owner is not sufficient to prevent split brain. During a network partition, an old owner might still be alive when a new owner is promoted.

We assign a monotonically increasing fencing epoch to each ownership term. The durable write path validates the epoch on writes and rejects stale owners. Promotion must use a coordination mechanism that prevents conflicting epochs from both being accepted as current.

If ownership is uncertain, briefly pause authoritative writes for the affected document rather than creating two valid-looking, divergent histories. Clients may continue typing into local pending queues while waiting to reconnect.

9. When is an edit actually saved?

This is one of the most important correctness boundaries in the design.

Consider a dangerous implementation:

Owner receives edit → applies in RAM → replies "saved" → crashes
                                          ↓
                              durable write never happened
Enter fullscreen mode Exit fullscreen mode

The user saw saved, but the edit disappeared. To avoid that failure, define a strict contract:

  1. Validate authorization, operation schema, version context, and duplicate identity.
  2. Reconcile the operation with the current authoritative document state.
  3. Assign the next document version and append the accepted operation to the durable log, using a write path that enforces the current fencing epoch.
  4. Wait for the configured durability policy to be satisfied.
  5. Only then acknowledge the edit as committed and publish it as an authoritative update.

A client may render the edit locally before step 4; that is optimistic UI, not durable acceptance. If the owner crashes after the durable append but before sending the acknowledgement, the client retries the same opId. Deduplication returns the original committed result rather than applying the edit twice.

The exact durability guarantee depends on storage and replication policy. A local disk append, a quorum-replicated commit, and synchronous cross-region replication provide different failure guarantees. Do not promise zero data loss during regional disasters unless the storage design actually supports it.

10. Operation log, snapshots, and version history

version history

Persisting a full document body after every tiny edit is expensive. Replaying its entire lifetime of operations every time someone opens it is also expensive.

Use a hybrid approach:

Snapshot at version 1,000
      + committed operations 1,001 ... 1,080
      = current document at version 1,080
Enter fullscreen mode Exit fullscreen mode

A snapshot contains a materialized document state at an exact authoritative version, with format/version metadata and integrity checks. The operation log records accepted changes. A bootstrap or recovering owner loads a verified snapshot and replays the verified log tail.

The materialized document view and the operation history may live in different stores and need not be updated in one distributed transaction. In this design, the durable operation log is authoritative; snapshot/materialized views are versioned projections that can be rebuilt idempotently. This is why recovery needs explicit version boundaries and reconciliation.

For user-visible version history, keep snapshots or revisions according to the product's retention requirements. Restoring an earlier revision should be a new authorized mutation at a new current version, not a silent rewind that invalidates connected clients' understanding of the timeline.

Why compaction is subtle

An old operation may still be needed for:

  • Transforming a client's stale OT operation.
  • Replaying an offline client's missing history.
  • Reconstructing a user-visible revision.
  • Meeting audit or retention requirements.

A snapshot does not automatically make all earlier operations disposable. Retention and compaction must be designed around these consumers, and long-offline clients need a defined fallback when the necessary history is no longer available.

11. Presence and cursors are not document truth

presence and cursors

A collaborator's cursor position, selection, and online indicator should not have the same durability requirements as a text edit.

A separate presence service tracks active sessions, not merely user IDs. One person may have the same document open in multiple tabs; closing one tab should not make the other disappear from the presence list. Heartbeats and TTLs help expire abandoned sessions.

Cursor updates can be sampled, coalesced, or dropped when clients are slow. A failed presence service may temporarily hide collaborator cursors, while committed document editing continues. Conversely, an accepted document edit must not be silently discarded just because a gateway's outbound queue is full.

Cursor positions also need reconciliation with document changes. An offset meaningful at version 100 may no longer identify the same location at version 105. A practical editor transforms selection anchors or uses position representations compatible with its document model.

12. Offline editing and reconnection

A disconnected user should usually be able to continue typing. The client applies changes locally and stores pending operations with stable IDs and sequencing information.

On reconnect:

  1. Authenticate and recheck current document permissions.
  2. Report the last known committed version and outstanding operation IDs.
  3. Determine which submitted operations were already committed but lost their acknowledgements.
  4. Fetch missing authoritative operations, if the relevant history is retained.
  5. Reconcile local pending operations against the updated state, respecting client-side ordering and the chosen OT/CRDT protocol.
  6. Resubmit unresolved operations idempotently and resume live subscriptions.

If the client is too far behind for incremental OT transformation, use a snapshot-based rebase only if the implementation has a valid way to map the local edits onto that snapshot. A snapshot by itself cannot reconstruct the user's original intent. For difficult conflicts, preserve the unsynced local copy and ask for manual resolution rather than silently dropping text.

If access was revoked while the user was offline, queued edits must not be accepted on reconnect. The UI may preserve local unsynced work where appropriate, but the backend enforces the current permission state.

13. Authorization must work during a live session

Checking permission only when a WebSocket opens is insufficient. An editor may be removed from a document while the socket remains connected.

Enforce authorization at the authoritative mutation path and ensure that permission changes invalidate or update live subscriptions promptly. A revoked viewer must stop receiving protected document updates; a revoked editor must stop submitting accepted edits. Reconnect and history/restore endpoints need their own authorization checks too.

Also validate every incoming operation's schema, size, ranges, base-version plausibility, and allowed document mutation type. A buggy or malicious client must not be able to corrupt shared state by submitting invalid operations.

14. Capacity planning: connections, operations, and fan-out

A collaborative editor is not sized by document storage alone. Here is an illustrative, not measured, workload:

Assumption Example
Concurrent actively edited documents 270,000
Average live editors per active document 3
Active editing connections 810,000
Average logical operations per active editor per second 0.8
Incoming edit operations per second 648,000
Additional peer deliveries at two peers per operation ~1.30 million/sec
Total open WebSocket connections including viewers 3 million
Heartbeat interval 20 seconds
Incoming heartbeat messages per second 150,000

The arithmetic is useful because it separates ingress edit throughput, outbound fan-out, and connection-management load. It is not a claim about Google Docs traffic.

A document with three editors is different from a document with three editors and 100,000 viewers. The latter may have little write traffic but enormous broadcast pressure.

To scale, distribute gateways horizontally, shard ownership by document, isolate hot documents, use gateway-local subscription fan-out, and keep per-connection buffers bounded. For large documents, consider incremental loading and efficient client rendering before attempting complex block-level authoritative sharding. Splitting a single document into independently writable blocks complicates cross-block edits, selections, and global operations.

15. Failure scenarios that decide whether the architecture is production-ready

Failure Correct behavior
WebSocket gateway crashes Reconnect to another gateway; retry unacknowledged ops using stable IDs
Owner crashes after durable commit but before ack New owner recovers committed log; retry deduplicates the op
Owner crashes before durable commit Do not claim the edit was saved; client keeps it pending
Two owners believe they are active Durable writer rejects stale fencing epoch; affected clients resync
Broadcast misses version 105 but delivers 106 Detect version gap and fetch missing history or snapshot
Snapshot disagrees with log Reject suspect snapshot; recover from last verified checkpoint and log
Presence service fails Degrade cursors/online indicators without corrupting edits
Permission revoked mid-session Stop writes and protected subscriptions promptly
Client reconnects months later Detect missing transform history; use supported rebase/manual conflict path
Slow subscriber accumulates updates Bound queue; drop ephemeral presence first; trigger authoritative resync
Reconnection storm after outage Backoff with jitter, admission control, prioritized recovery
Sync algorithm deployment introduces a bug Stop affected writes if necessary, roll back, validate history and repair
Region fails Recover from replicated history with higher fenced epoch; honor actual RPO/RTO

A particularly subtle case is broadcast loss. Pub/sub is a delivery mechanism, not the authoritative history. Every committed remote operation should carry a document version so a client can detect gaps and recover from the authoritative log. Even if the transport is generally reliable, application-level version checks remain necessary.

16. Trade-offs worth explaining in an interview

OT or CRDT? OT integrates naturally with a centrally ordered per-document log; CRDT can suit offline-first replication. Both require careful implementation and testing.

Single owner or multi-writer? A fenced owner simplifies ordering and debugging. Multi-writer designs may offer different availability/latency properties but introduce harder coordination and merge semantics.

Optimistic typing or server-confirmed rendering? Apply locally immediately, but distinguish pending from committed. Waiting for server round trips before rendering every keystroke harms usability.

Snapshots or operations? Combine versioned snapshots with a durable operation log. Compaction must respect replay, restore, and offline requirements.

One home region or active writers everywhere? A home-region owner keeps ordering manageable, at the cost of higher network round-trip time for distant collaborators. Multi-region active writing is a separate, more complex consistency design.

Always accept writes or pause during uncertain failover? Prefer a brief per-document write pause over divergent authoritative histories. Local pending edits can preserve the typing experience during that pause.

17. How to answer “Design Google Docs” in a system design interview

A clear interview narrative is more valuable than starting with a giant infrastructure diagram:

  1. Clarify scope: plain text or rich text? Typical collaborators? Offline support? History? Permissions?
  2. State invariants: eventual convergence, no falsely acknowledged edits, one authoritative version sequence per document in this design.
  3. Show the naive design: one buffer, autosave, and why whole-document writes lose concurrent changes.
  4. Introduce WebSockets: persistent transport, gateways, routing, and live fan-out.
  5. Explain concurrency: an actual conflicting-edit example, followed by OT or CRDT and its limitations.
  6. Make durability explicit: operation IDs, commit acknowledgements, durable history, snapshots, and replay.
  7. Add failure handling: owner fencing, reconnect, authorization changes, missing broadcasts.
  8. Scale the system: shard by document, separate presence, isolate hot documents, and manage large fan-out.

The high-signal follow-up is often not “What is OT?” but “What happens when the owner crashes after sending a saved acknowledgement?” Your answer should point to the durability boundary, not merely say that the client reconnects.

Frequently asked questions

How does a Google Docs-style editor prevent users from overwriting each other's changes?

It sends structured edits rather than blindly replacing the entire document. An OT or CRDT synchronization protocol reconciles concurrent operations so clients can converge on a shared state. The protocol still needs correct implementation, ordering or merge semantics, and durable storage.

Does Google Docs use Operational Transform or CRDT?

This article does not assert which algorithm Google's current internal implementation uses. It explains two established approaches for building a Google Docs-style editor and chooses a centralized OT-style architecture as a teaching example.

Why are WebSockets not enough for collaborative editing?

WebSockets move messages bidirectionally, but do not solve concurrent edits, authoritative ordering, duplicate submissions, persistence, permission revocation, or recovery from missed updates.

How are edits recovered after a disconnect?

Clients keep stable operation IDs and pending local edits. On reconnect they reauthenticate, fetch missing authoritative changes or a supported snapshot/rebase path, reconcile outstanding work, and retry idempotently. Severe conflicts may require user intervention.

How do you scale a document with thousands of viewers?

Keep one correctly fenced authoritative edit stream in the baseline design, but distribute already-committed updates through scalable fan-out infrastructure. Coalesce presence and isolate hot documents; do not confuse passive-viewer traffic with write-serialization load.

Final architecture: what really matters

A reliable collaborative editor combines instant local editing with authoritative, durable synchronization. The core path is:

Client → WebSocket Gateway → Shard Router → Document Owner
       → Validate / Deduplicate / Reconcile
       → Durable Operation Log → Commit Ack
       → Broadcast → Other Clients
Enter fullscreen mode Exit fullscreen mode

Around it sit fenced ownership, versioned snapshots, offline replay, authorization, presence, and observability. The difficult engineering is not choosing a WebSocket library. It is making sure users never receive a false saved signal, stale owners cannot create conflicting histories, and disconnected replicas have a safe path back to a converged state.

Explore the full visual system design on SeeItFlow

This article is a condensed architectural walkthrough. For the progressive diagrams, detailed trade-offs, production failure walkthroughs, and interview questions, see the complete six-part module:

SeeItFlow is a free visual learning platform for software engineering and system design.

Top comments (1)

Collapse
 
fintechrailsdigest profile image
Fintech Rails Digest •

the op protocol section is where most explainers stop early. opId for idempotent dedup across retries and baseVersion telling the server which version the op was prepared against are the two fields that decide whether a flaky websocket turns into duplicate text or a lost edit, and naming them explicitly is useful. the point about not trusting a client-supplied ownership epoch is well made too. an authoritative owner assigning per-document order keeps the transform logic tractable in a way full CRDT merge rules rarely stay, so the choice reads defensible rather than just traditional.