NebulaDB
Live showcase

Architecture

How a request becomes durable, searchable, replicated data

One process, one index, many front ends. This page follows a write, a query and a RAG call through NebulaDB, then shows how nodes and regions stay in step.

01 · The front door

Every request passes the same middleware chain

Ordering is deliberate. Audit runs before the follower guard and auth, so rejected writes are still recorded. Health and metrics bypass the chain so probes survive an abusive client.

Middleware chain: rate limit, audit, follower write guard, auth, body limit and trace, handler Rate limittoken bucket Auditmutations logged Follower guard409 on followers AuthAPI key · JWT Limits · tracetower-http HandlerREST · SSE /healthz AND /metrics ARE MOUNTED OUTSIDE THIS CHAIN

02 · Query planes

Three paths through one index

WRITE PATH

Ingest a document

  1. Pick a chunker for the document type
  2. Embed each chunk: LRU, then Redis, then the model
  3. Append to the WAL with CRC32 and fsync
  4. Apply to HNSW, BM25 and the document store
  5. Stream to followers and remote regions
nebula-chunk → nebula-embed → nebula-wal → nebula-index
READ PATH

Run SQL

  1. Parse into a typed plan tree
  2. Check the result cache
  3. Run semantic_match as HNSW top-k
  4. Apply residual filters, GROUP BY and joins
  5. Shape rows for REST, gRPC or pgwire
nebula-sql → nebula-index → nebula-pgwire
AI PATH

Answer a question

  1. Expand the query, embed it
  2. Hybrid retrieve, optionally rerank
  3. Emit context events
  4. Build the guarded prompt, stream LLM tokens
  5. Record time-to-first-token in metrics
nebula-rerank → nebula-bm25 + nebula-vector → nebula-llm

03 · Durability

Write-ahead log, snapshots, recovery

The index in RAM is a cache of the log. If the process dies at any instant, the next boot reconstructs exactly the acknowledged state.

WAL frame
┌─────────────┬─────────────┬───────────────────────┐
│ len (u32 LE)│ crc32 (u32) │ bincode(WalRecord)    │
└─────────────┴─────────────┴───────────────────────┘

/var/lib/nebula/<node>/
  wal/0000000001.nwal
  wal/0000000002.nwal
  snapshots/snap-00042.nsnap
  cluster/follower.cursor

  1. Load the newest committed snapshot

    Snapshots are written to a temporary file and atomically renamed, so a partial snapshot is never visible.

  2. Replay WAL segments past the snapshot

    Vectors are in the records, so no embedding calls are made during recovery.

  3. Truncate a torn tail

    A frame whose length or CRC does not check out is cut back to the last good record.

  4. Serve

    /admin/durability reports segments, size and sequence range.

04 · Multi-region

Home regions and XDCR

Each region accepts writes only for the buckets it homes, replicates both ways, and serves every read locally. There is one writer per bucket, so there is nothing to merge across oceans.

Two regions with leaders, followers and XDCR Region us-east-1 has a leader homing bucket A and a follower. Region eu-west-1 has a leader homing bucket B and a follower. Followers tail their local leader's WAL. XDCR streams run in both directions between the leaders. A client writing bucket B to us-east receives 421 and retries in eu-west. REGION US-EAST-1 Leaderhome: bucket Awrites A · reads all Followertails WAL · readspromote on failure TailWal REGION EU-WEST-1 Leaderhome: bucket Bwrites B · reads all Followertails WAL · readspromote on failure TailWal XDCR · A → ← B · XDCR nebula-clientwrite B to us-east→ 421 → retry eu-west

05 · Zero-downtime change

Swap rebalance as a state machine

Upgrading or replacing a node is a Kubernetes resource the operator drives through explicit phases, so a stuck step is visible and resumable rather than a half-finished script.

NebulaRebalance phases: Pending, Preflight, Snapshotting, Rebalancing, Promoting, Draining, Completed Pending Preflight Snapshotting Rebalancing Promoting Draining Completed ANY PHASE → FAILED, WITH THE REASON ON THE RESOURCE STATUS

06 · Kubernetes

Declarative operations

operator/examples/cluster-ha.yaml (excerpt)
apiVersion: nebula.nebuladb.io/v1alpha1
kind: NebulaCluster
metadata:
  name: nebula-prod
spec:
  replication:
    enabled: true
    followers: 2
    lagByteThreshold: 104857600   # 100 MiB
  autoscaling: { enabled: true, minFollowers: 2, maxFollowers: 5 }
  backup:
    enabled: true
    schedule: "0 */6 * * *"
    retain: 14
  security:
    apiKeysSecretRef: { name: nebula-auth, key: api_keys }
  regions:
  - name: us-east-1
  - name: us-west-2
  • NebulaClusterLeader and followers per region, storage, AI configuration, auth and upgrade policy.
  • NebulaBucketDeclarative bucket seeding against a cluster.
  • NebulaRebalanceOne-shot rebalance, failover or upgrade.
  • NebulaBackup · NebulaRegionFailoverScheduled backups and region-level failover.

More examples in operator/examples/: basic cluster, bucket seeding, backup schedule, swap, scale and failover rebalances, region failover.

Go deeper

Design records cover multi-region active-active, backup and restore, Raft, CDC, parallel replay, zero-downtime rebalance, graceful degradation, MCP and Azure AI Search compatibility.

Read the design docs All features