Nervos SMT Sidecar

The Nervos SMT sidecar is the default state-root engine for new NOOSChain chains. It replaces the legacy iden3-backed sparse Merkle tree as the component that folds deterministic state leaves into a block state root.

For the general explanation of state roots, state leaves, SMTs, and engine selection, start with State Root.

It does not replace NOOSChain execution, permissions, encryption, snapshots, replay, Raft, or operator governance. TypeScript still owns block execution and decides the ordered list of state mutations. The Rust sidecar owns only the Nervos sparse Merkle tree calculation and local checkpoint/load behavior.

Origin

"Nervos" here refers to the maintained Rust sparse-merkle-tree crate from the Nervos ecosystem. NOOSChain is not depending on Nervos CKB as a blockchain, is not bridging to CKB consensus, and is not outsourcing block validation to a Nervos network.

NOOSChain uses the Nervos SMT implementation as a local state-root engine. The production sidecar lives under:

external/state-root-sidecar-nervos/

The benchmark and candidate-evaluation utility lives under:

external/state-root-sidecar-nervos-tools/

The frozen engine contract identifies the hasher as the Nervos sparse-merkle-tree crate with Blake2bHasher. That choice is part of the nervos-smt-v2 state-root contract.

Why It Was Chosen

The legacy iden3-v1 path remains available for old blocks, replay, snapshots, and activation-boundary tests, but NOOSChain needed a stronger production path for new chains.

The Nervos sidecar was chosen because it provides:

• a maintained Rust sparse-Merkle-tree implementation;
• a clear 32-byte H256 key/value model;
• deterministic root calculation with pinned vectors;
• a natural external-process boundary for heavy SMT work;
• persistent checkpoint/load behavior for large state sets;
• a path toward state proof generation and stateless proof verification;
• less dependence on Node.js process memory during root calculation.

The tradeoff is important: nervos-smt-v2 is not byte-compatible with iden3-v1. It produces a different root scheme for the same logical state. That is why existing iden3 chains must migrate through an explicit activation height rather than by changing a local environment default.

What It Is Not

The Nervos SMT sidecar is not:

• Nervos CKB consensus;
• a blockchain bridge;
• a Raft or HashiCorp Raft sidecar;
• the smart-contract Wasmtime sidecar;
• a database replica;
• a transaction executor;
• a permission, encryption, key, or payload service;
• a way to reinterpret old iden3 blocks under new root rules.

It receives only state-root protocol messages and returns roots, checkpoints, and proof results.

What Changes

nervos-smt-v2 changes the state-root scheme. It is not byte-compatible with iden3-v1.

For new chains, the default state-root engine is Nervos from genesis. For existing iden3 chains, moving to Nervos is a protocol-visible migration and must happen at a recorded activation height. Operators must not continue an existing iden3 chain by changing a local environment default.

The practical changes are:

• block metadata records state_root_engine_version;
• snapshots and chain checkpoints preserve the engine metadata;
• replay selects the engine per block, not from the current process default;
• production admission requires noos state-root preflight to pass;
• after Nervos is active, the node fails closed if the sidecar is missing, wrong-version, unhealthy, or unable to load a trusted checkpoint.

Execution Split

The execution boundary is deliberately narrow:

1. Consensus orders a NOOSChain block envelope.
2. TypeScript verifies and executes the block deterministically.
3. TypeScript produces ordered state-root mutations from consensus state changes.
4. The selected state-root engine applies those mutations.
5. The returned root is stored on the block and checked by replay/sync.

The sidecar never receives raw database mutations, plaintext payloads, private keys, DEKs, bearer tokens, or Raft log control. It receives only the state-root engine protocol messages documented in Nervos State-Root Contract.

At runtime, the flow is:

NOOSChain block execution
  -> ordered state changes
  -> sidecar apply_batch
  -> sidecar root
  -> block state_root and state_root_engine_version

The transport is newline-delimited JSON over stdin/stdout. Production execution uses a supervised sidecar process and an exclusive execution lease so two block executions cannot interleave stateful sidecar operations.

Consensus-Critical Contract

The following are consensus-critical for nervos-smt-v2:

• engine id: nervos-smt-v2;
• sidecar protocol version: 1;
• checkpoint format version: 1;
• key encoding: lowercase 32-byte hex H256;
• value encoding: lowercase 32-byte hex H256;
• mutation ordering: caller-provided block state-change order;
• empty root semantics;
• delete semantics;
• hasher/tree implementation;
• stable error behavior for invalid inputs and checkpoint mismatches.

Changing any of those values requires a new engine id or an explicit protocol migration. A validator must not locally reinterpret a key, sort mutations, deduplicate writes, or fall back to another engine after activation.

Build

Build the production sidecar with:

npm run build:state-root-sidecar-nervos

That script runs a Cargo release build from:

external/state-root-sidecar-nervos/Cargo.toml

Build the sidecar plus benchmark/candidate tooling with:

npm run build:state-root-tools

That script builds both:

• external/state-root-sidecar-nervos/
• external/state-root-sidecar-nervos-tools/

On Windows, the default release binary path is:

external/state-root-sidecar-nervos/target/release/noos-state-root-sidecar-nervos.exe

On Unix-like hosts, the default release binary path is:

external/state-root-sidecar-nervos/target/release/noos-state-root-sidecar-nervos

Configuration

For new chains, set:

NOOS_STATE_ROOT_ENGINE=nervos-smt-v2

NOOSChain locates the sidecar with NOOS_STATE_ROOT_SIDECAR_COMMAND. If the variable is not set, the node uses the default release path under external/state-root-sidecar-nervos/target/release/.

Core sidecar settings:

Setting	Purpose
NOOS_STATE_ROOT_SIDECAR_COMMAND	Sidecar executable path.
NOOS_STATE_ROOT_SIDECAR_PROTOCOL_VERSION	Required sidecar protocol version; currently 1.
NOOS_STATE_ROOT_CHECKPOINT_DIR	Directory for verified local checkpoint files; defaults to ./state-root-checkpoints.
NOOS_STATE_ROOT_SIDECAR_TIMEOUT_MS	Per-request timeout; default is 60000.
NOOS_STATE_ROOT_SIDECAR_AUTO_RESTART	Whether persistent sessions restart after failure; default is true.
NOOS_STATE_ROOT_SIDECAR_MAX_RESTARTS	Restart limit; default is 3.
NOOS_STATE_ROOT_SIDECAR_RESTART_BACKOFF_MS	Restart delay; default is 2000.
NOOS_STATE_ROOT_SIDECAR_SUPERVISOR_ENABLED	Supervisor policy: auto, true, or false.
NOOS_STATE_ROOT_SIDECAR_HEALTH_INTERVAL_MS	Supervisor health-loop interval; default is 10000.
NOOS_STATE_ROOT_SIDECAR_PREWARM	Start the supervisor before Nervos is required or scheduled.

For existing iden3 chains, do not use these settings to silently continue the same chain with Nervos roots. Use an activation boundary instead.

Build And Verify Recipe

Use this sequence before enabling Nervos in a local or production-like node:

npm run build:state-root-sidecar-nervos
npm run test:state-root-nervos-vectors
npm run test:state-root-persistent-sidecar
npm run test:state-root-engine-safety
npm run test:state-root-activation-readiness
npm run noos -- state-root preflight --json

The vector tests pin empty-root and ordered-mutation behavior. Persistent sidecar tests exercise checkpoint load/export and crash/restart behavior. Activation-readiness and preflight checks prove that the node can execute the configured sidecar before it admits Nervos blocks.

Continuous Supervisor

Production nodes run a continuous sidecar supervisor when Nervos is required, scheduled, explicitly enabled, or prewarmed. The supervisor:

• starts the sidecar without shell interpolation;
• checks sidecar version and health;
• restarts with bounded backoff;
• exposes status through /node/observability/consensus;
• lends the running process to block execution through an exclusive lease;
• records degraded status and last errors for operators.

Normal post-activation execution uses the supervised process. Short-lived sidecar sessions may still be used by isolated tests or one-shot probes, but the production execution path is the continuous supervised sidecar.

Checkpoint Flow

For each Nervos block, TypeScript:

1. loads the latest verified local sidecar checkpoint at or before the previous height;
2. hydrates from state_leaves if no trusted checkpoint exists;
3. applies ordered block mutations;
4. reads the root;
5. exports a new sidecar checkpoint;
6. hashes the checkpoint file and records verified metadata.

Sidecar checkpoint files live under NOOS_STATE_ROOT_CHECKPOINT_DIR. They are local operational cache, not consensus state. Losing them is recoverable from materialized leaves but slower. Corrupt or wrong-version checkpoint files are rejected before execution.

Normal snapshots do not bundle local sidecar checkpoint files. Snapshot import preserves state-root engine metadata and state_leaves; after import, a Nervos node can hydrate the sidecar from those leaves and export a fresh verified local checkpoint.

Observability

Operators inspect Nervos through:

npm run noos -- state-root status --json
npm run noos -- state-root readiness --json
npm run noos -- state-root preflight --json
npm run noos -- state-root proof get --namespace bucket --id bucket-a --json
npm run noos -- state-root proof verify --file proof.json --json
npm run noos -- production monitor-report --json

The Admin GUI shows the same data on Dashboard and Membership/Consensus:

• active engine;
• scheduled activation;
• readiness checks;
• supervisor state;
• sidecar process status;
• sidecar version/protocol/checkpoint metadata;
• latest verified checkpoint.

The proof commands use the public /state/proof and /state/proof/verify routes. NOOSChain maps public namespace/entity ids to canonical state keys, then the Nervos sidecar generates or statelessly verifies the compiled nervos-smt-v2-proof-v1 proof.

Related Docs

• Nervos State-Root Contract
• State-Root Engine Migration
• State-Root Activation
• Nervos Activation Runbook
• Nervos Sidecar Recovery Runbook