Performance Benchmarks

Performance benchmarks are developer tools for measuring hot paths, catching regressions, and collecting release evidence. They complement tests and verification commands; they do not replace them.

Use this page when you need to answer one of these questions:

• Did a code change make block execution, sync, snapshots, or state roots slower?
• Which subsystem is responsible for a measured slowdown?
• Is a candidate state-root engine fast enough to keep evaluating?
• Does a release candidate still fit the current v1 performance budget?
• Which benchmark report should be attached to a release or optimization decision?

Benchmarks deliberately run outside the normal fast test loop. Most runners create isolated database schemas, seed synthetic or replay-shaped data, write a JSON report under benchmark-results, and drop their schema unless a KEEP_DB flag is set.

Quick Selection

Goal	Command	Use when
Run the release performance matrix	npm run benchmark:v1-readiness	Preparing a v1 release candidate or checking broad regressions.
Check the release matrix against budgets	npm run benchmark:v1-readiness:check	A readiness report has been produced and needs pass/fail evaluation.
Measure block commit hot paths	npm run benchmark:block-execution	Touching handlers, indexes, state leaves, SMT application, or block finalization.
Measure PostgreSQL query/write/storage hot paths	npm run benchmark:db-performance	Adding indexes, changing table shape, or investigating DB-heavy regressions.
Compare iden3 and Nervos state-root engines	npm run benchmark:state-root-engines	Evaluating state-root activation, replay, snapshots, and sidecar recovery.
Measure state-root candidate apply/rebuild behavior	npm run benchmark:state-root-engine	Testing an engine boundary before connecting it to real block execution.
Measure SMT adapter behavior	npm run benchmark:smt	Tuning persistent iden3 storage, cache behavior, or flush batching.
Evaluate future SMT candidates	npm run benchmark:smt:evaluation	Checking deterministic vectors and the external candidate process contract.
Measure sync catch-up	npm run benchmark:sync-catchup	Changing sync, payload backfill, peer selection, snapshots, or observer recovery.
Measure payload recovery writes	npm run benchmark:payload-backfill	Payload phases dominate sync catch-up.
Measure local payload-store reads and stats	npm run benchmark:payload-store	Changing payload lookup, payload GC, payload stats, or export-like scans.
Measure archive snapshot export/verify/import	npm run benchmark:snapshots	Changing snapshot format, import staging, verification, or restore logic.
Measure public-index search strategies	npm run benchmark:record-index-search	Changing record_indexes query plans or index design.
Measure signing/canonicalization cost	npm run benchmark:signing	Changing signatures, canonical JSON, key parsing, or signer caching.

package.json still contains benchmark:performance, but the referenced src/stability-test/benchmarks/benchmark-runner.ts file is not present in this workspace. Treat that command as legacy design until the runner is restored or removed.

How To Read Benchmark Reports

Prefer reports that explain where time went, not just how long the whole run took. The useful fields vary by runner, but most reports include a combination of:

• config: workload size, profile, batch sizes, feature flags, and report directory.
• timings: p50, p95, p99, average, min, and max latency for named phases.
• throughput: blocks/sec, transactions/sec, payloads/sec, rows/sec, or mutations/sec.
• storage: table and index size for hot database objects.
• verification: root equality, replay checks, payload availability checks, or counter consistency checks.
• recommendations: threshold-based hints from the runner.

When comparing reports:

1. Compare the same command, profile, environment variables, branch, database, sidecar binary, and machine class.
2. Prefer p95 and p99 for user-facing or operator-facing latency. Averages hide tails.
3. Record dataset size. A 2,000-row smoke profile and a 1,000,000-row snapshot profile are different evidence.
4. Treat fast local runs as development smoke only. Release gates need production-like hardware.
5. Run correctness tests after performance work. A faster path is not useful if replay, state-root verification, or snapshot restore breaks.

Common mistakes:

• comparing Windows laptop results with Linux CI or production-like hardware;
• accepting a mean-only improvement while p95/p99 regresses;
• changing indexes because one query improved while block writes slowed down;
• treating a benchmark pass as consensus-safety evidence;
• ignoring sidecar version, PostgreSQL configuration, or environment flags;
• hiding stale cached counters behind faster operator endpoints.

V1 Readiness Matrix

The v1 readiness matrix is the broad benchmark gate. It runs the most important focused benchmarks together and writes one combined report.

Fast local smoke:

$env:NOOS_V1_READINESS_FAST="true"
npm run benchmark:v1-readiness
npm run benchmark:v1-readiness:check
Remove-Item Env:\NOOS_V1_READINESS_FAST

Full matrix:

npm run benchmark:v1-readiness
npm run benchmark:v1-readiness:check

The matrix currently runs:

• build:state-root-sidecar-nervos
• benchmark:block-execution for configured profiles
• benchmark:state-root-engines
• benchmark:snapshots
• benchmark:sync-catchup
• benchmark:payload-backfill
• benchmark:state-root-persistent-sidecar

The runner writes reports under benchmark-results/v1-readiness. The checker uses test/benchmarks/v1-readiness/v1-performance-budgets.json unless NOOS_V1_BUDGET_FILE points somewhere else.

Useful matrix variables:

Variable	Meaning
NOOS_V1_READINESS_FAST	Uses smaller smoke profiles when true. Do not use this for sign-off.
NOOS_V1_READINESS_LOG_DIR	Combined report directory.
NOOS_V1_READINESS_BLOCK_PROFILES	Comma-separated block execution profiles.
NOOS_V1_READINESS_STATE_ROOT_PROFILES	Comma-separated state-root comparison profiles.
NOOS_V1_READINESS_SYNC_SCENARIOS	Comma-separated sync scenarios.
NOOS_V1_READINESS_SNAPSHOT_ROWS	Snapshot fixture size.
NOOS_V1_READINESS_PAYLOAD_BACKFILL_RECORDS	Payload-backfill fixture size.
NOOS_V1_READINESS_SIDECAR_LEAVES	Persistent sidecar initial leaf count.
NOOS_V1_READINESS_SIDECAR_UPDATES	Persistent sidecar update count.
NOOS_V1_READINESS_REPORT	Existing report consumed by the checker.
NOOS_V1_READINESS_CHECK_OUTPUT	Budget-check output path.

For operator-facing budgets and sign-off workflow, see V1 Performance Targets.

Execution And Storage Benchmarks

Block Execution

Run the default block execution benchmark:

npm run benchmark:block-execution

Common profiles:

$env:NOOS_BLOCK_EXEC_BENCH_PROFILE="small"
npm run benchmark:block-execution

$env:NOOS_BLOCK_EXEC_BENCH_PROFILE="large-block"
npm run benchmark:block-execution

$env:NOOS_BLOCK_EXEC_BENCH_PROFILE="many-indexes"
npm run benchmark:block-execution

Important variables:

Variable	Default	Meaning
NOOS_BLOCK_EXEC_BENCH_PROFILE	medium	small, medium, large-block, many-indexes, or low-cardinality-indexes.
NOOS_BLOCK_EXEC_BENCH_BLOCKS	profile-dependent	Blocks to build and execute.
NOOS_BLOCK_EXEC_BENCH_TXS_PER_BLOCK	profile-dependent	Transactions per block.
NOOS_BLOCK_EXEC_BENCH_INDEXES_PER_RECORD	profile-dependent	Public indexes per encrypted-record transaction.
NOOS_BLOCK_EXEC_BENCH_PAYLOAD_SIZE_BYTES	profile-dependent	Synthetic encrypted payload size.
NOOS_BLOCK_EXEC_BENCH_LOG_DIR	./benchmark-results/block-execution	Report directory.
NOOS_BLOCK_EXEC_BENCH_KEEP_DB	false	Keep the isolated schema for inspection.

The runner submits synthetic ADD_ENCRYPTED_RECORD transactions through the mempool, builds real blocks, and executes them through the normal executor. Watch these fields first:

• blockCommitTotalMs: total commit cost.
• executionHandlerMs: transaction handler work, including encrypted-record writes and public-index fanout.
• executionStateLeafMaterializeMs: cost of materializing state_leaves.
• executionStateSmtNetApplyMs and executionSmtFlushMs: SMT mutation and persistence cost.
• mempoolAdmissionMs and executionAuthMs: signer, nonce, and authorization cost.
• fanout.transactionTypes: which transaction types create state rows, unique state keys, and net SMT mutations.
• summary.smt: whether SMT dominates block time.

Use the fanout report before changing the SMT backend. If one handler creates many unique state keys per transaction, the cost is real state fanout. If many same-key mutations collapse inside the block, batching is already helping and the next optimization may be handler shape, not tree implementation.

Database Performance

Run:

npm run benchmark:db-performance

Smoke and larger profiles:

$env:NOOS_DB_PERF_ROWS="1000"
$env:NOOS_DB_PERF_ITERATIONS="3"
$env:NOOS_DB_PERF_WRITE_ROWS="300"
npm run benchmark:db-performance

$env:NOOS_DB_PERF_ROWS="50000"
$env:NOOS_DB_PERF_ITERATIONS="20"
$env:NOOS_DB_PERF_WRITE_ROWS="5000"
npm run benchmark:db-performance

Important variables:

Variable	Default	Meaning
NOOS_DB_PERF_ROWS	5000	Synthetic rows for read-path scenarios.
NOOS_DB_PERF_ITERATIONS	10	Query repetitions per scenario.
NOOS_DB_PERF_WRITE_ROWS	1000	Rows used in write-throughput scenarios.
NOOS_DB_PERF_RESULT_LIMIT	100	Result limit for bounded reads.
NOOS_DB_PERF_LOG_DIR	./benchmark-results/db-performance	Report directory.
NOOS_DB_PERF_KEEP_DB	false	Keep the isolated schema.
NOOS_DB_PERF_WRITE_DIAGNOSTICS	false	Run trigger/index write-cost diagnostics.

The report covers hot query plans, write throughput, table/index storage, and recommendations. Use it before adding indexes. An index that improves a rare query but slows encrypted-record writes or block execution can be a net loss.

When encrypted-record writes are slow, run:

$env:NOOS_DB_PERF_WRITE_DIAGNOSTICS="true"
npm run benchmark:db-performance

Interpret diagnostic variants this way:

• trigger-group speedup means cached-counter maintenance is suspect;
• all-trigger speedup with weak individual signals means cumulative trigger overhead is suspect;
• secondary-index speedup means the index set should be audited against real query plans;
• no meaningful speedup means the bottleneck is likely base row size, PostgreSQL settings, WAL/disk behavior, or the benchmark machine.

Record Index Search

Run:

npm run benchmark:record-index-search

Important variables:

Variable	Default	Meaning
NOOS_RECORD_INDEX_BENCH_RECORDS	10000	Synthetic records to seed.
NOOS_RECORD_INDEX_BENCH_ITERATIONS	20	Query iterations.
NOOS_RECORD_INDEX_BENCH_PROFILE	mixed_realistic	Search workload profile.
NOOS_RECORD_INDEX_BENCH_RESULT_LIMIT	100	Result limit.
NOOS_RECORD_INDEX_BENCH_RESULT_OFFSET	0	Result offset.
NOOS_RECORD_INDEX_BENCH_WRITE_RECORDS	5000	Write-side comparison rows.
NOOS_RECORD_INDEX_BENCH_LOG_DIR	./benchmark-results/record-index-search	Report directory.
NOOS_RECORD_INDEX_BENCH_KEEP_DB	false	Keep the isolated schema.

This benchmark compares the current record_indexes index set, targeted (bucket_id, key, typed_value) indexes, and a benchmark-only normalized-key prototype. Use it with Record Public Indexes when deciding whether a public-index query deserves schema work.

Signing

Run:

npm run benchmark:signing

Important variables:

Variable	Default	Meaning
NOOS_SIGNING_BENCH_ITERATIONS	10000	Iterations for small-payload operations.
NOOS_SIGNING_BENCH_LARGE_PAYLOAD_BYTES	65536	Synthetic large encryptedPayload size.
NOOS_SIGNING_BENCH_LOG_DIR	./benchmark-results/signing	Report directory.

The benchmark separates PEM parsing, canonical JSON serialization, native Node/OpenSSL Ed25519 signing/verification, signObject and verifyObjectSignature, cached signature paths, and full transaction sign/verify paths.

If native Ed25519 with pre-parsed keys is fast but signObject is slow, focus on key-object caching and canonicalization before considering a crypto backend replacement. Signature key caching is process-local and stores parsed Node KeyObjects only. It does not store signatures, transaction bodies, plaintext, DEKs, or payload bytes.

State-Root And SMT Benchmarks

State-root benchmarks are especially sensitive to engine version and sidecar binary. Record the active root engine, activation settings, platform, and sidecar build when comparing results.

State-Root Engine Comparison

Run:

npm run benchmark:state-root-engines

Smoke and multi-profile examples:

$env:NOOS_STATE_ROOT_COMPARE_PROFILES="small"
npm run benchmark:state-root-engines

$env:NOOS_STATE_ROOT_COMPARE_PROFILES="medium,large-block,many-indexes"
npm run benchmark:state-root-engines

Important variables:

Variable	Default	Meaning
NOOS_STATE_ROOT_COMPARE_PROFILES	medium	Comma-separated small, medium, large-block, or many-indexes.
NOOS_STATE_ROOT_COMPARE_BLOCKS	profile-dependent	Blocks per engine/profile pair.
NOOS_STATE_ROOT_COMPARE_TXS_PER_BLOCK	profile-dependent	Transactions per block.
NOOS_STATE_ROOT_COMPARE_INDEXES_PER_RECORD	profile-dependent	Public indexes per encrypted-record transaction.
NOOS_STATE_ROOT_COMPARE_PAYLOAD_SIZE_BYTES	profile-dependent	Synthetic payload size.
NOOS_STATE_ROOT_COMPARE_LOG_DIR	./benchmark-results/state-root-engines	Report, archive, and checkpoint directory.
NOOS_STATE_ROOT_COMPARE_KEEP_DB	false	Keep isolated schemas.
NOOS_STATE_ROOT_COMPARE_SKIP_SNAPSHOTS	false	Skip archive snapshot export/verify/import timings.
NOOS_STATE_ROOT_COMPARE_SKIP_RECOVERY	false	Skip Nervos sidecar restart/checkpoint recovery timing.
NOOS_STATE_ROOT_COMPARE_NERVOS_SIDECAR_COMMAND	platform default	Override the Nervos sidecar binary.

The benchmark executes the same workload with iden3-v1 and nervos-smt-v2. It compares block commit p95, state-root apply p95, replay verification, snapshot export/verify/import, and Nervos supervised sidecar restart recovery.

Build the sidecar first if needed:

npm run build:state-root-sidecar-nervos

Use deltas as directional evidence. They are not a production activation gate by themselves. Activation decisions also need replay verification, snapshot verification, operator recovery drills, and protocol-version review.

SMT Adapter

Run:

npm run benchmark:smt

Important variables:

Variable	Default	Meaning
NOOS_SMT_BENCH_LEAVES	1000	Unique leaves inserted.
NOOS_SMT_BENCH_UPDATES	1000	Existing leaves updated.
NOOS_SMT_BENCH_DELETES	100	Leaves deleted.
NOOS_SMT_BENCH_PROOF_SAMPLES	10	Proofs generated.
NOOS_SMT_BENCH_INCLUDE_UNBUFFERED	false	Also run persistent storage without buffered writes.
NOOS_SMT_BENCH_CACHE_MAX_ENTRIES	250000	Max PostgreSQL SMT adapter cache entries.
NOOS_SMT_BENCH_LOG_DIR	./benchmark-results/smt	Report directory.
NOOS_SMT_BENCH_KEEP_DB	false	Keep the isolated schema.

Compare persistent_buffered with in_memory. If persistent storage is much slower, inspect PostgreSQL cache misses, state_smt_kv reads, and flush behavior. If they are close, the dominant cost is likely iden3 traversal, hash-to-field conversion, or proof work.

Production SMT flushes also honor NOOS_SMT_FLUSH_BATCH_SIZE, default 1000. Tune it only after measuring flushMs.

NOOS_SMT_CACHE_MAX_ENTRIES bounds the block-local in-memory SMT cache. If a block reaches the guard, the adapter stops adding new cache entries for that block and continues through PostgreSQL plus the pending write buffer. This is a performance fallback, not a consensus change.

SMT Candidate Evaluation

Run:

npm run benchmark:smt:evaluation

Validate the external process contract with the included fixture:

$env:NOOS_SMT_EVAL_EXTERNAL_CANDIDATE_COMMAND=".\node_modules\.bin\tsx.cmd"
$env:NOOS_SMT_EVAL_EXTERNAL_CANDIDATE_ARGS_JSON='["test/benchmarks/smt-evaluation/external-iden3-fixture.ts","--vector","{vectorPath}"]'
npm run benchmark:smt:evaluation

Important variables:

Variable	Default	Meaning
NOOS_SMT_EVAL_LEAVES	1000	Unique keys in the vector.
NOOS_SMT_EVAL_UPDATE_ROUNDS	1	Full update passes after inserts.
NOOS_SMT_EVAL_DELETE_EVERY	10	Delete every Nth key, or 0 to disable deletes.
NOOS_SMT_EVAL_PROOF_SAMPLES	25	Proof operations appended for live keys.
NOOS_SMT_EVAL_PROFILE_INTERNAL_TIMINGS	true	Profile NOOSChain adapter internals.
NOOS_SMT_EVAL_EXTERNAL_CANDIDATE_COMMAND	unset	External executable for a candidate.
NOOS_SMT_EVAL_EXTERNAL_CANDIDATE_ARGS_JSON	unset	JSON args array. {vectorPath} is replaced.
NOOS_SMT_EVAL_LOG_DIR	./benchmark-results/smt-evaluation	Report directory.
NOOS_SMT_EVAL_WRITE_VECTOR	false	Preserve the generated vector.

The current production native path is Nervos, and Nervos is not iden3-compatible. It is a protocol-versioned root scheme. A future candidate is not eligible for production unless it produces deterministic roots, validates proof semantics, passes replay and snapshot vectors, runs consistently on Windows and Linux, and is backed by a maintained implementation.

See SMT Candidate Evaluation for the external candidate process contract.

State-Root Engine Candidate

Build tools and run:

npm run build:state-root-tools
npm run benchmark:state-root-engine

Important variables:

Variable	Default	Meaning
NOOS_STATE_ROOT_ENGINE_BENCH_LEAVES	1000	Initial materialized leaves.
NOOS_STATE_ROOT_ENGINE_BENCH_UPDATES	1000	Ordered block-style set mutations.
NOOS_STATE_ROOT_ENGINE_BENCH_DELETE_EVERY	10	Delete every Nth initial key, or 0 to disable deletes.
NOOS_STATE_ROOT_ENGINE_BENCH_LOG_DIR	./benchmark-results/state-root-engine	Report and vector directory.
NOOS_STATE_ROOT_ENGINE_BENCH_KEEP_VECTOR	false	Preserve generated vectors.
NOOS_STATE_ROOT_ENGINE_RUST_COMMAND	auto-detected	External native candidate executable.
NOOS_STATE_ROOT_ENGINE_RUST_ARGS_JSON	unset	JSON args array. {vectorPath} is replaced.

Larger profile:

$env:NOOS_STATE_ROOT_ENGINE_BENCH_LEAVES="50000"
$env:NOOS_STATE_ROOT_ENGINE_BENCH_UPDATES="50000"
$env:NOOS_STATE_ROOT_ENGINE_BENCH_DELETE_EVERY="20"
npm run benchmark:state-root-engine

This benchmark applies an ordered mutation batch and rebuilds from the final leaf set. A candidate passes its internal check only when apply and rebuild produce the same root for that engine. Root equality with iden3 is informational for new root schemes.

Real State-Root Vectors

Run:

npm run build:state-root-tools
npm run benchmark:state-root-real-vectors

The runner commits real signed transactions through the single-node consensus path, extracts block_state_changes as replay/apply vectors, extracts state_leaves as snapshot/rebuild vectors, and runs iden3 plus built external utilities against those vectors.

Important variables:

Variable	Default	Meaning
NOOS_STATE_ROOT_REAL_VECTOR_LOG_DIR	./benchmark-results/state-root-real-vectors	Report and vector directory.
NOOS_STATE_ROOT_REAL_VECTOR_KEEP_DB	false	Keep the isolated replay schema.
NOOS_STATE_ROOT_ENGINE_BENCH_KEEP_VECTOR	false	Preserve vectors.
NOOS_STATE_ROOT_ENGINE_RUST_COMMAND	auto-detected	Run one explicit external candidate.
NOOS_STATE_ROOT_ENGINE_RUST_ARGS_JSON	unset	JSON args array. {vectorPath} is replaced.

Persistent State-Root Sidecar

Build, test, and benchmark:

npm run build:state-root-sidecar-nervos
npm run test:state-root-persistent-sidecar
npm run benchmark:state-root-persistent-sidecar

Important variables:

Variable	Default	Meaning
NOOS_STATE_ROOT_SIDECAR_BENCH_LEAVES	5000	Initial leaves loaded before the first checkpoint.
NOOS_STATE_ROOT_SIDECAR_BENCH_UPDATES	5000	Ordered set mutations after the first checkpoint.
NOOS_STATE_ROOT_SIDECAR_BENCH_DELETE_EVERY	25	Delete every Nth initial key, or 0 to disable deletes.
NOOS_STATE_ROOT_SIDECAR_BENCH_BATCH_SIZE	1000	Mutations per apply_batch request.
NOOS_STATE_ROOT_SIDECAR_BENCH_LOG_DIR	./benchmark-results/state-root-persistent-sidecar	Report and temporary checkpoint directory.
NOOS_STATE_ROOT_SIDECAR_BENCH_KEEP_CHECKPOINT	false	Preserve checkpoint files.
NOOS_STATE_ROOT_SIDECAR_COMMAND	auto-detected	Explicit sidecar executable.

The benchmark measures initial load, update/delete batches, checkpoint export size/time, restart checkpoint-load time, and Node-side memory samples. The sidecar checkpoint is local sidecar state, not a NOOSChain consensus snapshot.

Sync, Payload, And Snapshot Benchmarks

Sync Catch-Up

Run:

npm run benchmark:sync-catchup

Important variables:

Variable	Default	Meaning
NOOS_SYNC_CATCHUP_BENCH_SCENARIO	fresh-observer	fresh-observer, lagged-observer, missing-payloads, snapshot-bootstrap, multi-peer-failover, policy-permission-changes, or observer-restart-recovery.
NOOS_SYNC_CATCHUP_BENCH_PAYLOAD_PROFILE	single-bucket	single-bucket or multi-policy.
NOOS_SYNC_CATCHUP_BENCH_SOURCE_BLOCKS	scenario-dependent	Source blocks committed.
NOOS_SYNC_CATCHUP_BENCH_PRE_SYNC_BLOCKS	scenario-dependent	Blocks synced before the measured window.
NOOS_SYNC_CATCHUP_BENCH_TXS_PER_BLOCK	10	Transactions per source block.
NOOS_SYNC_CATCHUP_BENCH_MAX_BLOCKS_PER_BATCH	50	Maximum blocks requested per HTTP batch.
NOOS_SYNC_CATCHUP_BENCH_PAYLOAD_SIZE_BYTES	512	Synthetic encrypted payload size.
NOOS_SYNC_CATCHUP_BENCH_RUN_PAYLOAD_BACKFILL	true	Run payload backfill during measured sync.
NOOS_SYNC_CATCHUP_BENCH_REPLAY_VERIFY	false	Run full replay verification after catch-up.
NOOS_SYNC_CATCHUP_BENCH_LOG_DIR	./benchmark-results/sync-catchup	Report directory.
NOOS_SYNC_CATCHUP_BENCH_KEEP_DB	false	Keep isolated schemas.

The runner creates a source chain, serves it through /chain/head, /chain/blocks, and /records/payloads, then syncs an isolated target node with the real sync service. It reports remote-head fetch, block-batch fetch, block execution, payload discovery, payload fetch, payload update, peer request counts, counter diagnostics, throughput, and final head/root equality.

Useful profiles:

$env:NOOS_SYNC_CATCHUP_BENCH_RUN_PAYLOAD_BACKFILL="false"
npm run benchmark:sync-catchup

$env:NOOS_SYNC_CATCHUP_BENCH_PAYLOAD_PROFILE="multi-policy"
npm run benchmark:sync-catchup

$env:NOOS_SYNC_CATCHUP_BENCH_SCENARIO="snapshot-bootstrap"
$env:NOOS_SYNC_CATCHUP_BENCH_PRE_SYNC_BLOCKS="100"
$env:NOOS_SYNC_CATCHUP_BENCH_SOURCE_BLOCKS="120"
$env:NOOS_SYNC_CATCHUP_BENCH_PAYLOAD_PROFILE="multi-policy"
$env:NOOS_SYNC_CATCHUP_BENCH_REPLAY_VERIFY="true"
npm run benchmark:sync-catchup

$env:NOOS_SYNC_CATCHUP_BENCH_SCENARIO="multi-peer-failover"
$env:NOOS_SYNC_CATCHUP_BENCH_PAYLOAD_PROFILE="multi-policy"
npm run benchmark:sync-catchup

$env:NOOS_SYNC_CATCHUP_BENCH_SCENARIO="policy-permission-changes"
npm run benchmark:sync-catchup

$env:NOOS_SYNC_CATCHUP_BENCH_SCENARIO="observer-restart-recovery"
npm run benchmark:sync-catchup

Interpretation:

• Block fetch dominates: inspect HTTP range size, DTO query plans, compression, and serialization.
• Block execution dominates: inspect target-side block execution and SMT metrics.
• Payload fetch dominates: inspect peer serving, batch size, and peer scoring.
• Payload update dominates: inspect local payload-store write batching, indexes, triggers, and availability counters.
• Counter diagnostics mismatch: treat the benchmark as failed. Stale counters can hide sync and backfill regressions.

Payload Backfill

Run:

npm run benchmark:payload-backfill

Important variables:

Variable	Default	Meaning
NOOS_PAYLOAD_BACKFILL_BENCH_RECORDS	1000	Missing encrypted-record rows to seed.
NOOS_PAYLOAD_BACKFILL_BENCH_PAYLOAD_SIZE_BYTES	512	Synthetic ciphertext size.
NOOS_PAYLOAD_BACKFILL_BENCH_HTTP_BATCH_SIZE	100	Record ids per peer request, capped by API contract.
NOOS_PAYLOAD_BACKFILL_BENCH_DB_UPDATE_BATCH_SIZE	500	Verified payload rows per local update.
NOOS_PAYLOAD_BACKFILL_BENCH_SEED_BATCH_SIZE	1000	Seed insert batch size.
NOOS_PAYLOAD_BACKFILL_BENCH_REMOTE_UNAVAILABLE_EVERY	0	Inject unavailable peer payloads.
NOOS_PAYLOAD_BACKFILL_BENCH_HASH_MISMATCH_EVERY	0	Inject hash mismatches.
NOOS_PAYLOAD_BACKFILL_BENCH_DISABLE_OBSERVABILITY_UPDATE_TRIGGER	false	Diagnostic-only trigger-overhead check.
NOOS_PAYLOAD_BACKFILL_BENCH_LOG_DIR	./benchmark-results/payload-backfill	Report directory.
NOOS_PAYLOAD_BACKFILL_BENCH_KEEP_DB	false	Keep the isolated schema.

The runner seeds records across replication policies, serves deterministic ciphertext from a local peer, and runs the real backfill service. The report includes peer request counts, skip reasons, HTTP/DB batch counts, payloads/sec, and verification that fetched rows match policy and injected-failure expectations.

Use the trigger diagnostic only in isolated benchmark schemas. It intentionally makes cached observability counters stale and is not a production optimization.

Payload Store

Run:

npm run benchmark:payload-store

Important variables:

Variable	Default	Meaning
NOOS_PAYLOAD_STORE_BENCH_ROWS	10000	Payload rows to seed.
NOOS_PAYLOAD_STORE_BENCH_PAYLOAD_SIZE_BYTES	512	Synthetic ciphertext size.
NOOS_PAYLOAD_STORE_BENCH_SEED_BATCH_SIZE	1000	Seed batch size.
NOOS_PAYLOAD_STORE_BENCH_LOOKUP_SAMPLES	1000	Single and batch lookup samples.
NOOS_PAYLOAD_STORE_BENCH_BATCH_LOOKUP_SIZE	100	Batch lookup size.
NOOS_PAYLOAD_STORE_BENCH_GC_LIMIT	1000	GC dry-run candidate limit.
NOOS_PAYLOAD_STORE_BENCH_WRITE_OVERHEAD_SAMPLE_ROWS	1000	Existing rows updated to measure trigger/cache overhead.
NOOS_PAYLOAD_STORE_BENCH_LOG_DIR	./benchmark-results/payload-store	Report directory.
NOOS_PAYLOAD_STORE_BENCH_KEEP_DB	false	Keep the isolated schema.

This benchmark measures read-side local payload behavior: single lookup, batch lookup, cached stats, realtime stats, update-over-existing-row overhead, GC preview/prune, and export-like scans. Use it when payload-store endpoints or operator stats change.

Snapshot Streaming

Run:

npm run benchmark:snapshots

Recommended large-file profile:

$env:NOOSCHAIN_SMALL_SNAPSHOT_FAST_PATH_BYTES="0"
$env:NOOS_SNAPSHOT_BENCH_RECORD_INDEX_ROWS="250000"
npm run benchmark:snapshots

Important variables:

Variable	Default	Meaning
NOOS_SNAPSHOT_BENCH_RECORD_INDEX_ROWS	250000	Synthetic record_indexes rows.
NOOS_SNAPSHOT_BENCH_INSERT_BATCH_SIZE	1000	Fixture-generation batch size.
NOOS_SNAPSHOT_BENCH_FORMAT	archive	archive or legacy json.
NOOS_SNAPSHOT_BENCH_LOG_DIR	./benchmark-results/snapshots	Report directory.
NOOS_SNAPSHOT_BENCH_KEEP_FILES	false	Keep temporary snapshot files.
NOOSCHAIN_SMALL_SNAPSHOT_FAST_PATH_BYTES	33554432	Set 0 to force the large streaming path.
NOOSCHAIN_SNAPSHOT_COPY_ENABLED	true	Use PostgreSQL COPY staging for large imports.
NOOSCHAIN_SNAPSHOT_COPY_BUFFER_BYTES	1048576	Bytes buffered per active COPY writer.
NOOSCHAIN_SNAPSHOT_HEADER_SCAN_BYTES	1048576	Prefix/tail bytes for metadata reads.
NOOSCHAIN_SNAPSHOT_MAINTENANCE_WORK_MEM	256MB	Transaction-local import index rebuild memory.
NOOSCHAIN_SNAPSHOT_IMPORT_SYNCHRONOUS_COMMIT	on	Transaction-local durability setting.
NOOSCHAIN_SNAPSHOT_REBUILD_RECORD_INDEXES_THRESHOLD	50000	Drop/rebuild secondary record_indexes indexes above this row count.
NOOSCHAIN_SNAPSHOT_DROP_RECORD_INDEX_CONSTRAINTS	true	Also drop/rebuild constraints during record-index import.

The report includes file size, export time, verification time, import time, rows imported per table, staging/COPY statistics, parse-and-stage timing, final table insertion timing, index rebuild timing, and persistent SMT rebuild timing. Use those phase timings to decide whether the next bottleneck is JSON parsing, archive table verification, PostgreSQL final insertion, index rebuild, or SMT rebuild.

Archive snapshots should be the default benchmark format. Legacy JSON remains useful for compatibility comparisons only.

Release Evidence Checklist

For release readiness or a major performance-sensitive change, keep these artifacts with the engineering decision:

• combined benchmark:v1-readiness report;
• benchmark:v1-readiness:check report;
• raw child benchmark reports under benchmark-results/v1-readiness/raw;
• hardware, OS, Node, PostgreSQL, and sidecar build details;
• environment variables and profiles used;
• linked correctness evidence, especially replay, state-root, snapshot, and sync verification;
• explanation for any budget change or accepted regression.

Do not loosen a budget only to make a release pass. First decide whether the workload envelope was wrong, the benchmark machine was not representative, or the implementation needs optimization.

When To Add A Benchmark

Add or extend a benchmark when a change introduces a new hot path, a new operator-facing budget, or a performance claim that tests cannot measure.

A useful benchmark should:

• run from package.json;
• write a machine-readable JSON report;
• record its workload configuration;
• use isolated schemas or temporary files;
• validate enough invariants to catch obviously invalid fast paths;
• separate major phases in the report;
• include recommendations only when they are tied to measured fields.

Do not put long-running soak, malformed-peer chaos, crash fuzzing, or fork conflict drills into a benchmark runner. Those belong in hardness, soak, or incident-response exercises.

Current Limits

The benchmark suite does not prove consensus safety, Byzantine tolerance, operator disaster recovery, or production readiness by itself. It also does not simulate every possible data shape, 10M-entry state, overnight resource leaks, malformed peers, or real network jitter.

Use benchmarks as one layer of evidence: fast enough, measured in the right place, and paired with correctness verification.