NOOSChain Docs

Appearance

Size Stress Testing ​

Current Status ​

package.json currently contains stability:size and stability:size:10m, but the referenced runner file src/stability-test/size-stress/size-stress-runner.ts is not present in this workspace. Treat this document as the intended size/load stress design until that runner is restored or reimplemented.

The intended harness lives under src/stability-test/size-stress.

This is not a normal E2E test. It is designed to grow the chain with a large stream of generated transactions while measuring block creation, execution, SMT updates, multi-node HTTP sync, snapshots, and payload availability behavior.

Safety ​

The intended runner refuses NODE_ENV=production. It creates isolated schemas named with noos_size_stress_<runId>_node_<index> and drops only those schemas unless NOOS_SIZE_STRESS_KEEP_DB=true.

The default run is configurable and does not require loading all transactions into memory. Transactions are generated, signed, submitted, committed, and discarded incrementally.

Intended 10M mode is guarded:

powershell
$env:NOOS_SIZE_STRESS_TX_COUNT="10000000"
$env:NOOS_SIZE_STRESS_ALLOW_10M="true"
$env:NOOS_SIZE_STRESS_TXS_PER_BLOCK="1000"
npm run stability:size

Bash:

bash
NOOS_SIZE_STRESS_TX_COUNT=10000000 NOOS_SIZE_STRESS_ALLOW_10M=true NOOS_SIZE_STRESS_TXS_PER_BLOCK=1000 npm run stability:size

Configuration ​

  • NOOS_SIZE_STRESS_TX_COUNT default 100000
  • NOOS_SIZE_STRESS_ALLOW_10M default false
  • NOOS_SIZE_STRESS_NODE_COUNT default 3
  • NOOS_SIZE_STRESS_TXS_PER_BLOCK default 1000
  • NOOS_SIZE_STRESS_SYNC_EVERY_BLOCKS default 10
  • NOOS_SIZE_STRESS_SNAPSHOT_EVERY_BLOCKS default 100
  • NOOS_SIZE_STRESS_VERIFY_EVERY_BLOCKS default 100
  • NOOS_SIZE_STRESS_REPLAY_EVERY_BLOCKS default 0
  • NOOS_SIZE_STRESS_PAYLOAD_SIZE_BYTES default 128
  • NOOS_SIZE_STRESS_INDEX_KEY_COUNT default 3
  • NOOS_SIZE_STRESS_BUCKET_COUNT default 6
  • NOOS_SIZE_STRESS_MIX_PER_RECORD_KEY default true
  • NOOS_SIZE_STRESS_MIX_PER_BUCKET_KEY default true
  • NOOS_SIZE_STRESS_MIX_METADATA_ONLY default true
  • NOOS_SIZE_STRESS_MIX_AUTHORIZED_ONLY default true
  • NOOS_SIZE_STRESS_KEEP_DB default false
  • NOOS_SIZE_STRESS_RESUME default false
  • NOOS_SIZE_STRESS_RUN_ID default generated
  • NOOS_SIZE_STRESS_EXPORT_REPORT_PATH default unset
  • NOOS_SIZE_STRESS_PROGRESS_INTERVAL_MS default 5000
  • NOOS_SIZE_STRESS_VERBOSE_EVENTS default false; set to true to print low-level size_stress_event JSON in addition to the human node timeline
  • NOOS_SIZE_STRESS_REAL_ENCRYPTION default true up to 100k
  • NOOS_SIZE_STRESS_FINAL_REPLAY_MAX_TX default 250000
  • NOOS_SIZE_STRESS_FORCE_FINAL_REPLAY default false

Runtime sync serving also honors NOOS_SYNC_MAX_BLOCK_RANGE, default 1000. Increase it cautiously for high-bandwidth local testing; the code refuses values above 10000. Larger ranges reduce HTTP round trips but increase response serialization and per-request memory pressure.

Nodes ​

The intended harness starts multiple local Fastify NOOSChain nodes with isolated schemas. Node 0 is the producer. Followers sync from node 0 over HTTP using the normal block sync path. Only node 0 generates and commits transactions.

By default, the intended size-stress harness focuses on block creation and finalized block sync. It inserts generated transactions directly into Node A's mempool, then Node A proposes blocks and followers catch up through /chain/blocks. This mode intentionally does not exercise transaction gossip on the hot path. Follower nodes are observers: they read, verify, sync, serve permitted data, and perform availability-layer payload backfill, but they reject transaction admission and gossip.

The intended size-stress harness disables Fastify's raw per-request logger for these temporary nodes and emits node-labelled lifecycle lines instead. Live output is intended to explain what each participant is doing, for example:

text
[2026-05-11T12:00:00.000Z] System: Creating and submitting transaction batch to Node A {"blockIndex":0,"transactionCount":1000}
[2026-05-11T12:00:01.000Z] Node A: Proposed and committed block {"height":"10","blocksCommitted":1}
[2026-05-11T12:00:02.000Z] Node B: Syncing committed blocks from Node A {"blocksCommitted":10}
[2026-05-11T12:00:02.150Z] Node B: HTTP GET /chain/blocks?from=10&to=19&limit=10 -> 200 {"durationMs":150}
[2026-05-11T12:00:02.151Z] Node B: Downloaded block batch from Node A {"fetchedBlocks":10,"firstHeight":"10","lastHeight":"19"}
[2026-05-11T12:00:02.152Z] Node B: Executing synced block locally {"height":"10","transactionCount":500}
[2026-05-11T12:00:04.000Z] Node B: Verified synced block locally {"height":"10","syncedBlocks":1}

If the log appears to pause after Node A: HTTP GET /chain/blocks ... -> 200, the network fetch is already complete. The follower is then doing local sync work: inserting the block, executing its transactions, updating SMT state, verifying the resulting block hash/state root, and finally fetching any authorized availability-layer ciphertext. The Executing synced block locally and Verified synced block locally lines show that progress.

The final benchmark report remains machine-readable JSON. Periodic progress lines also remain JSON so external tooling can still parse run progress. The older low-level size_stress_event JSON stream is now opt-in via NOOS_SIZE_STRESS_VERBOSE_EVENTS=true; this keeps the default console output focused on the node timeline.

The normal sync path uses bounded /chain/blocks ranges, batched block transaction loading, optional Brotli/gzip response compression, and batched payload backfill. These are the same mechanisms used outside the stress harness.

Genesis includes six buckets:

  • per-record key, replicate all
  • per-record key, authorized only
  • per-record key, metadata only
  • per-bucket key, replicate all
  • per-bucket key, authorized only
  • per-bucket key, metadata only

Per-bucket-key buckets receive initial bucket keys. Payload availability is checked on followers after sync.

Replay and Snapshots ​

Full replay is expensive at high transaction counts. The intended runner performs chain verification and snapshot verification periodically. Final replay runs only when TX_COUNT <= NOOS_SIZE_STRESS_FINAL_REPLAY_MAX_TX unless NOOS_SIZE_STRESS_FORCE_FINAL_REPLAY=true.

Snapshots are exported and verified at configured block intervals. Snapshot import is spot-checked for smaller runs. Large snapshot import currently needs chunked importer work to avoid PostgreSQL parameter pressure, so high-volume runs verify snapshot integrity without importing the full document.

Report ​

The final JSON report includes:

  • submitted/executed/failed transaction counts
  • committed blocks
  • throughput
  • latency summaries
  • storage/row counts
  • memory usage
  • chain/snapshot/replay/follower verification
  • replication policy checks
  • issues

Set NOOS_SIZE_STRESS_EXPORT_REPORT_PATH to write the full report to disk.

Current Limitations ​

  • No Raft, CometBFT, or libp2p.
  • No Byzantine consensus simulation.
  • Resume metadata is persisted in size_stress_runs; full resume of exact signer nonce/bucket rotation schedule is a first-pass implementation and will be expanded with stronger checkpoint manifests.
  • Snapshot import spot-checking is intentionally limited for larger runs until the snapshot importer uses chunked inserts.
  • Production object storage is not tested.

Audience-first NOOSChain documentation.

NOOSChain