← Back to Research
Research · July 22, 2026

Crashing an Agave validator during snapshot bootstrap with one oversized account length

Simon Morley·7 min read

An Agave validator that bootstraps from a snapshot served by an untrusted peer — the default configuration of agave-validator run when --known-validators is not set — can be made to panic and abort during snapshot load by a single malformed field: one account-storage file (AppendVec) that declares an account whose data_len exceeds the 10 MiB per-account maximum triggers a fatal error in accounts-index generation, before the snapshot hash is ever checked, from a crafted storage file as small as ~136 bytes.

Why this matters

The exposed party is any operator bringing a validator up from a snapshot: a newly-joining node, or one restarting without a usable local snapshot. In the default configuration — agave-validator run with no --known-validators — the node will accept a snapshot from a peer it has no reason to trust, and a peer that serves a crafted snapshot can crash it deterministically. The crash is availability-only: no memory corruption, no funds or consensus impact, and a validator already in steady state is unaffected.

Two properties make it worse than a one-off abort. First, it is a boot-time failure, so the usual steady-state defences do not apply — the node dies before it is a participant. Second, the snapshot's cryptographic hash gate does not save you: the hash that snapshot verification checks is produced by the same index-generation pass that panics, so the integrity check that would reject the malformed snapshot never runs. Re-fetching from the same or a colluding peer reproduces the crash, which is a boot-loop until the operator changes snapshot source or sets a known-validator constraint.

How the attack works

Three trust assumptions compose (mechanism cited to Agave v4.1.0):

  • Entry point: the snapshot download a bootstrapping validator fetches from a peer. The malformed field rides inside an AppendVec account-storage file in the snapshot archive.
  • Preconditions: the victim is in snapshot-fetch state (joining, or restarting without a local snapshot) and --known-validators is unset, so the attacker's node can be selected as the snapshot source. Remote, unauthenticated, pre-consensus.
  • Processing path:
    1. Reconstruction skips per-account validation. When a snapshot storage is rebuilt, AppendVec::new_for_startup trusts the storage length and, when it equals the file size (which the attacker controls), returns without running sanitize_layout_and_length(). The per-account sanitize() that does run checks only the executable byte and lamports — it never bounds data_len (accounts-db/src/append_vec.rs).
    2. Index generation treats an oversized data_len as fatal. Building the accounts index reads every account via a File-backed scan whose reader buffer is capped at STORE_META_OVERHEAD + MAX_PERMITTED_DATA_LENGTH. The scan requests STORE_META_OVERHEAD + data_len bytes with no cap on data_len; when data_len exceeds 10 MiB the reader returns a quota error and the scan returns Err. generate_index_for_slot calls .expect("must scan accounts storage") on that result — so it panics (accounts-db/src/accounts_db.rs).
  • Boundary crossed: attacker-controlled snapshot bytes reach a fatal .expect() on an internal path, before the snapshot hash gate — because the hash that gate checks is produced by the very pass that aborts.
  • Resulting behaviour: a deterministic crash/abort of the bootstrapping node. Index generation scans every storage, so one malicious AppendVec among many legitimate ones is enough; no valid account data is required.

The mechanism is a reachable panic on unvalidated input, not a memory-safety bug. MAX_PERMITTED_DATA_LENGTH is 10 MiB.

Affected systems

  • Severity: High for availability of bootstrapping nodes in default config; not steady-state, not RCE (per NR-2026-002).
  • Chain: Solana.
  • Implementation: Agave (agave-validator). Verified builds:
    • v4.1.0 – v4.1.2 — v4.1.2 is the current stable release; the two affected files (accounts-db/src/append_vec.rs, accounts-db/src/accounts_db.rs) are byte-identical at v4.1.0 and v4.1.2, and v4.1.0's copy is byte-identical to the build the crash was measured against (per NR-2026-002). Every v4.1.x release to date is affected.
    • 4.2.0-alpha.0 (3e9a16d6e6) — the build measured.
    • master (as of 2026-07-01) — present and, if anything, easier to reach: the per-account layout sanitisation was removed from the startup path (length is derived from file size, so the skip is unconditional) and the alternate Mmap storage backing was removed (File-backed only), so the panic is the sole outcome.
  • Component: snapshot-load AppendVec reconstruction (new_for_startup) → accounts-index generation (generate_index_for_slot).
  • Gated on: the victim being in snapshot-fetch state and --known-validators being unset. A validator in steady state, or one restricted to known snapshot sources, is not exposed by this path. No other chains or clients are claimed.

Evidence

  • Publicly documented: none as a CVE. Agave's SECURITY.md acknowledges maliciously-crafted snapshots as a known trust-on-first-use limitation with improvement "actively underway"; this brief adds a concrete, reproduced instance of that class, it does not claim a new public disclosure.
  • Independently reproduced by NullRabbit (lab): two unit-level tests run against the real Agave deserializer craft a 136-byte AppendVec with data_len = MAX_PERMITTED_DATA_LENGTH + 1 and drive it through the real new_for_startup (the path that skips per-account sanitisation). test_malformed_snapshot_oversized_data_len_scan_errors shows the real scan_accounts call index generation makes returns Err(Io(QuotaExceeded)); test_malformed_snapshot_oversized_data_len_panics_index_gen shows the exact scan_accounts(..).expect("must scan accounts storage") pattern panics with the production message.
  • Observed under controlled conditions: the measured panic is must scan accounts storage: Io(Custom { kind: QuotaExceeded, error: "requested more bytes than allowed capacity range" }), against 4.2.0-alpha.0 (byte-identical functions in v4.1.0). On recent master the new_for_startup signature changed, so the test bodies need a one-line signature update to compile there; the mechanism is unchanged.
  • Wire signature (traffic reproducer): a driver crafts the malicious snapshot archive (a version file, a bank-manifest stub, and accounts/<slot>.<id> = the 136-byte AppendVec whose StoredMeta.data_len = MAX_PERMITTED_DATA_LENGTH + 1), serves it over loopback HTTP, and fetches it — reproducing the download an untrusted peer would serve, with the malformed account length present on the wire. It captures the malicious serve/fetch, not a live validator crash; the crash itself is proven by the unit-level reproducer. The captured bundle ships in the NullRabbit/nr-bundles-public dataset (primitive sol_snapshot_oversized_datalen_indexgen_panic).
  • Not established: no live/mainnet validator was crashed. The crash is proven at the deserializer and index-generation level; the boot-loop consequence follows from the crash being deterministic, and is not separately load-tested.

Operator actions

Available today; no upstream fix required (per NR-2026-002):

  • Set --known-validators (primary). This constrains snapshot selection so an attacker's node cannot be chosen as your snapshot source, which removes the entry point.
  • When bootstrapping, prefer a trusted local or known snapshot source rather than an arbitrary peer.
  • Detection idea (unconfirmed): alert on validator aborts during snapshot load whose panic references must scan accounts storage, and on snapshot fetches from sources outside your known set. Treat this as a monitoring heuristic, not a validated signature.

Canonical references

Related attacks

  • Same technique class: NRDAX-T0352 (unvalidated field-length panic) also has instances in QUIC's quichequiche_0rtt_packet_len_panic and quiche_payload_overflow_crash — where an unvalidated length field likewise drives a panic. Those are separate, reverse-engineered public CVEs in a different implementation, linked here only by the shared mechanism shape, not asserted to be the same bug.
  • Same family: state_import_abuse — malformed bulk-state import reaching a reachable fault during import. NR-2026-002 is a single-finding advisory; no same-advisory sibling primitive is claimed.

Track attacks affecting Agave and Solana on NRDAX and in NullRabbit research.

Analysis plus reproducer. No weaponised proof-of-concept code.

solanaagavesnapshot-bootstrapaccounts-dbstate_import_abuseinfrastructure-security
Publication policy:Why we publish these findings →

Related Posts