A reorg-safe indexer for ERC-20 balances. It reads blocks, maintains a queryable projection of who owns what — and when the chain rewrites itself, it rewrites the projection to match. No phantom balances, no double-counted deposits, no history the chain has abandoned.
python3 scripts/demo.py # watch a balance survive a chain rewrite
make test # the whole reorg matrix, ~0.7s, no services
A projection is not the sum of the blocks you have seen. It is the sum of the blocks on the chain's current canonical branch — and that branch is allowed to change under you. A reorg is not a failure to handle; it is a protocol event to expect. An indexer that only ever appends is not indexing the chain. It is indexing one temporary version of it and hoping.
Most indexer bugs are the same bug: for block in chain: apply(block). That
loop is correct right up until the network orphans a block you already applied.
After that, your balances reflect transfers that no longer exist on any chain
anyone agrees on. On an exchange, that is a deposit credited twice, or one that
vanishes. rewind is built so that cannot happen, and the code is arranged so
you can see why.
the exchange is funded, then alice deposits 500
indexed on the original chain:
alice 500
exchange 999500
supply sum 0 (0 means value is conserved)
...a reorg hits. the block with alice's deposit is orphaned,
and the replacement branch pays bob instead of alice.
after the indexer reconciles:
alice 20
bob 500
exchange 999480
supply sum 0 (0 means value is conserved)
Alice's phantom 500 is gone the moment the indexer notices the ground moved. The reorg is logged, not hidden. And the books still sum to zero.
A block is never deleted. It moves between two states, and a reorg is just a status flip — which is exactly what makes reorg-then-back safe, because you cannot re-canonicalise a block you threw away.
stateDiagram-v2
[*] --> canonical: applied on the head
canonical --> orphaned: a reorg replaced this branch
orphaned --> canonical: a later reorg restored it
canonical --> [*]: finalised (past the reorg horizon)
note right of orphaned
marked, never deleted -
the audit trail of every
reorg the indexer survived
end note
Everything the code does exists to keep these true. The fuzzer checks all four after every one of hundreds of random operations.
| invariant | what it guarantees | |
|---|---|---|
| I1 | canonicity | the projection equals an independent fold of the canonical chain — not the chain the indexer happened to see first |
| I2 | reversibility | applying and reverting a block are exact mirrors; every applied transfer knows the deltas to undo it |
| I3 | conservation | the sum of all balances equals net mint minus burn; a reorg moves value, it never creates or destroys it |
| I4 | resumability | a kill -9 at any point resumes to the same state; the cursor moves only inside the transaction that moves the projection |
Every row is a test you can run. The nasty ones are called out because they are exactly where naive indexers break.
| scenario | what breaks a naive indexer | test |
|---|---|---|
| shallow reorg (depth 1) | phantom balance from the orphaned block | test_depth_1 |
| deep reorg | partial rewind leaves the projection inconsistent | test_deep_reorg |
| equal-length replacement | height-based detection sails right past it | test_equal_length_reorg |
| reorg, then reorg back | deleting orphans means you can't restore them | test_reorg_then_back |
| crash mid-reorg | half-applied branch, corrupt projection | test_crash_between_orphan_and_apply_rolls_back |
| fork past finality | silently rewriting finalised history | test_reorg_deeper_than_finality_is_refused |
| hundreds of random reorgs | the bug you didn't think to write | test_fuzz |
Why equal-length reorgs are the tell of a serious indexer
A reorg does not have to make the chain longer. A block at height N can be replaced by a different block at height N — same height, different contents, different hash. An indexer that tracks "highest block I've seen" never notices: the number didn't change.
rewind never trusts the number. Before extending its chain, it checks that
the incoming block's parent_hash equals the head it stored. When a block is
replaced in place, the new block's parent still matches, but its own hash
differs from what the projection recorded at that height — and walking the new
head's ancestry reveals the divergence. Detection is by cryptographic lineage,
not by counting. This one decision is the line between an indexer that survives
reorgs and one that merely hasn't met a bad one.
The conservation proof
Model the zero address as a real account whose balance is allowed to go
negative. A mint (from = 0x0) credits a user and debits zero; a burn does the
reverse. Then for any set of transfers, sum(all balances) = 0 if and only if
every unit that entered a balance left another — which is true by construction,
because each transfer adds exactly value to one balance and subtracts exactly
value from another.
A reorg reverts some transfers and applies others. Each revert and each apply
is itself balance-neutral by the same argument. So no sequence of reorgs can
change the sum. The invariant sum(balances) == 0 is therefore not a hopeful
check — it is a theorem, and the store evaluates it as a single SUM() after
every reorg. If it were ever nonzero, the reversal logic would be provably
wrong, and the fuzzer would have caught it.
Why a crash mid-reorg is survivable
The reorg — orphan the dead branch, apply the live one, move the cursor, record the event — is a single database transaction. A crash before commit rolls the whole thing back: the cursor still points at the pre-reorg head, so the next sync detects the identical reorg and replays it from scratch. A crash after commit is simply the next sync starting from the new head.
There is no third state, so there is no recovery mode and no repair script. The test arms a crash between orphaning and applying — the most dangerous instant, where a lesser design would leave half a reorg on disk — and asserts the projection is byte-for-byte unchanged afterward. Recovery here is not a feature. It is the absence of a way to be broken.
The same indexer runs against two stores through one interface, so the store is swappable and the algorithm is proven twice.
- SQLite (
make test) — zero services, runs anywhere Python does in under a second. This is where the entire reorg matrix and the fuzzer live, because determinism is what makes reorg tests trustworthy: a reorg is a method call, not a rare mainnet accident you hope to capture. - Postgres (
make test-pg) — the production store, with nativeNUMERIC(78,0)for uint256 balances and real MVCC, so an API reader mid-reorg sees the whole old canon or the whole new one, never a half-applied branch.
The mock chain hands the indexer exactly what a node's JSON-RPC would: a block by hash, a block by height on the current canon, and the head. The indexer has no idea its chain is a mock — which is what makes the mock a fair test and the RPC adapter a small future addition rather than a rewrite.
make test # sqlite tier: reorg matrix, crash test, primitives, api
make fuzz # property fuzzer across seven seeds
make demo # the narrated reorg above
# postgres tier
docker compose up -d db
make test-pg
The read API is a thin Flask layer over the projection:
GET /v1/balance/<address> current balance on the canonical chain
GET /v1/head indexed head hash, height, transfer count
GET /v1/reorgs every reorg the indexer has absorbed, with proof
GET /v1/metrics throughput, reorg-depth histogram, lag from head
GET /healthz
/v1/metrics is what turns "we handle reorgs" into something you can watch:
{
"blocks_applied": 128400,
"blocks_reverted": 47,
"reorgs": 31,
"deepest_reorg": 4,
"reorg_depth_histogram": {"1": 22, "2": 6, "3": 2, "4": 1},
"blocks_per_second": 9120.5
}Batched forward, atomic sideways. Catching up on a cold start means applying
thousands of blocks, and a commit per block turns that into thousands of fsyncs.
So contiguous new blocks apply in one transaction (batch_size, default 256) - a
backfill that would take minutes takes seconds. A reorg always breaks the batch:
you cannot batch across a fork you haven't reconciled, and correctness outranks
throughput every time. Each batch is still a clean block boundary, so a crash
keeps whole committed batches and the next sync resumes from the last one.
Reverse-deltas, not recomputation. Reverting a block subtracts exactly what applying it added — O(transfers in the block), not O(history). Balances never drift because reversal is the arithmetic inverse of application, and the code is written so the two are visibly mirror images.
Orphans are marked, not deleted. An orphaned block stays in the database
with status = 'orphaned'. This is what makes reorg-then-back correct, and it
doubles as the audit trail: /v1/reorgs is queryable history of every rewrite
the projection survived.
Bounded reconciliation. A fork deeper than the finality horizon is refused, not absorbed. Past finality, a reorg is not a routine event — it is a signal that something is badly wrong, and rewriting finalised history on that signal would be a worse bug than stopping.
No ORM. The SQL is raw and visible because the state transitions are the project. An ORM would bury the one transaction that matters — the atomic reorg — under session machinery.
No P2P networking, no gas accounting, no EVM execution, no full-node
reimplementation. rewind does one thing and proves it: a token-balance
projection that stays correct under reorgs. A real deployment would add a
JSON-RPC chain source (the interface is already the right shape), backfill,
and metrics — none of which change the answer to "why can't this double-count a
deposit?"
MIT.