How I took Swapstracker from a one-year backfill estimate on my first, TypeScript version to a 20-day backfill with a Go rewrite and TimescaleDB on a Radxa ROCK 5C.
Swapstracker follows DEX trading on EVM chains. It reads swap events from Uniswap V2, V3 and V4-style pools, prices each trade in USD and serves trades, pair statistics and OHLCV candles through an API. New trades are pushed live to a SvelteKit frontend.
This write-up covers two chains, Ethereum and BSC, over two years of history.
The platform supports more EVM chains. Every figure on this page refers only to Ethereum and BSC.
I wrote the first version of Swapstracker as a set of TypeScript microservices writing to MongoDB. Each service made sense on its own. Together they spent a large share of CPU and memory turning objects into messages and messages back into objects, only to hand the data to the next service.
I track progress in blocks per second, logged continuously. At the rate my first version sustained, the logs projected roughly a year to backfill two years of both chains. That number is a projection: I did not let the first version run for a year to confirm it.
This is not a Go versus TypeScript result. Architecture, data access and storage changed together.
The indexer is one process; the platform is not. The API, Redis, the database and the frontend run as their own services.
Inside the indexer, an allocator hands out consecutive block ranges, sized by the batch optimizer. Several workers take ranges at the same time. Each one fetches its logs, then resolves pair metadata and block timestamps concurrently, checking the local cache before asking the network.
Workers finish in whatever order the network allows, so their batches pass through a reorder buffer. A single committer prices and writes them strictly in block order, so the stored history grows contiguously even while extraction runs in parallel.
A fixed number of in-flight slots caps how many prepared batches can wait at once. When one range is slow, the other workers stop taking new work instead of filling memory.
The worker count adapts at runtime. The ordering does not depend on it: only the committer writes.
An illustration of the design, made with an image model from my description of the indexer. Not a screenshot.
With those changes, the two-year backfill of both chains finished in 20 days. The history now holds more than 3.1 billion trades in about 1 TB of compressed storage, and the indexer uses around 80% less RAM than my first version did.
I also saw CPU temperature drop by roughly 15 °C. Temperature depends on load and room conditions, so I keep it as an observation, not a result.
| 3.1+ billion | trades stored, two years of Ethereum and BSC | observed |
|---|---|---|
| 1 TB | compressed history, on a 2 TB SSD | observed |
| 20 days | to backfill those two years, both chains together | observed |
| ≈ 1 year | what my first version projected for the same job | estimate from logged blocks/s |
| 80% less | RAM after the rewrite | reported, no reproducible benchmark yet |
All of this runs on a Radxa ROCK 5C single-board computer with a 2 TB Samsung T7 SSD attached over USB. The compressed history takes about half of that disk. The same board also serves HabitsFlow and GroupBrain, and the three apps reach the internet through Cloudflare Tunnel.
The board does not run full EVM nodes. Chain data comes from RPC providers; the indexer’s job is to get as much as possible out of every request and do the rest locally.
singleflight. It collapses in-flight reads; it does not deduplicate stored trades.I call OHLCV latency low. I have not published a p95 yet, so there is no millisecond figure here.
Public RPC endpoints vary a lot. Some drop old history, some time out, some return incomplete logs. The indexer fails over between providers, and candidate endpoints are checked against baselines, known results for fixed block ranges, before they are trusted.
Pricing itself is in-memory math over pool state. RPC is needed for logs, token metadata and block timestamps, and the last two are cached locally. Batch sizes adapt to log density: the optimizer aims for about 35,000 logs per batch and backs off when a provider struggles.
35,000 is a batch-size target, not a throughput figure and not a hard limit.
The numbers above come from my own logs. To make them something another engineer can reproduce, these are next:
They will land on this page, with the method, once I have them.