Historical market data replay
Binance Vision
event server
On-demand fetch from data.binance.vision, parse ZIP/CSV archives, and stream Binance-shaped events over WebSocket, NDJSON, or SSE — so any app that already speaks Binance WS can replay history.
How it works
- Request — client or agent specifies market, symbol, data type, and date range.
- Resolve — server probes Vision; missing days (delisted coins, gaps) are reported, not fatal.
- Fetch — downloads only available ZIPs into a local cache (checksum verified).
- Stream — parses CSV → Binance WS JSON and paces events by timestamp × speed.
WebSocket streaming
Connect, send one JSON config, receive events. Control frames mark missing days.
const ws = new WebSocket(`ws://${location.host}/ws/replay`);
ws.onopen = () => ws.send(JSON.stringify({
market: "um",
data_type: "aggTrades",
symbol: "BTCUSDT",
start_date: "2024-01-01",
end_date: "2024-01-03",
speed: 100,
limit: 500
}));
ws.onmessage = (m) => {
const ev = JSON.parse(m.data);
if (ev.e === "datasetMissing") console.warn("gap", ev.date, ev.key);
else if (ev.e === "aggTrade") console.log(ev.s, ev.p, ev.q, ev.T);
else if (ev.e === "replayEnd") console.log("done");
};
| Endpoint | Shape |
|---|---|
WS /ws/replay | Raw Binance events |
WS /market/ws | Optional SUBSCRIBE, then replay config / REPLAY |
WS /market/stream | {"stream","data"} envelope |
HTTP API
| Method | Path | Purpose |
|---|---|---|
| GET | /health | Liveness |
| GET | /api/catalog | Known dataset families + schemas |
| GET | /api/symbols?market=um&data_type=aggTrades | Symbols that have that dataset |
| GET | /api/resolve?... | Which days exist in a range (sparse-safe) |
| GET | /api/browse?prefix=data/ | Live S3 listing |
| GET | /stream/ndjson?... | Paced NDJSON body |
| GET | /stream/sse?... | Server-Sent Events |
Shared query fields: market, data_type, symbol,
start_date, end_date, frequency, interval (klines),
speed, limit.
curl -N "http://${location.host}/stream/ndjson?\
market=um&data_type=metrics&symbol=BTCUSDT&\
start_date=2024-01-01&end_date=2024-01-02&speed=1000&limit=5"
Missing datasets
Not every symbol has every data type for every day. When an archive is absent,
the stream emits {"e":"datasetMissing","date":"...","key":"..."}
and continues. Use /api/resolve first to preview coverage.
Event shapes
// aggTrade
{"e":"aggTrade","E":1704067200038,"a":1965151407,"s":"BTCUSDT",
"p":"42313.9","q":"0.046","f":4426785111,"l":4426785111,"T":1704067200038,"m":true}
// kline (historical bars are closed: k.x=true)
{"e":"kline","s":"BTCUSDT","k":{"t":...,"T":...,"i":"1m","o":"...","c":"...","x":true}}