Pull historical market data

Outcome

Save one reproducible historical response for one venue family, symbol, data family, and bounded time window. For historical pulls that use Hyperliquid core coverage, start with the Hyperliquid core coverage and route-family overview before returning here for the bounded pull workflow.

Prerequisites

Complete Choose venue and market family when the namespace is not already known.
Check the selected symbol and data family in Data quality before a long pull.
Load the API key from the runtime environment.
Set a page budget and output path before the first request.

Inputs

Field	Bounded value
Venue family	Hyperliquid core
Symbol	`BTC`
Data family	Trades
Route	`/v1/hyperliquid/trades/BTC`
Window	`start=1767225600000&end=1767229200000`
Page size	`limit=1000`
Stop rule	Stop after the requested window is exhausted or the client page budget is reached

Steps

Check the window

Confirm the family, symbol, data family, coverage source, freshness state, and incident state for the requested window.

Pull the first page

curl -sS "https://api.0xarchive.io/v1/hyperliquid/trades/BTC?start=1767225600000&end=1767229200000&limit=1000" \
  -H "X-API-Key: $OXARCHIVE_API_KEY"

Follow the cursor within bounds

If the response has meta.next_cursor, send that exact value as cursor on the next request. Keep the original start, end, and limit, and stop at the declared page budget.

Save each page

Store the full query, response status, returned rows, cursor value, and request ID for every page. Keep the raw response separate from any transformed table.

Expected state

The first response has success, a data array of trade rows, and route metadata. meta.next_cursor may be absent when the window is exhausted. A completed pull has no remaining cursor inside the requested window and a request ID for each page.

Verification

Use integer Unix milliseconds for start, end, and the route’s documented cursor values. Do not mix ISO text and numeric bounds in one job. Check that the final row stays inside the requested window, the cursor chain is recorded, and the quality decision is attached to the output.

Failure and recovery

400: correct the named parameter, symbol, or time window before retrying.
401: repair the key source or header.
404: return to Venue coverage and check the family namespace.
429: honor Retry-After when present, otherwise use capped exponential backoff with jitter and lower concurrency.
A remaining cursor at the page budget: mark the output partial and resume with the saved cursor under a new bounded run.
A quality or incident failure: narrow, delay, or mark the output. Do not add synthetic rows.

Saved run metadata

{
  "run": "btc-trades-hour-001",
  "venue_family": "hyperliquid_core",
  "symbol": "BTC",
  "data_family": "trades",
  "route": "/v1/hyperliquid/trades/BTC",
  "query": {
    "start": 1767225600000,
    "end": 1767229200000,
    "limit": 1000
  },
  "cursor_chain": [],
  "request_ids": ["00000000-0000-0000-0000-000000000000"],
  "quality_state": "record the returned coverage, freshness, and incident decision",
  "output": "./artifacts/btc-trades-hour-001.json"
}

The zero UUID is a shape placeholder only. Replace it with the UUID returned by each response or client wrapper. Append each real cursor to cursor_chain; do not invent an opaque cursor.

Next task

Use Point-In-Time Backtesting for a research manifest, Reliability and data gaps for partial output policy, or Limits and throughput before widening the pull.

​Outcome

​Prerequisites

​Inputs

​Steps

​Expected state

​Verification

​Failure and recovery

​Saved run metadata

​Next task