Limits and throughput

Outcome

Run one bounded historical job that stays inside its request, concurrency, credit, retry, and page budgets.

Prerequisites

Choose one venue family, symbol, data family, and time window.
Read Rate limits and Errors and request IDs for current capacity and retry behavior.
Set a worker count, page budget, retry budget, and output path before the first request.
Keep the API key in the runtime environment and out of job logs.

Inputs

Field	Bounded value
Route	`/v1/hyperliquid/trades/BTC`
Window	`start=1767225600000&end=1767229200000`
Page size	`limit=100`
Workers	`1` for the first run
Retry budget	A finite count for `429`, transient `5xx`, and network timeouts
Stop rule	Window exhausted, page budget reached, or a non-retryable error

Steps

Run one baseline request

curl -sS "https://api.0xarchive.io/v1/hyperliquid/trades/BTC?start=1767225600000&end=1767229200000&limit=100" \
  -H "X-API-Key: $OXARCHIVE_API_KEY"

Save the status, row count, request ID, and any meta.next_cursor before adding workers.

Set finite workers and pages

Give each worker one family, symbol, window, and cursor chain. Stop when the window is exhausted or the page budget is reached. Do not let the agent or scheduler create an unbounded loop.

Apply bounded retries

Retry 429, transient 5xx, and network timeouts only within the retry budget. Honor Retry-After when present; otherwise use capped exponential backoff with jitter and lower concurrency.

Widen only after the run is inspectable

Compare returned rows, request IDs, latency, retries, and quality state with the input record. Increase symbols or workers only after this run has a clear stop state.

Expected state

Every request has a recorded status and request identifier. The run ends as complete, partial, or stopped with a reason. A remaining cursor, exhausted retry budget, or quality failure is visible in the output rather than hidden by a retry loop.

Verification

Treat a 403 as an exact endpoint/account/key response. Inspect the endpoint contract and verify the account/key response before changing the route or retrying. Capacity limits affect request rate, concurrency, credits, standard replay’s speed, or export volume; they do not by themselves establish route or dataset availability. Check that the route, numeric window, page size, worker count, retry budget, and stop rule are written beside the output. Keep request IDs per page, including failed attempts when the client exposes them.

Failure and recovery

400, invalid symbols, or invalid parameters: fix the input before another call.
401: repair the key source or X-API-Key header.
403: inspect the exact endpoint/account/key response and stop unchanged retries.
429: honor Retry-After when present, otherwise back off with jitter and lower concurrency.
5xx or timeout: retry within budget, then check Data quality and stop if the budget is exhausted.
Page budget reached with a cursor remaining: mark the output partial and resume from the saved cursor under a new bounded run.

Saved run metadata

{
  "run": "btc-trades-throughput-001",
  "venue_family": "hyperliquid_core",
  "symbol": "BTC",
  "data_family": "trades",
  "route": "/v1/hyperliquid/trades/BTC",
  "query": {
    "start": 1767225600000,
    "end": 1767229200000,
    "limit": 100
  },
  "workers": 1,
  "page_budget": "record the finite page count",
  "retry_budget": "record the finite attempt count",
  "request_ids": ["00000000-0000-0000-0000-000000000000"],
  "output": "./artifacts/btc-trades-throughput-001.json",
  "stop_reason": "record the observed completion or stop reason"
}

The zero UUID is a shape placeholder only. Replace it with the UUID returned by each response or client wrapper.

Next task

Use Historical market data for cursor handling, Reliability and data gaps for partial output policy, or Rate limits for current account capacity.

​Outcome

​Prerequisites

​Inputs

​Steps

​Expected state

​Verification

​Failure and recovery

​Saved run metadata

​Next task