Skip to main content
Reliable market-data jobs do not assume every source is continuous. They check the market, window, and data family before using results.

​
Pattern

​
Failure Behavior

​
Reliability Contract

Define the output status for every job. A result can be complete, stale, partial, skipped, retried, or failed. Do not leave the status implicit. That matters for research notebooks, generated dashboards, downstream agents, and backtests because they may otherwise treat partial market data as clean.

​
Gap Handling

When a gap appears, stop using the affected local state until it is rebuilt. For REST history, rerun the smallest window that contains the issue. For replay, restart from a safe checkpoint or snapshot. For local books, discard unsafe state and resync before applying more diffs.

​
Human And Agent Use

Humans should see the gap state in logs or UI. Coding agents should be instructed not to interpolate missing records unless the user explicitly asks for a transformation. The default behavior is to preserve truth, mark the output, and point to the data-quality route that justified the decision.

​
Before You Commit To A Data Window

When evaluating a historical data source, verify how coverage, freshness, incidents, replay gaps, and request IDs are represented before the data touches research notebooks, dashboards, alerts, or models. Those answers define whether the data is usable, not just available.

​
Quality Status Per Dataset

Every durable dataset should carry a quality status: clean, stale, partial, interrupted, or skipped. Store the data-quality response and any request IDs that produced that status alongside the output. That makes later charts, models, and reports auditable without re-running the job.
Last modified on June 28, 2026