> ## Documentation Index
> Fetch the complete documentation index at: https://docs.simplefunctions.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Snapshots and data exports

> Public HuggingFace datasets and live snapshot endpoints — for offline calibration, replay, and reproducible research on SimpleFunctions market state.

SimpleFunctions exports two kinds of historical state so you can reproduce research without scraping rendered pages or hitting the public API every loop.

| Need                             | Use                                                      |
| -------------------------------- | -------------------------------------------------------- |
| Live state right now             | The real-time data API at `data.simplefunctions.dev/v1`. |
| Historical state, offline, batch | The public HuggingFace datasets below.                   |

## Public HuggingFace datasets

All five datasets live under the [SimpleFunctions organization](https://huggingface.co/SimpleFunctions). They are public — `git clone` and `huggingface-cli download` work without auth.

| Dataset                                                                                                            | Contents                                                                            | Cadence  | Files                       |
| ------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------- | -------- | --------------------------- |
| [`SimpleFunctions/sf-index-history`](https://huggingface.co/datasets/SimpleFunctions/sf-index-history)             | SimpleFunctions Index time series across themes.                                    | Daily    | `sf-index-history.jsonl`    |
| [`SimpleFunctions/world-state-daily`](https://huggingface.co/datasets/SimpleFunctions/world-state-daily)           | Compact daily world snapshot — top markets, regime distribution, tail-risk roll-up. | Daily    | `YYYY-MM-DD.json`           |
| [`SimpleFunctions/world-awareness-bench`](https://huggingface.co/datasets/SimpleFunctions/world-awareness-bench)   | Question / answer pairs measuring whether a model "knows the world".                | Periodic | `benchmark_YYYY-MM-DD.json` |
| [`SimpleFunctions/settled-markets`](https://huggingface.co/datasets/SimpleFunctions/settled-markets)               | Resolved markets with terminal state — for calibration and backtesting.             | Monthly  | `YYYY-MM.jsonl`             |
| [`SimpleFunctions/calibration-scorecards`](https://huggingface.co/datasets/SimpleFunctions/calibration-scorecards) | Scoring of public theses and forecasts against settled outcomes.                    | Monthly  | `YYYY-MM.json`              |

### Pull a file

```bash theme={null}
# SimpleFunctions Index time series
curl -L "https://huggingface.co/datasets/SimpleFunctions/sf-index-history/resolve/main/sf-index-history.jsonl" \
  -o sf-index-history.jsonl

# A single day's world state
curl -L "https://huggingface.co/datasets/SimpleFunctions/world-state-daily/resolve/main/2026-05-01.json" \
  -o world-state-2026-05-01.json

# A month of settled markets
curl -L "https://huggingface.co/datasets/SimpleFunctions/settled-markets/resolve/main/2026-05.jsonl" \
  -o settled-2026-05.jsonl
```

### Clone the whole dataset

```bash theme={null}
git lfs install
git clone https://huggingface.co/datasets/SimpleFunctions/world-state-daily
```

### Use from `datasets`

```python theme={null}
from datasets import load_dataset
ds = load_dataset("SimpleFunctions/sf-index-history")
```

## Live snapshots

For current state, hit the realtime data API:

```bash theme={null}
curl "https://data.simplefunctions.dev/v1/snapshot"
curl "https://data.simplefunctions.dev/v1/markets/featured?n=20"
```

See [Real-time data](/reference/realtime-data) for response shapes, WebSocket topics, and rate limits.

## Provenance

Public exports include enough timestamp and source metadata to reconstruct the state that produced a public chart, answer, or backtest. Where a row carries a trace id, treat it as a support / audit handle, not as a public database schema. See [Provenance](/concepts/provenance).

## What is not exported

Internal storage, raw cron output, observability tables, and unsettled in-flight intents are not part of any public export. If you need a slice that is not on HuggingFace, open an issue or email `patrick@simplefunctions.dev` rather than assuming an internal path.

## Next steps

<CardGroup cols={2}>
  <Card title="Real-time data" href="/reference/realtime-data">
    REST and WebSocket for current market state.
  </Card>

  <Card title="Calibration" href="/api-reference/index-regime">
    SimpleFunctions Index, regime, and calibration HTTP endpoints.
  </Card>
</CardGroup>