# Blockchain monitoring

Source: https://docs.settlemint.com/docs/developers/developer-guides/operations/blockchain-monitoring
Monitor platform status from the DALP dapp and API, and inspect chain RPC and indexer health from the API or CLI.



Use blockchain monitoring when you need detailed chain RPC and indexer diagnostics. The API exposes sync lag, block age, finality lag, stall time, reindex state, and recent service status for each monitored chain service.

Blockchain monitoring combines platform status with chain RPC and indexer signals for each monitored DALP service.

<Mermaid
  chart="`
flowchart TD
Operator[&#x22;Platform operator&#x22;] --> Status[&#x22;Platform status API&#x22;]
Operator --> Monitoring[&#x22;Blockchain monitoring API&#x22;]
Status --> Rollup[&#x22;Environment health rollup&#x22;]
Monitoring --> RPC[&#x22;Chain RPC health&#x22;]
Monitoring --> Indexer[&#x22;Indexer sync health&#x22;]
Monitoring --> Events[&#x22;Live health events&#x22;]
RPC --> Decision[&#x22;Operate, wait, or investigate&#x22;]
Indexer --> Decision
Events --> Decision
`"
/>

Use platform status when you need the operator dashboard rollup. The platform status API combines data freshness, transaction infrastructure, API request health, and workflow health into one snapshot and a trailing history view.

This is operational data for a DALP environment. It helps you answer questions like:

* Is an indexer behind the chain head?
* Is a chain RPC endpoint stale or stalled?
* Which networks have degraded or critical health?
* Did a service recover after an incident?

<Callout type="info" title="Operator access required">
  Blockchain monitoring read endpoints require the global `admin` role or the organization `owner` role. Starting a
  reindex requires system operator permission. API keys inherit the permissions of the user that created them.
</Callout>

## What DALP monitors [#what-dalp-monitors]

DALP reports health for two service types:

| Service type | What it tells you                                                                   |
| ------------ | ----------------------------------------------------------------------------------- |
| `chain-rpc`  | Whether DALP can read fresh blocks from the configured chain RPC endpoint.          |
| `indexer`    | Whether the indexer is keeping up with chain head and serving current indexed data. |

Health status values are:

| Status       | Meaning                                                        |
| ------------ | -------------------------------------------------------------- |
| `healthy`    | The service is reporting normally.                             |
| `degraded`   | The service is lagging or stale enough to need attention.      |
| `critical`   | The service is unhealthy for the monitored range.              |
| `unknown`    | DALP does not have enough recent data to classify the service. |
| `reindexing` | The indexer is rebuilding data for a chain.                    |

DALP smooths status transitions to avoid flapping. A new `healthy`, `degraded`, or `critical` state becomes the displayed `status` after three consecutive matching snapshots. The raw snapshot classification is still available as `rawStatus`. The `reindexing` state is shown immediately so operators can see active indexer rebuilds without waiting for additional samples.

| Service type | Degraded when                                                                                                                                | Critical when                                                                                               |
| ------------ | -------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------- |
| `chain-rpc`  | Collector latency is over 1.5 seconds, or block age or head stall is 130 seconds or more.                                                    | The RPC call fails, collector latency is over 5 seconds, or block age or head stall is 190 seconds or more. |
| `indexer`    | Block lag is 10 or more, indexer state is 5 minutes stale, historical sync is incomplete, or the indexer reports an impossible negative lag. | No sync state exists, block lag is over 100, or indexer state is 15 minutes stale.                          |

For on-demand mining networks such as local development chains, DALP ignores block-age, head-stall, and indexer-staleness checks because those networks do not produce blocks without transactions. Latency, call success, block lag, historical sync, and reindex state still count.

## Get the platform status rollup [#get-the-platform-status-rollup]

Open Platform settings > Platform status in the dapp when you need the platform-wide operator dashboard view. DALP shows the same snapshot and history data from the API to users with the global administrator role.

The platform status snapshot endpoint is deprecated and will be removed after one release. It still returns a single environment-level health view for older clients, but new integrations should call the per-panel endpoints below and assemble the view client-side.

```bash
curl -G "https://your-platform.example.com/api/v2/platform-status/snapshot" \
  -H "X-Api-Key: sm_dalp_xxxxxxxxxxxxxxxx"
```

Use these replacements for new integrations:

```bash
curl -G "https://your-platform.example.com/api/v2/platform-status/data-freshness" \
  -H "X-Api-Key: sm_dalp_xxxxxxxxxxxxxxxx"

curl -G "https://your-platform.example.com/api/v2/platform-status/transactions" \
  -H "X-Api-Key: sm_dalp_xxxxxxxxxxxxxxxx"

curl -G "https://your-platform.example.com/api/v2/platform-status/platform-api" \
  -H "X-Api-Key: sm_dalp_xxxxxxxxxxxxxxxx"

curl -G "https://your-platform.example.com/api/v2/platform-status/workflows" \
  -H "X-Api-Key: sm_dalp_xxxxxxxxxxxxxxxx"

curl -G "https://your-platform.example.com/api/v2/platform-status/stat-cards" \
  -H "X-Api-Key: sm_dalp_xxxxxxxxxxxxxxxx"
```

The four panel endpoints each return `generatedAt`, `verdict`, `sparkline`, and panel-specific `stats`. Roll up the four `verdict` values in severity order (`outage`, `degraded`, `operational`, then `no_data`) to build the header verdict. Use the `stat-cards` response for the five 24-hour summary cards.

The stat cards summarize recent platform activity before you inspect the individual panels:

| Stat card            | What it means                                                                                                                        |
| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
| Operations completed | Workflows that reached a terminal status in the trailing 24 hours.                                                                   |
| Success rate         | Share of terminal workflows in the trailing 24 hours that completed successfully. Failed, killed, and cancelled workflows reduce it. |
| Completion time P95  | P95 wall-clock duration for successful workflow completions in the trailing 24 hours.                                                |
| Data delay           | Worst indexer sync lag, in blocks, from the latest indexer rollup bucket within the trailing 24 hours.                               |
| Blocks indexed       | Sum of the latest indexed block height reported by each indexer service.                                                             |

The workflow stat cards can be empty when no terminal workflows exist in the window or when the workflow admin query is unavailable. The indexer stat cards can be empty until indexer health snapshots or rollups exist, such as immediately after a fresh deployment.

The response groups health into four panels:

| Panel          | What it summarizes                                                         |
| -------------- | -------------------------------------------------------------------------- |
| Data freshness | Latest indexer rollups, chains in sync, and recent sync errors.            |
| Transactions   | Chain RPC service coverage.                                                |
| Platform API   | Request volume and 4xx/5xx rates over the trailing 24 hours.               |
| Workflows      | Completed and stalled workflow counts when workflow metrics are available. |

The platform status verdict is `operational`, `degraded`, `outage`, or `no_data`. `no_data` means DALP has not collected enough rollup data yet, such as immediately after a fresh deployment.

Use the history endpoint when you need the trailing daily severity grid shown in the dapp status page. The `days` query parameter accepts 1 to 30 days and defaults to 30.

```bash
curl -G "https://your-platform.example.com/api/v2/platform-status/history" \
  -H "X-Api-Key: sm_dalp_xxxxxxxxxxxxxxxx" \
  --data-urlencode "days=30"
```

Each history row returns oldest-first UTC day cells. DALP emits one row per indexer service, one row per chain RPC service, one Platform API row, and one Workflows row when those sources are available.

Empty cells use `no_data`. That means DALP has no observation for that row and UTC day, or the row's telemetry source was temporarily unavailable while the history response was assembled. DALP still returns the rest of the grid so operators can see the sources that were available.

For the Platform API row, DALP excludes authentication failures from the 4xx health rate when raw request logs cover the day. If raw logs no longer cover the full UTC day, DALP falls back to the retained 4xx rollup so older history does not hide real client-error storms.

If every row in the requested history window is `no_data`, DALP does not cache that empty response. A fresh deployment or recovering telemetry source can then populate the history grid as soon as observations arrive.

## Get a blockchain monitoring summary [#get-a-blockchain-monitoring-summary]

Use the blockchain monitoring summary endpoint for the current state per monitored chain service.

```bash
curl -G "https://your-platform.example.com/api/v2/blockchain-monitoring/health-metrics/summary" \
  -H "X-Api-Key: sm_dalp_xxxxxxxxxxxxxxxx" \
  --data-urlencode "from=2026-05-01T00:00:00.000Z" \
  --data-urlencode "to=2026-05-01T12:00:00.000Z"
```

The response contains one entry per monitored service:

```json
{
  "services": [
    {
      "serviceId": "01964f21-9f02-7000-9e8d-1e6d1fdc7f01",
      "serviceType": "indexer",
      "chainId": 11155111,
      "networkName": "Sepolia",
      "latestStatus": "healthy",
      "latestSampledAt": "2026-05-01T11:59:30.000Z",
      "snapshotCount": 48,
      "syncLag": 2,
      "blockHeight": 8154321,
      "blockAgeSeconds": null,
      "finalityLagBlocks": null,
      "stallSeconds": null,
      "recentLatencies": [124, 119, 131],
      "deploymentState": null
    }
  ]
}
```

Use these fields like this:

| Field               | Use it for                                                          |
| ------------------- | ------------------------------------------------------------------- |
| `latestStatus`      | The current classified health state.                                |
| `syncLag`           | Indexer lag in blocks. This is only set for `indexer` services.     |
| `blockAgeSeconds`   | Chain RPC freshness. This is only set for `chain-rpc` services.     |
| `finalityLagBlocks` | Chain RPC finality lag.                                             |
| `stallSeconds`      | How long the chain head has stopped advancing.                      |
| `deploymentState`   | Indexer version and reindex progress when an indexer is rebuilding. |

## Troubleshoot indexed data lag [#troubleshoot-indexed-data-lag]

Use these checks when an EVM transaction is visible in an explorer before DALP APIs, dashboards, or reports show the updated state. The transaction receipt proves the on-chain result; DALP read surfaces update after the Chain Indexer processes the relevant event range and writes the read model.

| Symptom                                                   | Check first                                                                                                              | What it means                                                                                     |
| --------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------- |
| Transaction appears in an explorer, but not in DALP reads | Compare the transaction block with the latest processed indexer block, `syncLag`, `latestStatus`, and `latestSampledAt`. | The on-chain operation can be final before the read model has processed the block.                |
| Newly deployed contract is missing from indexed views     | Confirm the contract discovery event was processed and any backfill work is complete.                                    | Newly discovered supported contracts can need follow-up indexing work before their events appear. |
| Indexed data changes after a recent block                 | Check reorg metrics and the finalized-block watermark.                                                                   | Recent indexed rows can be corrected when the chain reorganises inside the supported window.      |
| Reindex is active                                         | Check `deploymentState`, pending backfill progress, and live read availability.                                          | DALP can build replacement indexed state while the current served model remains available.        |

Do not start with a reindex. First confirm the transaction succeeded on the expected chain and contract, then check chain RPC health, indexer health, contract registration, and whether the checkpoint has reached the affected block. Fix RPC access, chain configuration, handler errors, or contract registration before rebuilding the indexed read model.

Read [Chain Indexer](/docs/architects/architecture/data-availability/chain-indexer) for the architecture explanation behind checkpoints, finality, reorg recovery, and reindexing.

## Investigate settlement creation gaps [#investigate-settlement-creation-gaps]

XvP settlement creation depends on both the factory event and read access to the newly deployed settlement contract at the
creation block. The indexer uses the factory event to discover the settlement address and creator, then reads the
settlement name, cutoff date, auto-execute flag, flows, hashlock, and external-flow flag from the settlement contract.

If XvP settlements are missing from indexed views after creation, check these before reindexing:

1. Confirm the chain RPC endpoint can serve historical reads at the settlement creation block.
2. Confirm the settlement contract address exists on the same chain and block range that the indexer is processing.
3. Check the indexer logs for a failed XvP settlement state read. A read failure means the settlement creation event was
   seen, but the indexer could not read the settlement details needed to populate the indexed settlement and leg records.
4. After fixing RPC access or chain configuration, re-run the affected indexer range so the settlement, legs, local
   approval placeholders, and indexed contract entry are rebuilt from the chain.

Use this check for XvP settlement creation only. Later approval, revoke, execute, cancel, expiry withdrawal, secret reveal,
and cancel-vote events are handled from their own settlement events.

## Trigger an indexer reindex [#trigger-an-indexer-reindex]

Start a rebuild only after you have fixed the root cause, such as RPC access or chain configuration. The endpoint requires a
positive `chainId`, then starts an indexer reindex operation. Treat this as indexer maintenance: run it in a controlled
window and wait for any active rebuild to finish before starting another one.

```bash
curl -X POST "https://your-platform.example.com/api/v2/admin/operator/blockchain-monitoring/reindex" \
  -H "X-Api-Key: sm_dalp_xxxxxxxxxxxxxxxx" \
  -H "Content-Type: application/json" \
  --data '{"chainId":11155111}'
```

A successful response confirms that DALP accepted the request:

```json
{
  "acknowledged": true
}
```

If another reindex is already running, the API returns a conflict. If the indexer service cannot be reached or rejects the
request, retry only after the service is available and the chain target is valid.

## Check a timeline [#check-a-timeline]

Use the timeline endpoint when you need chart data or want to see whether a problem is getting worse.

```bash
curl -G "https://your-platform.example.com/api/v2/blockchain-monitoring/health-metrics/timeline" \
  -H "X-Api-Key: sm_dalp_xxxxxxxxxxxxxxxx" \
  --data-urlencode "from=2026-04-24T00:00:00.000Z" \
  --data-urlencode "to=2026-05-01T00:00:00.000Z" \
  --data-urlencode "granularity=day"
```

The timeline accepts `hour` or `day` granularity. The queried range must be 31 days or less.

```json
{
  "buckets": [
    {
      "timestamp": "2026-04-30T00:00:00.000Z",
      "services": [
        {
          "serviceKey": "indexer-11155111",
          "serviceType": "indexer",
          "chainId": 11155111,
          "networkName": "Sepolia",
          "healthyPct": 95.8,
          "degradedPct": 4.2,
          "criticalPct": 0
        }
      ]
    }
  ]
}
```

## Inspect service health [#inspect-service-health]

Use the service breakdown when you need one row per monitored service with aggregate percentages.

```bash
curl -G "https://your-platform.example.com/api/v2/blockchain-monitoring/service-health-metrics" \
  -H "X-Api-Key: sm_dalp_xxxxxxxxxxxxxxxx" \
  --data-urlencode "from=2026-05-01T00:00:00.000Z" \
  --data-urlencode "to=2026-05-01T12:00:00.000Z"
```

The time range for summary and service breakdown calls must be 7 days or less.

## Inspect raw snapshots [#inspect-raw-snapshots]

Use snapshots when you need the underlying health log. You can filter by service, status, service type, network name, and time range.

```bash
curl -G "https://your-platform.example.com/api/v2/blockchain-monitoring/health-snapshots" \
  -H "X-Api-Key: sm_dalp_xxxxxxxxxxxxxxxx" \
  --data-urlencode "status=degraded" \
  --data-urlencode "serviceType=indexer" \
  --data-urlencode "limit=20"
```

Each snapshot includes the sampled time, service type, chain ID, network name, status, block metrics, and collector latency. Responses include `nextCursor` for pagination.

```json
{
  "items": [
    {
      "id": "01964f32-4cc1-7000-a05e-88a188d62921",
      "serviceType": "chain-rpc",
      "chainId": 11155111,
      "networkName": "Sepolia",
      "status": "degraded",
      "rawStatus": "degraded",
      "sampledAt": "2026-05-01T11:55:00.000Z",
      "blockHeight": 8154300,
      "chainHeadBlock": null,
      "syncLag": null,
      "entityCount": null,
      "blockAgeSeconds": 45,
      "finalityLagBlocks": 3,
      "stallSeconds": 30,
      "collectorLatencyMs": 148
    }
  ],
  "nextCursor": null,
  "totalCount": 1,
  "facets": {}
}
```

## Stream health events [#stream-health-events]

Use the stream endpoint when you want live updates in an operations screen or incident console.

```bash
curl -N "https://your-platform.example.com/api/v2/blockchain-monitoring/health-snapshots/stream" \
  -H "X-Api-Key: sm_dalp_xxxxxxxxxxxxxxxx"
```

The stream uses server-sent events. Every event includes `eventType`, so parse that first instead of assuming each message has snapshot fields.

Snapshot events use `eventType: "snapshot"` and carry the same health fields as snapshot list items, plus `serviceId` and optional `deploymentState`:

```json
{
  "eventType": "snapshot",
  "id": "01964f32-4cc1-7000-a05e-88a188d62921",
  "serviceId": "01964f21-9f02-7000-9e8d-1e6d1fdc7f01",
  "serviceType": "indexer",
  "chainId": 11155111,
  "networkName": "Sepolia",
  "status": "healthy",
  "rawStatus": "healthy",
  "sampledAt": "2026-05-01T11:59:30.000Z",
  "blockHeight": 8154321,
  "chainHeadBlock": 8154323,
  "syncLag": 2,
  "entityCount": null,
  "blockAgeSeconds": null,
  "finalityLagBlocks": null,
  "stallSeconds": null,
  "collectorLatencyMs": 124,
  "deploymentState": null
}
```

Block-progress events use `eventType: "block-progress"`. DALP emits these while an indexer processes block batches, including during reindexing:

```json
{
  "eventType": "block-progress",
  "chainId": 11155111,
  "blockNumber": 8154321
}
```

Keep the connection open and update your view based on the event type you receive.

## Use the CLI [#use-the-cli]

The same monitoring data is available from the DALP CLI:

```bash
dalp monitoring blockchain summary --from 2026-05-01T00:00:00.000Z --to 2026-05-01T12:00:00.000Z
dalp monitoring blockchain timeline --from 2026-04-24T00:00:00.000Z --to 2026-05-01T00:00:00.000Z
dalp monitoring blockchain services --from 2026-05-01T00:00:00.000Z --to 2026-05-01T12:00:00.000Z
dalp monitoring blockchain snapshots
dalp monitoring blockchain snapshots-stream
```

Use the CLI for quick checks. Use the API when you are building dashboards, alerting, or operational reports.

## Related operations [#related-operations]

* [Transaction tracking](/docs/developers/developer-guides/operations/transaction-tracking) - Check the status of a submitted transaction by hash.
* [Reconciliate balances](/docs/developers/developer-guides/operations/reconciliate-balances) - Compare on-chain balances with your internal records.
* [API reference](/docs/developers/developer-guides/api-integration/api-reference) - Browse generated API coverage.