> For the complete documentation index, see [llms.txt](https://emeditweb.gitbook.io/pulsar-stellar-sdk/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://emeditweb.gitbook.io/pulsar-stellar-sdk/indexer-guide.md). # Indexer Guide The Pulsar indexer is a Go daemon that polls Soroban RPC, decodes the events a tracked contract emits, and stores them in Postgres or SQLite past the roughly seven-day window RPC keeps. It is how the SDK's indexer path reads history older than the retention wall, and it is what the web explorer reads from. ## What is real today * **Polling and storage work.** The daemon loads its configuration, opens the database, applies migrations, registers its bootstrap contracts, and runs one polling loop per contract. Decoded events are written to the store. * **The HTTP surface is served.** The daemon mounts its REST and GraphQL read API on a listener alongside the poll loops, so a running indexer answers the routes below on a port. The read routes are public; the two write routes are gated. * **The event routes are built.** `GET /contracts/{id}/events` and `GET /events/{id}` serve the SDK's events-query and single-event methods. * **GraphQL is served.** `POST /graphql` exposes a read-only schema over the same data, with nesting the REST surface cannot express. * **There is no public hosted indexer.** Self-host it. The examples here use a local address. ## Self-hosting ### Requirements * **Go 1.23 or newer.** The module pins a newer toolchain in `go.mod`, which a 1.23 or newer `go` fetches and uses automatically on build, so you do not have to install that exact version by hand. * **A database, Postgres or SQLite.** The indexer supports both through a dual-driver design. SQLite needs no server and suits local development; Postgres is for a deployment. Migrations for each live in separate directories under `indexer/migrations/`, because their SQL differs (ADR-029). SQLite is the default, so you can start with no database server. ### Building The indexer is a separate Go module in the `pulsar-app` repository. It is not a pnpm workspace member. ```bash cd indexer go build ./... ``` ### Configuration The indexer reads its configuration from environment variables, validated once at startup. A missing required variable or an unusable value is a startup failure that names the variable, never a default quietly substituted. The full schema is in `.env.example` at the `pulsar-app` root; the keys you will set most often: | Variable | Required | Default | Purpose | | ------------------------------------ | -------- | -------- | --------------------------------------------------------------------------- | | `PULSAR_INDEXER_DB_URL` | yes | none | Database DSN. `file:./pulsar.db` for SQLite, a `postgres://` URL otherwise. | | `PULSAR_INDEXER_RPC_URL` | yes | none | Soroban RPC endpoint. Must be http or https. | | `PULSAR_INDEXER_NETWORK` | yes | none | One of `testnet`, `futurenet`, `mainnet`, `local`. | | `PULSAR_INDEXER_ADMIN_TOKEN` | yes | none | Bearer token guarding the write routes. At least 16 characters. | | `PULSAR_INDEXER_LISTEN_ADDR` | no | `:8080` | Address the HTTP surface binds. | | `PULSAR_INDEXER_DB_DRIVER` | no | `sqlite` | `sqlite` or `postgres`. | | `PULSAR_INDEXER_POLL_INTERVAL_SEC` | no | `5` | Seconds between RPC polls per contract. | | `PULSAR_INDEXER_BATCH_SIZE` | no | `100` | Events fetched per RPC page. | | `PULSAR_INDEXER_BOOTSTRAP_CONTRACTS` | no | empty | Comma-separated contract IDs tracked on first run. | | `PULSAR_INDEXER_WRITE_RATE_PER_SEC` | no | `5` | Sustained requests per second on the write surface. | | `PULSAR_INDEXER_WRITE_RATE_BURST` | no | `10` | Burst the write limiter tolerates. | | `PULSAR_INDEXER_READ_TIMEOUT_SEC` | no | `15` | Wall-clock ceiling on the time any request spends downstream. | The pool, TLS, and logging keys (`PULSAR_INDEXER_DB_POOL_MAX`, `PULSAR_INDEXER_DB_ALLOW_INSECURE_TLS`, `PULSAR_INDEXER_LOG_LEVEL`, and others) are in `.env.example` with their defaults. ### Running SQLite is the default, so the daemon needs no database server to stand up. It requires an admin token, so generate one: ```bash cd indexer PULSAR_INDEXER_ADMIN_TOKEN=$(openssl rand -hex 32) \ PULSAR_INDEXER_DB_URL=file:./pulsar.db \ PULSAR_INDEXER_RPC_URL=https://soroban-testnet.stellar.org \ PULSAR_INDEXER_NETWORK=testnet \ PULSAR_INDEXER_BOOTSTRAP_CONTRACTS=CDNWTVUDKCCGW7GOC6SBLUFXXUCD2YDHWRDUSXZ6CYBQKQWLCUYYWI5L \ go run ./cmd/pulsar-indexer ``` The daemon logs each startup stage, binds the listener before launching the poll loops (so a port already in use is a clear startup error), then runs until `SIGINT` or `SIGTERM`, drains in-flight requests, cancels every poll loop, and exits. Within one poll interval it begins filling the events table, and the read API is reachable on `:8080`. ## The read API Every JSON response, success or failure, is the same envelope, fixed by ADR-017. Every response also carries `X-Content-Type-Options: nosniff`. ### Response envelope A success carries the payload under `data`, with a `meta.took_ms` sibling, and a `next_cursor` on paginated routes: ```json { "data": { "ok": true, "version": "0.1.0", "latest_ledger": 1284913, "tracked_contracts": 1 }, "meta": { "took_ms": 0.42 } } ``` An error carries a wire class in `error.code`, a human-readable `error.message` that can change without notice, and a stable catalog code in `error.details.code` that a consumer branches on: ```json { "error": { "code": "not_found", "message": "The indexer is not tracking this contract.", "details": { "code": "NOT_FOUND_CONTRACT" } } } ``` The wire classes are `validation`, `not_found`, `rate_limited`, `internal`, and `unauthorized`. The HTTP status set is 200, 400, 401, 404, 429, and 500, plus a 204 with no body for a successful delete. ### Routes | Method | Path | Auth | Purpose | | -------- | ------------------------ | -------- | --------------------------------------------------- | | `GET` | `/health` | open | Liveness, `latest_ledger`, and `tracked_contracts`. | | `GET` | `/contracts` | open | List tracked contracts, oldest first. | | `POST` | `/contracts` | required | Register a contract to track. Idempotent (ADR-018). | | `GET` | `/contracts/{id}` | open | Read one tracked contract. | | `DELETE` | `/contracts/{id}` | required | Stop tracking a contract. Cascades to its events. | | `GET` | `/contracts/{id}/events` | open | That contract's events, newest first, paginated. | | `GET` | `/events/{id}` | open | A single event by its id. | | `POST` | `/graphql` | open | Read-only GraphQL over the same data (ADR-043). | ### GET /contracts/{id}/events Returns the contract's events, newest first, under `data.items`, with a `next_cursor` when more remain. The query parameters: | Parameter | Default | Meaning | | -------------------------- | ------- | ------------------------------------------------------------ | | `limit` | `50` | Page size, 1 to 500. | | `cursor` | none | Opaque cursor from a previous response's `next_cursor`. | | `order` | `desc` | `asc` or `desc`. | | `from_ledger`, `to_ledger` | none | Inclusive ledger-range bound. | | `name` | none | Exact event-name match. | | `topic_contains` | none | Case-sensitive substring match against decoded topic values. | An untracked contract is a 404 `NOT_FOUND_CONTRACT`, distinct from a tracked contract with no matching events, which is a 200 with an empty `items` (ADR-021). ### GET /events/{id} Returns one event by its id at 200, or a 404 `NOT_FOUND_EVENT` when no event has that id. The id is the indexer's own `BIGSERIAL` key, serialized as a string. ## GraphQL `POST /graphql` serves a read-only schema over the same stores, with `Contract.events` nesting the REST surface cannot express. It carries built-in depth and query-length limits and bounded parallelism as its denial-of-service guards, and introspection stays on because the schema is public (ADR-043). The root: ```graphql type Query { health: Health! contracts: [Contract!]! contract(id: ID!): Contract event(id: ID!): Event events(contractId: ID!, name: String, fromLedger: Int, toLedger: Int, topicContains: String, limit: Int, cursor: String, order: String): EventConnection! } ``` A request posts `{ "query": "...", "variables": { ... } }`: ```bash curl -X POST http://localhost:8080/graphql \ -H "Content-Type: application/json" \ -d '{"query":"{ contract(id:\"CDNWTVUDKCCGW7GOC6SBLUFXXUCD2YDHWRDUSXZ6CYBQKQWLCUYYWI5L\") { status events(limit:1) { items { id name } nextCursor } } }"}' ``` Responses follow the GraphQL `{ data, errors }` envelope, distinct from the REST envelope above. Each error carries the same catalog code in `errors[].extensions.code`, so a consumer branches on it the same way. An event `id` travels as a string, because a `BIGSERIAL` exceeds a JSON number's safe range (ADR-021). `contract(id:)` and `event(id:)` resolve to `null` when absent; `events(contractId:)` against an untracked contract is a `NOT_FOUND_CONTRACT` error, not an empty connection. ## Authentication The two write routes, `POST /contracts` and `DELETE /contracts/{id}`, require a static bearer token; every read route is public (ADR-044). ```bash curl -X POST http://localhost:8080/contracts \ -H "Authorization: Bearer $PULSAR_INDEXER_ADMIN_TOKEN" \ -H "Content-Type: application/json" \ -d '{"contract_id":"CDNWTVUDKCCGW7GOC6SBLUFXXUCD2YDHWRDUSXZ6CYBQKQWLCUYYWI5L"}' ``` * The token comes from `PULSAR_INDEXER_ADMIN_TOKEN`, is required with no default, and must be at least 16 characters. The daemon refuses to start otherwise, so a deployment cannot expose open writes by omission. Generate one with `openssl rand -hex 32`. * The presented and configured tokens are each hashed with SHA-256 and compared in constant time, so the comparison leaks neither length nor content. A missing or wrong token fails closed with a `401` and the `unauthorized` wire class, and a `WWW-Authenticate: Bearer` challenge. * A caller over the write rate limit gets a `429` with the `RATE_LIMITED` code. The limiter sits outside the auth check, so an unauthenticated flood is capped before the comparison runs. `DELETE` cascades: one authenticated delete removes the contract and its whole event history (`ON DELETE CASCADE`). That is why the write gate is a precondition of exposing the surface, not a later addition. ### Error catalog | Catalog code | Wire class | HTTP | When | | ------------------------- | -------------- | ---- | ------------------------------------------------------------------------------------- | | `VALIDATION_BODY` | `validation` | 400 | Body is missing, not the expected JSON object, or too large. | | `VALIDATION_CONTRACT_ID` | `validation` | 400 | Contract ID is not the letter C followed by 55 base32 characters. | | `VALIDATION_EVENT_ID` | `validation` | 400 | Event ID is not a string of digits. | | `VALIDATION_LIMIT` | `validation` | 400 | `limit` is out of the 1 to 500 range. | | `VALIDATION_ORDER` | `validation` | 400 | `order` is not `asc` or `desc`. | | `VALIDATION_CURSOR` | `validation` | 400 | `cursor` is malformed. | | `VALIDATION_LEDGER_RANGE` | `validation` | 400 | The ledger window is inverted or out of range. | | `VALIDATION_FILTER` | `validation` | 400 | `name` or `topic_contains` is not storable UTF-8. | | `NOT_FOUND_ROUTE` | `not_found` | 404 | No endpoint matches the path. | | `NOT_FOUND_METHOD` | `not_found` | 404 | The path exists but does not accept this method. | | `NOT_FOUND_CONTRACT` | `not_found` | 404 | The indexer is not tracking the requested contract. | | `NOT_FOUND_EVENT` | `not_found` | 404 | No event has the requested id. | | `UNAUTHORIZED` | `unauthorized` | 401 | A write route was called without a valid bearer token. | | `RATE_LIMITED` | `rate_limited` | 429 | The write rate limit was exceeded. | | `INTERNAL_STORE` | `internal` | 500 | The indexer could not reach its store. The cause is logged server-side. | | `INTERNAL_PANIC` | `internal` | 500 | A handler hit an unexpected condition and recovered. The cause is logged server-side. | ## Storage: SQLite versus Postgres The same schema runs on both engines, but each carries its own migration set rather than one shared file: SQLite accepts several Postgres declarations and then behaves differently (ADR-029). SQLite is the default and the local path; point `PULSAR_INDEXER_DB_URL` at a file. Postgres is the production path: set `PULSAR_INDEXER_DB_DRIVER=postgres` and a `postgres://` DSN whose `sslmode` is `require`, `verify-ca`, or `verify-full`, or the daemon refuses to start on an unencrypted connection unless `PULSAR_INDEXER_DB_ALLOW_INSECURE_TLS=true` is set for local use (ADR-031). --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://emeditweb.gitbook.io/pulsar-stellar-sdk/indexer-guide.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.