Skip to content

Document RPC BACKFILL flag and refresh sample config #2789

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
kaankacar wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Document RPC BACKFILL flag and refresh sample config #2789

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
c5d4e7d
Document RPC BACKFILL flag and refresh sample config
kaankacar Aug 24, 2026
7909d67
Match sample config to gen-config-file output
kaankacar Aug 24, 2026
543e70f
Do not promise exact backfill coverage
kaankacar Aug 24, 2026
d12ade0
add hints about datastore configurations
ElliotFriend Sep 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 40 additions & 1 deletion docs/data/apis/rpc/admin-guide/configuring.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,10 @@ The resulting configuration should look like this:
# from the Internet and does not use TLS. "" (default) disables the admin server
# ADMIN_ENDPOINT = ""

# Populates database with `history-retention-window` ledgers synchronously on
# startup. This defaults to a week of ledgers if unspecified
# BACKFILL = false

# path to additional configuration for the Stellar Core configuration file used
# by captive core. It must, at least, include enough details to define a quorum
# set
Expand Down Expand Up @@ -102,7 +106,7 @@ MAX_GET_LATEST_LEDGER_EXECUTION_DURATION = "5s"
# The maximum duration of time allowed for processing a getLedgers request. When
# that time elapses, the rpc server would return -32001 and abort the request's
# execution
MAX_GET_LEDGERS_EXECUTION_DURATION = "5s"
MAX_GET_LEDGERS_EXECUTION_DURATION = "10s"

# The maximum duration of time allowed for processing a getLedgerEntries
# request. When that time elapses, the rpc server would return -32001 and abort
Expand Down Expand Up @@ -154,6 +158,9 @@ MAX_SIMULATE_TRANSACTION_EXECUTION_DURATION = "15s"
# Maximum amount of transactions allowed in a single getTransactions response
MAX_TRANSACTIONS_LIMIT = 200

# Specifies the desired Stellar network, 'pubnet', 'testnet', or 'futurenet'.
# NETWORK = ""

# Network passphrase of the Stellar network transactions should be signed for.
# Commonly used values are "Test SDF Future Network ; October 2022", "Test SDF
# Network ; September 2015" and "Public Global Stellar Network ; September 2015"
Expand Down Expand Up @@ -222,6 +229,14 @@ SOROBAN_FEE_STATS_RETENTION_WINDOW = 50
# HTTP port for Captive Core to listen on (0 disables the HTTP server)
STELLAR_CAPTIVE_CORE_HTTP_PORT = 11626

# HTTP port for Captive Core to listen on for high-performance queries like
# /getledgerentry (must not conflict with CAPTIVE_CORE_HTTP_PORT)
STELLAR_CAPTIVE_CORE_HTTP_QUERY_PORT = 11628

# Size of ledger history in Captive Core's high-performance query server (don't
# touch unless you know what you are doing)
STELLAR_CAPTIVE_CORE_HTTP_QUERY_SNAPSHOT_LEDGERS = 4

# path to stellar core binary
STELLAR_CORE_BINARY_PATH = "/usr/bin/stellar-core"

Expand All @@ -235,8 +250,16 @@ STELLAR_CORE_TIMEOUT = "2s"
# fields in the config toml from being parsed.
# STRICT = false

# Fetch historical ledgers from the datastore if they're not available locally. This entry should be followed by a datastore configuration.
# SERVE_LEDGERS_FROM_DATASTORE = false
Comment on lines +253 to +254
```

:::info

Example datastore configurations can be found on the [data lake integration page](./data-lake-integration.mdx).

:::

Note that the above generated configuration contains the default values and you need to substitute them with proper values to run the image. For instance, when using a container, it is recommended to create a volume for the Captive Core and RPC persistent storage and point `CAPTIVE_CORE_STORAGE_PATH` and `DB_PATH` to it accordingly.

Then, you should create a configuration file for [Stellar Core](https://github.com/stellar/stellar-core). You can find sample configuration files for [Testnet](https://github.com/stellar/go-stellar-sdk/blob/main/ingest/ledgerbackend/configs/captive-core-testnet.cfg) and [Pubnet](https://github.com/stellar/go-stellar-sdk/blob/main/ingest/ledgerbackend/configs/captive-core-pubnet.cfg)
Expand All @@ -259,6 +282,22 @@ Then you would mount that volume using by adding the following parameter: `-v /U

Your running container would mount that volume at the path `/opt/stellar`

## Backfilling History on Startup

Set `BACKFILL = true` to populate the database with a trailing window of history synchronously on startup, before live ingestion begins. This option was added in RPC v25.1.0 and defaults to `false`.

`HISTORY_RETENTION_WINDOW` sets the target size of that window. The default is `120960` ledgers, which is about 7 days.

RPC treats this value as a target, not an exact count. Datastore coverage, ledgers already in the local database, and checkpoint boundaries can each reduce what the node fetches.

`BACKFILL` also requires datastore serving. Set `SERVE_LEDGERS_FROM_DATASTORE = true`. If you enable `BACKFILL` alone, RPC fails to start with this error:

```text
backfill requires serving ledgers from datastore to be enabled. See the `--serve-ledgers-from-datastore` flag
```

To set up the datastore, see [Data Lake Integration](./data-lake-integration.mdx). That page also explains which requests the datastore serves directly.

## Next Step

After installation is complete, you are now ready to proceed to [Running RPC](./running.mdx)!
Loading