Document RPC BACKFILL flag and refresh sample config by kaankacar · Pull Request #2789 · stellar/stellar-docs

kaankacar · 2026-08-24T14:27:14Z

Addresses the mechanical half of #2602, as scoped by @ElliotFriend and @kalepail in the issue thread.

What changed

configuring.mdx tells readers the TOML block is the output of gen-config-file. The block had drifted. I verified it against the real generator and corrected it.

Added flags:

# BACKFILL = false
# SERVE_LEDGERS_FROM_DATASTORE = false
# NETWORK = ""
STELLAR_CAPTIVE_CORE_HTTP_QUERY_PORT = 11628
STELLAR_CAPTIVE_CORE_HTTP_QUERY_SNAPSHOT_LEDGERS = 4

Corrected value:

MAX_GET_LEDGERS_EXECUTION_DURATION from "5s" to "10s" (source: DefaultValue: 10 * time.Second)

New section Backfilling History on Startup gives the version, the default, the datastore prerequisite, and the retention window that sets the size. It links to Data Lake Integration instead of restating it, per @kalepail's note.

How I verified

I could not run docker run stellar/stellar-rpc:latest gen-config-file. Instead I cloned stellar-rpc at v28.0.0 and built the gen-config-file body against internal/config, which needs no Rust preflight linkage. Then I diffed its output against the doc block key by key.

Result: every environment-independent key now matches the generator exactly, in value and in comment state.

Claim	Source in `internal/config/options.go`
`BACKFILL` default `false`	`Name: "backfill"`, `DefaultValue: false`
Requires datastore serving	`Validate:` errors when `cfg.Backfill && !cfg.ServeLedgersFromDatastore`
`SERVE_LEDGERS_FROM_DATASTORE` default `false`	`DefaultValue: false`
`HISTORY_RETENTION_WINDOW` default `120960`	`SevenDayOfLedgers = OneDayOfLedgers * 7`, `OneDayOfLedgers = 17280`
`MAX_GET_LEDGERS_EXECUTION_DURATION` default `10s`	`DefaultValue: 10 * time.Second`

Both usage strings and the startup error message are copied verbatim from the source.

Version: BACKFILL first appears in v25.1.0. v25.0.0 has no backfill option. This matches @ElliotFriend's finding.

Why the block cannot be regenerated verbatim on a laptop

Four defaults depend on the machine, not on the release:

defaultStellarCoreBinaryPath, _ := exec.LookPath("stellar-core")
defaultUintCPU   := uint(runtime.NumCPU())
defaultUint16CPU := uint16(runtime.NumCPU())

They feed STELLAR_CORE_BINARY_PATH, PREFLIGHT_WORKER_COUNT, PREFLIGHT_WORKER_QUEUE_SIZE and STELLAR_CAPTIVE_CORE_HTTP_QUERY_THREAD_POOL_SIZE. The current block holds container values, such as /usr/bin/stellar-core and 8 workers. My run produced "" and 15. Pasting my output would put my CPU count into the docs.

So a faithful full regeneration has to run inside the Docker image. I left these four untouched, and left out STELLAR_CAPTIVE_CORE_HTTP_QUERY_THREAD_POOL_SIZE for the same reason. Please regenerate them in the image when convenient.

Two more differences I found and did not change:

DB_PATH: the doc shows stellar_rpc.sqlite, the generator emits soroban_rpc.sqlite. The source carries // TODO: deprecate and rename to stellar_rpc.sqlite, so the doc matches the intent. Your call.
HISTORY_ARCHIVE_URLS: the generator comments this out, the doc leaves it live. Live is arguably better, because operators must set it.

Left for RPC team review

The issue also asks for startup-behavior and coverage-boundary wording. I did not write it. Both reviewers said it needs RPC-team input rather than source reading, and options.go cannot settle it:

where an ordinary fresh startup begins without BACKFILL
how datastore gaps and existing local state limit what gets materialized
whether release-note storage figures should be labelled as dated examples

Please keep #2602 open for that half.

Checks

prettier passes with the repo config
The sample block is still strictly alphabetical, 62 keys
scripts/check-relative-links.sh --staged reports "All links check out"

The sample config in configuring.mdx predates two flags. Add both, in alphabetical order, with the usage text from the RPC source. - BACKFILL = false - SERVE_LEDGERS_FROM_DATASTORE = false Add a section for BACKFILL. It gives the version, the default, the datastore prerequisite, and the retention window that sets the size. It links to Data Lake Integration for datastore setup. Partially addresses #2602

Copilot

Pull request overview

Documents RPC backfill configuration and startup behavior.

Changes:

Adds BACKFILL and datastore-serving flags.
Adds startup backfill prerequisites, defaults, and datastore guidance.

Recommendation: NEEDS-CHANGES — correct the incomplete generated-config claim, avoid guaranteeing exact backfill coverage, and label the code fence.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

stellar-jenkins-ci · 2026-08-24T14:35:17Z

Preview is available here:
http://developers-pr2789.previews.kube001.services.stellar-ops.com

Verified the sample block against the real generator. Built the gen-config-file body from stellar-rpc v28.0.0 and diffed it. - comment out BACKFILL and SERVE_LEDGERS_FROM_DATASTORE, because the generator comments out zero-value defaults - add NETWORK - add STELLAR_CAPTIVE_CORE_HTTP_QUERY_PORT and STELLAR_CAPTIVE_CORE_HTTP_QUERY_SNAPSHOT_LEDGERS - correct MAX_GET_LEDGERS_EXECUTION_DURATION from 5s to 10s Every environment-independent key now matches the generator exactly.

stellar-jenkins-ci · 2026-08-24T14:43:29Z

Preview is available here:
http://developers-pr2789.previews.kube001.services.stellar-ops.com

Copilot flagged the retention sentence as an over-promise. It is right. Verified in internal/ingest/backfill.go at v27.1.1 and v28.0.0: - nBackfill = min(retentionWindow, currentTipLedger) - start clamped by dsInfo.sequences.First - forward fill ends at PrevCheckpoint(currentTipLedger) - ledgers already in the local database are not refetched Upstream's own comment says the window is "at least approximately" met. So call the value a target and name the three limits. Also label the error code fence as text.

kaankacar · 2026-08-24T14:45:22Z

Pushed fixes for two of the three review comments.

Retention wording. The over-promise flag was correct. I checked internal/ingest/backfill.go at v27.1.1 and v28.0.0:

nBackfill = min(retentionWindow, currentTipLedger)
start clamped by dsInfo.sequences.First
forward fill ends at PrevCheckpoint(currentTipLedger)
ledgers already in the local database are not refetched

Upstream's own comment says the window is met "at least approximately". The section now calls the value a target and names the three limits.

I stopped short of documenting exact thresholds or the failure path. @ElliotFriend and @kalepail reserved coverage boundaries for RPC-team review, so that wording should come from them. verifyBounds and ledgerThreshold = 384 are the relevant code if they want it.

Code fence. Labelled as text.

Sample config completeness. I regenerated the block instead of reasoning about it. I built the gen-config-file body from stellar-rpc v28.0.0 against internal/config, which needs no Rust linkage, then diffed key by key.

Every environment-independent key now matches the generator exactly, in value and comment state. That pass also caught two things: my two new keys needed commenting out, and MAX_GET_LEDGERS_EXECUTION_DURATION was documented as 5s while the source sets 10 * time.Second.

Four keys still differ, and they cannot be fixed here. Their defaults depend on the host:

defaultStellarCoreBinaryPath, _ := exec.LookPath("stellar-core")
defaultUintCPU   := uint(runtime.NumCPU())
defaultUint16CPU := uint16(runtime.NumCPU())

The block currently holds container values, such as /usr/bin/stellar-core and 8 workers. My run produced "" and 15. Regenerating these needs to happen inside the Docker image, so I left them alone.

stellar-jenkins-ci · 2026-08-24T14:52:53Z

Preview is available here:
http://developers-pr2789.previews.kube001.services.stellar-ops.com

kaankacar · 2026-09-07T15:17:40Z

🤖 Automated message from Kaan's Automated Triage Bot.

I checked every key against stellar-rpc v28.0.1 options.go. The five added keys, their comment state, the 10s correction and the startup error text all match the generator. Please keep #2602 open for the RPC-team wording. Ready to merge.

Copilot

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 1 comment.

+# Fetch historical ledgers from the datastore if they're not available locally. This entry should be followed by a datastore configuration.
+# SERVE_LEDGERS_FROM_DATASTORE = false


github-actions · 2026-09-09T20:07:50Z

PR Preview: https://developers-pr-2789.previews.kube001.services.stellar-ops.com

Copilot AI balanced review requested due to automatic review settings August 24, 2026 14:27

Copilot started reviewing on behalf of kaankacar August 24, 2026 14:27 View session

Copilot AI reviewed Aug 24, 2026

View reviewed changes

Comment thread docs/data/apis/rpc/admin-guide/configuring.mdx Outdated

Comment thread docs/data/apis/rpc/admin-guide/configuring.mdx Outdated

Comment thread docs/data/apis/rpc/admin-guide/configuring.mdx Outdated

ElliotFriend requested a review from Shaptic August 31, 2026 15:14

kaankacar requested a review from ElliotFriend September 7, 2026 15:14

kaankacar added the bot:ready-to-merge Bot verified and approved; waiting for a maintainer to merge label Sep 7, 2026

kaankacar mentioned this pull request Sep 8, 2026

Triage bot daily report — 2026-09-07 #2829

Closed

Shaptic approved these changes Sep 8, 2026

View reviewed changes

Comment thread docs/data/apis/rpc/admin-guide/configuring.mdx Outdated

add hints about datastore configurations

d12ade0

Copilot AI review requested due to automatic review settings September 9, 2026 20:03

ElliotFriend approved these changes Sep 9, 2026

View reviewed changes

Copilot started reviewing on behalf of ElliotFriend September 9, 2026 20:04 View session

Copilot AI reviewed Sep 9, 2026

View reviewed changes

Comment thread docs/data/apis/rpc/admin-guide/configuring.mdx

Comment on lines +253 to +254

# Fetch historical ledgers from the datastore if they're not available locally. This entry should be followed by a datastore configuration.

# SERVE_LEDGERS_FROM_DATASTORE = false

github-actions Bot added the preview Preview builds for PRs by SDF employees. label Sep 9, 2026

Navigation Menu

Uh oh!

Document RPC BACKFILL flag and refresh sample config - #2789

Document RPC BACKFILL flag and refresh sample config#2789
kaankacar wants to merge 4 commits into
mainfrom
2602-document-stellar-rpc-backfill-startup-behavior

kaankacar commented Aug 24, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

stellar-jenkins-ci Bot commented Aug 24, 2026

Uh oh!

stellar-jenkins-ci Bot commented Aug 24, 2026

Uh oh!

kaankacar commented Aug 24, 2026

Uh oh!

stellar-jenkins-ci Bot commented Aug 24, 2026

Uh oh!

kaankacar commented Sep 7, 2026

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

github-actions Bot commented Sep 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

		# Fetch historical ledgers from the datastore if they're not available locally. This entry should be followed by a datastore configuration.
		# SERVE_LEDGERS_FROM_DATASTORE = false

Uh oh!

Conversation

kaankacar commented Aug 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changed

How I verified

Why the block cannot be regenerated verbatim on a laptop

Left for RPC team review

Checks

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

stellar-jenkins-ci Bot commented Aug 24, 2026

Uh oh!

stellar-jenkins-ci Bot commented Aug 24, 2026

Uh oh!

kaankacar commented Aug 24, 2026

Uh oh!

stellar-jenkins-ci Bot commented Aug 24, 2026

Uh oh!

kaankacar commented Sep 7, 2026

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

github-actions Bot commented Sep 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

kaankacar commented Aug 24, 2026 •

edited

Loading