Skip to content

feat(backend): diagnostic XDR error parser and ledger-partitioned eve… - #706

Merged
PeterOche merged 1 commit into
degenspot:mainfrom
feyishola:feature/be-003-004-event-partitioning-diagnostic-parser
Sep 27, 2026
Merged

PeterOche merged 1 commit into
degenspot:mainfrom
feyishola:feature/be-003-004-event-partitioning-diagnostic-parser

Conversation

@feyishola

Copy link
Copy Markdown
Contributor

closes #620
closes #621
closes #622
closes #623

(Soroban RPC Diagnostic Event & XDR Error Code Parser) and BE-003 (Event Store Range Partitioning by Ledger Sequence). All changes are in packages/backend.

BE-004 — Diagnostic & XDR error parser

  • src/indexer/diagnostic-parser.service.ts decodes diagnostic events, TransactionResult, TransactionMeta and simulation error strings into a structured report: contract error variant, authorization failures, footprint/budget exhaustion, archived entries, call stack.
  • All custom contract error codes translated: scripts/generate-contract-errors.js builds src/indexer/contract-errors.generated.ts from packages/contracts/*/src/errors.rs (20 contracts, 290 codes). A test asserts every code has a message and that the registry matches the Rust sources (pnpm --filter backend check:contract-errors).
  • Error codes overlap between contracts (e.g. #6 has 18 meanings), so contract IDs map to contracts via SOROBAN_CONTRACT_ID + SOROBAN_CONTRACT_KIND (default call_registry), OUTCOME_MANAGER_CONTRACT_ADDRESS, and SOROBAN_CONTRACT_ERROR_MAP="CID:contract,...". Unmapped contracts get a list of the possible variants.
  • Failed transactions (from submitTransaction ERROR responses and indexFailedTransaction(txHash)) get structured logging via the NestJS Logger, are stored in a new failed_transactions table, and the submitting user receives one TRANSACTION_FAILED notification.
  • Zero crash on malformed XDR: garbage, truncated and corrupted input and 1,000 random-byte fuzz cases all degrade to null or counters.

Note: ContractError::MarketExpired / StakeBelowMinimum from the issue don't exist in the contracts. The nearest real ones are CallRegistryError::CallEnded and MarketError::RolloverInsufficientAmount.

BE-003 — event_logs range partitioning

  • Migration 1760000070000-PartitionEventStore: PARTITION BY RANGE (ledger) with 100,000-ledger partitions, a DEFAULT partition as a safety net, and a (contractId, topic0, ledger) composite B-Tree index. It adds a topic0 column.
  • Automated partition creation: Postgres can't create partitions from a trigger on the table being inserted into. Instead, event_logs_maintain_partitions() pre-creates partitions ahead of the newest ledger and moves any stray rows out of the default partition. EventLogPartitionService runs it on boot and every 10 min; the migration also schedules it in pg_cron when installed.
  • Zero data loss: an existing table is copied and row-count verified before being dropped (a mismatch rolls back the migration). down() is lossless.
  • event_logs_max_ledger() keeps the indexer's latest-ledger lookup fast (a plain ORDER BY ledger DESC LIMIT 1 would scan every partition).
  • EventStoreService.getRawEventsByLedgerRange() does range scans with keyset pagination. Reconciliation now filters by ledger range in SQL instead of loading a contract's whole history into memory.
  • Also fixes resolveStartLedger treating the bigint ledger as a string ("100" + 1 → "1001").

Query plan evidence

Postgres 16, 5M rows across 20M ledgers (204 partitions), 20 contracts, 8 topics. Latency measured with pgbench (warm connection, extended protocol).

Range scan Before After Change
contract + topic, 100k ledgers, page of 1000 3.10 ms 0.26 ms −92%
contract count over 1M ledgers 19.9 ms 2.23 ms −89%
contract + topic count over 2M ledgers 92.4 ms 3.10 ms −97%
reconciliation fetch (2,500 full rows) 9.25 ms 4.61 ms −50%
latest event (via event_logs_max_ledger()) 0.18 ms 0.71 ms +0.5 ms

The reconciliation fetch is limited by reading the rows themselves; an extra (contractId, ledger, txOrder) index made no difference, so none was added. Buffer reads for the 1M-ledger count dropped from 8,902 to 290.

Migration checks on real Postgres: fresh DB, a table created by TypeORM synchronize, a 5M-row table (up → down → up, counts verified), out-of-range inserts parked in the default partition and moved by maintenance. TypeORM synchronize produces 0 statements against the partitioned table.

Testing

  • pnpm --filter backend test: 64 suites, 907 tests passing. indexer.service.ts coverage is 98% lines / 85% branches.
  • pnpm --filter backend build passes.

Notes for reviewers

  • The existing migration chain fails on a brand-new database at CreateFollowsTable1760000000000 (missing walletAddress column). This predates the PR; the new migrations were tested on their own.
  • New env vars: SOROBAN_CONTRACT_KIND, SOROBAN_CONTRACT_ERROR_MAP, EVENT_LOG_PARTITION_LOOKAHEAD (default 2).

…nt_logs

BE-004: Soroban RPC diagnostic event & XDR error code parser
- DiagnosticParserService decodes diagnostic events, TransactionResult,
  TransactionMeta and simulation error strings into structured reports
  (contract error variant, auth failure, footprint/budget exhaustion,
  archived entries); never throws on malformed XDR
- contract error registry generated from packages/contracts/*/src/errors.rs
  (20 contracts, 290 codes) with a sync test against the Rust sources
- failed transactions are logged, persisted to failed_transactions and the
  submitting user is notified once (TRANSACTION_FAILED)

BE-003: range partitioning of event_logs by ledger sequence
- PartitionEventStore migration: 100k-ledger partitions, default partition,
  (contractId, topic0, ledger) composite index, verified zero-loss copy of
  existing rows and a lossless down()
- event_logs_maintain_partitions() run by EventLogPartitionService on boot
  and every 10 minutes (and pg_cron when installed)
- event_logs_max_ledger() helper keeps the indexer's latest-ledger lookup
  fast on the partitioned table
- EventStoreService.getRawEventsByLedgerRange() range scan with keyset
  pagination; reconciliation now pushes its ledger range into SQL

Also fixes resolveStartLedger concatenating the bigint ledger string.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@drips-wave

drips-wave Bot commented Sep 27, 2026

Copy link
Copy Markdown

@feyishola Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@PeterOche
PeterOche merged commit 464e9b3 into degenspot:main Sep 27, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants