Skip to content

Fix fee estimation APIs: bounded waitTime estimate, consistent with getFee (#1884) - #2450

Open
Ergologica wants to merge 3 commits into
ergoplatform:masterfrom
Ergologica:fix/fee-estimation-1884
Open

Fix fee estimation APIs: bounded waitTime estimate, consistent with getFee (#1884)#2450
Ergologica wants to merge 3 commits into
ergoplatform:masterfrom
Ergologica:fix/fee-estimation-1884

Conversation

@Ergologica

Copy link
Copy Markdown

Closes #1884

The problem

As reported in #1884, /transactions/waitTime could return absurd values (e.g. ~20 years for a 1 KB tx paying the minimal fee), while /transactions/getFee simultaneously recommended a lower fee for a 2-minute confirmation. Two causes:

  1. getExpectedWaitTime and getRecommendedFee were computed from unrelated data. getRecommendedFee uses the mempool fee histogram (average fee of transactions taken from the pool, bucketed by how long they waited), while getExpectedWaitTime used position_in_pool × elapsed / takenTxns, where elapsed is the time since the start of the statistics measurement. The two could not be consistent with each other.
  2. The position-based estimate was unbounded. As @pragmaxim noted in the issue, the statistics only advance when transactions are removed from the pool via ErgoMemPool#remove; when transactions get stuck, elapsed grows without limit and the estimate diverges (the "20 years" effect). Additionally, MemPoolStatistics.measurementIntervalMsec was set to 60 * 1000 (one minute) while the comment and the pruning logic (keep data up to 2*measurementIntervalMsec) clearly intend one hour.

The fix

  • getExpectedWaitTime now estimates the wait time from the same fee histogram used by getRecommendedFee: the earliest wait-time bin whose average fee per KB does not exceed the queried transaction's fee per KB. This makes the two endpoints mutually consistent: a transaction paying getFee(t) gets waitTime <= t (covered by a test).
  • When no suitable histogram statistics exist, the previous position-based estimate is used as a fallback, with elapsed bounded by the statistics measurement window (2 × measurementIntervalMsec), so periods with no transactions taken from the pool can no longer produce unbounded estimates.
  • measurementIntervalMsec fixed to one hour (60 * 60 * 1000), matching its documentation and the histogram design (60 one-minute bins).

Testing

Four new tests in ErgoMemPoolSpec (there was no fee-estimation coverage before, addressing the testing gap mentioned by @kushti in the issue):

  • getRecommendedFee returns the minimal fee when no statistics is collected;
  • getRecommendedFee returns the histogram average for the first non-empty bin;
  • consistency: a transaction paying the recommended fee for a t-minute confirmation gets an expected wait time <= t (and exactly the bin the recommendation came from);
  • boundedness: with a year of statistics inactivity and a non-empty pool, the estimate stays within the measurement window bound.

All tests pass locally with sbt "testOnly org.ergoplatform.nodeView.mempool.ErgoMemPoolSpec".

@Ergologica

Copy link
Copy Markdown
Author

Flagging this before a reviewer finds it and reasonably closes the PR: #1884 already has a merged PR saying Closes #1884#2410, "Fix fee estimation bounds", merged 2026-07-13, seventeen days before I opened this. I did not see it when I started.

Having read both diffs against each other, they are not the same change.

#2410 (ErgoMemPool.scala +5/-3) bounds each endpoint on its own:

  • floors /transactions/getFee at the configured node minimal fee, so it cannot recommend below the accepted minimum;
  • caps the wait-time calculation to the mempool statistics window, so a long-idle node does not dominate the estimate.

This PR (ErgoMemPool.scala +30/-11, MemPoolStatistics.scala +1/-1, +57 in ErgoMemPoolSpec) goes at the part #2410 did not touch: getExpectedWaitTime and getRecommendedFee are derived from unrelated data, so bounding them separately still lets them contradict each other. /transactions/waitTime can report an implausible wait while /transactions/getFee simultaneously recommends a lower fee for a two-minute confirmation — which is the second half of what @dev2dev0 reported.

The clearest evidence they are complementary rather than competing: this branch merges clean on current master, and current master already contains #2410. It is built on top of that work.

So my reading is that #1884 is in the same state as #1870 — a merged PR did part of it, the issue stayed open because part of it was left. But that is my reading of my own PR, so treat it accordingly; if you look and conclude #2410 was sufficient, say so and I will close this without argument.

cc @glasgowm148 for the registry side — reservation is ErgoDevs/Ergo-Bounties#47, and I have written the general version of this up as ErgoDevs/Ergo-Bounties#60, since it is not the only entry in that state.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fee estimation APIs getExpectedWaitTime/getRecommendedFee return invalid values

1 participant