-
Notifications
You must be signed in to change notification settings - Fork 2
Description
Problem Overview
In pallet-revive's Ethereum compatibility mode, the gas estimate returned by eth_estimateGas (DRY_RUN) is insufficient when the actual transaction executes (REAL_TX), causing OutOfGas errors.
Environment Information
revive Version
polkadot-sdk commit: 1a788e01360730c36db448194bf8086a20e14b22
Test Project
- Project: aave-protocol-v2 (adapted for revm)
- Repository: https://github.com/papermoonio/aave-protocol-v2-revm
Reproduction Steps
1. Environment Setup
# Install nvm (if not already installed)
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash
# Reload shell configuration
source ~/.nvm/nvm.sh
# Install and use Node.js 18
nvm install 18
nvm use 18
# Verify Node.js version
node --version # Should display v18.x.x2. Clone and Compile Project
# Clone project
git clone https://github.com/papermoonio/aave-protocol-v2-revm.git
cd aave-protocol-v2-revm
# Install dependencies
npm install
# Compile contracts
npm run compile3. Start Local Nodes
In terminal 1, start revive-dev-node:
revive-dev-node --devIn terminal 2, start eth-rpc:
eth-rpc --dev4. Run Tests
# Ensure Node.js 18 is being used
source ~/.nvm/nvm.sh && nvm use 18
# Run tests (will fail before fix)
TS_NODE_TRANSPILE_ONLY=1 npx hardhat --network local test \
./test-suites/test-aave/__setup.spec.ts \
./test-suites/test-aave/weth-gateway.spec.ts \
--grep "stable WETH"Expected Result
Tests should pass, repayETH transaction executes successfully.
Actual Result (Before Fix)
Tests fail, repayETH transaction reverts due to OutOfGas:
Error: transaction failed [ See: https://links.ethers.org/v5-errors-CALL_EXCEPTION ]
(transactionHash="0x...", transaction={...}, receipt={"status":0,...}, code=CALL_EXCEPTION)
Hypothesis A: Timestamp/BlockNumber Difference (Primary Cause)
Investigation Process
Step 1: Observing the Phenomenon
When running Aave tests, repayETH transaction reports OutOfGas error:
Error: transaction failed [ See: https://links.ethers.org/v5-errors-CALL_EXCEPTION ]
(transactionHash="0x...", transaction={...}, receipt={"status":0,...}, code=CALL_EXCEPTION)
Initial observations:
- Gas value returned by
eth_estimateGasis insufficient to complete the actual transaction - Same transaction succeeds during estimation but fails during execution
Step 2: Log Comparison Analysis
Comparing DRY_RUN and REAL_TX execution through node logs (node.log):
[DRY_RUN] dry_run_eth_transact:
input: 0x02c5fcf8... (repayETH)
gas_limit: 18446744073709551615 (unlimited)
Result: Success, returns estimate 987,968
[REAL_TX] bare_call:
input: 0x02c5fcf8... (repayETH)
gas_limit: 987,968 (using estimated value)
Result: OutOfGas ❌
Key Finding: Same transaction input, DRY_RUN succeeds but REAL_TX fails.
Step 3: Deep Call Stack Analysis
Further analysis of output.log (30,710 lines containing complete opcode execution trace) reveals the call stack:
Frame#0: WETHGateway.repayETH()
│
├─> Frame#1: Helpers.getUserCurrentDebtMemory() [delegate_call]
│ │
│ └─> Frame#2: StableDebtToken.balanceOf(address)
│ │
│ └─> Internal call to calculateCompoundedInterest()
│ │
│ └─> Multiple calls to rayMul() ← Divergence happens here!
Step 4: Locating Opcode Execution Divergence
Using a Python script to compare DRY_RUN and REAL_TX opcode execution sequences line by line:
# Comparison results
First 1835 opcodes: Identical ✅
Opcode #1836 (PC 6747 JUMPI): Divergence point ❌
- DRY_RUN: Jump not taken → PC 6748
- REAL_TX: Jump taken → PC 6763Precise Divergence Point Location:
PC 6740: JUMPDEST ; Code block entry
PC 6741: SWAP1
PC 6742: POP
PC 6743: DUP1 ; Duplicate top stack element (condition value)
PC 6744: PUSH2 0x1a6b ; Push jump target (6763)
PC 6747: JUMPI ; ← Divergence point! Conditional jump
| Execution Mode | Condition Value | Jump? | Next PC |
|---|---|---|---|
| DRY_RUN | 0 | ❌ No jump | 6748 |
| REAL_TX | Non-zero | ✅ Jump | 6763 |
Step 5: Bytecode Reverse Engineering
Reverse engineering confirmed PC 6747 corresponds to Solidity code:
// WadRayMath.sol - rayMul function
function rayMul(uint256 a, uint256 b) internal pure returns (uint256) {
if (a == 0 || b == 0) { // ← PC 6747 JUMPI
return 0; // ← DRY_RUN takes this path
}
// ← REAL_TX takes this path (full overflow check)
require(a <= (type(uint256).max - halfRAY) / b, ...);
return (a * b + halfRAY) / RAY;
}Question: Why does the same function call have different values for a?
Step 6: Tracing the Source of Value a
The parameter a for rayMul comes from the result of calculateCompoundedInterest:
// MathUtils.sol
function calculateCompoundedInterest(uint256 rate, uint40 lastUpdateTimestamp, uint256 currentTimestamp) {
uint256 exp = currentTimestamp.sub(uint256(lastUpdateTimestamp));
// ↑ exp value depends on block.timestamp!
if (exp == 0) {
return WadRayMath.ray(); // Returns 1e27
}
// ... calculate compound interest using exp
uint256 basePowerTwo = ratePerSecond.rayMul(ratePerSecond);
// ↑ If exp=0, this might return 0
}Key Finding: The value of exp depends on block.timestamp!
Step 7: Verifying Timestamp Difference
Checking TIMESTAMP opcode (0x42) execution:
DRY_RUN: TIMESTAMP @ PC 5321 returns value = T1 (old block time)
REAL_TX: TIMESTAMP @ PC 5321 returns value = T2 (new block time, T2 > T1)
Causal chain confirmed:
DRY_RUN: timestamp = T1 (old) → exp = T1 - lastUpdate ≈ 0 → rayMul(0, x) → simplified path
REAL_TX: timestamp = T2 (new) → exp = T2 - lastUpdate > 0 → rayMul(non-zero, x) → full path
Step 8: Locating polkadot-sdk Code Issue
Tracing the timestamp source, located in eth_estimateGas implementation:
File: substrate/frame/revive/rpc/src/lib.rs
async fn estimate_gas(&self, transaction: GenericTransaction, block: Option<BlockNumberOrTag>) {
let block = block.unwrap_or_default(); // ← Problem is here!
// ...
}Checking BlockTag default value:
File: substrate/frame/revive/src/evm/api/rpc_types_gen.rs
pub enum BlockTag {
Earliest,
Finalized,
Safe,
#[default]
Latest, // ← Default is Latest, not Pending!
Pending,
}Continuing to trace dry_run function:
File: substrate/frame/revive/rpc/src/client/runtime_api.rs
pub async fn dry_run(&self, tx: GenericTransaction, block: BlockNumberOrTagOrHash) {
let timestamp_override = match block {
BlockNumberOrTagOrHash::BlockTag(BlockTag::Pending) =>
Some(Timestamp::current().as_millis()), // ← Only Pending sets this!
_ => None, // ← Latest goes here, timestamp_override = None
};
}Finally confirmed the issue in Stack::new():
File: substrate/frame/revive/src/exec.rs
let mut timestamp = T::Time::now();
// Only updates when timestamp_override exists
if let Some(timestamp_override) = exec_config.is_dry_run.as_ref().and_then(|cfg| cfg.timestamp_override) {
block_number = block_number.saturating_add(1u32.into());
timestamp = cmp::max(timestamp.saturating_add(delta), timestamp_override);
}
// If timestamp_override = None, uses old block's timestamp!Root Cause Summary
eth_estimateGas called without block parameter
↓
block.unwrap_or_default() → BlockTag::Latest
↓
In dry_run: Latest → timestamp_override = None
↓
Stack::new(): Uses confirmed block's timestamp (old)
↓
TIMESTAMP opcode returns old value T1
↓
calculateCompoundedInterest: exp = T1 - lastUpdate ≈ 0
↓
rayMul(0, x) → simplified path → low gas estimate
↓
Actual transaction executes in new block, timestamp = T2 > T1
↓
exp > 0 → rayMul(non-zero, x) → full path → needs more gas
↓
Estimated gas < Actual gas → OutOfGas ❌
Opcode Execution Statistics
Frame#2 Local Statistics (balanceOf function)
| Metric | DRY_RUN | REAL_TX | Difference |
|---|---|---|---|
| Total opcodes | 342 | 752 | +410 (2.2x) |
| Unique PC count | 328 | 472 | +144 |
| child_weight_consumed | 270,718,988 | 292,477,708 | +21,758,720 |
| child_gas_consumed | 5,415 | 5,850 | +435 |
Complete Transaction Statistics (output.log analysis)
| Metric | DRY_RUN | REAL_TX | Difference |
|---|---|---|---|
| Total opcode executions | 11,684 | 13,918 | +2,234 (+19.1%) |
| PC 5326-5332 loop | 18 times | 43 times | +25 times |
| PC 7040-7047 execution | 0 times | 30 times | +30 times |
| JUMPI@6747 taken | 0 times | 5 times | +5 times |
| JUMPI@6747 not taken | 5 times | 0 times | -5 times |
POC Verification: Simplified Contract Reproduces Issue
To verify the correctness of Hypothesis A, we created a simplified POC contract to reproduce this issue.
POC Repository
- repo url:
https://github.com/papermoonio/time-sensitive-poc - Contract:
TimeSensitiveContract.sol - Test:
test/test-standard.ts
POC Design Principle
The POC contract simulates Aave's time-sensitive behavior:
// Core logic: Choose different execution paths based on timeDelta
if (timeDelta < 1) {
// Simple path: Only calculate base interest (what eth_estimateGas sees)
emit ZeroDeltaPath(msg.sender, timeDelta, gasleft());
newBalance = balance + baseInterest;
} else {
// Complex path: Execute loop calculation (what actual transaction executes)
uint256 iterations = timeDelta * 100;
emit NonZeroDeltaPath(msg.sender, timeDelta, iterations, gasleft());
// ... execute iterations loop calculations
}Running POC
cd time-sensitive-poc
pnpm install
npx hardhat compile
# Ensure revive-dev-node and eth-rpc are running
npx hardhat test --network localPOC Results
======================================================================
POC: eth_estimateGas Timestamp Mismatch Issue
======================================================================
Contract deployed: 0x5CC307268a1393AB9A764A20DACE848AB8275c46
Account initialized with 1000 ETH balance
──────────────────────────────────────────────────────────────────────
Phase 1: Gas Estimation (uses latest block timestamp)
──────────────────────────────────────────────────────────────────────
timeDelta seen by estimator: 0 seconds
Expected path: Simple (no loop)
Estimated gas: 307892
──────────────────────────────────────────────────────────────────────
Phase 2: Actual Transaction (executes in new block)
──────────────────────────────────────────────────────────────────────
Using estimated gas limit: 307892
Result: ❌ Failed - OutOfGas
──────────────────────────────────────────────────────────────────────
Phase 3: Retry with 150% gas limit
──────────────────────────────────────────────────────────────────────
Using gas limit: 461838
Result: ✅ Succeeded
Actual gas used: 401586
──────────────────────────────────────────────────────────────────────
Analysis: Timestamp Difference Impact
──────────────────────────────────────────────────────────────────────
Estimation phase:
- timeDelta: 0 seconds
- Path taken: Simple (no complex calculation)
Execution phase:
- timeDelta: 2 seconds
- Path taken: Complex (200 loop iterations)
Gas comparison:
- Estimated: 307892
- Actual: 401586
- Difference: +93694 (30.43%)
======================================================================
Conclusion
======================================================================
✅ Successfully reproduced the timestamp mismatch issue!
Root Cause:
┌─────────────────────────────────────────────────────────────────┐
│ eth_estimateGas uses BlockTag::Latest (old timestamp) │
│ → timeDelta = 0 → Simple path → Low gas estimate │
│ │
│ Actual transaction runs in new block (new timestamp) │
│ → timeDelta ≥ 1 → Complex path → Higher gas needed │
│ │
│ Result: Estimated gas < Actual gas → OutOfGas error │
└─────────────────────────────────────────────────────────────────┘
This is exactly what happens in Aave's calculateCompoundedInterest!
POC Verification Conclusion
| Metric | Estimation Phase | Execution Phase | Description |
|---|---|---|---|
| timeDelta | 0 seconds | 2 seconds | Caused by BlockTag::Latest |
| Execution Path | Simple path | Complex path (200 loops) | Path divergence |
| Gas Consumption | 307,892 | 401,586 | +30.43% |
| Transaction Result | - | OutOfGas | Fails when using estimated value |
POC successfully reproduced the issue found in Aave: eth_estimateGas uses BlockTag::Latest (old timestamp) for estimation, but the actual transaction executes in a new block (new timestamp), leading to different code paths and OutOfGas error.
Hypothesis B: Weight Limit Metering Difference (Secondary Cause)
Investigation Process
Step 1: Observing Abnormal Gas Allocation in Logs
While analyzing node.log, found significant differences in gas allocation between DRY_RUN and REAL_TX:
[DRY_RUN] delegate_call:
gas_left=18446744073708943984 ← Near uint64::MAX!
[REAL_TX] delegate_call:
gas_left=396339 ← Normal value
Anomaly: DRY_RUN gas value is abnormally large (~1.8×10¹⁹), while REAL_TX is normal.
Step 2: Tracing Gas Decrement Pattern in Nested Calls
Extracting gas values from multi-level nested calls reveals completely different decrement patterns:
| Call Level | DRY_RUN gas | REAL_TX gas | Notes |
|---|---|---|---|
| 1 | 18,446,744,073,708,977,836 | 430,191 | |
| 5 | 18,446,744,073,708,937,093 | 389,013 | |
| 10 | 18,446,744,073,708,840,927 | 292,376 | |
| 15 | 18,446,744,073,708,802,627 | 253,640 | |
| 20 | 18,446,744,073,708,744,410 | 194,970 | |
| 25 | 18,446,744,073,708,925,474 | 309,832 | With refund |
| 30 | 18,446,744,073,708,901,100 | 267,480 | |
| 31 | 18,446,744,073,708,893,387 | 238,971 | |
| 32 | (Completed successfully) | OutOfGas ❌ |
Key Observations:
- DRY_RUN: Gas at each level is near
uint64::MAX, barely decrements - REAL_TX: Gas decreases from 430,191 down to 238,971, eventually exhausted at level 32
Step 3: Searching Gas Allocation Related Code
Searching for maybe_weight_limit keyword, found the entry point difference:
File: substrate/frame/revive/src/lib.rs
// DRY_RUN - dry_run_eth_transact function
let transaction_limits = TransactionLimits::EthereumGas {
eth_gas_limit: call_info.eth_gas_limit.saturated_into(),
maybe_weight_limit: None, // ← No weight limit!
eth_tx_info: EthTxInfo::new(call_info.encoded_len, base_weight),
};
// REAL_TX - eth_call function
TransactionLimits::EthereumGas {
eth_gas_limit: eth_gas_limit.saturated_into(),
maybe_weight_limit: Some(weight_limit), // ← Constrained by weight_limit!
eth_tx_info: EthTxInfo::new(encoded_len, extra_weight),
}Finding: DRY_RUN uses maybe_weight_limit: None, REAL_TX uses Some(weight_limit).
Step 4: Analyzing weight_limit Impact on Gas Calculation
Tracing weight_left calculation logic:
File: substrate/frame/revive/src/metering/math.rs
pub fn weight_left<T: Config, S: State>(...) -> Option<Weight> {
let unbounded_weight_left = eth_tx_info.weight_remaining(...)?;
// Critical branch!
Some(match meter.weight.weight_limit {
Some(weight_limit) => unbounded_weight_left.min(
weight_limit.checked_sub(&self_consumed_weight)?
),
None => unbounded_weight_left, // ← DRY_RUN takes this, returns unlimited
})
}Impact:
weight_limit = None→ Returnsunbounded_weight_left(nearly unlimited)weight_limit = Some(X)→ Returnsmin(unbounded, X - consumed)(constrained)
Step 5: Analyzing Nested Call Propagation Rules
Continuing to trace weight_limit propagation in nested calls:
File: substrate/frame/revive/src/metering/math.rs
pub fn new_nested_meter<T: Config, S: State>(...) {
let nested_weight_limit =
if meter.weight.weight_limit.is_none() {
None // Parent None → Child None
} else {
Some(weight_left) // Parent has limit → Child inherits remaining
};
}Propagation Rules:
- DRY_RUN: Parent
None→ All child calls areNone→ Never constrained - REAL_TX: Parent
Some(X)→ Child calls inherit remaining value → Decreases at each level
Root Cause Summary
DRY_RUN (maybe_weight_limit = None):
Root: weight_limit = None
↓
Call 1: nested_weight_limit = None (unconstrained)
↓
Call 2: nested_weight_limit = None (unconstrained)
↓
Call N: nested_weight_limit = None (always unconstrained) ✅ Success
REAL_TX (maybe_weight_limit = Some(X)):
Root: weight_limit = X
↓ consumed_1
Call 1: nested_weight_limit = min(unbounded, X - consumed_1)
↓ consumed_2
Call 2: nested_weight_limit = min(unbounded, prev - consumed_2)
↓ consumed_3
Call N: nested_weight_limit keeps decreasing... may < needed → OutOfGas! ❌
Problem Summary
Comparison Summary
| Dimension | Hypothesis A: Timestamp Difference (Primary) | Hypothesis B: Weight Limit Difference (Secondary) |
|---|---|---|
| Problem Level | eth-rpc → pallet-revive → EVM | pallet-revive metering layer |
| Direct Cause | BlockTag::Latest vs new block | maybe_weight_limit set differently |
| Impact Mechanism | Different timestamp → different calculation results → different code branches | Different weight decrement rules, differences accumulate |
| Quantified Impact | 2.2x opcode difference, +21.7M ref_time | Gas exhausted after 32 nested levels |
| Verification Status | ✅ Verified and fixed | 🟡 Pending independent verification |
Compound Effect Model
┌─────────────────────────────────────────────────────────────────────────┐
│ Problem Compound Model │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ DRY_RUN Phase: │
│ • BlockTag::Latest → timestamp = old value (Hypothesis A) │
│ • maybe_weight_limit = None → unconstrained (Hypothesis B) │
│ • Takes simplified path (342 opcodes) │
│ • Returns estimate: 987,968 gas │
│ │
│ REAL_TX Phase: │
│ • New block → timestamp = new value → different code path (A) │
│ • maybe_weight_limit = Some(X) → constrained, differences │
│ accumulate (Hypothesis B) │
│ • Takes full path (752 opcodes, 2.2x) │
│ • Actual need > estimate → OutOfGas │
│ │
│ Compound Effect: │
│ Hypothesis A causes: More opcodes executed (+410) │
│ Hypothesis B causes: Insufficient gas in deep calls │
│ Combined: OutOfGas triggered earlier and more severely │
│ │
└─────────────────────────────────────────────────────────────────────────┘
Verification Status
| Hypothesis | Status | Verification Method | Conclusion |
|---|---|---|---|
| A: Timestamp Difference | ✅ Verified and Fixed | Modified dryrun timestamp logic | All 44 identical tests pass |
| B: Weight Limit Difference | 🟡 Pending Independent Verification | Requires separate modification | Secondary factor |
Impact
- Gas value returned by
eth_estimateGasis unreliable for time-sensitive contracts - DeFi protocols with interest calculations (like AAVE) cannot execute normally
- Problem is more severe for deeply nested calls
- Users submitting transactions with estimated values will get OutOfGas
Suggested Fix Solutions
Solution 1: Change estimate_gas Default Block Tag
For Hypothesis A - Alternative Solution
// substrate/frame/revive/rpc/src/lib.rs
async fn estimate_gas(
&self,
transaction: GenericTransaction,
block: Option<BlockNumberOrTag>,
) -> RpcResult<U256> {
// Use Pending instead of Latest as default
let block = block.unwrap_or(BlockNumberOrTag::Tag(BlockTag::Pending));
// ...
}Fix Verification Results
Using Solution 1, running test suite after fix:
| Test File | Test Count | Status |
|---|---|---|
| weth-gateway.spec.ts | 11 | ✅ Pass |
| flashloan.spec.ts | 17 | ✅ Pass |
| liquidation-atoken.spec.ts | 6 | ✅ Pass |
| liquidation-underlying.spec.ts | 6 | ✅ Pass |
| atoken-transfer.spec.ts | 4 | ✅ Pass |
| Total | 44 | ✅ All Pass |
Solution 2: Add Safety Factor
Universal Solution
// Add buffer when returning estimate
let estimated_gas = dry_run_result.gas_used;
let safe_gas = estimated_gas * 150 / 100; // 1.5x safety factor