feat(sync-sim): add SimStorage with real Merkle tree for protocol testing

[xilosada]

Summary

Adds SimStorage infrastructure to the simulation framework, replacing the flat DigestCache HashMap with a real Merkle tree implementation. This enables accurate simulation of sync protocols that depend on tree structure (e.g., HashComparison).

Key changes:

• Add SimStorage with in-memory Merkle tree using Store + InMemoryDB
• Add RuntimeEnv bridge to connect storage Key operations
• Update SimNode to use hybrid storage: real tree + metadata cache
• Add insert_entity_hierarchical() for creating proper tree depth
• Make Index::get_index() and get_children_of() public for traversal
• Add tree structure verification tests for protocol selection

Entity semantics improvements:

• Entity counting now correctly excludes intermediate tree nodes
• iter_entities() returns only "real" entities (with metadata)
• has_entity() checks metadata cache, not storage (excludes intermediate nodes)

Protocol selection now uses real tree depth:

• SubtreePrefetch scenarios have max_depth > 3
• LevelWise scenarios have max_depth <= 2

Test Plan

• All 203 sync_sim tests pass
• Tree structure verification tests added
• Hierarchical insertion tests verify intermediate nodes aren't counted
• Protocol negotiation compliance tests still pass

Spec Reference

• Simulation Framework Spec §5: In-memory Storage
• Simulation Framework Spec §7: State Digest and Hashing
• Simulation Framework Spec §11: HashComparison Protocol

---

Note

Medium Risk
Moderate risk because it changes core simulation state/digest semantics and introduces new tree-building logic, plus exposes additional calimero-storage::Index APIs that could affect downstream users.

Overview
Simulation state now uses a real Merkle tree. SimNode replaces the DigestCache/HashMap-backed model with a new SimStorage (in-memory Store + InMemoryDB bridged via RuntimeEnv) and derives state_digest/root_hash from the storage root hash.

Entity semantics were tightened. The simulation now tracks “real” entities via a separate metadata cache, adds iter_entities()/entity_ids() and hierarchical insertion (insert_entity_hierarchical) to create intermediate tree nodes without counting them as entities; scenarios/assertions were updated accordingly and new tests verify depth-sensitive protocol negotiation.

Storage traversal support was opened up. crates/storage makes Index::get_index() and Index::get_children_of() public and adds EntityIndex accessor methods to support Merkle tree traversal from the simulation.

^{Written by Cursor Bugbot for commit 22815bb. This will update automatically on new commits. Configure here.}

[meroreviewer]

🤖 AI Code Reviewer

Reviewed by 3 agents | Quality score: 95% | Review time: 194.8s

💡 4 suggestions, 📝 3 nitpicks. See inline comments.

---

_{🤖 Generated by AI Code Reviewer | Review ID: review-d5aafdf7}

[meroreviewer]

🤖 AI Code Reviewer

Reviewed by 3 agents | Quality score: 92% | Review time: 265.6s

💡 6 suggestions, 📝 2 nitpicks. See inline comments.

---

_{🤖 Generated by AI Code Reviewer | Review ID: review-4c66ded3}

[meroreviewer]

🤖 AI Code Reviewer

Reviewed by 3 agents | Quality score: 100% | Review time: 229.6s

🟡 1 warnings, 💡 3 suggestions. See inline comments.

---

_{🤖 Generated by AI Code Reviewer | Review ID: review-ae5b05a0}

[meroreviewer]

🤖 AI Code Reviewer

Reviewed by 3 agents | Quality score: 89% | Review time: 235.6s

🟡 1 warnings, 💡 3 suggestions, 📝 2 nitpicks. See inline comments.

---

_{🤖 Generated by AI Code Reviewer | Review ID: review-32829545}

[cursor]

Cursor Bugbot has reviewed your changes and found 2 potential issues.

^{Bugbot Autofix is ON. A Cloud Agent has been kicked off to fix the reported issues.}

[cursor]

Bugbot Autofix prepared fixes for 2 of the 2 bugs found in the latest run.

• ✅ Fixed: Newly added entity_ids() method is never called
  • Removed the unused entity_ids() method from SimNode as it was dead code with no callers in the codebase.
• ✅ Fixed: Inconsistent entity existence checks across query methods
  • Updated has_entity() and entity_count() to check both entity_metadata AND storage data, ensuring consistency with get_entity() and iter_entities().

Or push these changes by commenting:

@cursor push f96fe4800f