supplyverifier: add pull syncer and non-universe node verification

[ffranr]

Extend the supplyverifier syncer to support pulling supply commitments from universe servers.

Add a supplyverifier state machine so that non-universe tapd nodes (regular node) can verify supply commitments by:

• Watching for on-chain spends
• Pull-syncing from universe servers

Fixes #1452
Fixes #1463

[Roasbeef]

Can be rebased now.

[Roasbeef]

Seems like this could be just fn.Result?

[Roasbeef]

Style nit: can move this one declaration below, to sit right above the PullSupplyCommitment method.

[ffranr]

Seems like this could be just fn.Result?

My intent is to keep both the fetch results (FetchResult) and a map of errors (ErrorMap) in the same structure, since there can be one error per universe server sync target. In that sense, it’s a custom alternative to fn.Result. I considered using an Either, but here both FetchResult and ErrorMap may be populated at the same time. But maybe the syncer doesn't need to return so much?

[Roasbeef]

This will likely trigger the "concurrent write to map" race condition error.

An alternative here would be to use a buffered results channel.

[ffranr]

Well spotted, thanks!

For now I'll add a mutex (because it seems like the smallest change). But I'll look into expanding ParSliceErrCollect with the buffered results channel like you suggest. Sounds like a nice addition.

[Roasbeef]

If I understand your future plans correctly, then another way we'd be able to determine this is if is ours or not is via the presence of an entry in the new pre commitment table.

[ffranr]

Yes, or through mint batch or seedling lookup. But with a potential distant recovery mode in mind, it might be better to rely solely on key access as a design principle. I haven’t thought that through in detail yet.

[Roasbeef]

So this is just a temp state? Or a way to move forward the state machine if we detect a spend right as it's being created?

[ffranr]

The high-level architecture I have in mind is:

• The state machine watches on-chain outpoints for spends.
• A spend triggers SyncVerifyState.SpendEvent.
• SpendEvent simply applies a delay before transitioning to SyncVerifyState.SyncVerifyEvent. The delay is needed because after confirmation, the issuer peer must publish the new supply commitment to the universe server, and the server needs time to verify and store it.
• After the delay, SyncVerifyState.SyncVerifyEvent can sync and verify immediately, with no further waiting.

[coveralls]

Pull Request Test Coverage Report for Build 17652734676

Details

• 710 of 1110 (63.96%) changed or added relevant lines in 22 files are covered.
• 71 unchanged lines in 18 files lost coverage.
• Overall coverage increased (+0.2%) to 57.142%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
universe/supplyverifier/verifier.go	9	10	90.0%
mssmt/node.go	8	10	80.0%
mssmt/proof.go	21	23	91.3%
tapcfg/server.go	3	6	50.0%
universe/archive.go	8	11	72.73%
universe/supplyverifier/events.go	0	4	0.0%
universe/supplycommit/env.go	16	23	69.57%
universe/supplycommit/mock.go	0	7	0.0%
universe/supplycommit/manager.go	23	33	69.7%
tapdb/supply_commit.go	37	51	72.55%

Files with Coverage Reduction	New Missed Lines	%
universe/supplyverifier/manager.go	1	63.28%
address/mock.go	2	96.2%
asset/group_key.go	2	72.15%
fn/iter.go	2	62.07%
tapdb/mssmt.go	2	90.45%
universe/supplycommit/env.go	2	74.3%
universe/supplycommit/transitions.go	3	84.64%
authmailbox/receive_subscription.go	4	77.89%
rpcserver.go	4	61.29%
tapdb/multiverse.go	4	79.52%

Totals
Change from base Build 17628726079:	0.2%
Covered Lines:	64058
Relevant Lines:	112103

---

💛 - Coveralls

[jtobin]

LGTM -- I think subsequent refinement can be made in other PR's. 👍

[Roasbeef]

Love how we can view the final generated schema for diffs (in addition to the migration ofc), makes larger migrations like this easier to review.

Alternatively, we could just change the existing versions in place....as this was just in master.

[Roasbeef]

👍

[Roasbeef]

Are you aware that we already have this?

taproot-assets/mssmt/encoding.go

Lines 70 to 100 in 8519931

	// Decode decodes the compressed proof encoded within Reader.
	func (p *CompressedProof) Decode(r io.Reader) error {
	var numNodes uint16
	if err := binary.Read(r, byteOrder, &numNodes); err != nil {
	return err
	}
	nodes := make([]Node, 0, numNodes)
	for i := uint16(0); i < numNodes; i++ {
	var keyBytes [sha256.Size]byte
	if _, err := r.Read(keyBytes[:]); err != nil {
	return err
	}
	var sum uint64
	if err := binary.Read(r, byteOrder, &sum); err != nil {
	return err
	}
	nodes = append(nodes, NewComputedNode(NodeHash(keyBytes), sum))
	}
	var bitsBytes [MaxTreeLevels / 8]byte
	if _, err := r.Read(bitsBytes[:]); err != nil {
	return err
	}
	bits := UnpackBits(bitsBytes[:])
	*p = CompressedProof{
	Bits: bits,
	Nodes: nodes,
	}
	return nil
	}

[Roasbeef]

Ah ok, yeah it's just an integer index, it's a map key. I think it's fine as is.

[Roasbeef]

A ref to come back to: this needs to spend the spent pre commits in the DB.

[Roasbeef]

Why do we need this sleep at all? Is it just an itest artefact?

[Roasbeef]

Alternatively, we can implement a delay/retry loop when we go back to the sync verify state if/when we fail to fetch the supply commit. We also have this handy function:

taproot-assets/fn/retry.go

Lines 37 to 86 in b25c053

	// RetryFuncN executes the provided function with exponential backoff retry
	// logic. This is particularly useful for RPC calls in load-balanced
	// environments where nodes may temporarily return inconsistent results. The
	// function respects context cancellation and returns immediately if the context
	// is cancelled.
	func RetryFuncN[T any](ctx context.Context,
	config RetryConfig, fn func() (T, error)) (T, error) {
	var (
	result T
	err error
	)
	backoff := config.InitialBackoff
	// We'll retry the function up to MaxRetries times, backing off each
	// time until it succeeds.
	for attempt := 0; attempt <= config.MaxRetries; attempt++ {
	result, err = fn()
	if err == nil {
	return result, nil
	}
	if attempt == config.MaxRetries {
	return result, err
	}
	// Cap the backoff at the configured maximum to prevent
	// excessive delays.
	if backoff > config.MaxBackoff {
	backoff = config.MaxBackoff
	}
	// Wait for the backoff duration or until the context is
	// cancelled, whichever comes first.
	select {
	case <-ctx.Done():
	return result, ctx.Err()
	case <-time.After(backoff):
	// Apply the multiplier to implement exponential
	// backoff.
	backoff = time.Duration(
	float64(backoff) * config.BackoffMultiplier,
	)
	}
	}
	return result, err
	}

[Roasbeef]

Looks like we don't ever read or act on this field.

[Roasbeef]

It doesn't look like we're handling the case of many of the pre commitments being spent. Or rather it can cause us to run the verification loop many times, potentially inserting the same commitment several times. Doesn't look dire, but I think we can tighten up the logic here.

[Roasbeef]

👍

[Roasbeef]

We should wrap this in a helper assertion (optional VerifyFirst) and apply it to the other test as well, since that creates several supply commitments.