Verify sharding support

[ahtotruu]

Verify that the BFT core, as currently implemented, can support sharded partitions and includes valid shard tree certificates in the responses returned to the shard validators.

Starting and running a partition in pre-defined sharding configuration is sufficient for now, splitting and joining of shards in an existing partition is not required for the MVP.

[lploom]

Issue 1

The root block proposal fails because the leader, while constructing the block proposal message for the current round, calls IsValid() on its own proposal. This validation fails because only one InputRecordChangeRequest is allowed per partition.

err="invalid proposal: invalid block: invalid payload: duplicate requests for partition 00000003"

func (x *Payload) IsValid() error {
	// There can only be one request per partition identifier in a block
	sysIdSet := map[types.PartitionID]struct{}{}

	for _, req := range x.Requests {
		if err := req.IsValid(); err != nil {
			return fmt.Errorf("invalid IR change request for %s: %w", req.Partition, err)
		}
		// Timeout requests do not contain proof
		if req.CertReason == T2Timeout && len(req.Requests) > 0 {
			return fmt.Errorf("partition %s timeout proof contains requests", req.Partition)
		}
		if _, found := sysIdSet[req.Partition]; found {
			return fmt.Errorf("duplicate requests for partition %s", req.Partition)
		}
		sysIdSet[req.Partition] = struct{}{}
	}
	return nil
}

Meanwhile, the InputRecordChangeRequestBuffer correctly allows partition + shard-level granularity:

type IrReqBuffer struct {
	irChgReqBuffer map[types.PartitionShardID]*irChange
	log            *slog.Logger
}

type PartitionShardID struct {
	PartitionID PartitionID
	ShardID     string // types.ShardID is not comparable; use ShardID.Key() here
}

Issue 2

If the above issue is fixed, it is possible to extract and verify UCs for both shards. However, UC.Verify is missing a ShardID check, there is even a TODO comment:

// TODO: verify shardID also
func (x *UnicityCertificate) Verify(tb RootTrustBase, algorithm crypto.Hash, partitionID PartitionID, shardID ShardID, shardConfHash []byte) error {
	if err := x.IsValid(partitionID, shardConfHash); err != nil {
		return fmt.Errorf("invalid unicity certificate: %w", err)
	}

	shardTreeRoot, err := x.ShardTreeCertificate.ComputeCertificateHash(x.InputRecord, x.TRHash, x.ShardConfHash, algorithm)
	if err != nil {
		return err
	}
	unicityTreeRoot, err := x.UnicityTreeCertificate.EvalAuthPath(shardTreeRoot, algorithm)
	if err != nil {
		return fmt.Errorf("evaluating unicity tree certificate: %w", err)
	}
	rootHash := x.UnicitySeal.Hash
	if !bytes.Equal(unicityTreeRoot, rootHash) {
		return fmt.Errorf("unicity seal hash %X does not match with the root hash of the unicity tree %X", rootHash, unicityTreeRoot)
	}

	if err := x.UnicitySeal.Verify(tb); err != nil {
		return fmt.Errorf("verifying unicity seal: %w", err)
	}
	return nil
}