pc/wallet rescan

[paolocappelletti]

This PR adds reindex feature to a sidechain node.
The reindex resets state and wallet (except the secret storage) to the initial state , then all the blocks from the history are applied again one by one to them.
Reindexing is performed in an asynch way, so that the node is not blocked during the process.
All "write" operations (send of transactions, forging, tx or blocks received by other nodes) are not executed during the reindex.
All "read" operations, including sending local blocks to other peers, are allowed.

Two new endpoints has been added to the wallet/ , to launch the reindex and get info about its progress.
Furthermore, a new debug endpoint is added only in regtest, to allow to perform step by step the reindex, for debug pourposes

If the node is stopped during reindex, it will restart from the reached height at the next restart.

Mode details here: https://www.notion.so/horizen-labs/Wallet-rescan-d2ffe9298bfd468681088977cb4095d4

related PR: HorizenOfficial/developer-horizen-global#69

[sgonta]

Could you please mark variable, what is null?

[paolocappelletti]

the null variable is the new SidechainNodeViewReindexer, since it is not used in this test I did not mocked it

[i-Alex]

I guess Sergiy meant something like val sidechainWalletApiRoute: Route = SidechainWalletApiRoute(mockedRESTSettings, mockedSidechainNodeViewHolderRef, sidechainNodeViewReindexer=null, sidechainSecretsCompanion).route
to clearly understand what was nullified.

[MarcoOl94]

It looks good. Just few comments:

1. Maybe it can be good to check if the reindex is in progress also in the endpoints that use the StateStorage (e.g. hashCeased, getSidechainBlockIdForBackup).
2. There can be issues in case of having a reindex in progress at the same time of a Certificate generation?
3. Do we want to add also an optional parameter to the endpoints importSecret and importSecrets that can launch automatically a reindex (defualt=False)?

[i-Alex]

Tricky feature, a lot of curvy points there.
The overall design is very good!

[i-Alex]

I guess Sergiy meant something like val sidechainWalletApiRoute: Route = SidechainWalletApiRoute(mockedRESTSettings, mockedSidechainNodeViewHolderRef, sidechainNodeViewReindexer=null, sidechainSecretsCompanion).route
to clearly understand what was nullified.

[paolocappelletti]

hope to have collected all the cases.. see also here @i-Alex :
https://docs.google.com/document/d/1Ipn4umbwVqJep-UfS5gdM7HSKnbCcK6JVGiJRLBIy8c/edit#