Long open time Bitcoin Core Qt

[mtmtmt123]

Please make a new version to fix it , long or no open

[hidaloop]

Hello! Please sorry for such a long response time.
Could you please a couple of questions (if you are still here).

• Was it your very first time launching a Qt wallet?
• Was it your consecutive opening after initial syncronization?

Looking forward to hearing back. Cheers.

[hidaloop]

We have attempted to fix long chain scanning time. Currently the fix is obtc-nightly branch.
The first launch of the program after a fresh rebuild is not immediate, but further once indexation happens it should work faster. If you have any issues with builds please let us know.

[asof]

This is because the function used to calculate whether a matrix has full rank is very slow:
https://github.com/PoWx-Org/obtc-core/blob/master/src/util/matrixchecks.cpp#L17
bool heavyhash::checks::IsFullRank(const uint64_t matrix_[64*64])

You should consider using the function is_full_rank in the code of the miner, which is much faster:
https://github.com/PoWx-Org/obtc-miner/blob/master/algo/heavyhash/heavyhash-gate.c#L79
static inline bool is_full_rank(const uint16_t matrix[64][64])

[hidaloop]

Hello @asof! Thank you for your note.
The reason we went for rank revealing SVD is the numerical stability, which is really important for core implementation. Whereas Gaussian elimination is less numerically stable.

I also wanted to kindly ask if you have any proofs, tests, benchmarks that showcase your claim? Because from theoretical point of view both algorithms for nxn matricies are of O(n^3) complexity.

Although, it would be nice if you could present some proofs, it feels that performing matrix checks in general may be wasteful for old blocks. The term 'old' may be quite arbitrary, as long as security and consensus are guaranteed. So I think researching this issue is a very feasible task for the further round of updates.
Thank you for noting once again.

In the meantime, I cordially invite you to test an implementation of PoW cache database, that we pushed to the obtc-nightly branch, as it may speed the work of your wallet greatly!

[asof]

The Gaussian elimination function finished in 0.116 ms while the SVD version needed 19.5 ms.
Benchmark code: https://github.com/asof/discuss/blob/main/IsFullRank/IsFullRank.cpp

The obtc-nightly branch runs well on my PC. It starts instantly and takes a little longer to shut down.

[hidaloop]

Thank you very much for putting work and generating the results, @asof! This is a substantial difference of x200.

However, as I said Gaussian elimination can be unstable:

One possible problem is numerical instability, caused by the possibility of dividing by very small numbers. If, for example, the leading coefficient of one of the rows is very close to zero, then to row-reduce the matrix, one would need to divide by that number. This means that any error existed for the number that was close to zero would be amplified. (Source)

Such instability can result in potential loop holes - where a miner exploits the non-full rank property of a matrix and does less computation, and another one is the different results on different machines which leads to non-unique hashes.
And from this point of view consensus is more important. Therefore, we probably will not immediately change the algorithm, but definitely look into a more efficient way of managing matrix checks.

The obtc-nightly branch runs well on my PC. It starts instantly and takes a little longer to shut down.

Thank you for checking and validating. The shutdown is slower due to an extra leveldb closing gracefully. For now this will be the aid for the long start-up time and re-scanning. But again, we will push a patch on matrix checks after some extra research.

I hope that this concludes the thread on the long open time for the Qt wallet.

[hidaloop]

@asof, can I kindly ask you to open another issue that is specifically devoted for long rank checking algorithm?
That would be really great. Cheers!