File blockchain_header gets corrupted

[swat69]

Electrum version v4 alpha2 : #a53a8db

after a few minutes, the run_electrum ls logging this error in the console and the connection is lost to the server:

E/i | interface.[electrum.marpmedev.xyz:50002] | Exception in wrapper_func: TypeError("unsupported operand type(s) for -: 'int' and 'str'",)
Traceback (most recent call last):
File "/home/ben/electrum2/electrum/util.py", line 988, in wrapper
return await func(*args, **kwargs)
File "/home/ben/electrum2/electrum/interface.py", line 334, in wrapper_func
return await func(self, *args, **kwargs)
File "/home/ben/electrum2/electrum/interface.py", line 358, in run
await self.open_session(ssl_context)
File "/home/ben/electrum2/electrum/interface.py", line 465, in open_session
await group.spawn(self.monitor_connection)
File "/home/ben/.local/lib/python3.6/site-packages/aiorpcx/curio.py", line 242, in aexit
await self.join()
File "/home/ben/.local/lib/python3.6/site-packages/aiorpcx/curio.py", line 211, in join
raise task.exception()
File "/home/ben/electrum2/electrum/interface.py", line 502, in run_fetch_blocks
await self._process_header_at_tip()
File "/home/ben/electrum2/electrum/interface.py", line 514, in _process_header_at_tip
_, height = await self.step(height, header)
File "/home/ben/electrum2/electrum/interface.py", line 572, in step
return await self._resolve_potential_chain_fork_given_forkpoint(good, bad, bad_header)
File "/home/ben/electrum2/electrum/interface.py", line 622, in _resolve_potential_chain_fork_given_forkpoint
b = forkfun(bad_header) # type: Blockchain
File "/home/ben/electrum2/electrum/blockchain.py", line 264, in fork
self.save_header(header)
File "/home/ben/electrum2/electrum/blockchain.py", line 188, in func_wrapper
return func(self, *args, **kwargs)
File "/home/ben/electrum2/electrum/blockchain.py", line 447, in save_header
self.swap_with_parent()
File "/home/ben/electrum2/electrum/blockchain.py", line 357, in swap_with_parent
if not self._swap_with_parent():
File "/home/ben/electrum2/electrum/blockchain.py", line 375, in _swap_with_parent
if self.parent.get_chainwork() >= self.get_chainwork():
File "/home/ben/electrum2/electrum/blockchain.py", line 188, in func_wrapper
return func(self, *args, **kwargs)
File "/home/ben/electrum2/electrum/blockchain.py", line 565, in get_chainwork
work_in_single_header = self.chainwork_of_header_at_height(cached_height)
File "/home/ben/electrum2/electrum/blockchain.py", line 544, in chainwork_of_header_at_height
work = ((2 ** 256 - target - 1) // (target + 1)) + 1
TypeError: unsupported operand type(s) for -: 'int' and 'str'
E/i | interface.[electrum.marpmedev.xyz:50002] | Exception in wrapper_func: AssertionError((0, 1),)
Traceback (most recent call last):
File "/home/ben/electrum2/electrum/util.py", line 988, in wrapper
return await func(*args, **kwargs)
File "/home/ben/electrum2/electrum/interface.py", line 334, in wrapper_func
return await func(self, *args, **kwargs)
File "/home/ben/electrum2/electrum/interface.py", line 358, in run
await self.open_session(ssl_context)
File "/home/ben/electrum2/electrum/interface.py", line 465, in open_session
await group.spawn(self.monitor_connection)
File "/home/ben/.local/lib/python3.6/site-packages/aiorpcx/curio.py", line 242, in aexit
await self.join()
File "/home/ben/.local/lib/python3.6/site-packages/aiorpcx/curio.py", line 211, in join
raise task.exception()
File "/home/ben/electrum2/electrum/interface.py", line 502, in run_fetch_blocks
await self._process_header_at_tip()
File "/home/ben/electrum2/electrum/interface.py", line 514, in _process_header_at_tip
_, height = await self.step(height, header)
File "/home/ben/electrum2/electrum/interface.py", line 572, in step
return await self._resolve_potential_chain_fork_given_forkpoint(good, bad, bad_header)
File "/home/ben/electrum2/electrum/interface.py", line 622, in _resolve_potential_chain_fork_given_forkpoint
b = forkfun(bad_header) # type: Blockchain
File "/home/ben/electrum2/electrum/blockchain.py", line 264, in fork
self.save_header(header)
File "/home/ben/electrum2/electrum/blockchain.py", line 188, in func_wrapper
return func(self, *args, **kwargs)
File "/home/ben/electrum2/electrum/blockchain.py", line 444, in save_header
assert delta == self.size(), (delta, self.size())
AssertionError: (0, 1)

After deleting the file blockchain header the client is able to connect again until the error occurs again, the issue is reproducible.

[marpme]

do you have old data inside your electrum folder? if so can you still reproduce with everything disabled

I was not able to reproduce with the latest master. Could you please try again

[swat69]

no old data in folder .electrum-xvg.

new test with master branch:
2 seperate computer (ubuntu18 and debian10), one connected to electrum.marpmedev.xyz, one to node1.verge-electrum.com:
Error after 60 min, next try (after deleting blockchain_header) it takes only 2 minutes up to the error, then 90 minutes.
It looks like the error occur on both computers/wallets at the same time.
I attach an invalid blockchain_header
_blockchain_headers.gz

[swat69]

File "/home/ben/electrum2/electrum/blockchain.py", line 544, in chainwork_of_header_at_height
work = ((2 ** 256 - target - 1) // (target + 1)) + 1
TypeError: unsupported operand type(s) for -: 'int' and 'str'

reason crash: variable target is str/undefined

540 def chainwork_of_header_at_height(self, height: int) -> int:
........"""work done by single header at given height"""
........ chunk_idx = height // 2016 - 1
........ target = self.get_target(chunk_idx)
........ work = ((2 ** 256 - target - 1) // (target + 1)) + 1
........ return work

494 def get_target(self, index: int) -> int:
........ # compute target from chunk x, used in chunk x+1
........ if constants.net.TESTNET:
................ return 0
........ if index == -1:
................ return MAX_TARGET
........ if index < len(self.checkpoints):
................ h, t = self.checkpoints[index]
................ return t

h,t ---> t = undefined, because of file checkpoints.json (index[0])

[
["00000000007a5b03d0cde55844032c8c890c9e6bac0d7836aad53c9e251f301d", ""],
["0000000009076a32f65518d94c76fd3d281b86dc3185030800ea07ff8ffba559", ""],

[marpme]

Well we have to somehow adjust targeting for sure. Right now I've disabled the target checks in electrum since bitcoin got a quite easy approach to check em but we have a huge calculcation for upcoming targets and also do we have retargeting each and every block instead of once every 2016 Blocks.

Seems like a huge task to get this all done, but it's also quite necessary in order to ensure a solid working product.