Validator set is not properly updated after a node is restarted

[dzhelezov]

A Validator Candidate transaction is marked as INVALID after a node is restarted (e.g. went offline then back online), even if the db is fully up-to-date.

This is probably due to the fact the the local validator set in CandidateTrackerImpl is not properly updated and the initial set of validators is being used

[dzhelezov]

@dnck please provide more info/logs here

[dnck]

Might there be an issue with how the docker container volumes are mapped? I doubt it, but it might be worth detailing how it seems to works. @dt93 can you confirm this is not the issue?

The Dockerfile starts pendulum from a root data directory. Thus, the {}netdb, {}netdb-log, {}net-spent-addresses-db, {}net-spent-addresses-db-log, {}net-snapshot are all persisted in the data directory.

The docker-compose.yml file on the Validator maps a seed file found on the docker host directory to the data/resources directory in the running container. Thus, each validator key file is created and persisted in the running container data/resources directory.

If the validator is stopped, and restarted, does the validator resume with the key file in the mapped data/resources directory?