Consider options to allow Scrapyd restarts #442
Description
When Scrapyd restarts, it no longer has a connection to the running process scrapy/scrapyd#521. The job isn't listed as running, and this error occurs:
data_registry.exceptions.RecoverableError: Unable to find status of Scrapyd job ea951fa8eb8911f0af289c6b004cdfa7
Assuming the process is still running, we could perhaps use the log to determine the job's status.
That way, we don't need to be as careful about deployments.