Discovery between different process

[asmodehn]

Refer to https://github.com/asmodehn/pyzmp/blob/docs/docs/tutorial.rst

We need to have a way to do service discovery on a system, even if the different process have been generated previously, from different interpreters...

While direct addressing the process using the zmp socket URL is probably trivial enough, we need to design some system to be able to connect to a set of processes and discover services, without previous knowledge of the system...

[asmodehn]

Currently services are registered into a manager from there : https://github.com/asmodehn/pyzmp/blob/master/pyzmp/service.py#L30 so discovery is very simple in one interpreter...

We need to improve that...

[asmodehn]

The properway to do that is likely to store our node details in /var/run/pyzmp/ somehow...
This would be the "well-known" location where another (unrelated) process can gather information about the nodes running on this machine, and establish connection via hte zmp socket urls...

[asmodehn]

Note : for authorization we need to rely on process authorization from OS, and not implement our own on top of it (or nightmare will ensue...)

[asmodehn]

Also relates to #8

[asmodehn]

TODO : Deep dive into : https://www.gitbook.com/book/gritzko/swarm-the-protocol/details

[asmodehn]

Idea is to implement something similar to SWARM, in python, but at a slightly lower level... Maybe something around python pickled named tuples...
This should allow to have a smooth, eventually consistent, distributed key value store, that seems appropriate to keep a list of nodes currently in the system...
Remember also https://sans-io.readthedocs.io/ so the protocol lib should not be implemented here with zmq and other IO things, but somewhere else...

[asmodehn]

It seems we need first to implement something much simpler, for a start : #26

[asmodehn]

We should ultimately rely on colos for the "machine-local" discovery (identifying compatible coprocesses locally)

But as we move into the pyzmp layer, with network enabled, some distributed storage seems necessary to implement service discovery across multiple machine, that is multiple instances of colos.