Use nostr as cache system

[KoalaSat]

What does this PR do?

This is a first POC. I'm exploring to use nostr not as primary DB but as a cache system of a decentralized network such as the federation of Robosats, right now a client has to directly connect to every single coordinator book to fetch its specific orders.

Centralizing this system would create a single point of failure, so I'm exploring a federation cache network, where the full book is aditioanlly stored and maintained by coordinators, allowing users to initially connect just to 2-3 coordinators. any feedback is welcome 🙂

For now this PR just creates a coordinator relay and sends an event every time a new order is created. The note follows this on going NIP nostr-protocol/nips#1331

As additional win, this will allow Robosats to quickly integrate others' platforms orders with just a single implementation

Federation cache

Every relay of the federation will be in sync with the others, containing all orders available in the federation:

flowchart TD;
    classDef coordinator fill:#F7931A
    classDef nostr fill:#8e30eb
    subgraph 1
    Coor1:::coordinator-->Relay1;
    end
    subgraph 2
    Coor2:::coordinator-->Relay2;
    end
    subgraph 3
    Coor3:::coordinator-->Relay3;
    end

    Relay1:::nostr<-->Relay2;
    Relay2:::nostr<-->Relay3;
    Relay3:::nostr<-->Relay1;
    Relay2<-->ExternalRelay;

    Relay1-->Client;
    Relay2-->Client;

Loading

The federation will keep a strong negentropy circle between their relays, but any other external relay will be able to fetch or create new order events. Because it's just cache, we can leave the creation of events open to anyone, if it becomes a spam problem, it can be cleaned up automatically by coordinators.

Clients will always send requests filtering by coordinators' public keys, that way spam or fake orders will never reach to the user.

How to test

Publish an order. Connect your WS tools to ws://localhost:12596/nostr and send:

["REQ", "111", {"kinds":[38383]}]

You should get in return:

["EVENT", "111", <your order note>]
["EOSE","111"]  # End of response

Any change on the order should remove previous note and create a new one (identified over time by its d tag). To keep privacy, Robosats will only have 2 order statuses: pending and success.

Checklist before merging

• Install pre-commit and initialize it: pip install pre-commit, then pre-commit install. Pre-commit installs git hooks that automatically check the codebase. If pre-commit fails when you commit your changes, please fix the problems it points out.

[jerryfletcher21]

@KoalaSat nice idea, and really needed once the number of coordinators starts to grow. :)
An other thing that we will have to change related to this improvement is that in the main client when creating a robot, the robot is created in every coordinator, again making a requests to each of them. This is not really needed since the robot will then be used on just one coordinator.

[KoalaSat]

@KoalaSat nice idea, and really needed once the number of coordinators starts to grow. :) An other thing that we will have to change related to this improvement is that in the main client when creating a robot, the robot is created in every coordinator, again making a requests to each of them. This is not really needed since the robot will then be used on just one coordinator.

The reason behind is that wherever the token comes from, we ask for coordinators if the token exists, in case there are active or past orders for it. Certainty, we know when the token is created but that would involve to fetch the coordinator every time the user tries to create an order, affecting the waiting time (unknown robots can´t create an order).

It's tricky to find an efficient way and don´t affect other parts, specially because the app/api assumes my robot is known, and for example, we would have to handle it in case the user visits their profile.

Anyways itś good to raise this topic from time to time to figure out if there are better ways

[jerryfletcher21]

In roboauto I have that when creating a robot, the user have to specifiy a coordinator, and when recovering a robot, together with the secret also the coordinator have to be specified.
A similar solution could also be implemented in the main client, especially once the number of coordinators starts to grow.
It can also be added an additional button that searches across the coordinator if the robot with the specified secret is present, to be used just when the user is recovering a robot and does not remember his coordinator.
In this way except when pressing this button the client never makes a requests to every coordinator.

[KoalaSat]

In roboauto I have that when creating a robot, the user have to specifiy a coordinator, and when recovering a robot, together with the secret also the coordinator have to be specified. A similar solution could also be implemented in the main client, especially once the number of coordinators starts to grow. It can also be added an additional button that searches across the coordinator if the robot with the specified secret is present, to be used just when the user is recovering a robot and does not remember his coordinator. In this way except when pressing this button the client never makes a requests to every coordinator.

Good to know, my plan with this cache service was also very related to this topic, cache book can also store on goings orders, and if you download it and just search on it your robot name thatś another nice way to know there is an order for your robot, as far the cache system is reliable enought 😄

[jerryfletcher21]

Good to know, my plan with this cache service was also very related to this topic, cache book can also store on goings orders, and if you download it and just search on it your robot name thatś another nice way to know there is an order for your robot, as far the cache system is reliable enought 😄

Ah yes this is even better, if the cache book keeps also track of ongoing orders, the user needs just the secret. The client can calculate the robot name from the secret, then search for his coordinator in the cache book. So there is never the need to search all coordinators to check if the robot is present and have an order.
But I agree is better to first work on making the cache system reliable :)

[KoalaSat]

torfication of sync calls are done as part of the cron command to avoid mistakes if the coordinator modifies sync.sh https://github.com/RoboSats/robosats/pull/1362/files#diff-366e6549a96aec09107bfc5f3f3804f92cfee8d9804bc8b222136d0a0a15d08b

[KoalaSat]

tested and working in Satstralia 👍

[KoalaSat]

Tested in Satstralia for 3 weeks already and fully working! 🚀