Radicle Node

[lorenzleutgeb]

Hey! Prompted by this discussion on Discourse I would like to propose that the NixOS infra team sets up a Radicle node (services.radicle.enable) and uses that to mirror repositories under the NixOS namespace, most notably nixpkgs, to the Radicle network.

I am a Radicle team member, a proficient Nix user (maybe @fricklerhandwerk can vouch for me and confirm that?!), a co-maintainer of the radicle-node package in nixpkgs, and willing to support.

The node itself could receive updates from GitHub via webhook, or just periodically fetch changes from GitHub, and announce those to the Radicle network. Quite simple really, and the resource requirements are minimal. The biggest load would be other nodes fetching nixpkgs, with initial clones probably generating the most load. We could tell people to git clone --reference, though. Talking about load here is premature though, since it's really hard to estimate how many nodes would be fetching.

What's valuable here is that this would be the official mirror of nixpkgs (and friends), signed with a key that's controlled by the infra team. We could of course also start out with a repo that gets less traffic, such as this one.

I am happy to file a PR, if there's interest.

cc:

• @ju1m who co-maintains the NixOS module.
• @amesgen who co-maintains the package.
• @FintanH on the Radicle Team.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/could-somebody-setup-an-authorative-mirror-of-nixpkgs-on-radicle/63999/8

[winterqt]

Not infra, but a few questions that come to mind:

1. Will other nodes “mindlessly” grab the repo(s); as in, would other nodes only be pulling if a user requests it? Or can other nodes be set up as mirrors, periodically pulling from the authoritative node?
2. Is it possible to disable PRs and issues in the Radicle repo?

[lorenzleutgeb]

1. Will other nodes “mindlessly” grab the repo(s); as in, would other nodes only be pulling if a user requests it? Or can other nodes be set up as mirrors, periodically pulling from the authoritative node?

It depends... When a node keeps a copy of a repository in its local storage, we say it is "seeding" it. Node operators may configure different seeding policies, which then in turn influence how a node reacts to announcements of updates for that repository. The biggest discriminator here is permissive vs. selective seeding policy. Nodes with a permissive policy will start replicating all repositories that they "hear about". Nodes with a selective policy will only replicate a list of repositories defined by the user.

So this should clarify to some degree whether a node is "interested in" a repository at all, and how it reacts to "hearing about" new repositories. Now that we got this out of the way, we can look at the next layer, which is: Given the node is seeding a repository, how will it react to updates?

Nodes gossip a subscription message to the network. (They compute a bloom filter of all the repository identifiers they are seeding. This way, other nodes can with some probability, know, whether another node is interested in updates.) And they also gossip (to whom depends on the known subscriptions), announcements of updates to their refs (in the Git sense: heads of branches, and tags).

When it comes to fetching the actual contents, i.e. do the git fetch, I'm afraid I have to go read some code. I think that nodes relay the heads of references of others, but I am not sure whether node A will also be able to fetch the head for node B from node C.

2. Is it possible to disable PRs and issues in the Radicle repo?

No. But there is a concept of "following" (search for "followed" in the User Guide). And the NixOS node could just not follow anyone, for the time being.

[zimbatm]

Hey! Thanks for reaching out.

I think the main blocker is that importing nixpkgs into radicle takes forever. Or at least the last time I tried, a few months back, I wasn't able to complete the operation. The project looks really interesting to me (but I am not the decision make in the infra team).

Did you already setup a demo node somewhere we could take a look at?

[lorenzleutgeb]

I think the main blocker is that importing nixpkgs into radicle takes forever. Or at least the last time I tried, a few months back, I wasn't able to complete the operation.

With the upcoming release it'll still take quite long, but it'll be much faster than forever:

[I] lorenz@wsl ~/s/g/N/nixpkgs (master)> pwd
/home/lorenz/src/github.com/NixOS/nixpkgs
[I] lorenz@wsl ~/s/g/N/nixpkgs (master)> git rev-parse HEAD
84969609f889fefb506af31c4dda1ec4eb0b6011
[I] lorenz@wsl ~/s/g/N/nixpkgs (master)> fish_add_path --prepend $HOME/src/rad/heartwood/target/debug
set -U fish_user_paths /home/lorenz/src/rad/heartwood/target/debug
[I] lorenz@wsl ~/s/g/N/nixpkgs (master)> time rad init --name nixpkgs --description "nixpkgs test" --default-branch master --private --no-confirm

Initializing private radicle 👾 repository in /home/lorenz/src/github.com/NixOS/nixpkgs..

✓ Repository nixpkgs created.

Your Repository ID (RID) is rad:z3mS3NCbgGEAr95RtRdqJfGta81qk.
You can show it any time by running `rad .` from this directory.

You have created a private repository.
This repository will only be visible to you, and to peers you explicitly allow.

To make it public, run `rad publish`.
To push changes, run `git push rad master`.

________________________________________________________
Executed in  626.03 secs    fish           external
   usr time   19.29 mins    0.80 millis   19.29 mins
   sys time    1.10 mins    1.09 millis    1.10 mins

(This is on a ThinkPad in WSL.)

Did you already setup a demo node somewhere we could take a look at?

Yeah, I am running a node, and I can show any interested folks around, for example screensharing on Jitsi. If you have something in particular you want to "take a look at", please be more specific so I can get a better feeling for what you are looking for.