Privacy option

[aaronshaf]

I propose an option to make the service's dat url private, and to offer a GET endpoint for retrieving dat archives for a particular application.

[RangerMauve]

What's the use case for the GET endpoint? To get data out without the DAT URL being exposed?

Personally, I'd want to avoid centralization as much as possible. Mostly to support the offline-first use case.

If your application relies on sending GET requests anywhere, it won't work as soon as your network goes down, whereas a P2P application can still get updates from the local network.

I think that if the URL is private and you need to send GET requests for the data, there's no point in storing data in a Dat in the first place. At that point you might as well have a regular HTTP server for discovery.

I'm all for hiding the URL from GET requests, though. Then the URL would need to be given to somebody explicitly.

[aaronshaf]

How about:

• For every app a different Dat archive is created to store discovery info
• GET /{application key} returns url for respective archive

With the above one could subscribe to changes made to a discovery dat archive relevant to their app, and not receive notifications on changes made to a dat archive that stores info for potentially hundreds/thousands of apps.

This also could help optionally enable the pub service not to broadcast the URLs of all apps for which it does discovery for.

/ might better serve as a place for documentation anyway.

[RangerMauve]

Yeah, that's a good idea.

Reasoning for why I went with the single dat:

Originally, I was thinking of having a single dat to make it more likely that it would get replicated.

Even if a given application didn't have that many peers seeding the discovery, other applications would be there helping it out.

Also, I wanted to have pubs publish their informotaion to other pubs so that applications could automatically discover more sources, but I'm not sure if that's a good idea.

Regarding subscribing to changes:

Applications can filter file activity by which application directory they're intersted in.

On the other hand, the network activity and metadata stream is going to get replicated regardless.

Honestly, I'm not sure how much activity there would be for an active application, since the goal is to have most of the data in the users' dats anyways.