From b84d4257d40317f70298a2283bbfaaae2ecff279 Mon Sep 17 00:00:00 2001 From: theWebalyst Date: Tue, 22 Nov 2016 22:39:03 +0000 Subject: [PATCH 1/8] First draft --- .../00000000-nfs-file-sharing-by-hash-url.md | 170 ++++++++++++++++++ 1 file changed, 170 insertions(+) create mode 100644 text/0000-nfs-file-sharing-by-hash-url/00000000-nfs-file-sharing-by-hash-url.md diff --git a/text/0000-nfs-file-sharing-by-hash-url/00000000-nfs-file-sharing-by-hash-url.md b/text/0000-nfs-file-sharing-by-hash-url/00000000-nfs-file-sharing-by-hash-url.md new file mode 100644 index 0000000..ede70e8 --- /dev/null +++ b/text/0000-nfs-file-sharing-by-hash-url/00000000-nfs-file-sharing-by-hash-url.md @@ -0,0 +1,170 @@ +# NFS File Sharing By Hash URL + +- Status: proposed ("proposed", "agreed", "active", "implemented" or "rejected") +- Type: new feature +- Related components: SAFE Beaker Browser, safe-js, safe_core +- Start Date: 22-11-2016 +- Discussion: (Preliminary to be replaced) - [File sharing at the level of safe: URLs](https://forum.safedev.org/t/file-sharing-at-the-level-of-safe-urls/288?u=happybeing) +- Supersedes: n/a +- Superseded by: n/a + +## Summary + +Provide a way to obtain a "safe:" URL for any file stored using SAFE NFS that is based on the file hash (network address) and so does not involve a SAFE public ID or SAFE DNS. Such a *SAFE Hash URL* would be valid to use as part of a normal HTML page or as a file download link in SAFE Beaker Browser. + +This would allow, for example, an image captured by a SAFE web app and stored in a user's SAFE NFS storage, to be loaded into the DOM of a web page/app, and displayed within by the browser as part of the HTML/app UI. + +Also, a user could request to *Share file...* and be returned a SAFE Hash URL for that file (or any file stored in their shared drive). This can be sent to anyone, who provided the file is public, could then access the image with just a SAFE web browser. Private files could also be accessed this way but only by the authorised owner, as part of their browser history, or as quick access links in other applications or documents. + +An important feature is avoiding the need for anything other than a standard SAFE Browser, such as a special file sharing app, or for the user to have made a public ID and service for the purpose of sharing files (see *Alternatives*). + +## Conventions +- The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC 2119](http://tools.ietf.org/html/rfc2119). + +## Motivation + +Easy sharing of files via web links is one of the "killer" features of the current web and it would be valuable to provide this feature, while making it more functional (e.g. permanent links) and easier to use than file sharing is currently (no need to have a cloud service, no need to upload any file once stored on SAFE NFS). + +Without this, the SAFE NFS user would need to use a specialist file sharing application to share a file, and possibly also to access a file shared by someone else depending on the way the sharing application is implemented. + +An additional aim is to make it easy for SAFE web apps to create content, save it to the network, and to have a flexible and easy to use API for displaying it within the application. Native applications will also benefit from easier ways to share data they generate with users (e.g. as download links) or is easily sent to other applications or web apps, using a standard that every application can easily support. + +Providing access to SAFE files, without requiring publication via SAFE DNS, makes it easy to port an existing app, or for someone who already knows how to create and load files on a traditional web server to integrate this functionality. Thus we can expect many apps to take advantage of these widely expected features, similar to, but even easier to use than the current web (no need to upload or use a file sharing service). It also makes it possible for this feature to implemented as permanent URLs, so that immutable data will be accessible forever from such as URL. + +Example Use Cases: + +1) File Sharing: users will find it useful if they can share files without first creating a public id and service. They would need to have a way of identifying the file to be shared and obtaining a URL for it, which would require an application to provide a suitable URL for a given file. This RFC recommends extending the API so that applications can do this, but doesn't consider any such applications or how they work. + +2) Web Apps That Create Files: if a web app creates a file it may wish to present the file by embedding a link to it within the HTML of the app - by creating a URL for the file and embedding a link to it in the DOM. For example, an application might capture images using the webcam, store these on the network, and display them within the app by embedding image links within its HTML. To do this using SAFE NFS would be difficult, either needing a public ID and service to be created (probably one per such application). Alternatively the app have to implement a way to read files stored on SAFE NFS and render them directly on the page, which is difficult and would make it hard for the developer to take advantage of this functionality, and indeed all the advantages of doing this directly in HTML. + +## Detailed design + +We need a format of safe: URL which SAFE Beaker (and other compatible browsers) can recognise and use to access content stored on SAFE network without reference to a SAFE DNS public ID. Each file stored on SAFE network is addressed using a cryptographic hash, so we can use that, but we need to consider different kinds of file and how to provide expected and appropriate behaviour, depending on how the file is stored (e.g. as immutable data, mutable data), how it is identified by the user (e.g. via a directory listing obtained from SAFE NFS), and the type of the file (e.g. text, PDF, HTML, binary, video etc.). + +This is tricky - to cover all cases with sensible defaults so the following is a proposal as a starting point which will probably be subject to discussion and revision: + +#### SAFE Web URL Structure + +A safe: URL is similar to a standard web URL, but always begins with ```safe:```, which is the link protocol which indicates a resource locator on the SAFE network. For a website published on SAFE network, we create a public ID and service using SAFE NFS, and this allows SAFE Beaker to interpret URLs of the following form: + +```safe://[service.]publicid[/filepath]``` + +```service``` is optional, and defaults to ```www``` +```filepath``` is also optional, and if service is ommitted, or is given as ```www``` it defaults to ```index.html``` + +So for example, the following URLs would all access the same file assuming it exists in a directory that has been published at the given service and public ID: + + +```safe://rfcs +safe://www.rfcs +safe://rfcs/index.html +safe://www.rfcs/index.html``` + + +All the above is existing functionality, so the format of hash URLs must always be distinguishable from this. + +### SAFE Hash URL Structure + +SAFE Beaker must be able to recognise a Hash URL, parse it, and determine what method to use to access the file, and what if any meta data to provide with the HTTP response to the browser engine. We can also choose to provide a way for a human readable filename to be conveyed as part of the URL, rather than base this on the hash or require the downloading user to type it in. + +Indicating a hash URL. + +**A Hash URL always begins with ```safe::```** This allows us to continue to permit a normal ```safe:``` URL to omit or include the conventional but redundant '//' that we are accustomed to in web URLs. + +In the following, square brackets denote an optional component. So ```[type:]hash``` means either a value denoting a ```type```, followed by a ':', followed by a value denoting a ```hash```, or just a value denoting a ```hash```. + +The complete structure of a Hash URL is then. + +```safe::type:hash[:filename]``` + +Where: + +- ```type``` is one of ```file:``` or ```folder:``` or ```dns:```. So far I have only discussed retrieving regular files stored with SAFE NFS, but including a type would instruct the client on the type of data to access and how to make use of it, and could be extended to further types with backwards compatibility. + +- ```hash``` is the hash address in hexadecimal which resolves to a [FILE](https://github.com/maidsafe/safe_core/blob/eb172c2718aa43b78e644f998cf65a5ff92dfa4b/src/nfs/file.rs#L29) entry + +- ```filename``` is the URI encoded value of a string that must not include '/' (i.e it is a filename without a path), with an optional file extension. + +Examples might be: + +```safe:file:4e1243bd22c66e76c2ba9eddc1f91394e57f9f83:happybeing.png``` + + +```safe:file:4e1243bd22c66e76c2ba9eddc1f91394e57f9f83:Safecoin-white-paper.pdf``` + + +```safe:file:4e1243bd22c66e76c2ba9eddc1f91394e57f9f83``` + + +This RFC does not consider the details or use of ```folder:``` or ```dns:``` or other types as these can be defined later. For now we only need implement type ```file:```. + +The author doesn't adequately understand the underlying implementation of SAFE NFS storage, but believes that this method of file addressing will be feasible according to [RFC0046-new-auth-flow](https://github.com/maidsafe/rfcs/tree/master/text/0046-new-auth-flow) which uses a system of [Containers](https://github.com/maidsafe/rfcs/blob/master/text/0046-new-auth-flow/containers.md#nfs-convention) to emulate filesystem at the NFS level. This though will depend on the decisions taken around implementing emulation NFS and the underlying file structures. + +### Behaviour Options - Discussion + +Disregarding any limitations imposed by the implementation we should consider the viability and usefulness SAFE Hash URLs being either permanent or temporary. Assuming the shared file is immutable, we might assume the URL would be permanent. If the file were either mutable or deliberately referenced via a mutable data structure, then the URL could be impermanent. + +At this point I'll just assume all options are on the table and open up the question of what is expected and desirable. + +#### Permanent URLs + + +If all SAFE Hash URL's are valid forever, we have the advantage of certainty, and the feature of permanent access to all files shared in this manner. + +It wouldn't ensure that SAFE websites were always available forever and immutable, because generally these contain utilise ordinary SAFE URLs that map to a mutable file structure, but the advent of *Permanent URLs* would make it *possible* to create a SAFE web app or website that was easy to inspect and verify as immutable. + +One question is whether we would want to limit SAFE Hash URLs to only allow sharing of immutable data, or to allow permanent *and* impermanent URLs (see next). + +A guarantee of permanence for all SAFE Hash URLs would have to be verifiable (i.e. within safe_core), because it would not be desirable for people to assume that these were permanent and immutable if there was a way of subverting this with a customised SAFE Hash URL maker). + +#### Impermanent URLs + +There are many good things about permanent URLs, but also the disadvantage for users who accidentally share something that can never subsequently be unshared. There are also things people want to be able to share temporarily - so as to allow the intended recipient access but later invalidate to reduce the chance that someone else later obtains the URL and gains access to the file. Also, what one shares one day, or in one state of mind, might later be regretted and so for all these situations it might also be useful to provide a way to generate URLs that can be invalidated. + +If we allow both permanent and impermanent variations it would be sensible to indicate this as part of the URL so that users know what they are sharing and receiving. In this case we might extend the type to indicate one or both variations. For example adding '-p' or '-i' to the end of the ```type``` as in ```safe:file-p:4e1243bd22c66e76c2ba9eddc1f91394e57f9f83:Safecoin-white-paper.pdf``` + +Impermanent URLs would require a *delete Hash URL* API, which raises a further question as to whether or not we should allow different files to appear at the same address at different times. + +For simplicity's sake, this RFC mandates only permanent URLs (and so only type ```file:``` and never ```file-p``` or ```file-i```), but this assumes we can enforce permanance, and is up for debate. + +### SAFE NFS API Changes + +SAFE NFS API to be extended with: + +```getPermanentLink(fileName: String ) -> Future``` + + + ```fileName``` - the full path of an existing file in the user's NFS storage + + +This returns a permanently valid URL for fileName formatted as a SAFE Hash URL including ```type```, ```hash``` and ```filename``` elements. + +### safe-js Changes + +safe-js to be extended with ```safeNFS.getPermanentLink()``` which returns a Promise that resolves to a SAFE Hash URL as above. + +### SAFE Beaker Changes + +SAFE Beaker will detect every SAFE Hash URL, parse it according to the above, and use this information to: + +a) access the file on the network +b) provide error handling where retrieving the file fails or times out +c) deliver the content appropriately (e.g. embedded in HTML content, embedded download link, or location bar access) including where provided: applying metadata to assist in rendering embedded content, and supplying a default filename when offering to download. + +## Drawbacks + +Why should we *not* do this? + +## Alternatives + +>>> What other designs have been considered? What is the impact of not doing this? + +### Application based or shared DNS + +Rather than addressing a file using a hash, we might use either the existing SAFE DNS to share files saved in the user or application managed public storage. SAFE NFS at present would be cumbersome to use for this, requiring every app which wants to generate file URLs to have access to a user's registerd public ID and service so this is not practical. + +There are changes being considered which may make a DNS based sharing URLs more practical (see [this dev forum post](https://forum.safedev.org/t/file-sharing-at-the-level-of-safe-urls/288/7?u=happybeing) but these have not been decided and have not been considered as part of this RFC. So once clear, it may be worthwhile reviewing this option. + +## Unresolved questions + +What parts of the design are still to be done? From 30482763b00f5fbede7ff416716db164ebce3cd6 Mon Sep 17 00:00:00 2001 From: theWebalyst Date: Tue, 22 Nov 2016 22:39:58 +0000 Subject: [PATCH 2/8] First draft --- .../00000000-nfs-file-sharing-by-hash-url.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-nfs-file-sharing-by-hash-url/00000000-nfs-file-sharing-by-hash-url.md b/text/0000-nfs-file-sharing-by-hash-url/00000000-nfs-file-sharing-by-hash-url.md index ede70e8..cb5d1fb 100644 --- a/text/0000-nfs-file-sharing-by-hash-url/00000000-nfs-file-sharing-by-hash-url.md +++ b/text/0000-nfs-file-sharing-by-hash-url/00000000-nfs-file-sharing-by-hash-url.md @@ -1,6 +1,6 @@ # NFS File Sharing By Hash URL -- Status: proposed ("proposed", "agreed", "active", "implemented" or "rejected") +- Status: proposed - Type: new feature - Related components: SAFE Beaker Browser, safe-js, safe_core - Start Date: 22-11-2016 From 2b77be3855f2e54fea2bf4da146464b76befbe59 Mon Sep 17 00:00:00 2001 From: theWebalyst Date: Wed, 23 Nov 2016 16:17:01 +0000 Subject: [PATCH 3/8] First full draft Added missing sections (small), reworded some text and improved formatting. --- .../00000000-nfs-file-sharing-by-hash-url.md | 130 +++++++++--------- 1 file changed, 68 insertions(+), 62 deletions(-) diff --git a/text/0000-nfs-file-sharing-by-hash-url/00000000-nfs-file-sharing-by-hash-url.md b/text/0000-nfs-file-sharing-by-hash-url/00000000-nfs-file-sharing-by-hash-url.md index cb5d1fb..01a5626 100644 --- a/text/0000-nfs-file-sharing-by-hash-url/00000000-nfs-file-sharing-by-hash-url.md +++ b/text/0000-nfs-file-sharing-by-hash-url/00000000-nfs-file-sharing-by-hash-url.md @@ -8,140 +8,141 @@ - Supersedes: n/a - Superseded by: n/a + ## Summary -Provide a way to obtain a "safe:" URL for any file stored using SAFE NFS that is based on the file hash (network address) and so does not involve a SAFE public ID or SAFE DNS. Such a *SAFE Hash URL* would be valid to use as part of a normal HTML page or as a file download link in SAFE Beaker Browser. +Provide a way to obtain a "safe:" URL based on the file hash (network address) for any file stored using SAFE NFS, and so is not reliant on a public ID or SAFE DNS. Such a *SAFE Hash URL* would be valid to use as part of a normal HTML page or as a file download link in SAFE Beaker Browser. + +This would allow, for example: -This would allow, for example, an image captured by a SAFE web app and stored in a user's SAFE NFS storage, to be loaded into the DOM of a web page/app, and displayed within by the browser as part of the HTML/app UI. +- an image captured by a SAFE web app and stored in a user's SAFE NFS storage, to be loaded into the DOM of the app, or an HTML document generated by the app, and displayed within by the browser as part of the app UI or saved HTML. -Also, a user could request to *Share file...* and be returned a SAFE Hash URL for that file (or any file stored in their shared drive). This can be sent to anyone, who provided the file is public, could then access the image with just a SAFE web browser. Private files could also be accessed this way but only by the authorised owner, as part of their browser history, or as quick access links in other applications or documents. +- a *Share file...* command in any app, which offers up a SAFE Hash URL for an arbitrary file (any file stored in their shared drive). This can be sent to anyone who, provided the file is public, could then access the image with just a SAFE web browser. Private files could also be accessed this way but only by the authorised owner, as part of their browser history, or as quick access links in other applications or documents. An important feature is avoiding the need for anything other than a standard SAFE Browser, such as a special file sharing app, or for the user to have made a public ID and service for the purpose of sharing files (see *Alternatives*). + ## Conventions - The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC 2119](http://tools.ietf.org/html/rfc2119). + ## Motivation -Easy sharing of files via web links is one of the "killer" features of the current web and it would be valuable to provide this feature, while making it more functional (e.g. permanent links) and easier to use than file sharing is currently (no need to have a cloud service, no need to upload any file once stored on SAFE NFS). +Easy sharing of files via web links is one of the "killer" features of the current web and it is important for SAFE to avoid losing expected and valuable functionality in order to encourage and accelerate adoption. This proposal in fact would exceed the current functionality (e.g. by making links permanent) and be easier to use because all SAFE users will already have everything they need (no need to sign up to a separate cloud service, and no need to upload any file once stored on SAFE NFS). + +Without this, the SAFE NFS user would need to use a specialist file sharing application to share a file, and files accessed through such links would be less secure, hard to trust or guarantee permanence. -Without this, the SAFE NFS user would need to use a specialist file sharing application to share a file, and possibly also to access a file shared by someone else depending on the way the sharing application is implemented. +An additional aim is to make it easy for SAFE web apps to create content, save it to the network, and to have a flexible and easy to use API for displaying it within the application. Native applications will also benefit from easier ways to share data they generate with users (e.g. as download links) or which is easily sent to other applications or web apps, using a standard that every application can easily support. -An additional aim is to make it easy for SAFE web apps to create content, save it to the network, and to have a flexible and easy to use API for displaying it within the application. Native applications will also benefit from easier ways to share data they generate with users (e.g. as download links) or is easily sent to other applications or web apps, using a standard that every application can easily support. +Providing access to SAFE files, without requiring publication via SAFE DNS, makes it easy to port existing apps, or for someone who already knows how to create and load files on a traditional web server to offer this functionality in new apps. Thus we can expect many apps to take advantage of these widely expected features, similar to, but more functional and easier to use than the current web. The guaranteed permanence of download links will no doubt also lead to the creation of new and innovative kinds of application. -Providing access to SAFE files, without requiring publication via SAFE DNS, makes it easy to port an existing app, or for someone who already knows how to create and load files on a traditional web server to integrate this functionality. Thus we can expect many apps to take advantage of these widely expected features, similar to, but even easier to use than the current web (no need to upload or use a file sharing service). It also makes it possible for this feature to implemented as permanent URLs, so that immutable data will be accessible forever from such as URL. -Example Use Cases: +## Required Functionality 1) File Sharing: users will find it useful if they can share files without first creating a public id and service. They would need to have a way of identifying the file to be shared and obtaining a URL for it, which would require an application to provide a suitable URL for a given file. This RFC recommends extending the API so that applications can do this, but doesn't consider any such applications or how they work. -2) Web Apps That Create Files: if a web app creates a file it may wish to present the file by embedding a link to it within the HTML of the app - by creating a URL for the file and embedding a link to it in the DOM. For example, an application might capture images using the webcam, store these on the network, and display them within the app by embedding image links within its HTML. To do this using SAFE NFS would be difficult, either needing a public ID and service to be created (probably one per such application). Alternatively the app have to implement a way to read files stored on SAFE NFS and render them directly on the page, which is difficult and would make it hard for the developer to take advantage of this functionality, and indeed all the advantages of doing this directly in HTML. +2) Apps That Create Files: if a web app creates a file it may wish to present the file by embedding a link to it within the HTML of the app - by creating a URL for the file and embedding a link to it in the DOM. For example, an application might capture images using the webcam, store these on the network, and display them within the app by embedding image links within its HTML. To do this using SAFE NFS would be difficult, either needing a public ID and service to be created (probably one per such application). Alternatively the app have to implement a way to read files stored on SAFE NFS and then render them directly on the page, which is difficult and would make it harder for the developer to take advantage of this functionality, and lose the advantages of doing this directly in HTML. + ## Detailed design -We need a format of safe: URL which SAFE Beaker (and other compatible browsers) can recognise and use to access content stored on SAFE network without reference to a SAFE DNS public ID. Each file stored on SAFE network is addressed using a cryptographic hash, so we can use that, but we need to consider different kinds of file and how to provide expected and appropriate behaviour, depending on how the file is stored (e.g. as immutable data, mutable data), how it is identified by the user (e.g. via a directory listing obtained from SAFE NFS), and the type of the file (e.g. text, PDF, HTML, binary, video etc.). +We need a format of safe: URL which SAFE Beaker (and other compatible browsers) can recognise and use to access content stored on SAFE network without reference to a SAFE public ID. Each file stored on SAFE network is addressed using a cryptographic hash, so we can use that, but we need to consider different kinds of file and how to provide expected and appropriate behaviour, depending on how the file is stored (e.g. as immutable data, mutable data, key-value store etc.), how it is identified by the user (e.g. via a directory listing obtained from SAFE NFS), and the type of the file (e.g. text, PDF, HTML, binary, video etc.). -This is tricky - to cover all cases with sensible defaults so the following is a proposal as a starting point which will probably be subject to discussion and revision: +This is tricky - to cover all cases with sensible defaults so the following is a proposal as a starting point which will probably be subject to discussion and revision. -#### SAFE Web URL Structure +### SAFE Web URL Structure -A safe: URL is similar to a standard web URL, but always begins with ```safe:```, which is the link protocol which indicates a resource locator on the SAFE network. For a website published on SAFE network, we create a public ID and service using SAFE NFS, and this allows SAFE Beaker to interpret URLs of the following form: +To recap existing functionality, a `safe:` website URL is similar to a standard web URL, but always begins with `safe:` rather than `http` or `https`. This link protocol indicates a resource located on the SAFE network. For a website published on SAFE network, we create a public ID and service using SAFE NFS, and this allows SAFE Beaker to interpret URLs of the following form: -```safe://[service.]publicid[/filepath]``` +In the following, square brackets denote an optional component. So `[service.]publicid` means either a value denoting a `service` followed by a '.' followed by a value denoting a `publicid`, or just a value denoting a `publicid`. -```service``` is optional, and defaults to ```www``` -```filepath``` is also optional, and if service is ommitted, or is given as ```www``` it defaults to ```index.html``` + `safe://[service.]publicid[/filepath]` -So for example, the following URLs would all access the same file assuming it exists in a directory that has been published at the given service and public ID: +`service` is optional, and defaults to `www` +`filepath` is also optional, and if service is ommitted or given as `www`, the filepath defaults to `index.html` -```safe://rfcs -safe://www.rfcs -safe://rfcs/index.html -safe://www.rfcs/index.html``` +So for example, the following URLs would all access the same file assuming it exists in a directory that has been published at the given service and public ID: -All the above is existing functionality, so the format of hash URLs must always be distinguishable from this. + `safe://rfcs` + `safe://www.rfcs` + `safe://rfcs/index.html` + `safe://www.rfcs/index.html` -### SAFE Hash URL Structure -SAFE Beaker must be able to recognise a Hash URL, parse it, and determine what method to use to access the file, and what if any meta data to provide with the HTTP response to the browser engine. We can also choose to provide a way for a human readable filename to be conveyed as part of the URL, rather than base this on the hash or require the downloading user to type it in. +All the above is existing functionality, so the format of *SAFE Hash URLs* must always be distinguishable from this. -Indicating a hash URL. +### SAFE Hash URL Structure -**A Hash URL always begins with ```safe::```** This allows us to continue to permit a normal ```safe:``` URL to omit or include the conventional but redundant '//' that we are accustomed to in web URLs. +SAFE Beaker must be able to recognise a Hash URL, parse it, and determine what method to use to access the file, and what if any meta data to provide with the response it delivers. We can also choose to provide a way for a human readable filename to be conveyed as part of the URL, rather than base this on the hash or require the downloading user to type it in. -In the following, square brackets denote an optional component. So ```[type:]hash``` means either a value denoting a ```type```, followed by a ':', followed by a value denoting a ```hash```, or just a value denoting a ```hash```. +**A Hash URL always begins with `safe::`** This allows us to continue to permit a normal `safe:` URL to omit or include the conventional but redundant '//' that we are accustomed to in web URLs. -The complete structure of a Hash URL is then. +The complete structure of a Hash URL is then: -```safe::type:hash[:filename]``` + `safe::type:hash[:filename]` Where: -- ```type``` is one of ```file:``` or ```folder:``` or ```dns:```. So far I have only discussed retrieving regular files stored with SAFE NFS, but including a type would instruct the client on the type of data to access and how to make use of it, and could be extended to further types with backwards compatibility. +- `type` is one of `file:` or `folder:` or `dns:`. So far I have only discussed retrieving regular files stored with SAFE NFS, but including a type instructs the client on the type of data being retrieved access and how to make use of it, while providing for support to added later for additional types, with backwards compatibility. -- ```hash``` is the hash address in hexadecimal which resolves to a [FILE](https://github.com/maidsafe/safe_core/blob/eb172c2718aa43b78e644f998cf65a5ff92dfa4b/src/nfs/file.rs#L29) entry +- `hash` is the hash address in hexadecimal which resolves to a [FILE](https://github.com/maidsafe/safe_core/blob/eb172c2718aa43b78e644f998cf65a5ff92dfa4b/src/nfs/file.rs#L29) entry. -- ```filename``` is the URI encoded value of a string that must not include '/' (i.e it is a filename without a path), with an optional file extension. +- `filename` is the URI encoded value of a string that must not include the path separator character '/' (i.e it is a filename without a path, and with an optional file extension). Examples might be: -```safe:file:4e1243bd22c66e76c2ba9eddc1f91394e57f9f83:happybeing.png``` - + `safe:file:4e1243bd22c66e76c2ba9eddc1f91394e57f9f83:happybeing.png` + `safe:file:4e1243bd22c66e76c2ba9eddc1f91394e57f9f83:Safecoin-white-paper.pdf` + `safe:file:4e1243bd22c66e76c2ba9eddc1f91394e57f9f83` -```safe:file:4e1243bd22c66e76c2ba9eddc1f91394e57f9f83:Safecoin-white-paper.pdf``` +This RFC does not consider the details or use of `folder:` or `dns:` or other types as these can be defined later. For now we only need implement type `file:` -```safe:file:4e1243bd22c66e76c2ba9eddc1f91394e57f9f83``` - - -This RFC does not consider the details or use of ```folder:``` or ```dns:``` or other types as these can be defined later. For now we only need implement type ```file:```. - -The author doesn't adequately understand the underlying implementation of SAFE NFS storage, but believes that this method of file addressing will be feasible according to [RFC0046-new-auth-flow](https://github.com/maidsafe/rfcs/tree/master/text/0046-new-auth-flow) which uses a system of [Containers](https://github.com/maidsafe/rfcs/blob/master/text/0046-new-auth-flow/containers.md#nfs-convention) to emulate filesystem at the NFS level. This though will depend on the decisions taken around implementing emulation NFS and the underlying file structures. +The author believes that this method of addressing SAFE NFS stored files will be feasible according to [RFC0046-new-auth-flow](https://github.com/maidsafe/rfcs/tree/master/text/0046-new-auth-flow) which uses a system of [Containers](https://github.com/maidsafe/rfcs/blob/master/text/0046-new-auth-flow/containers.md#nfs-convention) to emulate filesystem at the NFS level. This though will depend on the decisions taken around implementing emulation NFS and the underlying file structures. ### Behaviour Options - Discussion -Disregarding any limitations imposed by the implementation we should consider the viability and usefulness SAFE Hash URLs being either permanent or temporary. Assuming the shared file is immutable, we might assume the URL would be permanent. If the file were either mutable or deliberately referenced via a mutable data structure, then the URL could be impermanent. +Disregarding any limitations imposed by the implementation, we should in reviewing this RFC consider the viability and usefulness SAFE Hash URLs being either permanent or temporary. Assuming the shared file is immutable, we should be able to guarantee that the URL will also be permanent. If the file were either mutable or deliberately referenced via a mutable data structure, then the URL could be designed to be impermanent. -At this point I'll just assume all options are on the table and open up the question of what is expected and desirable. +**IMPORTANT:** At this point both options are on the table so we can debate what is expected and desirable, but as is stands this RFC proposes to implement only permanent SAFE Hash URLs. #### Permanent URLs - If all SAFE Hash URL's are valid forever, we have the advantage of certainty, and the feature of permanent access to all files shared in this manner. -It wouldn't ensure that SAFE websites were always available forever and immutable, because generally these contain utilise ordinary SAFE URLs that map to a mutable file structure, but the advent of *Permanent URLs* would make it *possible* to create a SAFE web app or website that was easy to inspect and verify as immutable. - -One question is whether we would want to limit SAFE Hash URLs to only allow sharing of immutable data, or to allow permanent *and* impermanent URLs (see next). +It wouldn't ensure that SAFE websites were always available forever and immutable, because generally these utilise ordinary SAFE URLs that map to a mutable file structure, but the advent of *Permanent URLs* would make it *possible* to create a SAFE web app or website that was easy to inspect and verify as permanent and immutable. -A guarantee of permanence for all SAFE Hash URLs would have to be verifiable (i.e. within safe_core), because it would not be desirable for people to assume that these were permanent and immutable if there was a way of subverting this with a customised SAFE Hash URL maker). +The question is whether we would want to limit SAFE Hash URLs to only allow sharing of immutable data, or to allow permanent *and* impermanent URLs (see next). A guarantee of permanence for all SAFE Hash URLs would have to be verifiable (i.e. within safe_core), because it would not be desirable for people to assume that these were permanent and immutable if there was a way of subverting this with a customised SAFE Hash URL maker). #### Impermanent URLs -There are many good things about permanent URLs, but also the disadvantage for users who accidentally share something that can never subsequently be unshared. There are also things people want to be able to share temporarily - so as to allow the intended recipient access but later invalidate to reduce the chance that someone else later obtains the URL and gains access to the file. Also, what one shares one day, or in one state of mind, might later be regretted and so for all these situations it might also be useful to provide a way to generate URLs that can be invalidated. +There are many good things about permanent URLs, but also the disadvantage for users who accidentally share something that can never subsequently be unshared. There are also things people want to be able to share temporarily. For example, so an intended recipient access them, but later be invalidated to reduce the chance that someone else later obtains the URL and gains access to the file. Also, what one shares one day, or in one state of mind, might later be regretted and so in certain circumstances it might also be useful to provide a way to generate URLs that can be invalidated. Impermanence is also likely to be expected behaviour, so if it is not, we might need to find ways to make that very clear. -If we allow both permanent and impermanent variations it would be sensible to indicate this as part of the URL so that users know what they are sharing and receiving. In this case we might extend the type to indicate one or both variations. For example adding '-p' or '-i' to the end of the ```type``` as in ```safe:file-p:4e1243bd22c66e76c2ba9eddc1f91394e57f9f83:Safecoin-white-paper.pdf``` +If we allow both permanent and impermanent URLs it would be sensible to indicate this as part of the URL so that users know what they are sharing and receiving. For example, by adding '-p' or '-i' to the end of the `type` as in: -Impermanent URLs would require a *delete Hash URL* API, which raises a further question as to whether or not we should allow different files to appear at the same address at different times. + `safe:file-p:4e1243bd22c66e76c2ba9eddc1f91394e57f9f83:Safecoin-white-paper.pdf` -For simplicity's sake, this RFC mandates only permanent URLs (and so only type ```file:``` and never ```file-p``` or ```file-i```), but this assumes we can enforce permanance, and is up for debate. +Impermanent URLs would require a *delete Hash URL* API, which raises a further question as to whether or not we should allow different files to appear at the same address at different times, and if not how this could be prevented. -### SAFE NFS API Changes +**IMPORTANT:** For simplicity's sake, this RFC mandates only permanent URLs (and so only type `file:` and never `file-p` or `file-i`), but this assumes we can enforce permanence, and is up for debate. -SAFE NFS API to be extended with: +### SAFE NFS API Changes -```getPermanentLink(fileName: String ) -> Future``` +Assuming only *permanent SAFE Hash URLs* are supported, SAFE NFS API to be extended with: + `getPermanentLink(fileName: String) -> Future` - ```fileName``` - the full path of an existing file in the user's NFS storage + `fileName` - the full path of an existing file in the user's NFS storage +This returns a permanently valid URL for fileName formatted as a SAFE Hash URL including `type`, `hash` and `filename` elements. -This returns a permanently valid URL for fileName formatted as a SAFE Hash URL including ```type```, ```hash``` and ```filename``` elements. +If we later support impermanent URLs a suitably named function can be added which would use a type of `file-i` instead of `file` ### safe-js Changes -safe-js to be extended with ```safeNFS.getPermanentLink()``` which returns a Promise that resolves to a SAFE Hash URL as above. +safe-js to be extended with `safeNFS.getPermanentLink()` which returns a Promise that resolves to a SAFE Hash URL as above. ### SAFE Beaker Changes @@ -151,20 +152,25 @@ a) access the file on the network b) provide error handling where retrieving the file fails or times out c) deliver the content appropriately (e.g. embedded in HTML content, embedded download link, or location bar access) including where provided: applying metadata to assist in rendering embedded content, and supplying a default filename when offering to download. + ## Drawbacks -Why should we *not* do this? +It relies on the implementation of SAFE NFS storage and might be difficult to maintain if that ever changes. + +There might be privacy or tracking issues associated with sharing URLs if they could somehow be linked back to a user or account, though none are immediately apparent, save ill considered naming of files. + ## Alternatives ->>> What other designs have been considered? What is the impact of not doing this? +Not doing this would require users to find other ways of sharing files, which since they are not built in to safe_core and SAFE Beaker will be more cumbersome (for users and for app developers) and impossible to ensure permanence, and harder to trust in terms of security (i.e. from tracking and surveillance). Making such a widely used feature harder than necessary would reduce the rate and potential level of adoption, thereby diminishing its positive overall impact on security and privacy. ### Application based or shared DNS -Rather than addressing a file using a hash, we might use either the existing SAFE DNS to share files saved in the user or application managed public storage. SAFE NFS at present would be cumbersome to use for this, requiring every app which wants to generate file URLs to have access to a user's registerd public ID and service so this is not practical. +Rather than addressing a file using a hash, we might use either the existing SAFE DNS to share files saved in the user or application managed public storage. SAFE NFS at present would be cumbersome to use for this, requiring every app which wants to generate file URLs to have access to a user's registered public ID and service so this is not practical. There are changes being considered which may make a DNS based sharing URLs more practical (see [this dev forum post](https://forum.safedev.org/t/file-sharing-at-the-level-of-safe-urls/288/7?u=happybeing) but these have not been decided and have not been considered as part of this RFC. So once clear, it may be worthwhile reviewing this option. + ## Unresolved questions -What parts of the design are still to be done? +It needs to be confirmed that proposed modifications to the implementation of SAFE NFS [Containers](https://github.com/maidsafe/rfcs/blob/master/text/0046-new-auth-flow/containers.md#nfs-convention) would support this functionality if adopted. \ No newline at end of file From 4c70f0131fcc97bec987c6bf4671dbef7c011706 Mon Sep 17 00:00:00 2001 From: theWebalyst Date: Wed, 23 Nov 2016 17:11:21 +0000 Subject: [PATCH 4/8] Added "Extensibility", many minor changes. ...to punctuation, readability and grammar. --- .../00000000-nfs-file-sharing-by-hash-url.md | 66 +++++++++++-------- 1 file changed, 39 insertions(+), 27 deletions(-) diff --git a/text/0000-nfs-file-sharing-by-hash-url/00000000-nfs-file-sharing-by-hash-url.md b/text/0000-nfs-file-sharing-by-hash-url/00000000-nfs-file-sharing-by-hash-url.md index 01a5626..e1ca2a9 100644 --- a/text/0000-nfs-file-sharing-by-hash-url/00000000-nfs-file-sharing-by-hash-url.md +++ b/text/0000-nfs-file-sharing-by-hash-url/00000000-nfs-file-sharing-by-hash-url.md @@ -11,13 +11,13 @@ ## Summary -Provide a way to obtain a "safe:" URL based on the file hash (network address) for any file stored using SAFE NFS, and so is not reliant on a public ID or SAFE DNS. Such a *SAFE Hash URL* would be valid to use as part of a normal HTML page or as a file download link in SAFE Beaker Browser. +Provide a way to obtain a 'safe:' URL based on the file hash (network address) for any file stored using SAFE NFS, and so is not reliant on a public ID or SAFE DNS. Such a *SAFE Hash URL* would be valid to use as part of a normal HTML page or as a file download link in SAFE Beaker Browser. This would allow, for example: -- an image captured by a SAFE web app and stored in a user's SAFE NFS storage, to be loaded into the DOM of the app, or an HTML document generated by the app, and displayed within by the browser as part of the app UI or saved HTML. +- an image captured by a SAFE web app and stored in a user's SAFE NFS storage, to be loaded into the DOM of the app, or into an HTML document generated by the app, and displayed within by the browser as part of the app UI or saved HTML. -- a *Share file...* command in any app, which offers up a SAFE Hash URL for an arbitrary file (any file stored in their shared drive). This can be sent to anyone who, provided the file is public, could then access the image with just a SAFE web browser. Private files could also be accessed this way but only by the authorised owner, as part of their browser history, or as quick access links in other applications or documents. +- a *Share file...* command, easily implemented in any app, which offers up a SAFE Hash URL for a file saved to or chosen from NFS 'drive'. This Hash URL can be sent to anyone who, provided the file is public, could then access the image with just a SAFE web browser. Private files could also be accessed this way but only by the authorised owner, as part of their browser history, or as quick access links in other applications or documents. An important feature is avoiding the need for anything other than a standard SAFE Browser, such as a special file sharing app, or for the user to have made a public ID and service for the purpose of sharing files (see *Alternatives*). @@ -28,33 +28,30 @@ An important feature is avoiding the need for anything other than a standard SAF ## Motivation -Easy sharing of files via web links is one of the "killer" features of the current web and it is important for SAFE to avoid losing expected and valuable functionality in order to encourage and accelerate adoption. This proposal in fact would exceed the current functionality (e.g. by making links permanent) and be easier to use because all SAFE users will already have everything they need (no need to sign up to a separate cloud service, and no need to upload any file once stored on SAFE NFS). +Easy sharing of files via web links is one of the "killer" features of the current web and it is important for SAFE to avoid losing expected and valuable functionality in order to encourage and accelerate adoption. This proposal would exceed current expected functionality (e.g. by making links permanent) and be easier to use because all SAFE users will already have everything they need (no need to sign up to a separate cloud service, and no need to upload any file once stored on SAFE NFS). Sharing SAFE URLs with those not already on SAFE network would not be so smooth because the recipient will have to join the network (or perhaps wait for a SAFE Browser to load on activating the link), but this is not wholly negative because it in part encourages adoption, and then a default of having a SAFE Browser loaded. -Without this, the SAFE NFS user would need to use a specialist file sharing application to share a file, and files accessed through such links would be less secure, hard to trust or guarantee permanence. +Without this feature, the SAFE NFS user would need to use a specialist file sharing application to share a file, and files accessed through such links would be less secure, hard to trust or guarantee permanence. -An additional aim is to make it easy for SAFE web apps to create content, save it to the network, and to have a flexible and easy to use API for displaying it within the application. Native applications will also benefit from easier ways to share data they generate with users (e.g. as download links) or which is easily sent to other applications or web apps, using a standard that every application can easily support. - -Providing access to SAFE files, without requiring publication via SAFE DNS, makes it easy to port existing apps, or for someone who already knows how to create and load files on a traditional web server to offer this functionality in new apps. Thus we can expect many apps to take advantage of these widely expected features, similar to, but more functional and easier to use than the current web. The guaranteed permanence of download links will no doubt also lead to the creation of new and innovative kinds of application. +An additional aim is to make it easy for SAFE web apps to create content, save it to the network, and to have a flexible and easy to use API for displaying it within the application. Native applications will also benefit from easier ways to share data they generate with users (e.g. as download links) or which is easily sent to other applications or web apps, using a standard that every application can support with minimal development effort. +Providing access to SAFE files without requiring publication via SAFE DNS, makes it easy to port existing apps, or for someone who already knows how to create and load files using traditional methods to offer this functionality in new apps. Thus we can expect many apps to take advantage of these widely expected features, similar to, but more functional and easier to use than the current web. The guaranteed permanence of download links will no doubt also lead to the creation of new and innovative kinds of application. ## Required Functionality -1) File Sharing: users will find it useful if they can share files without first creating a public id and service. They would need to have a way of identifying the file to be shared and obtaining a URL for it, which would require an application to provide a suitable URL for a given file. This RFC recommends extending the API so that applications can do this, but doesn't consider any such applications or how they work. +**1) File Sharing:** users will find it useful if they can share files without first creating a public id and service. They would need to have a way of identifying the file to be shared and obtaining a URL for it, which would require an application to provide a suitable URL for a given file. This RFC recommends extending the API so that applications can do this, but doesn't consider any such applications or how they offer this functionality. -2) Apps That Create Files: if a web app creates a file it may wish to present the file by embedding a link to it within the HTML of the app - by creating a URL for the file and embedding a link to it in the DOM. For example, an application might capture images using the webcam, store these on the network, and display them within the app by embedding image links within its HTML. To do this using SAFE NFS would be difficult, either needing a public ID and service to be created (probably one per such application). Alternatively the app have to implement a way to read files stored on SAFE NFS and then render them directly on the page, which is difficult and would make it harder for the developer to take advantage of this functionality, and lose the advantages of doing this directly in HTML. +**2) Apps That Create Files:** if a web app creates a file it may wish to present the file by embedding a link to it within the HTML of the app, by embedding the link in its DOM. For example, an application might capture images using the webcam, store these on the network, and display them within the app by embedding image links within its HTML. To do this using SAFE NFS would be difficult, either needing a public ID and service to be created (possibly one per such application). Alternatively the app have to implement a way to read files stored on SAFE NFS and then render them directly on the page, which is difficult and would make it harder for the developer to take advantage of this functionality, and lose the advantages of doing this directly in HTML. ## Detailed design -We need a format of safe: URL which SAFE Beaker (and other compatible browsers) can recognise and use to access content stored on SAFE network without reference to a SAFE public ID. Each file stored on SAFE network is addressed using a cryptographic hash, so we can use that, but we need to consider different kinds of file and how to provide expected and appropriate behaviour, depending on how the file is stored (e.g. as immutable data, mutable data, key-value store etc.), how it is identified by the user (e.g. via a directory listing obtained from SAFE NFS), and the type of the file (e.g. text, PDF, HTML, binary, video etc.). +We need a format of `safe:` URL which SAFE Beaker (and other compatible browsers) can recognise and use to access content stored on SAFE network without reference to a SAFE public ID. Each file stored on SAFE network is addressed using a cryptographic hash, so we can use that, but we need to consider different kinds of file and how to provide expected and appropriate behaviour, depending on how the file is stored (e.g. as immutable data, mutable data, key-value store etc.), how it is identified by the user (e.g. via a directory listing obtained from SAFE NFS), and the type of the file (e.g. text, PDF, HTML, binary, video etc.). This is tricky - to cover all cases with sensible defaults so the following is a proposal as a starting point which will probably be subject to discussion and revision. ### SAFE Web URL Structure -To recap existing functionality, a `safe:` website URL is similar to a standard web URL, but always begins with `safe:` rather than `http` or `https`. This link protocol indicates a resource located on the SAFE network. For a website published on SAFE network, we create a public ID and service using SAFE NFS, and this allows SAFE Beaker to interpret URLs of the following form: - -In the following, square brackets denote an optional component. So `[service.]publicid` means either a value denoting a `service` followed by a '.' followed by a value denoting a `publicid`, or just a value denoting a `publicid`. +To recap existing functionality, a `safe:` website URL is similar to a standard web URL, but always begins with `safe` rather than `http` or `https`. This link protocol indicates the resource is to be located on the SAFE network. For a website published on SAFE network, we create a public ID and service using SAFE NFS, and this allows SAFE Beaker to interpret URLs of the following form: `safe://[service.]publicid[/filepath]` @@ -62,6 +59,8 @@ In the following, square brackets denote an optional component. So `[service.]pu `filepath` is also optional, and if service is ommitted or given as `www`, the filepath defaults to `index.html` +In the above and subsequent notation, square brackets denote an optional component. So `[service.]publicid` means either a value which equates to a `service` followed by a '.' followed by a value for a `publicid`, or just a value for a `publicid`. + So for example, the following URLs would all access the same file assuming it exists in a directory that has been published at the given service and public ID: @@ -81,7 +80,7 @@ SAFE Beaker must be able to recognise a Hash URL, parse it, and determine what m The complete structure of a Hash URL is then: - `safe::type:hash[:filename]` + `safe:type:hash[:filename]` Where: @@ -102,31 +101,40 @@ This RFC does not consider the details or use of `folder:` or `dns:` or other ty The author believes that this method of addressing SAFE NFS stored files will be feasible according to [RFC0046-new-auth-flow](https://github.com/maidsafe/rfcs/tree/master/text/0046-new-auth-flow) which uses a system of [Containers](https://github.com/maidsafe/rfcs/blob/master/text/0046-new-auth-flow/containers.md#nfs-convention) to emulate filesystem at the NFS level. This though will depend on the decisions taken around implementing emulation NFS and the underlying file structures. +#### Extensibility +This URL scheme is simple and retains the potential to be extended, but to provide for additional data types, and to control their retrieval and interpretation. + +For example, values within a key-value store (in a MutableData object) could be addressed directly by extending the Hash URL with parameters. So for a given application, appending `?key=postaladdress&encoding=json` to a URL could be used to retrieve the value of `name` from a MutableData object and parse it ready for use. + ### Behaviour Options - Discussion -Disregarding any limitations imposed by the implementation, we should in reviewing this RFC consider the viability and usefulness SAFE Hash URLs being either permanent or temporary. Assuming the shared file is immutable, we should be able to guarantee that the URL will also be permanent. If the file were either mutable or deliberately referenced via a mutable data structure, then the URL could be designed to be impermanent. +Assuming the shared file is immutable, we should be able to guarantee that the URL will also be permanent. If the file were either mutable or deliberately referenced via a mutable data structure, then the URL could be impermanent by design. So disregarding at this stage any limitations imposed by the implementation, we should in reviewing this RFC consider the viability and usefulness designing SAFE Hash URLs to be *either* permanent or temporary. -**IMPORTANT:** At this point both options are on the table so we can debate what is expected and desirable, but as is stands this RFC proposes to implement only permanent SAFE Hash URLs. +**IMPORTANT:** At this point both options are on the table so we can debate what is expected and desirable, but as is stands this RFC proposes to implement only permanent SAFE Hash URLs. Some points for discussion are presented below. #### Permanent URLs If all SAFE Hash URL's are valid forever, we have the advantage of certainty, and the feature of permanent access to all files shared in this manner. -It wouldn't ensure that SAFE websites were always available forever and immutable, because generally these utilise ordinary SAFE URLs that map to a mutable file structure, but the advent of *Permanent URLs* would make it *possible* to create a SAFE web app or website that was easy to inspect and verify as permanent and immutable. +It wouldn't ensure that SAFE websites were always available forever and immutable, because generally these utilise ordinary SAFE URLs that map to a mutable file structure, but the advent of *Permanent URLs* would make it *possible* to create a SAFE web app or website that could be inspected to verify it as permanent and immutable. The question is whether we would want to limit SAFE Hash URLs to only allow sharing of immutable data, or to allow permanent *and* impermanent URLs (see next). A guarantee of permanence for all SAFE Hash URLs would have to be verifiable (i.e. within safe_core), because it would not be desirable for people to assume that these were permanent and immutable if there was a way of subverting this with a customised SAFE Hash URL maker). #### Impermanent URLs -There are many good things about permanent URLs, but also the disadvantage for users who accidentally share something that can never subsequently be unshared. There are also things people want to be able to share temporarily. For example, so an intended recipient access them, but later be invalidated to reduce the chance that someone else later obtains the URL and gains access to the file. Also, what one shares one day, or in one state of mind, might later be regretted and so in certain circumstances it might also be useful to provide a way to generate URLs that can be invalidated. Impermanence is also likely to be expected behaviour, so if it is not, we might need to find ways to make that very clear. +There are many good things about permanent URLs, but also the disadvantage for users who accidentally share something that can never subsequently be unshared. There are also things people want to be able to share temporarily. For example, in order that the intended recipient access them, but later be invalidated to reduce the chance that someone else who discovers the URL can gain access to the file. -If we allow both permanent and impermanent URLs it would be sensible to indicate this as part of the URL so that users know what they are sharing and receiving. For example, by adding '-p' or '-i' to the end of the `type` as in: +In general, what one shares one day, or in one state of mind, might later be regretted and so in certain circumstances it might also be useful to provide a way to generate URLs that can be invalidated. Impermanence is also likely to be expected behaviour, so if it is not, we might need to find ways to make that very clear. + +On the other hand, applications could be developed to extend file sharing to allow the creation of deletable URLs if this is seen as desired, which also helps make it clear that the default is permanence. + +If we create built-in support for *both* permanent and impermanent URLs it would be sensible to indicate this as part of the URL so that users know what they are sharing and receiving. For example, by adding '-p' or '-i' to the end of the `type` as in: `safe:file-p:4e1243bd22c66e76c2ba9eddc1f91394e57f9f83:Safecoin-white-paper.pdf` -Impermanent URLs would require a *delete Hash URL* API, which raises a further question as to whether or not we should allow different files to appear at the same address at different times, and if not how this could be prevented. +Built-in support for impermanent URLs would require a *delete Hash URL* API, which raises a further question as to whether or not we should allow different files to appear at the same address at different times, and if not how this could be prevented. -**IMPORTANT:** For simplicity's sake, this RFC mandates only permanent URLs (and so only type `file:` and never `file-p` or `file-i`), but this assumes we can enforce permanence, and is up for debate. +**IMPORTANT:** For simplicity's sake, this RFC mandates only permanent URLs (and so only type `file:` and never `file-p:` or `file-i:`), but this assumes we can enforce permanence, and is up for debate. ### SAFE NFS API Changes @@ -148,9 +156,11 @@ safe-js to be extended with `safeNFS.getPermanentLink()` which returns a Promise SAFE Beaker will detect every SAFE Hash URL, parse it according to the above, and use this information to: -a) access the file on the network -b) provide error handling where retrieving the file fails or times out -c) deliver the content appropriately (e.g. embedded in HTML content, embedded download link, or location bar access) including where provided: applying metadata to assist in rendering embedded content, and supplying a default filename when offering to download. + a) access the file on the network + + b) provide error handling where retrieving the file fails or times out + + c) deliver the content appropriately (e.g. embedded in HTML content, embedded download link, or location bar access) including where provided: applying metadata to assist in rendering embedded content, and supplying a default filename when offering to download. ## Drawbacks @@ -162,13 +172,15 @@ There might be privacy or tracking issues associated with sharing URLs if they c ## Alternatives -Not doing this would require users to find other ways of sharing files, which since they are not built in to safe_core and SAFE Beaker will be more cumbersome (for users and for app developers) and impossible to ensure permanence, and harder to trust in terms of security (i.e. from tracking and surveillance). Making such a widely used feature harder than necessary would reduce the rate and potential level of adoption, thereby diminishing its positive overall impact on security and privacy. +Not doing this would require users to find other ways of sharing files, which since they are not built in to safe_core and SAFE Beaker will be more cumbersome (for users and for app developers), impossible to ensure permanence, and harder to trust in terms of security (i.e. from tracking and surveillance). Making such a widely used feature harder than necessary would reduce the rate and potential level of adoption, thereby diminishing its positive overall impact on security and privacy. + +Alternatives would probably consume more network resources than providing support for direct links within safe_core, increasing user PUT costs and network load for a feature we can expect would be used by almost every SAFE user. ### Application based or shared DNS Rather than addressing a file using a hash, we might use either the existing SAFE DNS to share files saved in the user or application managed public storage. SAFE NFS at present would be cumbersome to use for this, requiring every app which wants to generate file URLs to have access to a user's registered public ID and service so this is not practical. -There are changes being considered which may make a DNS based sharing URLs more practical (see [this dev forum post](https://forum.safedev.org/t/file-sharing-at-the-level-of-safe-urls/288/7?u=happybeing) but these have not been decided and have not been considered as part of this RFC. So once clear, it may be worthwhile reviewing this option. +There are changes being considered which may make a DNS based sharing URLs more practical (see [this dev forum post](https://forum.safedev.org/t/file-sharing-at-the-level-of-safe-urls/288/7?u=happybeing)) but these have not been decided and have not been considered as part of this RFC. Once clear, it may be worthwhile reviewing this option. ## Unresolved questions From 4b19bddd527eddb3939bc5e1c319c532c5d32a7e Mon Sep 17 00:00:00 2001 From: theWebalyst Date: Wed, 23 Nov 2016 21:14:03 +0000 Subject: [PATCH 5/8] Renamed and revised after feedback from @ben: you can easily reduce the motivation to: having links point directly network addresses makes it much more accessible and easy to share content. The point you are currently making is a little vage and could easily be done with DNS/NFS without "direct links".Options Generally don't talk about what currently is the case or how it doesn't work. that isn't the focus on the RFC, the focus is what the new feature provides.Options you can mention it in the motivation but, no need to reiterate that in the requirements.Options this shouldn't be an argument for the future but rather a description of it.Options the rest looks very good!Options one side note: I learnt today that in order to retrieve a proper network address we need to have a full DataIdentifier - So rather than saying url by hash, let's call it "url via network address" and define the "hash" as a base64 encoded serialisation of a DataIdentifier as defined in https://github.com/maidsafe/routing/blob/7c00dd14a2c4e4b2a3f7813a13edad119c0efa83/src/data/mod.rs#L111 maidsafe/routing - GitHub routing - Routing - specialised storage DHT Options by doing it this way, this will also be automatically be possible with any future additions of new data types (as they all always must be able to be addressed as DataIdentifiers within the network). So if you could update and use that instead, we are future proof in it.Options I might sound harsh or direct, so I want to state this very clearly: this is a great first draft! Thank you for doing this, it is very well done.Options I'd only like to cut down its length (as it is always easier to get people to read shorter docs rather than longer ones ;) ) and focus it a little clearer. But it is a great first go at it!Options Does my feedback make sense? Let me know if you'd like me to clarify anything or get more details or don't understand it or whatever helps you move this forward.Options One content question: so you don't want any mutable links at all? --- .../0000-file-sharing-by-safe-address-url.md | 185 +++++++++++++++++ .../00000000-nfs-file-sharing-by-hash-url.md | 188 ------------------ 2 files changed, 185 insertions(+), 188 deletions(-) create mode 100644 text/0000-file-sharing-by-safe-address-url/0000-file-sharing-by-safe-address-url.md delete mode 100644 text/0000-nfs-file-sharing-by-hash-url/00000000-nfs-file-sharing-by-hash-url.md diff --git a/text/0000-file-sharing-by-safe-address-url/0000-file-sharing-by-safe-address-url.md b/text/0000-file-sharing-by-safe-address-url/0000-file-sharing-by-safe-address-url.md new file mode 100644 index 0000000..0d57af7 --- /dev/null +++ b/text/0000-file-sharing-by-safe-address-url/0000-file-sharing-by-safe-address-url.md @@ -0,0 +1,185 @@ +# File Sharing By SAFE Address URL + +- Status: proposed +- Type: new feature +- Related components: SAFE Beaker Browser, safe-js, safe_core +- Start Date: 22-11-2016 +- Discussion: (Preliminary to be replaced) - [File sharing at the level of safe: URLs](https://forum.safedev.org/t/file-sharing-at-the-level-of-safe-urls/288?u=happybeing) +- Supersedes: n/a +- Superseded by: n/a + + +## Summary + +Provide a way to obtain a 'safe:' URL based on the SAFE network address for any file stored using SAFE NFS, and so is not reliant on a public ID or SAFE DNS. Such a *SAFE Address URL* would be valid to use as part of a normal HTML page or as a file download link in SAFE Beaker Browser. This avoids the need for anything other than a standard SAFE Browser, such as a special file sharing app, or for the user to have made a public ID and service for the purpose of sharing files (see *Alternatives*). + +## Conventions +- The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC 2119](http://tools.ietf.org/html/rfc2119). + + +## Motivation + +Allowing web applications and SAFE Beaker to access files directly using a SAFE Address URL would allow, for example: + +- an image captured by a SAFE web app and stored in a user's SAFE NFS storage, to be loaded into the DOM of the app, or into an HTML document generated by the app, and displayed within by the browser as part of the app UI or saved HTML. + +- a *Share file...* command, easily implemented in any app, which offers up a SAFE Address URL for a file saved to or chosen from NFS 'drive'. This URL can be sent to anyone who, provided the file is public, could then access the image with just a SAFE web browser. Private files could also be accessed this way but only by the authorised owner, as part of their browser history, or as quick access links in other applications or documents. + +Without this feature, the SAFE NFS user would need to use a specialist file sharing application to share a file, and files accessed through such links would be less secure, hard to trust or guarantee permanence. + +Providing access to SAFE files without requiring publication via SAFE DNS, makes it easy to port existing apps, or for someone who already knows how to create and load files using traditional methods to offer this functionality in new apps. + +The ability to guaranteed permanence of download links will no doubt also lead to the creation of new and innovative kinds of application. + +## Required Functionality + +A way of identifying the file to be shared and obtaining a SAFE Address URL for it. + +For SAFE Beaker Browser to recognise and process a SAFE Address URL differently from an ordinary `safe:` URL. + + +## Detailed design + +We need a format of `safe:` URL which SAFE Beaker (and other compatible browsers) can recognise and use to access content stored on SAFE network without reference to a SAFE public ID. + +Each file stored on SAFE network is addressed using a [DataIdentifier](https://github.com/maidsafe/routing/blob/7c00dd14a2c4e4b2a3f7813a13edad119c0efa83/src/data/mod.rs#L111), so we can use that to address the file on the network. + +We must also consider different kinds of file and how to provide expected and appropriate behaviour, depending on how the file is stored (e.g. as immutable data, mutable data, key-value store etc.), how it is identified by the user (e.g. via a directory listing obtained from SAFE NFS), and the type of the file (e.g. text, PDF, HTML, binary, video etc.). + +### SAFE Web URL Structure + +To recap existing functionality, a `safe:` website URL is similar to a standard web URL, but always begins with `safe` rather than `http` or `https`. This link protocol indicates the resource is to be located on the SAFE network. For a website published on SAFE network, we create a public ID and service using SAFE NFS, and this allows SAFE Beaker to interpret URLs of the following form: + + `safe://[service.]publicid[/filepath]` + +`service` is optional, and defaults to `www` + +`filepath` is also optional, and if service is ommitted or given as `www`, the filepath defaults to `index.html` + +NOTE: In the above and subsequent notation, square brackets denote an optional component. So `[service.]publicid` means either a value which equates to a `service` followed by a '.' followed by a value for a `publicid`, or just a value for a `publicid`. So for example, the following URLs would all access the same file assuming it exists in a directory that has been published at the given service and public ID: + + `safe://rfcs` + `safe://www.rfcs` + `safe://rfcs/index.html` + `safe://www.rfcs/index.html` + +All the above is existing functionality, so the format of *SAFE Address URLs* must always be distinguishable from this. + +### SAFE Address URL Structure + +SAFE Beaker must be able to recognise a SAFE Address URL, parse it, and determine what method to use to access the file, and what if any meta data to provide with the response it delivers. We can also choose to provide a way for a human readable filename to be conveyed as part of the URL, rather than base this on the address or having the user type it in. + +**A SAFE Address URL always begins with `safe::`** This allows us to continue to permit a normal `safe:` URL to omit or include the conventional but redundant '//' that we are accustomed to in web URLs. + +The complete structure of a SAFE Address URL is then: + + `safe:type:address[:filename]` + +Where: + +- `type` is one of `file:` or `folder:` or `dns:`. So far I have only discussed retrieving regular files stored with SAFE NFS, but including a type instructs the client on the type of data being retrieved access and how to make use of it, while providing for support to added later for additional types, with backwards compatibility. + +- `address` is the a base64 encoded serialisation of a [DataIdentifier](https://github.com/maidsafe/routing/blob/7c00dd14a2c4e4b2a3f7813a13edad119c0efa83/src/data/mod.rs#L111). + +- `filename` is the URI encoded value of a string that must not include the path separator character '/' (i.e it is a filename without a path, and with an optional file extension). + +Examples might be: + + `safe:file:U2VjdXJlIEFjY2VzcyBGb3IgRXZlcnlvbmU=:happybeing.png` + `safe:file:U2VjdXJlIEFjY2VzcyBGb3IgRXZlcnlvbmU=:Safecoin-white-paper.pdf` + `safe:file:U2VjdXJlIEFjY2VzcyBGb3IgRXZlcnlvbmU=` + + +This RFC does not consider the details or use of `folder:` or `dns:` or other types as these can be defined later. For now we only need implement type `file:` + +The author believes that this method of addressing SAFE NFS stored files will be feasible according to [RFC0046-new-auth-flow](https://github.com/maidsafe/rfcs/tree/master/text/0046-new-auth-flow) which uses a system of [Containers](https://github.com/maidsafe/rfcs/blob/master/text/0046-new-auth-flow/containers.md#nfs-convention) to emulate file system at the NFS level. + +This though will depend on the decisions taken around implementing emulation NFS and the underlying NFS file structures. + +**Permanent URLs:** SAFE NFS files are immutable data, so a SAFE Address URL will by default be a permanent link to immutable data. + +#### Extensibility +This URL scheme is simple and retains the potential to be extended, but to provide for additional data types, and to control their retrieval and interpretation. + +For example, values within a key-value store (in a MutableData object) could be addressed directly by extending the Address URL with parameters. So for a given application, appending `?key=postaladdress&encoding=json` to a URL could be used to retrieve the value of `name` from a MutableData object and parse it ready for use. + +### Behaviour Options - Discussion + +Assuming the shared file is immutable, we can guarantee that the SAFE Address URL will also be permanent. If the file were either mutable or deliberately referenced via a mutable data structure, then the URL could be impermanent by design. So disregarding at this stage any limitations imposed by the implementation, we should in reviewing this RFC consider the viability and usefulness designing SAFE Address URLs to be *either* permanent or impermanent. + +**IMPORTANT:** At this point both options are on the table so we can debate what is expected and desirable, but as is stands this RFC proposes to implement only permanent SAFE Address URLs. Some points for discussion are presented below. + +#### Permanent URLs + +If all SAFE Address URLs are valid forever, we have the advantage of certainty, and the feature of permanent access to all files shared in this manner. + +It wouldn't ensure that SAFE websites were always available forever and immutable, because generally these utilise ordinary SAFE URLs that map to a mutable file structure, but the advent of *Permanent URLs* would make it *possible* to create a SAFE web app or website that could be inspected to verify it as permanent and immutable. + +The question is whether we would want to limit SAFE Address URLs to only allow sharing of immutable data, or to allow permanent *and* impermanent URLs (see next). A guarantee of permanence for all SAFE Address URLs would have to be verifiable (i.e. within safe_core), because it would not be desirable for people to assume that these were permanent and immutable if there was a way of subverting this with a customised SAFE Address URL maker). + +#### Impermanent URLs + +There are many good things about permanent URLs, but also the disadvantage for users who accidentally share something that can never subsequently be unshared. There are also things people want to be able to share temporarily. For example, in order that the intended recipient access them, but later be invalidated to reduce the chance that someone else who discovers the URL can gain access to the file. + +In general, what one shares one day, or in one state of mind, might later be regretted and so in certain circumstances it might also be useful to provide a way to generate URLs that can be invalidated. Impermanence is also likely to be expected behaviour, so if it is not, we might need to find ways to make that very clear. + +On the other hand, applications could be developed to extend file sharing to allow the creation of deletable URLs if this is seen as desired, which also helps make it clear that the default is permanence. + +If we create built-in support for *both* permanent and impermanent URLs it would be sensible to indicate this as part of the URL so that users know what they are sharing and receiving. For example, by adding '-p' or '-i' to the end of the `type` as in: + + `safe:file-p:U2VjdXJlIEFjY2VzcyBGb3IgRXZlcnlvbmU=:Safecoin-white-paper.pdf` + +Adding support for impermanent URLs would require a *delete SAFE Address URL* API, which raises a further question as to whether or not we should allow different files to appear at the same address at different times, and if not then how this could be prevented. It would mean that SAFE Address URLs could not always be guaranteed to be permanent, which might be undesirable. It also creates a problem for anyone in possession of a SAFE Address URL: not being able to tell if it is permanent or not. + +**IMPORTANT:** For simplicity's sake, this RFC mandates only permanent URLs (and so only type `file:` and never `file-p:` or `file-i:`) which means that with this proposal we can always guarantee SAFE Address URLs are permanent, and we can leave the tricky questions for a later upgrade discussion. + +### SAFE NFS API Changes + +Assuming only *permanent SAFE Address URLs* are supported, SAFE NFS API to be extended with: + + `getPermanentLink(fileName: String) -> Future` + + `fileName` - the full path of an existing file in the user's NFS storage + +This returns a permanently valid URL for fileName formatted as a SAFE Address URL including `type`, `address` and `filename` elements. + +If we later support impermanent URLs a suitably named function can be added which would use a type of `file-i` instead of `file` + +### safe-js Changes + +safe-js to be extended with `safeNFS.getPermanentLink()` which returns a Promise that resolves to a SAFE Address URL as above. + +### SAFE Beaker Changes + +SAFE Beaker will detect every SAFE Address URL, parse it according to the above, and use this information to: + + a) access the file on the network + + b) provide error handling where retrieving the file fails or times out + + c) deliver the content appropriately (e.g. embedded in HTML content, embedded download link, or location bar access) including where provided: applying metadata to assist in rendering embedded content, and supplying a default filename when offering to download. + + +## Drawbacks + +Relying on the implementation of SAFE NFS storage means this functionality would be difficult to maintain if that ever changes. + +There might be privacy or tracking issues associated with sharing URLs if they could somehow be linked back to a user or account, though none are immediately apparent, except for ill considered naming of files. + + +## Alternatives + +Not doing this would require users to find other ways of sharing files, which since they are not built in to safe_core and SAFE Beaker will be more cumbersome (for users and for app developers), impossible to ensure permanence, and harder to trust in terms of security (i.e. from tracking and surveillance). Making such a widely used feature harder than necessary would reduce the rate and potential level of adoption, thereby diminishing its positive overall impact on security and privacy. + +Alternatives would probably consume more network resources than providing support for direct links within safe_core, increasing user PUT costs and network load for a feature we can expect would be used by almost every SAFE user. + +### Application based or shared DNS + +Rather than addressing a file using a network address, we might use either the existing SAFE DNS to share files saved in the user or application managed public storage. SAFE NFS at present would be cumbersome to use for this, requiring every app which wants to generate file URLs to have access to a user's registered public ID and service so this is not practical. + +There are changes being considered which may make a DNS based sharing URLs more practical (see [this dev forum post](https://forum.safedev.org/t/file-sharing-at-the-level-of-safe-urls/288/7?u=happybeing)) but these have not been decided and have not been considered as part of this RFC. Once clear, it may be worthwhile reviewing this option. + + +## Unresolved questions + +It needs to be confirmed that proposed modifications to the implementation of SAFE NFS [Containers](https://github.com/maidsafe/rfcs/blob/master/text/0046-new-auth-flow/containers.md#nfs-convention) would support this functionality if adopted. \ No newline at end of file diff --git a/text/0000-nfs-file-sharing-by-hash-url/00000000-nfs-file-sharing-by-hash-url.md b/text/0000-nfs-file-sharing-by-hash-url/00000000-nfs-file-sharing-by-hash-url.md deleted file mode 100644 index e1ca2a9..0000000 --- a/text/0000-nfs-file-sharing-by-hash-url/00000000-nfs-file-sharing-by-hash-url.md +++ /dev/null @@ -1,188 +0,0 @@ -# NFS File Sharing By Hash URL - -- Status: proposed -- Type: new feature -- Related components: SAFE Beaker Browser, safe-js, safe_core -- Start Date: 22-11-2016 -- Discussion: (Preliminary to be replaced) - [File sharing at the level of safe: URLs](https://forum.safedev.org/t/file-sharing-at-the-level-of-safe-urls/288?u=happybeing) -- Supersedes: n/a -- Superseded by: n/a - - -## Summary - -Provide a way to obtain a 'safe:' URL based on the file hash (network address) for any file stored using SAFE NFS, and so is not reliant on a public ID or SAFE DNS. Such a *SAFE Hash URL* would be valid to use as part of a normal HTML page or as a file download link in SAFE Beaker Browser. - -This would allow, for example: - -- an image captured by a SAFE web app and stored in a user's SAFE NFS storage, to be loaded into the DOM of the app, or into an HTML document generated by the app, and displayed within by the browser as part of the app UI or saved HTML. - -- a *Share file...* command, easily implemented in any app, which offers up a SAFE Hash URL for a file saved to or chosen from NFS 'drive'. This Hash URL can be sent to anyone who, provided the file is public, could then access the image with just a SAFE web browser. Private files could also be accessed this way but only by the authorised owner, as part of their browser history, or as quick access links in other applications or documents. - -An important feature is avoiding the need for anything other than a standard SAFE Browser, such as a special file sharing app, or for the user to have made a public ID and service for the purpose of sharing files (see *Alternatives*). - - -## Conventions -- The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC 2119](http://tools.ietf.org/html/rfc2119). - - -## Motivation - -Easy sharing of files via web links is one of the "killer" features of the current web and it is important for SAFE to avoid losing expected and valuable functionality in order to encourage and accelerate adoption. This proposal would exceed current expected functionality (e.g. by making links permanent) and be easier to use because all SAFE users will already have everything they need (no need to sign up to a separate cloud service, and no need to upload any file once stored on SAFE NFS). Sharing SAFE URLs with those not already on SAFE network would not be so smooth because the recipient will have to join the network (or perhaps wait for a SAFE Browser to load on activating the link), but this is not wholly negative because it in part encourages adoption, and then a default of having a SAFE Browser loaded. - -Without this feature, the SAFE NFS user would need to use a specialist file sharing application to share a file, and files accessed through such links would be less secure, hard to trust or guarantee permanence. - -An additional aim is to make it easy for SAFE web apps to create content, save it to the network, and to have a flexible and easy to use API for displaying it within the application. Native applications will also benefit from easier ways to share data they generate with users (e.g. as download links) or which is easily sent to other applications or web apps, using a standard that every application can support with minimal development effort. - -Providing access to SAFE files without requiring publication via SAFE DNS, makes it easy to port existing apps, or for someone who already knows how to create and load files using traditional methods to offer this functionality in new apps. Thus we can expect many apps to take advantage of these widely expected features, similar to, but more functional and easier to use than the current web. The guaranteed permanence of download links will no doubt also lead to the creation of new and innovative kinds of application. - -## Required Functionality - -**1) File Sharing:** users will find it useful if they can share files without first creating a public id and service. They would need to have a way of identifying the file to be shared and obtaining a URL for it, which would require an application to provide a suitable URL for a given file. This RFC recommends extending the API so that applications can do this, but doesn't consider any such applications or how they offer this functionality. - -**2) Apps That Create Files:** if a web app creates a file it may wish to present the file by embedding a link to it within the HTML of the app, by embedding the link in its DOM. For example, an application might capture images using the webcam, store these on the network, and display them within the app by embedding image links within its HTML. To do this using SAFE NFS would be difficult, either needing a public ID and service to be created (possibly one per such application). Alternatively the app have to implement a way to read files stored on SAFE NFS and then render them directly on the page, which is difficult and would make it harder for the developer to take advantage of this functionality, and lose the advantages of doing this directly in HTML. - - -## Detailed design - -We need a format of `safe:` URL which SAFE Beaker (and other compatible browsers) can recognise and use to access content stored on SAFE network without reference to a SAFE public ID. Each file stored on SAFE network is addressed using a cryptographic hash, so we can use that, but we need to consider different kinds of file and how to provide expected and appropriate behaviour, depending on how the file is stored (e.g. as immutable data, mutable data, key-value store etc.), how it is identified by the user (e.g. via a directory listing obtained from SAFE NFS), and the type of the file (e.g. text, PDF, HTML, binary, video etc.). - -This is tricky - to cover all cases with sensible defaults so the following is a proposal as a starting point which will probably be subject to discussion and revision. - -### SAFE Web URL Structure - -To recap existing functionality, a `safe:` website URL is similar to a standard web URL, but always begins with `safe` rather than `http` or `https`. This link protocol indicates the resource is to be located on the SAFE network. For a website published on SAFE network, we create a public ID and service using SAFE NFS, and this allows SAFE Beaker to interpret URLs of the following form: - - `safe://[service.]publicid[/filepath]` - -`service` is optional, and defaults to `www` - -`filepath` is also optional, and if service is ommitted or given as `www`, the filepath defaults to `index.html` - -In the above and subsequent notation, square brackets denote an optional component. So `[service.]publicid` means either a value which equates to a `service` followed by a '.' followed by a value for a `publicid`, or just a value for a `publicid`. - -So for example, the following URLs would all access the same file assuming it exists in a directory that has been published at the given service and public ID: - - - `safe://rfcs` - `safe://www.rfcs` - `safe://rfcs/index.html` - `safe://www.rfcs/index.html` - - -All the above is existing functionality, so the format of *SAFE Hash URLs* must always be distinguishable from this. - -### SAFE Hash URL Structure - -SAFE Beaker must be able to recognise a Hash URL, parse it, and determine what method to use to access the file, and what if any meta data to provide with the response it delivers. We can also choose to provide a way for a human readable filename to be conveyed as part of the URL, rather than base this on the hash or require the downloading user to type it in. - -**A Hash URL always begins with `safe::`** This allows us to continue to permit a normal `safe:` URL to omit or include the conventional but redundant '//' that we are accustomed to in web URLs. - -The complete structure of a Hash URL is then: - - `safe:type:hash[:filename]` - -Where: - -- `type` is one of `file:` or `folder:` or `dns:`. So far I have only discussed retrieving regular files stored with SAFE NFS, but including a type instructs the client on the type of data being retrieved access and how to make use of it, while providing for support to added later for additional types, with backwards compatibility. - -- `hash` is the hash address in hexadecimal which resolves to a [FILE](https://github.com/maidsafe/safe_core/blob/eb172c2718aa43b78e644f998cf65a5ff92dfa4b/src/nfs/file.rs#L29) entry. - -- `filename` is the URI encoded value of a string that must not include the path separator character '/' (i.e it is a filename without a path, and with an optional file extension). - -Examples might be: - - `safe:file:4e1243bd22c66e76c2ba9eddc1f91394e57f9f83:happybeing.png` - `safe:file:4e1243bd22c66e76c2ba9eddc1f91394e57f9f83:Safecoin-white-paper.pdf` - `safe:file:4e1243bd22c66e76c2ba9eddc1f91394e57f9f83` - - -This RFC does not consider the details or use of `folder:` or `dns:` or other types as these can be defined later. For now we only need implement type `file:` - -The author believes that this method of addressing SAFE NFS stored files will be feasible according to [RFC0046-new-auth-flow](https://github.com/maidsafe/rfcs/tree/master/text/0046-new-auth-flow) which uses a system of [Containers](https://github.com/maidsafe/rfcs/blob/master/text/0046-new-auth-flow/containers.md#nfs-convention) to emulate filesystem at the NFS level. This though will depend on the decisions taken around implementing emulation NFS and the underlying file structures. - -#### Extensibility -This URL scheme is simple and retains the potential to be extended, but to provide for additional data types, and to control their retrieval and interpretation. - -For example, values within a key-value store (in a MutableData object) could be addressed directly by extending the Hash URL with parameters. So for a given application, appending `?key=postaladdress&encoding=json` to a URL could be used to retrieve the value of `name` from a MutableData object and parse it ready for use. - -### Behaviour Options - Discussion - -Assuming the shared file is immutable, we should be able to guarantee that the URL will also be permanent. If the file were either mutable or deliberately referenced via a mutable data structure, then the URL could be impermanent by design. So disregarding at this stage any limitations imposed by the implementation, we should in reviewing this RFC consider the viability and usefulness designing SAFE Hash URLs to be *either* permanent or temporary. - -**IMPORTANT:** At this point both options are on the table so we can debate what is expected and desirable, but as is stands this RFC proposes to implement only permanent SAFE Hash URLs. Some points for discussion are presented below. - -#### Permanent URLs - -If all SAFE Hash URL's are valid forever, we have the advantage of certainty, and the feature of permanent access to all files shared in this manner. - -It wouldn't ensure that SAFE websites were always available forever and immutable, because generally these utilise ordinary SAFE URLs that map to a mutable file structure, but the advent of *Permanent URLs* would make it *possible* to create a SAFE web app or website that could be inspected to verify it as permanent and immutable. - -The question is whether we would want to limit SAFE Hash URLs to only allow sharing of immutable data, or to allow permanent *and* impermanent URLs (see next). A guarantee of permanence for all SAFE Hash URLs would have to be verifiable (i.e. within safe_core), because it would not be desirable for people to assume that these were permanent and immutable if there was a way of subverting this with a customised SAFE Hash URL maker). - -#### Impermanent URLs - -There are many good things about permanent URLs, but also the disadvantage for users who accidentally share something that can never subsequently be unshared. There are also things people want to be able to share temporarily. For example, in order that the intended recipient access them, but later be invalidated to reduce the chance that someone else who discovers the URL can gain access to the file. - -In general, what one shares one day, or in one state of mind, might later be regretted and so in certain circumstances it might also be useful to provide a way to generate URLs that can be invalidated. Impermanence is also likely to be expected behaviour, so if it is not, we might need to find ways to make that very clear. - -On the other hand, applications could be developed to extend file sharing to allow the creation of deletable URLs if this is seen as desired, which also helps make it clear that the default is permanence. - -If we create built-in support for *both* permanent and impermanent URLs it would be sensible to indicate this as part of the URL so that users know what they are sharing and receiving. For example, by adding '-p' or '-i' to the end of the `type` as in: - - `safe:file-p:4e1243bd22c66e76c2ba9eddc1f91394e57f9f83:Safecoin-white-paper.pdf` - -Built-in support for impermanent URLs would require a *delete Hash URL* API, which raises a further question as to whether or not we should allow different files to appear at the same address at different times, and if not how this could be prevented. - -**IMPORTANT:** For simplicity's sake, this RFC mandates only permanent URLs (and so only type `file:` and never `file-p:` or `file-i:`), but this assumes we can enforce permanence, and is up for debate. - -### SAFE NFS API Changes - -Assuming only *permanent SAFE Hash URLs* are supported, SAFE NFS API to be extended with: - - `getPermanentLink(fileName: String) -> Future` - - `fileName` - the full path of an existing file in the user's NFS storage - -This returns a permanently valid URL for fileName formatted as a SAFE Hash URL including `type`, `hash` and `filename` elements. - -If we later support impermanent URLs a suitably named function can be added which would use a type of `file-i` instead of `file` - -### safe-js Changes - -safe-js to be extended with `safeNFS.getPermanentLink()` which returns a Promise that resolves to a SAFE Hash URL as above. - -### SAFE Beaker Changes - -SAFE Beaker will detect every SAFE Hash URL, parse it according to the above, and use this information to: - - a) access the file on the network - - b) provide error handling where retrieving the file fails or times out - - c) deliver the content appropriately (e.g. embedded in HTML content, embedded download link, or location bar access) including where provided: applying metadata to assist in rendering embedded content, and supplying a default filename when offering to download. - - -## Drawbacks - -It relies on the implementation of SAFE NFS storage and might be difficult to maintain if that ever changes. - -There might be privacy or tracking issues associated with sharing URLs if they could somehow be linked back to a user or account, though none are immediately apparent, save ill considered naming of files. - - -## Alternatives - -Not doing this would require users to find other ways of sharing files, which since they are not built in to safe_core and SAFE Beaker will be more cumbersome (for users and for app developers), impossible to ensure permanence, and harder to trust in terms of security (i.e. from tracking and surveillance). Making such a widely used feature harder than necessary would reduce the rate and potential level of adoption, thereby diminishing its positive overall impact on security and privacy. - -Alternatives would probably consume more network resources than providing support for direct links within safe_core, increasing user PUT costs and network load for a feature we can expect would be used by almost every SAFE user. - -### Application based or shared DNS - -Rather than addressing a file using a hash, we might use either the existing SAFE DNS to share files saved in the user or application managed public storage. SAFE NFS at present would be cumbersome to use for this, requiring every app which wants to generate file URLs to have access to a user's registered public ID and service so this is not practical. - -There are changes being considered which may make a DNS based sharing URLs more practical (see [this dev forum post](https://forum.safedev.org/t/file-sharing-at-the-level-of-safe-urls/288/7?u=happybeing)) but these have not been decided and have not been considered as part of this RFC. Once clear, it may be worthwhile reviewing this option. - - -## Unresolved questions - -It needs to be confirmed that proposed modifications to the implementation of SAFE NFS [Containers](https://github.com/maidsafe/rfcs/blob/master/text/0046-new-auth-flow/containers.md#nfs-convention) would support this functionality if adopted. \ No newline at end of file From 48e1c5bcf4af2e23697ce7b8c32014b15dfdfa7c Mon Sep 17 00:00:00 2001 From: theWebalyst Date: Wed, 23 Nov 2016 21:31:59 +0000 Subject: [PATCH 6/8] Minor rewording and formatting tweaks. --- .../0000-file-sharing-by-safe-address-url.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/text/0000-file-sharing-by-safe-address-url/0000-file-sharing-by-safe-address-url.md b/text/0000-file-sharing-by-safe-address-url/0000-file-sharing-by-safe-address-url.md index 0d57af7..400bd1d 100644 --- a/text/0000-file-sharing-by-safe-address-url/0000-file-sharing-by-safe-address-url.md +++ b/text/0000-file-sharing-by-safe-address-url/0000-file-sharing-by-safe-address-url.md @@ -11,7 +11,7 @@ ## Summary -Provide a way to obtain a 'safe:' URL based on the SAFE network address for any file stored using SAFE NFS, and so is not reliant on a public ID or SAFE DNS. Such a *SAFE Address URL* would be valid to use as part of a normal HTML page or as a file download link in SAFE Beaker Browser. This avoids the need for anything other than a standard SAFE Browser, such as a special file sharing app, or for the user to have made a public ID and service for the purpose of sharing files (see *Alternatives*). +Provide a way to obtain a `safe:` URL based on the SAFE network address for any file stored using SAFE NFS, and which does not rely on a public ID or the SAFE DNS. Such a *SAFE Address URL* would be valid to use as part of a normal HTML page or as a file download link in SAFE Beaker Browser. This avoids the need for a special file sharing app, or for the user to have made a public ID and service for the purpose of sharing files (see *Alternatives*). All that is needed is a way to generate a SAFE Address URL for a given file. It can then be downloaded using only a standard SAFE Browser, or accessed by any SAFE app. ## Conventions - The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC 2119](http://tools.ietf.org/html/rfc2119). @@ -23,7 +23,7 @@ Allowing web applications and SAFE Beaker to access files directly using a SAFE - an image captured by a SAFE web app and stored in a user's SAFE NFS storage, to be loaded into the DOM of the app, or into an HTML document generated by the app, and displayed within by the browser as part of the app UI or saved HTML. -- a *Share file...* command, easily implemented in any app, which offers up a SAFE Address URL for a file saved to or chosen from NFS 'drive'. This URL can be sent to anyone who, provided the file is public, could then access the image with just a SAFE web browser. Private files could also be accessed this way but only by the authorised owner, as part of their browser history, or as quick access links in other applications or documents. +- a *Share file...* command, easily implemented in any app, which offers up a SAFE Address URL for a file saved to or chosen from NFS 'drive'. This URL can be sent to anyone who, provided the file is public, could then access the file with just a SAFE web browser. Private files could also be accessed this way but only by the authorised owner, as part of their browser history, or as quick access links in other applications or documents. Without this feature, the SAFE NFS user would need to use a specialist file sharing application to share a file, and files accessed through such links would be less secure, hard to trust or guarantee permanence. @@ -99,13 +99,13 @@ This though will depend on the decisions taken around implementing emulation NFS **Permanent URLs:** SAFE NFS files are immutable data, so a SAFE Address URL will by default be a permanent link to immutable data. #### Extensibility -This URL scheme is simple and retains the potential to be extended, but to provide for additional data types, and to control their retrieval and interpretation. +This URL scheme is simple and retains the potential to be extended to provide for additional data types, and to control their retrieval and interpretation. For example, values within a key-value store (in a MutableData object) could be addressed directly by extending the Address URL with parameters. So for a given application, appending `?key=postaladdress&encoding=json` to a URL could be used to retrieve the value of `name` from a MutableData object and parse it ready for use. ### Behaviour Options - Discussion -Assuming the shared file is immutable, we can guarantee that the SAFE Address URL will also be permanent. If the file were either mutable or deliberately referenced via a mutable data structure, then the URL could be impermanent by design. So disregarding at this stage any limitations imposed by the implementation, we should in reviewing this RFC consider the viability and usefulness designing SAFE Address URLs to be *either* permanent or impermanent. +Assuming the shared file is immutable, we can guarantee that the SAFE Address URL will also be permanent. If the file were either mutable or deliberately referenced via a mutable data structure, then the URL could be impermanent by design. So disregarding at this stage any limitations imposed by the implementation, we should in reviewing this RFC consider the viability and usefulness of designing SAFE Address URLs to be *either* permanent or impermanent. **IMPORTANT:** At this point both options are on the table so we can debate what is expected and desirable, but as is stands this RFC proposes to implement only permanent SAFE Address URLs. Some points for discussion are presented below. @@ -129,9 +129,9 @@ If we create built-in support for *both* permanent and impermanent URLs it would `safe:file-p:U2VjdXJlIEFjY2VzcyBGb3IgRXZlcnlvbmU=:Safecoin-white-paper.pdf` -Adding support for impermanent URLs would require a *delete SAFE Address URL* API, which raises a further question as to whether or not we should allow different files to appear at the same address at different times, and if not then how this could be prevented. It would mean that SAFE Address URLs could not always be guaranteed to be permanent, which might be undesirable. It also creates a problem for anyone in possession of a SAFE Address URL: not being able to tell if it is permanent or not. +Adding support for impermanent URLs would require a *delete SAFE Address URL* API, which raises a further question as to whether or not we should allow different files to appear at the same address at different times, and if not then how this could be prevented. It would mean that SAFE Address URLs could not always be guaranteed to be permanent, which might be undesirable. It also creates a problem for anyone in possession of a SAFE Address URL: how to tell if it is permanent or not. -**IMPORTANT:** For simplicity's sake, this RFC mandates only permanent URLs (and so only type `file:` and never `file-p:` or `file-i:`) which means that with this proposal we can always guarantee SAFE Address URLs are permanent, and we can leave the tricky questions for a later upgrade discussion. +**IMPORTANT:** For simplicity's sake, this RFC mandates only permanent URLs (so only type `file:` and never `file-p:` or `file-i:`) which means that with this proposal we can always guarantee SAFE Address URLs are permanent, and we can leave the tricky questions for a later upgrade discussion. ### SAFE NFS API Changes @@ -153,11 +153,11 @@ safe-js to be extended with `safeNFS.getPermanentLink()` which returns a Promise SAFE Beaker will detect every SAFE Address URL, parse it according to the above, and use this information to: - a) access the file on the network +a) access the file on the network - b) provide error handling where retrieving the file fails or times out +b) provide error handling where retrieving the file fails or times out - c) deliver the content appropriately (e.g. embedded in HTML content, embedded download link, or location bar access) including where provided: applying metadata to assist in rendering embedded content, and supplying a default filename when offering to download. +c) deliver the content appropriately (e.g. embedded in HTML content, embedded download link, or location bar access) including where provided: applying metadata to assist in rendering embedded content, and supplying a default filename when offering to download. ## Drawbacks From 23c73927c4e236684db475fd67264faffb8999b4 Mon Sep 17 00:00:00 2001 From: theWebalyst Date: Thu, 24 Nov 2016 14:21:53 +0000 Subject: [PATCH 7/8] Terms revised Address, URL -> Data, URI --- .../0000-file-sharing-by-safe-data-uri.md} | 80 +++++++++---------- 1 file changed, 40 insertions(+), 40 deletions(-) rename text/{0000-file-sharing-by-safe-address-url/0000-file-sharing-by-safe-address-url.md => 0000-file-sharing-by-safe-data-uri/0000-file-sharing-by-safe-data-uri.md} (61%) diff --git a/text/0000-file-sharing-by-safe-address-url/0000-file-sharing-by-safe-address-url.md b/text/0000-file-sharing-by-safe-data-uri/0000-file-sharing-by-safe-data-uri.md similarity index 61% rename from text/0000-file-sharing-by-safe-address-url/0000-file-sharing-by-safe-address-url.md rename to text/0000-file-sharing-by-safe-data-uri/0000-file-sharing-by-safe-data-uri.md index 400bd1d..7221dc5 100644 --- a/text/0000-file-sharing-by-safe-address-url/0000-file-sharing-by-safe-address-url.md +++ b/text/0000-file-sharing-by-safe-data-uri/0000-file-sharing-by-safe-data-uri.md @@ -1,17 +1,17 @@ -# File Sharing By SAFE Address URL +# File Sharing By SAFE Data URI - Status: proposed - Type: new feature - Related components: SAFE Beaker Browser, safe-js, safe_core - Start Date: 22-11-2016 -- Discussion: (Preliminary to be replaced) - [File sharing at the level of safe: URLs](https://forum.safedev.org/t/file-sharing-at-the-level-of-safe-urls/288?u=happybeing) +- Discussion: (Preliminary to be replaced) - [File sharing at the level of safe: URIs](https://forum.safedev.org/t/file-sharing-at-the-level-of-safe-URIs/288?u=happybeing) - Supersedes: n/a - Superseded by: n/a ## Summary -Provide a way to obtain a `safe:` URL based on the SAFE network address for any file stored using SAFE NFS, and which does not rely on a public ID or the SAFE DNS. Such a *SAFE Address URL* would be valid to use as part of a normal HTML page or as a file download link in SAFE Beaker Browser. This avoids the need for a special file sharing app, or for the user to have made a public ID and service for the purpose of sharing files (see *Alternatives*). All that is needed is a way to generate a SAFE Address URL for a given file. It can then be downloaded using only a standard SAFE Browser, or accessed by any SAFE app. +Provide a way to obtain a `safe:` URI based on the SAFE network address for any file stored using SAFE NFS, and which does not rely on a public ID or the SAFE DNS. Such a *SAFE Data URI* would be valid to use as part of a normal HTML page or as a file download link in SAFE Beaker Browser. This avoids the need for a special file sharing app, or for the user to have made a public ID and service for the purpose of sharing files (see *Alternatives*). All that is needed is a way to generate a SAFE Data URI for a given file. It can then be downloaded using only a standard SAFE Browser, or accessed by any SAFE app. ## Conventions - The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC 2119](http://tools.ietf.org/html/rfc2119). @@ -19,11 +19,11 @@ Provide a way to obtain a `safe:` URL based on the SAFE network address for any ## Motivation -Allowing web applications and SAFE Beaker to access files directly using a SAFE Address URL would allow, for example: +Allowing web applications and SAFE Beaker to access files directly using a SAFE Data URI would allow, for example: - an image captured by a SAFE web app and stored in a user's SAFE NFS storage, to be loaded into the DOM of the app, or into an HTML document generated by the app, and displayed within by the browser as part of the app UI or saved HTML. -- a *Share file...* command, easily implemented in any app, which offers up a SAFE Address URL for a file saved to or chosen from NFS 'drive'. This URL can be sent to anyone who, provided the file is public, could then access the file with just a SAFE web browser. Private files could also be accessed this way but only by the authorised owner, as part of their browser history, or as quick access links in other applications or documents. +- a *Share file...* command, easily implemented in any app, which offers up a SAFE Data URI for a file saved to or chosen from NFS 'drive'. This URI can be sent to anyone who, provided the file is public, could then access the file with just a SAFE web browser. Private files could also be accessed this way but only by the authorised owner, as part of their browser history, or as quick access links in other applications or documents. Without this feature, the SAFE NFS user would need to use a specialist file sharing application to share a file, and files accessed through such links would be less secure, hard to trust or guarantee permanence. @@ -33,22 +33,22 @@ The ability to guaranteed permanence of download links will no doubt also lead t ## Required Functionality -A way of identifying the file to be shared and obtaining a SAFE Address URL for it. +A way of identifying the file to be shared and obtaining a SAFE Data URI for it. -For SAFE Beaker Browser to recognise and process a SAFE Address URL differently from an ordinary `safe:` URL. +For SAFE Beaker Browser to recognise and process a SAFE Data URI differently from an ordinary `safe:` URI. ## Detailed design -We need a format of `safe:` URL which SAFE Beaker (and other compatible browsers) can recognise and use to access content stored on SAFE network without reference to a SAFE public ID. +We need a format of `safe:` URI which SAFE Beaker (and other compatible browsers) can recognise and use to access content stored on SAFE network without reference to a SAFE public ID. Each file stored on SAFE network is addressed using a [DataIdentifier](https://github.com/maidsafe/routing/blob/7c00dd14a2c4e4b2a3f7813a13edad119c0efa83/src/data/mod.rs#L111), so we can use that to address the file on the network. We must also consider different kinds of file and how to provide expected and appropriate behaviour, depending on how the file is stored (e.g. as immutable data, mutable data, key-value store etc.), how it is identified by the user (e.g. via a directory listing obtained from SAFE NFS), and the type of the file (e.g. text, PDF, HTML, binary, video etc.). -### SAFE Web URL Structure +### SAFE Web URI Structure -To recap existing functionality, a `safe:` website URL is similar to a standard web URL, but always begins with `safe` rather than `http` or `https`. This link protocol indicates the resource is to be located on the SAFE network. For a website published on SAFE network, we create a public ID and service using SAFE NFS, and this allows SAFE Beaker to interpret URLs of the following form: +To recap existing functionality, a `safe:` website URI is similar to a standard web URI, but always begins with `safe` rather than `http` or `https`. This link protocol indicates the resource is to be located on the SAFE network. For a website published on SAFE network, we create a public ID and service using SAFE NFS, and this allows SAFE Beaker to interpret URIs of the following form: `safe://[service.]publicid[/filepath]` @@ -56,22 +56,22 @@ To recap existing functionality, a `safe:` website URL is similar to a standard `filepath` is also optional, and if service is ommitted or given as `www`, the filepath defaults to `index.html` -NOTE: In the above and subsequent notation, square brackets denote an optional component. So `[service.]publicid` means either a value which equates to a `service` followed by a '.' followed by a value for a `publicid`, or just a value for a `publicid`. So for example, the following URLs would all access the same file assuming it exists in a directory that has been published at the given service and public ID: +NOTE: In the above and subsequent notation, square brackets denote an optional component. So `[service.]publicid` means either a value which equates to a `service` followed by a '.' followed by a value for a `publicid`, or just a value for a `publicid`. So for example, the following URIs would all access the same file assuming it exists in a directory that has been published at the given service and public ID: `safe://rfcs` `safe://www.rfcs` `safe://rfcs/index.html` `safe://www.rfcs/index.html` -All the above is existing functionality, so the format of *SAFE Address URLs* must always be distinguishable from this. +All the above is existing functionality, so the format of *SAFE Data URIs* must always be distinguishable from this. -### SAFE Address URL Structure +### SAFE Data URI Structure -SAFE Beaker must be able to recognise a SAFE Address URL, parse it, and determine what method to use to access the file, and what if any meta data to provide with the response it delivers. We can also choose to provide a way for a human readable filename to be conveyed as part of the URL, rather than base this on the address or having the user type it in. +SAFE Beaker must be able to recognise a SAFE Data URI, parse it, and determine what method to use to access the file, and what if any meta data to provide with the response it delivers. We can also choose to provide a way for a human readable filename to be conveyed as part of the URI, rather than base this on the address or having the user type it in. -**A SAFE Address URL always begins with `safe::`** This allows us to continue to permit a normal `safe:` URL to omit or include the conventional but redundant '//' that we are accustomed to in web URLs. +**A SAFE Data URI always begins with `safe::`** This allows us to continue to permit a normal `safe:` URI to omit or include the conventional but redundant '//' that we are accustomed to in web URIs. -The complete structure of a SAFE Address URL is then: +The complete structure of a SAFE Data URI is then: `safe:type:address[:filename]` @@ -96,62 +96,62 @@ The author believes that this method of addressing SAFE NFS stored files will be This though will depend on the decisions taken around implementing emulation NFS and the underlying NFS file structures. -**Permanent URLs:** SAFE NFS files are immutable data, so a SAFE Address URL will by default be a permanent link to immutable data. +**Permanent URIs:** SAFE NFS files are immutable data, so a SAFE Data URI will by default be a permanent link to immutable data. #### Extensibility -This URL scheme is simple and retains the potential to be extended to provide for additional data types, and to control their retrieval and interpretation. +This URI scheme is simple and retains the potential to be extended to provide for additional data types, and to control their retrieval and interpretation. -For example, values within a key-value store (in a MutableData object) could be addressed directly by extending the Address URL with parameters. So for a given application, appending `?key=postaladdress&encoding=json` to a URL could be used to retrieve the value of `name` from a MutableData object and parse it ready for use. +For example, values within a key-value store (in a MutableData object) could be addressed directly by extending the Data URI with parameters. So for a given application, appending `?key=postaladdress&encoding=json` to a URI could be used to retrieve the value of `name` from a MutableData object and parse it ready for use. ### Behaviour Options - Discussion -Assuming the shared file is immutable, we can guarantee that the SAFE Address URL will also be permanent. If the file were either mutable or deliberately referenced via a mutable data structure, then the URL could be impermanent by design. So disregarding at this stage any limitations imposed by the implementation, we should in reviewing this RFC consider the viability and usefulness of designing SAFE Address URLs to be *either* permanent or impermanent. +Assuming the shared file is immutable, we can guarantee that the SAFE Data URI will also be permanent. If the file were either mutable or deliberately referenced via a mutable data structure, then the URI could be impermanent by design. So disregarding at this stage any limitations imposed by the implementation, we should in reviewing this RFC consider the viability and usefulness of designing SAFE Data URIs to be *either* permanent or impermanent. -**IMPORTANT:** At this point both options are on the table so we can debate what is expected and desirable, but as is stands this RFC proposes to implement only permanent SAFE Address URLs. Some points for discussion are presented below. +**IMPORTANT:** At this point both options are on the table so we can debate what is expected and desirable, but as is stands this RFC proposes to implement only permanent SAFE Data URIs. Some points for discussion are presented below. -#### Permanent URLs +#### Permanent URIs -If all SAFE Address URLs are valid forever, we have the advantage of certainty, and the feature of permanent access to all files shared in this manner. +If all SAFE Data URIs are valid forever, we have the advantage of certainty, and the feature of permanent access to all files shared in this manner. -It wouldn't ensure that SAFE websites were always available forever and immutable, because generally these utilise ordinary SAFE URLs that map to a mutable file structure, but the advent of *Permanent URLs* would make it *possible* to create a SAFE web app or website that could be inspected to verify it as permanent and immutable. +It wouldn't ensure that SAFE websites were always available forever and immutable, because generally these utilise ordinary SAFE URIs that map to a mutable file structure, but the advent of *Permanent URIs* would make it *possible* to create a SAFE web app or website that could be inspected to verify it as permanent and immutable. -The question is whether we would want to limit SAFE Address URLs to only allow sharing of immutable data, or to allow permanent *and* impermanent URLs (see next). A guarantee of permanence for all SAFE Address URLs would have to be verifiable (i.e. within safe_core), because it would not be desirable for people to assume that these were permanent and immutable if there was a way of subverting this with a customised SAFE Address URL maker). +The question is whether we would want to limit SAFE Data URIs to only allow sharing of immutable data, or to allow permanent *and* impermanent URIs (see next). A guarantee of permanence for all SAFE Data URIs would have to be verifiable (i.e. within safe_core), because it would not be desirable for people to assume that these were permanent and immutable if there was a way of subverting this with a customised SAFE Data URI maker). -#### Impermanent URLs +#### Impermanent URIs -There are many good things about permanent URLs, but also the disadvantage for users who accidentally share something that can never subsequently be unshared. There are also things people want to be able to share temporarily. For example, in order that the intended recipient access them, but later be invalidated to reduce the chance that someone else who discovers the URL can gain access to the file. +There are many good things about permanent URIs, but also the disadvantage for users who accidentally share something that can never subsequently be unshared. There are also things people want to be able to share temporarily. For example, in order that the intended recipient access them, but later be invalidated to reduce the chance that someone else who discovers the URI can gain access to the file. -In general, what one shares one day, or in one state of mind, might later be regretted and so in certain circumstances it might also be useful to provide a way to generate URLs that can be invalidated. Impermanence is also likely to be expected behaviour, so if it is not, we might need to find ways to make that very clear. +In general, what one shares one day, or in one state of mind, might later be regretted and so in certain circumstances it might also be useful to provide a way to generate URIs that can be invalidated. Impermanence is also likely to be expected behaviour, so if it is not, we might need to find ways to make that very clear. -On the other hand, applications could be developed to extend file sharing to allow the creation of deletable URLs if this is seen as desired, which also helps make it clear that the default is permanence. +On the other hand, applications could be developed to extend file sharing to allow the creation of deletable URIs if this is seen as desired, which also helps make it clear that the default is permanence. -If we create built-in support for *both* permanent and impermanent URLs it would be sensible to indicate this as part of the URL so that users know what they are sharing and receiving. For example, by adding '-p' or '-i' to the end of the `type` as in: +If we create built-in support for *both* permanent and impermanent URIs it would be sensible to indicate this as part of the URI so that users know what they are sharing and receiving. For example, by adding '-p' or '-i' to the end of the `type` as in: `safe:file-p:U2VjdXJlIEFjY2VzcyBGb3IgRXZlcnlvbmU=:Safecoin-white-paper.pdf` -Adding support for impermanent URLs would require a *delete SAFE Address URL* API, which raises a further question as to whether or not we should allow different files to appear at the same address at different times, and if not then how this could be prevented. It would mean that SAFE Address URLs could not always be guaranteed to be permanent, which might be undesirable. It also creates a problem for anyone in possession of a SAFE Address URL: how to tell if it is permanent or not. +Adding support for impermanent URIs would require a *delete SAFE Data URI* API, which raises a further question as to whether or not we should allow different files to appear at the same address at different times, and if not then how this could be prevented. It would mean that SAFE Data URIs could not always be guaranteed to be permanent, which might be undesirable. It also creates a problem for anyone in possession of a SAFE Data URI: how to tell if it is permanent or not. -**IMPORTANT:** For simplicity's sake, this RFC mandates only permanent URLs (so only type `file:` and never `file-p:` or `file-i:`) which means that with this proposal we can always guarantee SAFE Address URLs are permanent, and we can leave the tricky questions for a later upgrade discussion. +**IMPORTANT:** For simplicity's sake, this RFC mandates only permanent URIs (so only type `file:` and never `file-p:` or `file-i:`) which means that with this proposal we can always guarantee SAFE Data URIs are permanent, and we can leave the tricky questions for a later upgrade discussion. ### SAFE NFS API Changes -Assuming only *permanent SAFE Address URLs* are supported, SAFE NFS API to be extended with: +Assuming only *permanent SAFE Data URIs* are supported, SAFE NFS API to be extended with: `getPermanentLink(fileName: String) -> Future` `fileName` - the full path of an existing file in the user's NFS storage -This returns a permanently valid URL for fileName formatted as a SAFE Address URL including `type`, `address` and `filename` elements. +This returns a permanently valid URI for fileName formatted as a SAFE Data URI including `type`, `address` and `filename` elements. -If we later support impermanent URLs a suitably named function can be added which would use a type of `file-i` instead of `file` +If we later support impermanent URIs a suitably named function can be added which would use a type of `file-i` instead of `file` ### safe-js Changes -safe-js to be extended with `safeNFS.getPermanentLink()` which returns a Promise that resolves to a SAFE Address URL as above. +safe-js to be extended with `safeNFS.getPermanentLink()` which returns a Promise that resolves to a SAFE Data URI as above. ### SAFE Beaker Changes -SAFE Beaker will detect every SAFE Address URL, parse it according to the above, and use this information to: +SAFE Beaker will detect every SAFE Data URI, parse it according to the above, and use this information to: a) access the file on the network @@ -164,7 +164,7 @@ c) deliver the content appropriately (e.g. embedded in HTML content, embedded do Relying on the implementation of SAFE NFS storage means this functionality would be difficult to maintain if that ever changes. -There might be privacy or tracking issues associated with sharing URLs if they could somehow be linked back to a user or account, though none are immediately apparent, except for ill considered naming of files. +There might be privacy or tracking issues associated with sharing URIs if they could somehow be linked back to a user or account, though none are immediately apparent, except for ill considered naming of files. ## Alternatives @@ -175,9 +175,9 @@ Alternatives would probably consume more network resources than providing suppor ### Application based or shared DNS -Rather than addressing a file using a network address, we might use either the existing SAFE DNS to share files saved in the user or application managed public storage. SAFE NFS at present would be cumbersome to use for this, requiring every app which wants to generate file URLs to have access to a user's registered public ID and service so this is not practical. +Rather than addressing a file using a network address, we might use either the existing SAFE DNS to share files saved in the user or application managed public storage. SAFE NFS at present would be cumbersome to use for this, requiring every app which wants to generate file URIs to have access to a user's registered public ID and service so this is not practical. -There are changes being considered which may make a DNS based sharing URLs more practical (see [this dev forum post](https://forum.safedev.org/t/file-sharing-at-the-level-of-safe-urls/288/7?u=happybeing)) but these have not been decided and have not been considered as part of this RFC. Once clear, it may be worthwhile reviewing this option. +There are changes being considered which may make a DNS based sharing URIs more practical (see [this dev forum post](https://forum.safedev.org/t/file-sharing-at-the-level-of-safe-URIs/288/7?u=happybeing)) but these have not been decided and have not been considered as part of this RFC. Once clear, it may be worthwhile reviewing this option. ## Unresolved questions From 1926aff4dbaa55503756b29c026f26cf1c014138 Mon Sep 17 00:00:00 2001 From: theWebalyst Date: Thu, 24 Nov 2016 14:58:47 +0000 Subject: [PATCH 8/8] Clarify perm/imperm point and other tweaks Fix link broken by last commit! Minor formatting, typos and minor re-wording for clarity. --- .../0000-file-sharing-by-safe-data-uri.md | 23 +++++++++++-------- 1 file changed, 13 insertions(+), 10 deletions(-) diff --git a/text/0000-file-sharing-by-safe-data-uri/0000-file-sharing-by-safe-data-uri.md b/text/0000-file-sharing-by-safe-data-uri/0000-file-sharing-by-safe-data-uri.md index 7221dc5..82fc1e8 100644 --- a/text/0000-file-sharing-by-safe-data-uri/0000-file-sharing-by-safe-data-uri.md +++ b/text/0000-file-sharing-by-safe-data-uri/0000-file-sharing-by-safe-data-uri.md @@ -4,14 +4,14 @@ - Type: new feature - Related components: SAFE Beaker Browser, safe-js, safe_core - Start Date: 22-11-2016 -- Discussion: (Preliminary to be replaced) - [File sharing at the level of safe: URIs](https://forum.safedev.org/t/file-sharing-at-the-level-of-safe-URIs/288?u=happybeing) +- Discussion: (Preliminary to be replaced) - [File sharing at the level of safe: URIs](https://forum.safedev.org/t/file-sharing-at-the-level-of-safe-urls/288?u=happybeing) - Supersedes: n/a - Superseded by: n/a ## Summary -Provide a way to obtain a `safe:` URI based on the SAFE network address for any file stored using SAFE NFS, and which does not rely on a public ID or the SAFE DNS. Such a *SAFE Data URI* would be valid to use as part of a normal HTML page or as a file download link in SAFE Beaker Browser. This avoids the need for a special file sharing app, or for the user to have made a public ID and service for the purpose of sharing files (see *Alternatives*). All that is needed is a way to generate a SAFE Data URI for a given file. It can then be downloaded using only a standard SAFE Browser, or accessed by any SAFE app. +Provide a way to obtain a `safe:` URI based on the SAFE network address for any file stored using SAFE NFS, and which does not rely on a public ID or the SAFE DNS. Such a *SAFE Data URI* would be valid to use as part of a normal HTML page or as a file download link in SAFE Beaker Browser. This makes sharing files much easier by avoiding the need for a special file sharing app, or for the user to have made a public ID and service for the purpose (see *Alternatives*). All that is needed is a way to generate a SAFE Data URI for a given file. It can then be downloaded using only a standard SAFE Browser, or accessed by any SAFE app. ## Conventions - The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC 2119](http://tools.ietf.org/html/rfc2119). @@ -23,13 +23,14 @@ Allowing web applications and SAFE Beaker to access files directly using a SAFE - an image captured by a SAFE web app and stored in a user's SAFE NFS storage, to be loaded into the DOM of the app, or into an HTML document generated by the app, and displayed within by the browser as part of the app UI or saved HTML. + - a *Share file...* command, easily implemented in any app, which offers up a SAFE Data URI for a file saved to or chosen from NFS 'drive'. This URI can be sent to anyone who, provided the file is public, could then access the file with just a SAFE web browser. Private files could also be accessed this way but only by the authorised owner, as part of their browser history, or as quick access links in other applications or documents. Without this feature, the SAFE NFS user would need to use a specialist file sharing application to share a file, and files accessed through such links would be less secure, hard to trust or guarantee permanence. Providing access to SAFE files without requiring publication via SAFE DNS, makes it easy to port existing apps, or for someone who already knows how to create and load files using traditional methods to offer this functionality in new apps. -The ability to guaranteed permanence of download links will no doubt also lead to the creation of new and innovative kinds of application. +The ability to guarantee permanence of download links will no doubt also lead to the creation of new and innovative kinds of application. ## Required Functionality @@ -40,7 +41,7 @@ For SAFE Beaker Browser to recognise and process a SAFE Data URI differently fro ## Detailed design -We need a format of `safe:` URI which SAFE Beaker (and other compatible browsers) can recognise and use to access content stored on SAFE network without reference to a SAFE public ID. +We need a format of `safe:` URI which SAFE Beaker (and all SAFE compatible browsers) can recognise and use to access content stored on SAFE network without reference to a SAFE public ID. Each file stored on SAFE network is addressed using a [DataIdentifier](https://github.com/maidsafe/routing/blob/7c00dd14a2c4e4b2a3f7813a13edad119c0efa83/src/data/mod.rs#L111), so we can use that to address the file on the network. @@ -67,7 +68,7 @@ All the above is existing functionality, so the format of *SAFE Data URIs* must ### SAFE Data URI Structure -SAFE Beaker must be able to recognise a SAFE Data URI, parse it, and determine what method to use to access the file, and what if any meta data to provide with the response it delivers. We can also choose to provide a way for a human readable filename to be conveyed as part of the URI, rather than base this on the address or having the user type it in. +SAFE Beaker must be able to recognise a SAFE Data URI, parse it, and determine what method to use to access the file, and what if any metadata to provide with the response it delivers. We can also choose to provide a way for a human readable filename to be conveyed as part of the URI, rather than base this on the address or having the user type it in. **A SAFE Data URI always begins with `safe::`** This allows us to continue to permit a normal `safe:` URI to omit or include the conventional but redundant '//' that we are accustomed to in web URIs. @@ -77,7 +78,7 @@ The complete structure of a SAFE Data URI is then: Where: -- `type` is one of `file:` or `folder:` or `dns:`. So far I have only discussed retrieving regular files stored with SAFE NFS, but including a type instructs the client on the type of data being retrieved access and how to make use of it, while providing for support to added later for additional types, with backwards compatibility. +- `type` is one of `file:` or `folder:` or `dns:`. So far this RFC has only mentioned retrieving regular files accessed using SAFE NFS, but including a type instructs the client on the type of data being retrieved access and how to make use of it, while providing for additional types to be added later, with backwards compatibility. - `address` is the a base64 encoded serialisation of a [DataIdentifier](https://github.com/maidsafe/routing/blob/7c00dd14a2c4e4b2a3f7813a13edad119c0efa83/src/data/mod.rs#L111). @@ -105,7 +106,7 @@ For example, values within a key-value store (in a MutableData object) could be ### Behaviour Options - Discussion -Assuming the shared file is immutable, we can guarantee that the SAFE Data URI will also be permanent. If the file were either mutable or deliberately referenced via a mutable data structure, then the URI could be impermanent by design. So disregarding at this stage any limitations imposed by the implementation, we should in reviewing this RFC consider the viability and usefulness of designing SAFE Data URIs to be *either* permanent or impermanent. +Assuming the shared file is immutable, we can guarantee that the SAFE Data URI will also be permanent. If the file were either mutable or deliberately referenced via a mutable data structure then a URI could be made impermanent by design. So disregarding at this stage any limitations imposed by the implementation, we should in reviewing this RFC consider the viability and usefulness of designing SAFE Data URIs to be *either* permanent or impermanent. **IMPORTANT:** At this point both options are on the table so we can debate what is expected and desirable, but as is stands this RFC proposes to implement only permanent SAFE Data URIs. Some points for discussion are presented below. @@ -119,7 +120,7 @@ The question is whether we would want to limit SAFE Data URIs to only allow shar #### Impermanent URIs -There are many good things about permanent URIs, but also the disadvantage for users who accidentally share something that can never subsequently be unshared. There are also things people want to be able to share temporarily. For example, in order that the intended recipient access them, but later be invalidated to reduce the chance that someone else who discovers the URI can gain access to the file. +There are many good things about permanent URIs, but also the disadvantage for users who accidentally share something that can never subsequently be unshared. There are also things people want to be able to share temporarily. For example, sharing a link in order that the intended recipient access a file, which can later be invalidated to reduce the chance that someone else who discovers the URI can also obtain the file. In general, what one shares one day, or in one state of mind, might later be regretted and so in certain circumstances it might also be useful to provide a way to generate URIs that can be invalidated. Impermanence is also likely to be expected behaviour, so if it is not, we might need to find ways to make that very clear. @@ -129,6 +130,8 @@ If we create built-in support for *both* permanent and impermanent URIs it would `safe:file-p:U2VjdXJlIEFjY2VzcyBGb3IgRXZlcnlvbmU=:Safecoin-white-paper.pdf` +But this would allow someone to create a Data URI, and then deceive recipients by changing the `file-i` to `file-p` (or vice versa). So there are problems with allowing both: adding the option of impermanent URIs may affect the value and usefulness of permanence. + Adding support for impermanent URIs would require a *delete SAFE Data URI* API, which raises a further question as to whether or not we should allow different files to appear at the same address at different times, and if not then how this could be prevented. It would mean that SAFE Data URIs could not always be guaranteed to be permanent, which might be undesirable. It also creates a problem for anyone in possession of a SAFE Data URI: how to tell if it is permanent or not. **IMPORTANT:** For simplicity's sake, this RFC mandates only permanent URIs (so only type `file:` and never `file-p:` or `file-i:`) which means that with this proposal we can always guarantee SAFE Data URIs are permanent, and we can leave the tricky questions for a later upgrade discussion. @@ -169,13 +172,13 @@ There might be privacy or tracking issues associated with sharing URIs if they c ## Alternatives -Not doing this would require users to find other ways of sharing files, which since they are not built in to safe_core and SAFE Beaker will be more cumbersome (for users and for app developers), impossible to ensure permanence, and harder to trust in terms of security (i.e. from tracking and surveillance). Making such a widely used feature harder than necessary would reduce the rate and potential level of adoption, thereby diminishing its positive overall impact on security and privacy. +Not doing this would require users to find other ways of sharing files, which since they are not built in to safe_core and SAFE Beaker will be more cumbersome (for users and for app developers). Also, it would be impossible to ensure permanence, and harder to trust in terms of security (i.e. from tracking and surveillance). Making such a widely used feature harder than necessary would reduce the rate and potential level of adoption of the SAFE Network, thereby diminishing its positive overall impact on security and privacy. Alternatives would probably consume more network resources than providing support for direct links within safe_core, increasing user PUT costs and network load for a feature we can expect would be used by almost every SAFE user. ### Application based or shared DNS -Rather than addressing a file using a network address, we might use either the existing SAFE DNS to share files saved in the user or application managed public storage. SAFE NFS at present would be cumbersome to use for this, requiring every app which wants to generate file URIs to have access to a user's registered public ID and service so this is not practical. +Rather than locating a file using a network address, we might use either the existing SAFE DNS to share files saved in the user or application managed storage. SAFE NFS at present would be cumbersome to use for this, requiring every app which wants to generate file URIs to have access to a user's registered public ID and service so this is not practical. There are changes being considered which may make a DNS based sharing URIs more practical (see [this dev forum post](https://forum.safedev.org/t/file-sharing-at-the-level-of-safe-URIs/288/7?u=happybeing)) but these have not been decided and have not been considered as part of this RFC. Once clear, it may be worthwhile reviewing this option.