From fb8c7d2e2a82e5637a7dff670413e039c98d6151 Mon Sep 17 00:00:00 2001 From: Paul Frazee Date: Tue, 5 Jun 2018 20:46:08 -0500 Subject: [PATCH 1/9] Add proposals/0000-ephemeral-message.md --- proposals/0000-ephemeral-message.md | 55 +++++++++++++++++++++++++++++ 1 file changed, 55 insertions(+) create mode 100644 proposals/0000-ephemeral-message.md diff --git a/proposals/0000-ephemeral-message.md b/proposals/0000-ephemeral-message.md new file mode 100644 index 0000000..b47b45d --- /dev/null +++ b/proposals/0000-ephemeral-message.md @@ -0,0 +1,55 @@ + +Title: **DEP-0000: Ephemeral Message (Extension Message)** + +Short Name: `0000-ephemeral-message-extension` + +Type: Informative + +Status: Undefined (as of 2018-06-05) + +Github PR: + +Authors: [Paul Frazee](https://github.com/pfrazee) + + +# Summary +[summary]: #summary + +This DEP defines the non-standard `em` extension message used in the Dat replication protocol. This message provides a way to send arbitrary application data to a peer through an existing connection. + + +# Motivation +[motivation]: #motivation + +While Dat is effective at sharing persistent datasets, applications frequently need to transmit extra information which does not need to persist. This kind of information is known as "ephemeral." Examples include: sending chat messages, proposing changes to a dat, alerting peers to events, broadcasting identity information, and sharing the URLs of related datasets. + +This extension message will establish a common mechanism for sending ephemeral messages of an arbitrary composition. + + +# Reference Documentation +[reference-documentation]: #reference-documentation + +This DEP is implemented using the Dat replication protocol's "extension messages." In order to broadcast support for this DEP, a client should declare the `'em'` extension in the replication handshake. + +Ephemeral messages can be sent at any time after the connection is established by sending an extension message of type `'em'`. The message may include a payload up to 256 bytes in length. Any additional bytes should be truncated by the receiving client. The payload is a buffer of any encoding. + +The client may respond to the message by emitting an event, so that it may be handled by the client's application logic. + +No acknowledgment of receipt will be provided (no "ACK"). + +After publishing this DEP, the "Beaker Browser" will implement a Web API for exposing the `'em'` protocol to applications. It will restrict access so that the application code of a `dat://` site will only be able to send ephemeral messages on connections related to its own content. + + +# Drawbacks +[drawbacks]: #drawbacks + +- This DEP may present privacy concerns, as it may be used to track users in a similar fashion to HTTP Cookies, or be used to exfiltrate data. +- The payload of the `'session-data'` message is not authenticated in any way. The lack of trust must be considered by applications which leverage the data. +- If the recipient of the `'session-data'` message is not authenticated (as is currently the case in all Dat replication connections) the client will not know who is receiving the payload and may broadcast sensitive information. + + +# Changelog +[changelog]: #changelog + +- 2018-05-31: First complete draft submitted for review + From 9d092b413da36b25134bea2a446ce9b0671ef802 Mon Sep 17 00:00:00 2001 From: Paul Frazee Date: Tue, 5 Jun 2018 20:48:02 -0500 Subject: [PATCH 2/9] Set PR link on proposals/0000-ephemeral-message.md --- proposals/0000-ephemeral-message.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/0000-ephemeral-message.md b/proposals/0000-ephemeral-message.md index b47b45d..3662b61 100644 --- a/proposals/0000-ephemeral-message.md +++ b/proposals/0000-ephemeral-message.md @@ -7,7 +7,7 @@ Type: Informative Status: Undefined (as of 2018-06-05) -Github PR: +Github PR: https://github.com/datprotocol/DEPs/pull/28 Authors: [Paul Frazee](https://github.com/pfrazee) @@ -51,5 +51,5 @@ After publishing this DEP, the "Beaker Browser" will implement a Web API for exp # Changelog [changelog]: #changelog -- 2018-05-31: First complete draft submitted for review +- 2018-06-05: First complete draft submitted for review From e683b6037cc42e79a127557756aa091a4250ab7f Mon Sep 17 00:00:00 2001 From: Paul Frazee Date: Tue, 5 Jun 2018 20:48:51 -0500 Subject: [PATCH 3/9] Fix copy-pasta typos in proposals/0000-ephemeral-message.md --- proposals/0000-ephemeral-message.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/0000-ephemeral-message.md b/proposals/0000-ephemeral-message.md index 3662b61..aef33ac 100644 --- a/proposals/0000-ephemeral-message.md +++ b/proposals/0000-ephemeral-message.md @@ -44,8 +44,8 @@ After publishing this DEP, the "Beaker Browser" will implement a Web API for exp [drawbacks]: #drawbacks - This DEP may present privacy concerns, as it may be used to track users in a similar fashion to HTTP Cookies, or be used to exfiltrate data. -- The payload of the `'session-data'` message is not authenticated in any way. The lack of trust must be considered by applications which leverage the data. -- If the recipient of the `'session-data'` message is not authenticated (as is currently the case in all Dat replication connections) the client will not know who is receiving the payload and may broadcast sensitive information. +- The payload of the `'em'` message is not authenticated in any way. The lack of trust must be considered by applications which leverage the data. +- If the recipient of the `'em'` message is not authenticated (as is currently the case in all Dat replication connections) the client will not know who is receiving the payload and may broadcast sensitive information. # Changelog From c2bdace8bce57408b0630dfd1079ef94837aa74c Mon Sep 17 00:00:00 2001 From: Paul Frazee Date: Wed, 6 Jun 2018 00:15:34 -0500 Subject: [PATCH 4/9] Added a header with 'encoding' values and increased the max payload size from 256b to 10kb. --- proposals/0000-ephemeral-message.md | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/proposals/0000-ephemeral-message.md b/proposals/0000-ephemeral-message.md index aef33ac..c316f0f 100644 --- a/proposals/0000-ephemeral-message.md +++ b/proposals/0000-ephemeral-message.md @@ -31,9 +31,20 @@ This extension message will establish a common mechanism for sending ephemeral m This DEP is implemented using the Dat replication protocol's "extension messages." In order to broadcast support for this DEP, a client should declare the `'em'` extension in the replication handshake. -Ephemeral messages can be sent at any time after the connection is established by sending an extension message of type `'em'`. The message may include a payload up to 256 bytes in length. Any additional bytes should be truncated by the receiving client. The payload is a buffer of any encoding. +Ephemeral messages can be sent at any time after the connection is established by sending an extension message of type `'em'`. The message includes 1 byte of header information followed by a payload no longer than 10 kilobytes. Any additional bytes should be truncated by the receiving client. Therefore the message is encoded as follows: -The client may respond to the message by emitting an event, so that it may be handled by the client's application logic. +``` +header (1 byte) +payload (0-10000 bytes) +``` + +The first 6 header bits are reserved. The payload's encoding may be specified in the last two header bits. Possible values are: + + - `00` - binary + - `01` - utf8 + - `10` - json + +The receiving client should attempt to decode the payload according to this encoding-value. The client may respond to the message by emitting an event, so that it may be handled by the client's application logic. No acknowledgment of receipt will be provided (no "ACK"). @@ -51,5 +62,6 @@ After publishing this DEP, the "Beaker Browser" will implement a Web API for exp # Changelog [changelog]: #changelog +- 2018-06-05: Added a header with 'encoding' values and increased the max payload size from 256b to 10kb. - 2018-06-05: First complete draft submitted for review From 7fc180a29abdb3a21a2397fd18b7a4e8001e8b7c Mon Sep 17 00:00:00 2001 From: Paul Frazee Date: Sun, 10 Jun 2018 17:49:25 -0500 Subject: [PATCH 5/9] Change the message identifier to 'ephemeral' --- proposals/0000-ephemeral-message.md | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/proposals/0000-ephemeral-message.md b/proposals/0000-ephemeral-message.md index c316f0f..606b227 100644 --- a/proposals/0000-ephemeral-message.md +++ b/proposals/0000-ephemeral-message.md @@ -15,7 +15,7 @@ Authors: [Paul Frazee](https://github.com/pfrazee) # Summary [summary]: #summary -This DEP defines the non-standard `em` extension message used in the Dat replication protocol. This message provides a way to send arbitrary application data to a peer through an existing connection. +This DEP defines the non-standard `ephemeral` extension message used in the Dat replication protocol. This message provides a way to send arbitrary application data to a peer through an existing connection. # Motivation @@ -29,9 +29,9 @@ This extension message will establish a common mechanism for sending ephemeral m # Reference Documentation [reference-documentation]: #reference-documentation -This DEP is implemented using the Dat replication protocol's "extension messages." In order to broadcast support for this DEP, a client should declare the `'em'` extension in the replication handshake. +This DEP is implemented using the Dat replication protocol's "extension messages." In order to broadcast support for this DEP, a client should declare the `'ephemeral'` extension in the replication handshake. -Ephemeral messages can be sent at any time after the connection is established by sending an extension message of type `'em'`. The message includes 1 byte of header information followed by a payload no longer than 10 kilobytes. Any additional bytes should be truncated by the receiving client. Therefore the message is encoded as follows: +Ephemeral messages can be sent at any time after the connection is established by sending an extension message of type `'ephemeral'`. The message includes 1 byte of header information followed by a payload no longer than 10 kilobytes. Any additional bytes should be truncated by the receiving client. Therefore the message is encoded as follows: ``` header (1 byte) @@ -48,20 +48,21 @@ The receiving client should attempt to decode the payload according to this enco No acknowledgment of receipt will be provided (no "ACK"). -After publishing this DEP, the "Beaker Browser" will implement a Web API for exposing the `'em'` protocol to applications. It will restrict access so that the application code of a `dat://` site will only be able to send ephemeral messages on connections related to its own content. +After publishing this DEP, the "Beaker Browser" will implement a Web API for exposing the `'ephemeral'` protocol to applications. It will restrict access so that the application code of a `dat://` site will only be able to send ephemeral messages on connections related to its own content. # Drawbacks [drawbacks]: #drawbacks - This DEP may present privacy concerns, as it may be used to track users in a similar fashion to HTTP Cookies, or be used to exfiltrate data. -- The payload of the `'em'` message is not authenticated in any way. The lack of trust must be considered by applications which leverage the data. -- If the recipient of the `'em'` message is not authenticated (as is currently the case in all Dat replication connections) the client will not know who is receiving the payload and may broadcast sensitive information. +- The payload of the `'ephemeral'` message is not authenticated in any way. The lack of trust must be considered by applications which leverage the data. +- If the recipient of the `'ephemeral'` message is not authenticated (as is currently the case in all Dat replication connections) the client will not know who is receiving the payload and may broadcast sensitive information. # Changelog [changelog]: #changelog +- 2018-06-10: Change the message identifier to `'ephemeral'` - 2018-06-05: Added a header with 'encoding' values and increased the max payload size from 256b to 10kb. - 2018-06-05: First complete draft submitted for review From 5299afb729b6a1318314512b8a817d36d3682f9c Mon Sep 17 00:00:00 2001 From: Paul Frazee Date: Sun, 10 Jun 2018 17:53:54 -0500 Subject: [PATCH 6/9] Add more to the motivation of proposals/0000-ephemeral-message.md --- proposals/0000-ephemeral-message.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/proposals/0000-ephemeral-message.md b/proposals/0000-ephemeral-message.md index 606b227..39b3a85 100644 --- a/proposals/0000-ephemeral-message.md +++ b/proposals/0000-ephemeral-message.md @@ -23,7 +23,9 @@ This DEP defines the non-standard `ephemeral` extension message used in the Dat While Dat is effective at sharing persistent datasets, applications frequently need to transmit extra information which does not need to persist. This kind of information is known as "ephemeral." Examples include: sending chat messages, proposing changes to a dat, alerting peers to events, broadcasting identity information, and sharing the URLs of related datasets. -This extension message will establish a common mechanism for sending ephemeral messages of an arbitrary composition. +This DEP was motivated by the need for a quick solution to these use-cases. It establishes a mechanism for sending ephemeral messages over existing Dat connections. At time of writing, it is unclear whether this mechanism will be used in the long-term, or superseded by a more flexible messaging channel. + +A specific use case for this extension is to enable a new Web API which will expose peer message-passing channels to in-browser applications. Such an API would restrict access so that the application code of a `dat://` site will only be able to send ephemeral messages on connections related to its own content. # Reference Documentation @@ -62,6 +64,7 @@ After publishing this DEP, the "Beaker Browser" will implement a Web API for exp # Changelog [changelog]: #changelog +- 2018-06-10: Expand on the motivation of this DEP - 2018-06-10: Change the message identifier to `'ephemeral'` - 2018-06-05: Added a header with 'encoding' values and increased the max payload size from 256b to 10kb. - 2018-06-05: First complete draft submitted for review From 630a69141f80ea218ff20aa6353b7da67ba4d849 Mon Sep 17 00:00:00 2001 From: Paul Frazee Date: Sun, 10 Jun 2018 17:59:15 -0500 Subject: [PATCH 7/9] Change the payload encoding to protobuf and provide a more flexible content-type field --- proposals/0000-ephemeral-message.md | 21 ++++++++------------- 1 file changed, 8 insertions(+), 13 deletions(-) diff --git a/proposals/0000-ephemeral-message.md b/proposals/0000-ephemeral-message.md index 39b3a85..bb569df 100644 --- a/proposals/0000-ephemeral-message.md +++ b/proposals/0000-ephemeral-message.md @@ -33,24 +33,18 @@ A specific use case for this extension is to enable a new Web API which will exp This DEP is implemented using the Dat replication protocol's "extension messages." In order to broadcast support for this DEP, a client should declare the `'ephemeral'` extension in the replication handshake. -Ephemeral messages can be sent at any time after the connection is established by sending an extension message of type `'ephemeral'`. The message includes 1 byte of header information followed by a payload no longer than 10 kilobytes. Any additional bytes should be truncated by the receiving client. Therefore the message is encoded as follows: +Ephemeral messages can be sent at any time after the connection is established by sending an extension message of type `'ephemeral'`. The message payload is a protobuf with the following schema: ``` -header (1 byte) -payload (0-10000 bytes) +message EphemeralMessage { + optional string contentType = 1; + required bytes payload = 2; +} ``` -The first 6 header bits are reserved. The payload's encoding may be specified in the last two header bits. Possible values are: +The `contentType` string should provide a valid MIME type. If none is specified, the encoding should be considered `application/octet-stream`. - - `00` - binary - - `01` - utf8 - - `10` - json - -The receiving client should attempt to decode the payload according to this encoding-value. The client may respond to the message by emitting an event, so that it may be handled by the client's application logic. - -No acknowledgment of receipt will be provided (no "ACK"). - -After publishing this DEP, the "Beaker Browser" will implement a Web API for exposing the `'ephemeral'` protocol to applications. It will restrict access so that the application code of a `dat://` site will only be able to send ephemeral messages on connections related to its own content. +The client may respond to the message by emitting an event, so that it may be handled by the client's application logic. No acknowledgment of receipt will be provided (no "ACK"). # Drawbacks @@ -64,6 +58,7 @@ After publishing this DEP, the "Beaker Browser" will implement a Web API for exp # Changelog [changelog]: #changelog +- 2018-06-10: Change the payload encoding to protobuf and provide a more flexible content-type field. - 2018-06-10: Expand on the motivation of this DEP - 2018-06-10: Change the message identifier to `'ephemeral'` - 2018-06-05: Added a header with 'encoding' values and increased the max payload size from 256b to 10kb. From c7e10ac8c06b705b095ba41646cc9855adb19487 Mon Sep 17 00:00:00 2001 From: Paul Frazee Date: Wed, 20 Jun 2018 15:38:39 -0500 Subject: [PATCH 8/9] Add size-limit suggestion to ephemeral-message DEP --- proposals/0000-ephemeral-message.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/proposals/0000-ephemeral-message.md b/proposals/0000-ephemeral-message.md index bb569df..56d85b4 100644 --- a/proposals/0000-ephemeral-message.md +++ b/proposals/0000-ephemeral-message.md @@ -46,6 +46,8 @@ The `contentType` string should provide a valid MIME type. If none is specified, The client may respond to the message by emitting an event, so that it may be handled by the client's application logic. No acknowledgment of receipt will be provided (no "ACK"). +It's suggested that an encoded `EphemeralMessage` should be no larger than 2kb, to avoid creating too much work for the receiving peer to handle. + # Drawbacks [drawbacks]: #drawbacks @@ -58,6 +60,7 @@ The client may respond to the message by emitting an event, so that it may be ha # Changelog [changelog]: #changelog +- 2018-06-20: Add a size-limit suggestion - 2018-06-10: Change the payload encoding to protobuf and provide a more flexible content-type field. - 2018-06-10: Expand on the motivation of this DEP - 2018-06-10: Change the message identifier to `'ephemeral'` From b11523384d1980daf3941d7ae1fd1319175052e9 Mon Sep 17 00:00:00 2001 From: Paul Frazee Date: Mon, 2 Jul 2018 12:20:15 -0500 Subject: [PATCH 9/9] Add "Privacy, security, and reliability" section --- proposals/0000-ephemeral-message.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/proposals/0000-ephemeral-message.md b/proposals/0000-ephemeral-message.md index 56d85b4..59bbb2c 100644 --- a/proposals/0000-ephemeral-message.md +++ b/proposals/0000-ephemeral-message.md @@ -49,6 +49,18 @@ The client may respond to the message by emitting an event, so that it may be ha It's suggested that an encoded `EphemeralMessage` should be no larger than 2kb, to avoid creating too much work for the receiving peer to handle. +# Privacy, security, and reliability +[privacy-security-and-reliability]: #privacy-security-and-reliability + +Users of ephemeral messages should be conscious of the privacy, security, and reliability properties of the channel. Ephemeral messages are designed to be a minimal stopgap solution while better solutions are developed. This "minimal design" is reflected by the limited privacy, security, and reliability. + +At time of writing, the Dat messaging channel is encrypted using the public key of the first hypercore to be exchanged over the channel. As a result, all traffic can be decrypted and/or modified by an intermediary which possesses the public key. For typical hypercore messages, the ability to modify the messages is a non-issue because all hypercore data is authenticated. Ephemeral messages however have no authentication, and may be modified or monitored by an intermediary. + +Applications using the ephemeral message should not assume any privacy, nor should they trust that a peer is "who they say they are." + +Applications should also not assume connectivity will occur between all peers who have "joined the swarm" for a hypercore. There are many factors which may cause a peer not to connect: failed NAT traversal, the client running out of available sockets, or even the intentional blocking of a peer by the discovery network. + + # Drawbacks [drawbacks]: #drawbacks @@ -60,6 +72,7 @@ It's suggested that an encoded `EphemeralMessage` should be no larger than 2kb, # Changelog [changelog]: #changelog +- 2018-07-02: Add "Privacy, security, and reliability" section - 2018-06-20: Add a size-limit suggestion - 2018-06-10: Change the payload encoding to protobuf and provide a more flexible content-type field. - 2018-06-10: Expand on the motivation of this DEP