From 107c5ff25a47a9c0a386f1881f79956f209a844e Mon Sep 17 00:00:00 2001 From: Pedro Ferreira Date: Thu, 22 Sep 2022 14:58:12 +0100 Subject: [PATCH 1/4] feat: add best practices for use cases RFC --- ...000-use-case-integration-best-practices.md | 177 ++++++++++++++++++ 1 file changed, 177 insertions(+) create mode 100644 text/0000-use-case-integration-best-practices.md diff --git a/text/0000-use-case-integration-best-practices.md b/text/0000-use-case-integration-best-practices.md new file mode 100644 index 0000000..142cf00 --- /dev/null +++ b/text/0000-use-case-integration-best-practices.md @@ -0,0 +1,177 @@ +- Feature Name: use_case_integration_best_practices +- Status: Draft +- Start Date: 2022-09-20 +- RFC PR: (leave this empty) +- Hathor Issue: (leave this empty) +- Author: Pedro Ferreira + +# Summary +[summary]: #summary + +This document presents best practices for use cases that integrate with the Hathor Network, in order to improve the reliability of their operation. + +# Guide-level explanation +[guide-level-explanation]: #guide-level-explanation + +There are multiple ways that a use case can integrate with Hathor, here is a basic architecture of a common integration. + +- The use case has a software that runs operations offchain and is used by its customer. +- This software connects with a headless wallet, that executes operations in the Hathor network. +- The headless wallet is connected to a full node, that belongs to the Hathor peer-to-peer network. + +
+┌─────────────┐      ┌──────────┐                     ┌─────────────┐        ┌─────────────┐
+│             │      │          │◀────────────────────│             │        │             │
+│  Hathor     │      │ Headless │                     │  Use Case   │        │  Customer   │
+│  Full Node  │◀────▶│  Wallet  │                     │  Software   │◀──────▶│             │
+│             │      │          │────────────────────▶│             │        │             │
+└─────────────┘      └──────────┘                     └─────────────┘        └─────────────┘
+
+ +## Run more than one node + +Even though the architecture described above works fine for a simple use case, we strongly recommend any use case to run more than one node, in order to increase the reliability of the operation. + +During normal network activity, any transaction will be propagated through all peers and they will get to the same consensus. However, there are some situations where this might not be true, e.g. if an attack is happening to a single peer. To improve the reliability of the operation at any moment, it's important that all use cases run more than one full node and they **can't be connected among them**, only to the rest of the network. + +Let's assume an exchange wants to run nodes to identify deposits in the Hathor network, so a recommended approach for the integration would be to run at least two full nodes (node1 and node2), which are not connected between them (node1 must have node2 in the blacklist and vice versa). In that architecture, if any deposit is identified in node1, then the exchange must check that it's also a valid transaction in node2 and in one of the public nodes. With this approach, if one of the nodes is compromised and with a different consensus/state, an alert must be raised, so we must investigate what happened. + +### Validate new transactions on more than one full node before accepting them + +When running more than one node, any new transaction must be accepted only after validating that it's a valid transaction in all the nodes. + +### Validate all your full nodes have the same best block + +It's also important to regularly check if all nodes have the same best block. If they have different best blocks, this might be a clue that one of the nodes could be a target of an attack because is running a separate fork of the blockchain. + +### Recommended architecture + +
+                       ┌─────────────┐      ┌──────────┐                     ┌─────────────┐        ┌─────────────┐
+                       │             │      │          │◀────────────────────┤             │        │             │
+                       │  Hathor     │      │ Headless │                     │  Use Case   │        │  Customer   │
+                       │ Full Node 1 │◀────▶│  Wallet  │                     │  Software   │◀──────▶│             │
+                       │             │      │          ├────────────────────▶│             │        │             │
+                       └────┬────────┘      └──────────┘                     └─────────────┘        └─────────────┘
+                            │
+┌────────────┐              │
+│            │◀─────────────┘
+│  Rest of   │
+│  Network   │
+│            │◀─────────────┐
+└────────────┘              │
+                       ┌────┴────────┐
+                       │             │               
+                       │  Hathor     │
+                       │ Full Node 2 │
+                       │             │
+                       └─────────────┘
+
+
+ +## Peer-id + +The peer-id is a unique identifier of your full node in Hathor's p2p network. Even though it does not contain any private key information, keeping it secret is a security measure that prevents attackers from targeting directly your full nodes. + +Because of that, it's important to never share it in public chats, only in private groups with the Hathor team. If you think your peer-id has been exposed, you should generate a new peer-id and replace the exposed ones. + +## Be alert for weird behavior + +### Correctly validate transactions + +The transactions in the Hathor network have many fields that must be checked to guarantee that a transaction is valid for your use case. For more details about the fields of a transaction, check the [Transaction Anatomy RFC](https://github.com/HathorNetwork/rfcs/blob/master/text/0015-anatomy-of-tx.md). + +#### Version + +Transactions have `version=1`, blocks have versions `0` or `3`, and token creation transactions have `version=2`. + +#### Voided + +A voided transaction is **not** a valid transaction. You must only accept transactions that have `is_voided=false`. + +#### Outputs + +The outputs contain the fields used to identify that the funds belong to your wallet. + +##### Token + +The output has the token information, so you can check if it has the expected tokens. For example, exchanges expect the deposits to be of HTR, so `token = '00'` and `token_data=0`. + +##### Value + +The value of the output is an integer, so `1000` means `10.00`. + +##### Address + +The output address must be one of your wallet's addresses. + +##### Timelock + +The output must have a timelock, and if it's bigger than the current timestamp, you won't be able to spend this output until this moment is reached. Because of that, it's essential to validate that `timelock=null` before validating this transaction. + +#### Number of confirmations + +Some use cases might handle transactions with huge amounts, so it's essential to wait for some blocks to confirm the transaction before accepting it as a valid one. The more blocks confirm a transaction, the more guarantee there is that this transaction won't become invalid in the future. + +### Check if an unusual amount of deposits or withdrawals are being made + +Many use cases have withdrawals/deposits in their set of features through blockchain transactions. It's important to be alerted in case any unusual amount of deposits/withdrawals are being made because this could be caused by an attack. + +For that situation, it's crucial to have an easy way to block specific accounts that have unusual behavior and might be part of an attack. + +### Check if one of your full nodes gets out-of-sync + +One other relevant aspect of full node to always check for weird behavior is the sync among them. We recommend use cases to regularly validate that all their full nodes are in sync among them and in sync with at least one public node as well. + +This validation is important to guarantee the node is not isolated from the rest of the network with a fork of the blockchain. + +# Reference-level explanation +[reference-level-explanation]: #reference-level-explanation + +## Run a full node with a blacklist of peer-ids + +Following the recommended architecture, you will need to run more than one full node and they shouldn't be connected among them. To achieve that, each of your nodes must have a blacklist of peer-ids, containing the ids of the other nodes. + +For example, you run node1 (peerid1), node2 (peerid2), and node3 (peerid3). The node1 should have peerid2 and peerid3 in the blacklist, node2 should have peerid1 and peerid3 in the blacklist, and node3 should have peerid1 and peerid2 in the blacklist. + +To create this blacklist in the full node there are some possible approaches: + +### CLI command + +If you use the full node parameters directly in the command line, then you should add: + +``` +--peer-id-blacklist peerid1 peerid2 +``` + +### Environment variables + +If you use the full node parameters using env vars, then you should add: + +``` +export HATHOR_PEER_ID_BLACKLIST=[peerid1, peerid2] +``` + +### API + +There is also an API to add a peer-id to the blacklist while the node is running, however this is not recommended because if you restart your node you will lose this blacklist. + +``` +POST to /v1a/p2p/netfilter + +{ + "chain": { + "name": "post_peerid", + }, + "target": { + "type": "NetfilterReject", + "target_params": {} + }, + "match": { + "type": "NetfilterMatchPeerId", + "match_params": { + "peer_id": "peerid1" + } + } +} +``` \ No newline at end of file From 1b233389d0e5afae8a56af2d1533323e32cf5b34 Mon Sep 17 00:00:00 2001 From: Pedro Ferreira Date: Fri, 21 Oct 2022 19:39:45 -0300 Subject: [PATCH 2/4] feat: improvements proposed in the review --- ...000-use-case-integration-best-practices.md | 78 +++++++++---------- 1 file changed, 38 insertions(+), 40 deletions(-) diff --git a/text/0000-use-case-integration-best-practices.md b/text/0000-use-case-integration-best-practices.md index 142cf00..e3856e1 100644 --- a/text/0000-use-case-integration-best-practices.md +++ b/text/0000-use-case-integration-best-practices.md @@ -8,43 +8,16 @@ # Summary [summary]: #summary -This document presents best practices for use cases that integrate with the Hathor Network, in order to improve the reliability of their operation. +This document presents security best practices for use cases that integrate with the Hathor Network, in order to improve the reliability of their operation. # Guide-level explanation [guide-level-explanation]: #guide-level-explanation -There are multiple ways that a use case can integrate with Hathor, here is a basic architecture of a common integration. +There are multiple ways that a use case can integrate with Hathor, here is a common architecture compliant to our recommendations - The use case has a software that runs operations offchain and is used by its customer. -- This software connects with a headless wallet, that executes operations in the Hathor network. -- The headless wallet is connected to a full node, that belongs to the Hathor peer-to-peer network. - -
-┌─────────────┐      ┌──────────┐                     ┌─────────────┐        ┌─────────────┐
-│             │      │          │◀────────────────────│             │        │             │
-│  Hathor     │      │ Headless │                     │  Use Case   │        │  Customer   │
-│  Full Node  │◀────▶│  Wallet  │                     │  Software   │◀──────▶│             │
-│             │      │          │────────────────────▶│             │        │             │
-└─────────────┘      └──────────┘                     └─────────────┘        └─────────────┘
-
- -## Run more than one node - -Even though the architecture described above works fine for a simple use case, we strongly recommend any use case to run more than one node, in order to increase the reliability of the operation. - -During normal network activity, any transaction will be propagated through all peers and they will get to the same consensus. However, there are some situations where this might not be true, e.g. if an attack is happening to a single peer. To improve the reliability of the operation at any moment, it's important that all use cases run more than one full node and they **can't be connected among them**, only to the rest of the network. - -Let's assume an exchange wants to run nodes to identify deposits in the Hathor network, so a recommended approach for the integration would be to run at least two full nodes (node1 and node2), which are not connected between them (node1 must have node2 in the blacklist and vice versa). In that architecture, if any deposit is identified in node1, then the exchange must check that it's also a valid transaction in node2 and in one of the public nodes. With this approach, if one of the nodes is compromised and with a different consensus/state, an alert must be raised, so we must investigate what happened. - -### Validate new transactions on more than one full node before accepting them - -When running more than one node, any new transaction must be accepted only after validating that it's a valid transaction in all the nodes. - -### Validate all your full nodes have the same best block - -It's also important to regularly check if all nodes have the same best block. If they have different best blocks, this might be a clue that one of the nodes could be a target of an attack because is running a separate fork of the blockchain. - -### Recommended architecture +- This software connects to a headless wallet which communicates with a full node. The full node is connected to the Hathor's p2p network. +- The headless wallet will handle all events from the full node to keep its balance and transaction history updated. The headless software have APIs to manage a wallet, e.g. get addresses, create and push transactions to the network, create new tokens, and some other features described in the project [repository](https://github.com/HathorNetwork/hathor-wallet-headless). 
                        ┌─────────────┐      ┌──────────┐                     ┌─────────────┐        ┌─────────────┐
@@ -69,23 +42,44 @@ It's also important to regularly check if all nodes have the same best block. If
-## Peer-id +## Run more than one node + +We strongly recommend use cases to run two or more full nodes as a protection to direct attacks to their full nodes. + +Use case's full nodes **should not** be connected among them. This is important to mitigate some attack vectors. Remember that the transactions will be propagate by the p2p network and all use case's full nodes will receive the transactions eventually during normal network activity. + +### Validate new transactions on more than one full node before accepting them + +Let's assume an exchange wants to run nodes to identify deposits in the Hathor network, so a recommended approach for the integration would be to run at least two full nodes (node1 and node2), which are not connected between them (node1 must have node2 in the blacklist and vice versa). In that architecture, if any deposit is identified in node1, then the exchange must check that it's also a valid transaction in node2 and in one of the public nodes. With this approach, if one of the nodes is compromised and with a different consensus/state, an alert must be raised, so we must investigate what happened. -The peer-id is a unique identifier of your full node in Hathor's p2p network. Even though it does not contain any private key information, keeping it secret is a security measure that prevents attackers from targeting directly your full nodes. +### Validate all your full nodes have the same best block + +Use cases should regularly check whether the best block is the same on all their full nodes. If full node have different best blocks, the validation must be done again some seconds later because this might happen depending on the network block propagation time. If the differente continue, the nodes might be under attack and the use case should consider blocking deposits. + + +## Peer-id -Because of that, it's important to never share it in public chats, only in private groups with the Hathor team. If you think your peer-id has been exposed, you should generate a new peer-id and replace the exposed ones. +The peer-id is a unique identifier of your full node in Hathor's p2p network. You must keep your peer-id secret to prevent attackers from directly targeting your full nodes. Do not tell anyone your peer-ids, and do not publish them on public channels, only in private groups with the Hathor team. If you think your peer-id has been exposed, you should generate a new peer-id and replace the exposed ones. ## Be alert for weird behavior -### Correctly validate transactions +### How to validate a new transaction The transactions in the Hathor network have many fields that must be checked to guarantee that a transaction is valid for your use case. For more details about the fields of a transaction, check the [Transaction Anatomy RFC](https://github.com/HathorNetwork/rfcs/blob/master/text/0015-anatomy-of-tx.md). +- [Version](#version) +- [Voided state](#voided-state) +- Output + - [Token](#token) + - [Value](#value) + - [Address](#address) + - [Timelock](#timelock) + #### Version Transactions have `version=1`, blocks have versions `0` or `3`, and token creation transactions have `version=2`. -#### Voided +#### Voided state A voided transaction is **not** a valid transaction. You must only accept transactions that have `is_voided=false`. @@ -111,20 +105,24 @@ The output must have a timelock, and if it's bigger than the current timestamp, #### Number of confirmations -Some use cases might handle transactions with huge amounts, so it's essential to wait for some blocks to confirm the transaction before accepting it as a valid one. The more blocks confirm a transaction, the more guarantee there is that this transaction won't become invalid in the future. +Some use cases might handle transactions with huge amounts, so it's essential to wait for some blocks to confirm the transaction before accepting it as a valid one. The more blocks confirm a transaction, the more guarantee there is that this transaction won't become invalid in the future. As a reference, Bitcoin's use cases usually require a six confirmations before accepting a new deposit. ### Check if an unusual amount of deposits or withdrawals are being made -Many use cases have withdrawals/deposits in their set of features through blockchain transactions. It's important to be alerted in case any unusual amount of deposits/withdrawals are being made because this could be caused by an attack. +Many use cases have withdrawals/deposits in their set of features through blockchain transactions. It's important to check for unusual levels of deposits and withdrawals because this could be caused by an attack. -For that situation, it's crucial to have an easy way to block specific accounts that have unusual behavior and might be part of an attack. +For that situation, it's crucial to have an easy way to block specific accounts that have unusual behavior and might be part of an attack. You might also consider to limit the number of operations a user can do in a time window. + +This validation and user block must be done in the use-case software application. ### Check if one of your full nodes gets out-of-sync -One other relevant aspect of full node to always check for weird behavior is the sync among them. We recommend use cases to regularly validate that all their full nodes are in sync among them and in sync with at least one public node as well. +One other relevant aspect of full node is to always check for weird behavior is the sync among them. We recommend use cases to regularly validate that all their full nodes are in sync among them and in sync with at least one public node as well. This validation is important to guarantee the node is not isolated from the rest of the network with a fork of the blockchain. +Besides that, it's also important to validate that the timestamp of the best block of the node is recent, which means that the node's blockchain is not halted in the past. + # Reference-level explanation [reference-level-explanation]: #reference-level-explanation From 7d5b0c4552cfa12299e29dbe2aa10c7d89cbef20 Mon Sep 17 00:00:00 2001 From: Pedro Ferreira Date: Mon, 31 Oct 2022 12:04:47 -0300 Subject: [PATCH 3/4] feat: improved text with latest review --- ...000-use-case-integration-best-practices.md | 52 +++++++++---------- 1 file changed, 24 insertions(+), 28 deletions(-) diff --git a/text/0000-use-case-integration-best-practices.md b/text/0000-use-case-integration-best-practices.md index e3856e1..f31d1dc 100644 --- a/text/0000-use-case-integration-best-practices.md +++ b/text/0000-use-case-integration-best-practices.md @@ -46,11 +46,11 @@ There are multiple ways that a use case can integrate with Hathor, here is a com We strongly recommend use cases to run two or more full nodes as a protection to direct attacks to their full nodes. -Use case's full nodes **should not** be connected among them. This is important to mitigate some attack vectors. Remember that the transactions will be propagate by the p2p network and all use case's full nodes will receive the transactions eventually during normal network activity. +Use case's full nodes **should not** be connected among them. This is important to mitigate some attack vectors. Remember that the transactions will be propagated by the p2p network and all use case's full nodes will receive the transactions eventually during normal network activity. ### Validate new transactions on more than one full node before accepting them -Let's assume an exchange wants to run nodes to identify deposits in the Hathor network, so a recommended approach for the integration would be to run at least two full nodes (node1 and node2), which are not connected between them (node1 must have node2 in the blacklist and vice versa). In that architecture, if any deposit is identified in node1, then the exchange must check that it's also a valid transaction in node2 and in one of the public nodes. With this approach, if one of the nodes is compromised and with a different consensus/state, an alert must be raised, so we must investigate what happened. +Let's assume an exchange wants to run nodes to identify deposits in the Hathor network, so a recommended approach for the integration would be to run at least two full nodes (node1 and node2), which are not connected between them (node1 must have node2 in the blacklist and vice versa). In that architecture, if any deposit is identified in node1, then the exchange must check that it's also a valid transaction in node2 and in one of the public nodes. In this approach, if an attacker successfully compromises one of your full nodes, your validation would fail and the deposit will not be accepted. ### Validate all your full nodes have the same best block @@ -59,11 +59,9 @@ Use cases should regularly check whether the best block is the same on all their ## Peer-id -The peer-id is a unique identifier of your full node in Hathor's p2p network. You must keep your peer-id secret to prevent attackers from directly targeting your full nodes. Do not tell anyone your peer-ids, and do not publish them on public channels, only in private groups with the Hathor team. If you think your peer-id has been exposed, you should generate a new peer-id and replace the exposed ones. +The peer-id is a unique identifier of your full node in Hathor's p2p network. You must keep your peer-id secret to prevent attackers from directly targeting your full nodes. Do not tell anyone your peer-ids, and do not publish them on public channels. If you think your peer-id has been exposed, you should generate a new peer-id and replace the exposed ones. -## Be alert for weird behavior - -### How to validate a new transaction +## How to validate a new transaction The transactions in the Hathor network have many fields that must be checked to guarantee that a transaction is valid for your use case. For more details about the fields of a transaction, check the [Transaction Anatomy RFC](https://github.com/HathorNetwork/rfcs/blob/master/text/0015-anatomy-of-tx.md). @@ -75,37 +73,35 @@ The transactions in the Hathor network have many fields that must be checked to - [Address](#address) - [Timelock](#timelock) -#### Version - -Transactions have `version=1`, blocks have versions `0` or `3`, and token creation transactions have `version=2`. - -#### Voided state +### Version -A voided transaction is **not** a valid transaction. You must only accept transactions that have `is_voided=false`. +Version identifies the type of vertex. The possible choices are: -#### Outputs +- Block: 0 +- Regular Transaction: 1 +- Token Creation Transaction: 2 +- Merged Mined Block: 3 -The outputs contain the fields used to identify that the funds belong to your wallet. +Depending on your use case you must accept one or more types. -##### Token +### Voided state -The output has the token information, so you can check if it has the expected tokens. For example, exchanges expect the deposits to be of HTR, so `token = '00'` and `token_data=0`. +A voided transaction is cancelled and should **never** be accepted. You must validate that the transaction is not voided asserting that `is_voided == false`. -##### Value +### Outputs -The value of the output is an integer, so `1000` means `10.00`. +Hathor supports multi-token transactions. You must confirm that your outputs are correctly placed as follows: -##### Address +- Token id is correct. If you accept only HTR token, token id must be `"00"`. +- Token data is equal to `0`, if it's HTR token. +- Value matches the expected value. Note that it is an integer and `12.34` is represented by `1234`. +- Timelock must be `null`; otherwise your funds might be locked. -The output address must be one of your wallet's addresses. +### Number of confirmations -##### Timelock +Some use cases might handle transactions with huge amounts, so it's essential to wait for some blocks to confirm the transaction before accepting it as a valid one. The more blocks confirm a transaction, the more guarantee there is that this transaction won't become voided in the future. As a reference, Bitcoin's use cases usually require six confirmations before accepting a new deposit. -The output must have a timelock, and if it's bigger than the current timestamp, you won't be able to spend this output until this moment is reached. Because of that, it's essential to validate that `timelock=null` before validating this transaction. - -#### Number of confirmations - -Some use cases might handle transactions with huge amounts, so it's essential to wait for some blocks to confirm the transaction before accepting it as a valid one. The more blocks confirm a transaction, the more guarantee there is that this transaction won't become invalid in the future. As a reference, Bitcoin's use cases usually require a six confirmations before accepting a new deposit. +## Be alert for weird behavior ### Check if an unusual amount of deposits or withdrawals are being made @@ -113,11 +109,11 @@ Many use cases have withdrawals/deposits in their set of features through blockc For that situation, it's crucial to have an easy way to block specific accounts that have unusual behavior and might be part of an attack. You might also consider to limit the number of operations a user can do in a time window. -This validation and user block must be done in the use-case software application. +This validation and user throttling must be done in the use-case software application. ### Check if one of your full nodes gets out-of-sync -One other relevant aspect of full node is to always check for weird behavior is the sync among them. We recommend use cases to regularly validate that all their full nodes are in sync among them and in sync with at least one public node as well. +Always check for weird behavior in the synchronization among full nodes. We recommend use cases to regularly validate that all their full nodes are in sync among them and in sync with at least one public node as well. This validation is important to guarantee the node is not isolated from the rest of the network with a fork of the blockchain. From 8aac52dc23ffffe9251b1313c1117ec1343d3e42 Mon Sep 17 00:00:00 2001 From: Pedro Ferreira Date: Mon, 31 Oct 2022 13:29:15 -0300 Subject: [PATCH 4/4] fix links --- text/0000-use-case-integration-best-practices.md | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/text/0000-use-case-integration-best-practices.md b/text/0000-use-case-integration-best-practices.md index f31d1dc..9facee4 100644 --- a/text/0000-use-case-integration-best-practices.md +++ b/text/0000-use-case-integration-best-practices.md @@ -67,11 +67,8 @@ The transactions in the Hathor network have many fields that must be checked to - [Version](#version) - [Voided state](#voided-state) -- Output - - [Token](#token) - - [Value](#value) - - [Address](#address) - - [Timelock](#timelock) +- [Outputs](#outputs) +- [Number of confirmations](#number-of-confirmations) ### Version