diff --git a/httpDatastore.adoc b/httpDatastore.adoc new file mode 100644 index 0000000..1c93bb7 --- /dev/null +++ b/httpDatastore.adoc @@ -0,0 +1,102 @@ +How to create an HTTP datastore environment +=========================================== +Assumptions Made +---------------- +* GRR servers will be running on Ubuntu 16.04 LTS following +https://github.com/google/grr-doc/blob/master/installfrompip.adoc#installing-grr-server-for-dev-ie-tracking-head[Install +from Pip] documentation +* A “main server” will be hosting all the other services in the infrastructure. +Virtual Environment created is called “GRR_NEW” +* It is desired to have every data server host through port tcp:7000 (this is not +a necessity) +* All services are down for this configuration change. +* On every server you work with in this document, you activated your virtualenv +and changed into grr/ + +Set up the master dataserver +---------------------------- +1. Copy the `/install_data/etc/server.local.yaml` file from your main server to your +master dataserver +2. Edit the `server.local.yaml` with either changing or adding the following: ++ +[source,yaml] +Datastore.location: +Datastore.implementation: SQLiteDataStore +Dataserver.server_list: + - http://:7000 + - http://:7000 + - http://:7000 +Dataserver.client_credentials: + - ::rw +Dataserver.server_username: +Dataserver.server_password: + +3. Run `python grr/server/data_server/data_server.py +--config=install_data/etc/grr-server.yaml --master` to start the service and +verify it starts correctly. Keep this running + +Set up the slave dataservers +---------------------------- +1. Copy `install_data/etc/server.local.yaml` from the main server to your +dataserver slave +2. Edit `install_data/etc/server.local.yaml`. Change or add the following: ++ +[source,yaml] +Dataserver.server_list: + - :7000 +Dataserver.server_username: +Dataserver.server_password: +Datastore.implementation: SqliteDataStore +Datastore.location: + +* Notes: + - The `server_username` and `server_password` were configured in the setup of the +master dataserver. + - The configuration lines may not be in a sensible order, make sure to search for + them within the document to avoid duplicates. `Dataserver.server_username` and + `Dataserver.server_password` will need to be added in. The others will be modified +3. Run the command `grr_server --component dataserver_slave --verbose` to see the +data server slave connect. +4. Repeat for each dataserver slave you wish to create +5. *On the master server* in the management interface, after adding and connecting +all the data servers, run `rebalance` and confirm that you would like to reshard +the database. + +Update the main server configuration +------------------------------------ +1. Edit `install_data/etc/server.local.yaml` ++ +[source,yaml] +Dataserver.server_list: + - http://:7000 + - http://:7000 +Datastore.implementation: HTTPDataStore +HTTPDataStore.username: +HTTPDataStore.password: + + +Add more dataserver slaves after initial setup +---------------------------------------------- +Change configuration on the master dataserver +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +1. Run the command `python grr/server/data_server/manager.py +--config=install_data/etc/grr-server.yaml` to start the dataserver management +interface + - While in the interface, run `addserver 7000` and confirm that +you would like to add it. Repeat for all dataserver slaves. + - Keep this interface up +2. In a separate tab edit `install_data/etc/server.local.yaml`. Change the +`Dataserver.server_list` section to add any dataserver slaves you want to add ++ +[source,yaml] +Dataserver.server_list: + - :7000 + - :7000 + - :7000 + +3. Run the command `python grr/server/data_server/data_server.py +--config=install_data/etc/grr-server.yaml --master` + - Leave this command running, keep the screen somewhere you can monitor it. +4. Set up the dataserver slaves as you did above +5. Update the main server configuration as you did above with the new +dataserver slaves