From 39c6a822c96e9a52edecfe20d27bb5da55451c7e Mon Sep 17 00:00:00 2001 From: Trayan Azarov Date: Mon, 18 Dec 2023 19:35:01 +0200 Subject: [PATCH 1/3] feat: Cloud Deployment Blueprints - Adding AWS, GCP and Digital Ocean cloud deployment blueprints instead of the cloudformation template Refs: chroma-core/chroma@1479 --- docs/deployment.md | 283 ++++++++++++++++++++++++++++----------------- 1 file changed, 177 insertions(+), 106 deletions(-) diff --git a/docs/deployment.md b/docs/deployment.md index 6ea29758..62ec1478 100644 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -4,7 +4,8 @@ title: "☁️ Deployment" --- :::caution Alpha Status -Chroma Server is currently in Alpha. We are working hard to move Chroma from an in-memory single-process oriented library to a distributed production-grade DB! +Chroma Server is currently in Alpha. We are working hard to move Chroma from an in-memory single-process oriented +library to a distributed production-grade DB! - [x] Alpha <- Currently - [ ] Technical Preview - ~1 month away, powered by a completely new backend @@ -34,14 +35,16 @@ Fill out the survey to jump the wait-list. Coming Q3 2023. You can run a Chroma server in a Docker container. -You can get the Chroma Docker image from [Docker Hub](https://hub.docker.com/r/chromadb/chroma), or from the [Chroma GitHub Container Registry](https://github.com/chroma-core/chroma/pkgs/container/chroma) +You can get the Chroma Docker image from [Docker Hub](https://hub.docker.com/r/chromadb/chroma), or from +the [Chroma GitHub Container Registry](https://github.com/chroma-core/chroma/pkgs/container/chroma) ```sh docker pull chromadb/chroma docker run -p 8000:8000 chromadb/chroma ``` -You can also build the Docker image yourself from the Dockerfile in the [Chroma GitHub repository](https://github.com/chroma-core/chroma) +You can also build the Docker image yourself from the Dockerfile in +the [Chroma GitHub repository](https://github.com/chroma-core/chroma) ```sh git clone git@github.com:chroma-core/chroma.git @@ -53,159 +56,227 @@ The Chroma client can then be configured to connect to the server running in the ```python import chromadb + chroma_client = chromadb.HttpClient(host='localhost', port=8000) ``` ### Authentication with Docker -By default, the Docker image will run with no authentication. Follow the [Authentication](./usage-guide#authentication) section of the Usage Guide to configure authentication in the Docker container. +By default, the Docker image will run with no authentication. Follow the [Authentication](./usage-guide#authentication) +section of the Usage Guide to configure authentication in the Docker container. -You can also create a `.chroma_env` file setting the required environment variables and pass it to the Docker container with the `--env-file` flag when running the container. +You can also create a `.chroma_env` file setting the required environment variables and pass it to the Docker container +with the `--env-file` flag when running the container. ```sh docker run --env-file ./.chroma_env -p 8000:8000 chromadb/chroma ``` -## Simple AWS Deployment - -:warning: Chroma and its underlying database need at least 2gb of RAM, -which means it won't fit on the 1gb instances provided as part of the -AWS Free Tier. This template uses a `t3.small` EC2 instance, which -costs about two cents an hour, or $15 for a full month. If you follow these -instructions, AWS will bill you accordingly. - -:warning: This basic stack doesn't support any kind of authentication; -anyone who knows your server IP will be able to add and query for -embeddings. To secure this endpoint, you'll need to put it behind -[AWS API Gateway](https://aws.amazon.com/api-gateway/) or add your own -authenticating proxy. - -:warning: By default, this template saves all data on a single -volume. When you delete or replace it, the data will disappear. For -serious production use (with high availability, backups, etc.) please -read and understand the CloudFormation template and use it as a basis -for what you need, or reach out to the Chroma team for assistance. +## Cloud Deployments -### Step 1: Get an AWS Account +Below are instructions for deploying Chroma with Terraform on the following public cloud providers: -You will need an AWS Account. You can use one you already have, or -[create a new one](https://aws.amazon.com). +- [AWS](#aws) +- [Google Cloud Platform](#gcp-google-cloud-platform) +- [Digital Ocean](#digital-ocean) -### Step 2: Get credentials +### Prerequisites -For this example, we will be using the AWS command line -interface. There are -[several ways](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-prereqs.html) -to configure the AWS CLI, but for the purposes of these examples we -will presume that you have -[obtained an AWS access key](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html) -and will be using environment variables to configure AWS. - -Export the `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` environment variables in your shell: - -``` +- [Terraform CLI v1.3.4+](https://developer.hashicorp.com/terraform/tutorials/gcp-get-started/install-cli) -export AWS_ACCESS_KEY_ID=**\*\***\*\*\*\***\*\*** -export AWS_SECRET_ACCESS_KEY=****\*\*****\*\*****\*\***** +Generate a key: +```bash +ssh-keygen -t RSA -b 4096 -C "Chroma SSH Keypair" -N "" -f ./chroma_id_rsa && chmod 400 ./chroma_id_rsa ``` -You can also configure AWS to use a region of your choice using the -`AWS_REGION` environment variable: - +The key pair will allow you to connect to your Chroma instance via SSH (applicable only for AWS, GCP and DO +deployments). In addition, the keypair is used for post provisioning steps such as formatting the Chroma data volume ( +applicable only for AWS deployments). + +### AWS + +AWS Configuration variables + +| Parameter Name | Description | Default Value | +|--------------------------------------------|---------------------------------------------------------------------------------------------------------------------|-----------------------| +| chroma_release | The chroma release to deploy | 0.4.20 | +| region | AWS Region | us-west-1 | +| instance_type | AWS EC2 Instance Type | t3.medium | +| public_access | Enable public ingress on port 8000 | true | +| enable_auth | Enable authentication | true | +| auth_type | Authentication type | token | +| import_keypair | Whether to import the keypair in AWS. Set this to `false` if your keypair is already available in AWS. | true | +| keypair_name | The name of the keypair to import | chroma_keypair | +| ssh_public_key | SSH Public Key | ./chroma_id_rsa.pub | +| ssh_private_key | SSH Private Key | ./chroma_id_rsa | +| chroma_instance_volume_size | The size of the instance volume - the root volume | 30 | +| chroma_data_volume_size | EBS Volume Size of the attached data volume where your chroma data is stored | 20 | +| chroma_data_volume_snapshot_before_destroy | Take a snapshot of the chroma data volume before destroying it | false | +| chroma_data_restore_from_snapshot_id | Restore the chroma data volume from a snapshot | null | +| chroma_port | The port that chroma listens on | 8000 | +| source_ranges | List of CIDR ranges to allow through the firewall | ["0.0.0.0/0", "::/0"] | +| mgmt_source_ranges | List of CIDR ranges to allow for management of the Chroma instance. This is used for SSH incoming traffic filtering | ["0.0.0.0/0", "::/0"] | + +> Note: All of the above variables can be exported via environment variables (`TF_=`) or set in a `.tfvars` file. + +Set up your Terraform variables and deploy your instance: + +```bash +export TF_VAR_AWS_ACCESS_KEY= +export TF_VAR_AWS_SECRET_ACCESS_KEY= +export TF_ssh_public_key="./chroma_id_rsa.pub" +export TF_ssh_private_key="./chroma_id_rsa" +export TF_VAR_chroma_release=0.4.20 +export TF_VAR_region="us-west-1" +export TF_VAR_public_access="true" +export TF_VAR_enable_auth="true" +export TF_VAR_auth_type="token" +export TF_VAR_chroma_data_volume_snapshot_before_destroy="true" +terraform apply -auto-approve ``` -export AWS_REGION=us-east-1 +Verify that your instance is up and running: +```bash +export instance_public_ip=$(terraform output instance_public_ip | sed 's/"//g') +curl -v http://$instance_public_ip:8000/api/v1/heartbeat ``` -### Step 3: Run CloudFormation - -Chroma publishes Cloudformation templates to S3 for each release. +> Note: Depending on your OS the `sed` command might not work. In that case, you can manually copy the public IP from +> the Terraform output. -To launch the template using AWS CloudFormation, run the following command line invocation: +To get the auth token generated during the setup: +```bash +terraform output chroma_auth_token ``` -aws cloudformation create-stack --stack-name my-chroma-stack --template-url https://s3.amazonaws.com/public.trychroma.com/cloudformation/latest/chroma.cf.json - -``` +For more details check +our [Terraform AWS deployment blueprint](https://github.com/chroma-core/chroma/tree/main/examples/deployments/aws-terraform) -Replace `--stack-name my-chroma-stack` with a different stack name, if you wish. - -Wait a few minutes for the server to boot up, and Chroma will be -available! You can get the public IP address of your new Chroma server using the AWS console, or using the following command: - -``` +### GCP (Google Cloud Platform) -aws cloudformation describe-stacks --stack-name my-chroma-stack --query 'Stacks[0].Outputs' +You will need to install [gcloud CLI](https://cloud.google.com/sdk/gcloud) and authenticate with your GCP account: +```bash +gcloud auth application-default login ``` -### Step 4: Customize the Stack (optional) - -The CloudFormation template allows you to pass particular key/value -pairs to override aspects of the stack. Available keys are: - -- `InstanceType` - the AWS instance type to run (default: `t3.small`) -- `KeyName` - the AWS EC2 KeyPair to use, allowing to access the instance via SSH (default: none) - -To set a CloudFormation stack's parameters using the AWS CLI, use the -`--parameters` command line option. Parameters must be specified using -the format `ParameterName={parameter},ParameterValue={value}`. - -For example, the following command launches a new stack similar to the -above, but on a `m5.4xlarge` EC2 instance, and adding a KeyPair named -`mykey` so anyone with the associated private key can SSH into the -machine: - +GCP Configuration parameters: + +| Parameter Name | Description | Default Value | +|-----------------------------------|----------------------------------------------------------------------------------------|------------------------| +| project_id | The project id to deploy to | N/A | +| chroma_release | The chroma release to deploy | 0.4.20 | +| zone | N/A | us-central1-a | +| image | The image to use for the instance | debian-cloud/debian-11 | +| vm_user | The user to use for connecting to the instance. This is usually the default image user | debian | +| machine_type | N/A | e2-small | +| public_access | Enable public ingress on port 8000 | true | +| enable_auth | Enable authentication | true | +| auth_type | Authentication type | token | +| ssh_public_key | SSH Public Key | ./chroma_id_rsa.pub | +| ssh_private_key | SSH Private Key | ./chroma_id_rsa | +| chroma_instance_volume_size | The size of the instance volume - the root volume | 30 | +| chroma_data_volume_size | Volume Size of the attached data volume where your chroma data is stored | 20 | +| chroma_data_volume_device_name | The device name of the chroma data volume | chroma-disk-0 | +| prevent_chroma_data_volume_delete | Prevent the chroma data volume from being deleted when the instance is terminated | false | +| disk_type | The type of disk to use for the instance. Can be either pd-standard or pd-ssd | pd-ssd | +| labels | Labels to apply to all resources in this example | {environment: 'dev'} | +| chroma_port | The port that chroma listens on | 8000 | +| source_ranges | List of CIDR ranges to allow through the firewall | ["0.0.0.0/0"] | + +> Note: All of the above variables can be exported via environment variables (`TF_=`) or set in a `.tfvars` file. + +Set up your Terraform variables and deploy your instance: + +```bash +export TF_VAR_project_id= +export TF_ssh_public_key="./chroma_id_rsa.pub" +export TF_ssh_private_key="./chroma_id_rsa" +export TF_VAR_chroma_release="0.4.20" +export TF_VAR_zone="us-central1-a" +export TF_VAR_public_access="true" +export TF_VAR_enable_auth="true" +export TF_VAR_auth_type="token" +terraform apply -auto-approve ``` -aws cloudformation create-stack --stack-name my-chroma-stack --template-url https://s3.amazonaws.com/public.trychroma.com/cloudformation/latest/chroma.cf.json \ - --parameters ParameterKey=KeyName,ParameterValue=mykey \ - ParameterKey=InstanceType,ParameterValue=m5.4xlarge +Verify that your instance is up and running: +```bash +export instance_public_ip=$(terraform output instance_public_ip | sed 's/"//g') +curl -v http://$instance_public_ip:8000/api/v1/heartbeat ``` -### Step 5: Configure the Chroma Library - -When you launch the Chroma client library to actually use Chroma, all -you need to do is configure it to use the server's IP address and port -`8000`. You can do this in two ways: - -###### Using Environment Variables +> Note: Depending on your OS the `sed` command might not work. In that case, you can manually copy the public IP from +> the Terraform output. -``` - -export CHROMA_API_IMPL=rest -export CHROMA_SERVER_HOST= -export CHROMA_SERVER_HTTP_PORT=8000 +To get the auth token generated during the setup: +```bash +terraform output chroma_auth_token ``` -###### In Code - +For more details check +our [Terraform GCP deployment blueprint](https://github.com/chroma-core/chroma/tree/main/examples/deployments/google-cloud-compute) + +### Digital Ocean + +Digital Ocean Configuration parameters: + +| Parameter Name | Description | Default Value | +|-------------------------|---------------------------------------------------------------------------------------------------------------------|-----------------------| +| instance_image | The image to use for the instance | ubuntu-22-04-x64 | +| chroma_release | The chroma release to deploy | 0.4.20 | +| region | DO Region | nyc2 | +| instance_type | Droplet size | s-2vcpu-4gb | +| public_access | Enable public ingress on port 8000 | true | +| enable_auth | Enable authentication | true | +| auth_type | Authentication type | token | +| ssh_public_key | SSH Public Key | ./chroma-do.pub | +| ssh_private_key | SSH Private Key | ./chroma-do | +| chroma_data_volume_size | EBS Volume Size of the attached data volume where your chroma data is stored | 20 | +| chroma_port | The port that chroma listens on | 8000 | +| source_ranges | List of CIDR ranges to allow through the firewall | ["0.0.0.0/0", "::/0"] | +| mgmt_source_ranges | List of CIDR ranges to allow for management of the Chroma instance. This is used for SSH incoming traffic filtering | ["0.0.0.0/0", "::/0"] | + +> Note: All of the above variables can be exported via environment variables (`TF_=`) or set in a `.tfvars` file. + +Set up your Terraform variables and deploy your instance: + +```bash +export TF_VAR_do_token= +export TF_ssh_public_key="./chroma_id_rsa.pub" +export TF_ssh_private_key="./chroma_id_rsa" +export TF_VAR_chroma_release="0.4.20" +export TF_VAR_region="ams2" +export TF_VAR_public_access="true" +export TF_VAR_enable_auth="true" +export TF_VAR_auth_type="token" +terraform apply -auto-approve ``` -import chromadb -from chromadb.config import Settings -chroma = chromadb.HttpClient(host=, port=8000) +Verify that your instance is up and running: +```bash +export instance_public_ip=$(terraform output instance_public_ip | sed 's/"//g') +curl -v http://$instance_public_ip:8000/api/v1/heartbeat ``` -### Step 6: Clean Up (optional). - -To destroy the stack and remove all AWS resources, use the AWS CLI `delete-stack` command. - -``` +> Note: Depending on your OS the `sed` command might not work. In that case, you can manually copy the public IP from +> the Terraform output. -aws cloudformation delete-stack --stack-name my-chroma-stack +To get the auth token generated during the setup: +```bash +terraform output chroma_auth_token ``` -:warning: This will destroy all the data in your Chroma database, -unless you've taken a snapshot or otherwise backed it up. +For more details check +our [Terraform Digital Ocean deployment blueprint](https://github.com/chroma-core/chroma/tree/main/examples/deployments/do-terraform) -### Troubleshooting +### Need help or have questions? -If you get an error saying `No default VPC for this user` when creating `ChromaInstanceSecurityGroup`, head to [AWS VPC section](https://us-east-1.console.aws.amazon.com/vpc/home?region=us-east-1#vpcs) and create a default VPC for your user. +Reach out to us on [Discord](https://discord.gg/MMeYNTmh3x) \ No newline at end of file From 0b9804ffcff5850f897d3faadb6e79087346cd69 Mon Sep 17 00:00:00 2001 From: Trayan Azarov Date: Tue, 19 Dec 2023 09:49:29 +0200 Subject: [PATCH 2/3] fix: Fixed source CIRDs for AWS template Refs: chroma-core/chroma@1479 --- docs/deployment.md | 52 +++++++++++++++++++++++++--------------------- 1 file changed, 28 insertions(+), 24 deletions(-) diff --git a/docs/deployment.md b/docs/deployment.md index 62ec1478..c7b7aa48 100644 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -87,7 +87,8 @@ Below are instructions for deploying Chroma with Terraform on the following publ Generate a key: ```bash -ssh-keygen -t RSA -b 4096 -C "Chroma SSH Keypair" -N "" -f ./chroma_id_rsa && chmod 400 ./chroma_id_rsa +ssh-keygen -t RSA -b 4096 -C "Chroma SSH Keypair" -N "" -f ./chroma_id_rsa && \ +chmod 400 ./chroma_id_rsa ``` The key pair will allow you to connect to your Chroma instance via SSH (applicable only for AWS, GCP and DO @@ -98,27 +99,28 @@ applicable only for AWS deployments). AWS Configuration variables -| Parameter Name | Description | Default Value | -|--------------------------------------------|---------------------------------------------------------------------------------------------------------------------|-----------------------| -| chroma_release | The chroma release to deploy | 0.4.20 | -| region | AWS Region | us-west-1 | -| instance_type | AWS EC2 Instance Type | t3.medium | -| public_access | Enable public ingress on port 8000 | true | -| enable_auth | Enable authentication | true | -| auth_type | Authentication type | token | -| import_keypair | Whether to import the keypair in AWS. Set this to `false` if your keypair is already available in AWS. | true | -| keypair_name | The name of the keypair to import | chroma_keypair | -| ssh_public_key | SSH Public Key | ./chroma_id_rsa.pub | -| ssh_private_key | SSH Private Key | ./chroma_id_rsa | -| chroma_instance_volume_size | The size of the instance volume - the root volume | 30 | -| chroma_data_volume_size | EBS Volume Size of the attached data volume where your chroma data is stored | 20 | -| chroma_data_volume_snapshot_before_destroy | Take a snapshot of the chroma data volume before destroying it | false | -| chroma_data_restore_from_snapshot_id | Restore the chroma data volume from a snapshot | null | -| chroma_port | The port that chroma listens on | 8000 | -| source_ranges | List of CIDR ranges to allow through the firewall | ["0.0.0.0/0", "::/0"] | -| mgmt_source_ranges | List of CIDR ranges to allow for management of the Chroma instance. This is used for SSH incoming traffic filtering | ["0.0.0.0/0", "::/0"] | - -> Note: All of the above variables can be exported via environment variables (`TF_=`) or set in a `.tfvars` file. +| Parameter Name | Description | Default Value | +|--------------------------------------------|---------------------------------------------------------------------------------------------------------------------|---------------------| +| chroma_release | The chroma release to deploy | 0.4.20 | +| region | AWS Region | us-west-1 | +| instance_type | AWS EC2 Instance Type | t3.medium | +| public_access | Enable public ingress on port 8000 | true | +| enable_auth | Enable authentication | true | +| auth_type | Authentication type | token | +| import_keypair | Whether to import the keypair in AWS. Set this to `false` if your keypair is already available in AWS. | true | +| keypair_name | The name of the keypair to import | chroma_keypair | +| ssh_public_key | SSH Public Key | ./chroma_id_rsa.pub | +| ssh_private_key | SSH Private Key | ./chroma_id_rsa | +| chroma_instance_volume_size | The size of the instance volume - the root volume | 30 | +| chroma_data_volume_size | EBS Volume Size of the attached data volume where your chroma data is stored | 20 | +| chroma_data_volume_snapshot_before_destroy | Take a snapshot of the chroma data volume before destroying it | false | +| chroma_data_restore_from_snapshot_id | Restore the chroma data volume from a snapshot | null | +| chroma_port | The port that chroma listens on | 8000 | +| source_ranges | List of CIDR ranges to allow through the firewall | ["0.0.0.0/0"] | +| mgmt_source_ranges | List of CIDR ranges to allow for management of the Chroma instance. This is used for SSH incoming traffic filtering | ["0.0.0.0/0"] | + +> Note: All of the above variables can be exported via environment variables (`TF_=`) or set +> in a `.tfvars` file. Set up your Terraform variables and deploy your instance: @@ -187,7 +189,8 @@ GCP Configuration parameters: | chroma_port | The port that chroma listens on | 8000 | | source_ranges | List of CIDR ranges to allow through the firewall | ["0.0.0.0/0"] | -> Note: All of the above variables can be exported via environment variables (`TF_=`) or set in a `.tfvars` file. +> Note: All of the above variables can be exported via environment variables (`TF_=`) or set +> in a `.tfvars` file. Set up your Terraform variables and deploy your instance: @@ -242,7 +245,8 @@ Digital Ocean Configuration parameters: | source_ranges | List of CIDR ranges to allow through the firewall | ["0.0.0.0/0", "::/0"] | | mgmt_source_ranges | List of CIDR ranges to allow for management of the Chroma instance. This is used for SSH incoming traffic filtering | ["0.0.0.0/0", "::/0"] | -> Note: All of the above variables can be exported via environment variables (`TF_=`) or set in a `.tfvars` file. +> Note: All of the above variables can be exported via environment variables (`TF_=`) or set +> in a `.tfvars` file. Set up your Terraform variables and deploy your instance: From 019ae20b71d5a0f83e163f1cd55101136d411d9e Mon Sep 17 00:00:00 2001 From: Trayan Azarov Date: Mon, 22 Jan 2024 19:23:46 +0200 Subject: [PATCH 3/3] refactor: Refactored Deployments into its own category Refs: chroma-core/chroma#1479 --- docs/deployment.md | 286 ------------------------------- docs/deployment/aws.md | 67 ++++++++ docs/deployment/digital-ocean.md | 61 +++++++ docs/deployment/gcp.md | 73 ++++++++ docs/deployment/index.md | 104 +++++++++++ sidebars.js | 26 ++- 6 files changed, 330 insertions(+), 287 deletions(-) delete mode 100644 docs/deployment.md create mode 100644 docs/deployment/aws.md create mode 100644 docs/deployment/digital-ocean.md create mode 100644 docs/deployment/gcp.md create mode 100644 docs/deployment/index.md diff --git a/docs/deployment.md b/docs/deployment.md deleted file mode 100644 index c7b7aa48..00000000 --- a/docs/deployment.md +++ /dev/null @@ -1,286 +0,0 @@ ---- -sidebar_position: 9 -title: "☁️ Deployment" ---- - -:::caution Alpha Status -Chroma Server is currently in Alpha. We are working hard to move Chroma from an in-memory single-process oriented -library to a distributed production-grade DB! - -- [x] Alpha <- Currently -- [ ] Technical Preview - ~1 month away, powered by a completely new backend -- [ ] Full production -- [ ] GA - General Availability - -::: - -# ☁️ Deployment - -You can also deploy Chroma on a long-running server, and connect to it -remotely. - -There are many possible configurations, but for convenience we have -provided a very simple AWS CloudFormation template to experiment with -deploying Chroma to EC2 on AWS. - -## Hosted Chroma - -We want to offer hosted Chroma, and we need your help. - -Fill out the survey to jump the wait-list. Coming Q3 2023. - -[📝 30 second survey](https://airtable.com/shrOAiDUtS2ILy5vZ) - -## Docker - -You can run a Chroma server in a Docker container. - -You can get the Chroma Docker image from [Docker Hub](https://hub.docker.com/r/chromadb/chroma), or from -the [Chroma GitHub Container Registry](https://github.com/chroma-core/chroma/pkgs/container/chroma) - -```sh -docker pull chromadb/chroma -docker run -p 8000:8000 chromadb/chroma -``` - -You can also build the Docker image yourself from the Dockerfile in -the [Chroma GitHub repository](https://github.com/chroma-core/chroma) - -```sh -git clone git@github.com:chroma-core/chroma.git -cd chroma -docker-compose up -d --build -``` - -The Chroma client can then be configured to connect to the server running in the Docker container. - -```python -import chromadb - -chroma_client = chromadb.HttpClient(host='localhost', port=8000) -``` - -### Authentication with Docker - -By default, the Docker image will run with no authentication. Follow the [Authentication](./usage-guide#authentication) -section of the Usage Guide to configure authentication in the Docker container. - -You can also create a `.chroma_env` file setting the required environment variables and pass it to the Docker container -with the `--env-file` flag when running the container. - -```sh -docker run --env-file ./.chroma_env -p 8000:8000 chromadb/chroma -``` - -## Cloud Deployments - -Below are instructions for deploying Chroma with Terraform on the following public cloud providers: - -- [AWS](#aws) -- [Google Cloud Platform](#gcp-google-cloud-platform) -- [Digital Ocean](#digital-ocean) - -### Prerequisites - -- [Terraform CLI v1.3.4+](https://developer.hashicorp.com/terraform/tutorials/gcp-get-started/install-cli) - -Generate a key: - -```bash -ssh-keygen -t RSA -b 4096 -C "Chroma SSH Keypair" -N "" -f ./chroma_id_rsa && \ -chmod 400 ./chroma_id_rsa -``` - -The key pair will allow you to connect to your Chroma instance via SSH (applicable only for AWS, GCP and DO -deployments). In addition, the keypair is used for post provisioning steps such as formatting the Chroma data volume ( -applicable only for AWS deployments). - -### AWS - -AWS Configuration variables - -| Parameter Name | Description | Default Value | -|--------------------------------------------|---------------------------------------------------------------------------------------------------------------------|---------------------| -| chroma_release | The chroma release to deploy | 0.4.20 | -| region | AWS Region | us-west-1 | -| instance_type | AWS EC2 Instance Type | t3.medium | -| public_access | Enable public ingress on port 8000 | true | -| enable_auth | Enable authentication | true | -| auth_type | Authentication type | token | -| import_keypair | Whether to import the keypair in AWS. Set this to `false` if your keypair is already available in AWS. | true | -| keypair_name | The name of the keypair to import | chroma_keypair | -| ssh_public_key | SSH Public Key | ./chroma_id_rsa.pub | -| ssh_private_key | SSH Private Key | ./chroma_id_rsa | -| chroma_instance_volume_size | The size of the instance volume - the root volume | 30 | -| chroma_data_volume_size | EBS Volume Size of the attached data volume where your chroma data is stored | 20 | -| chroma_data_volume_snapshot_before_destroy | Take a snapshot of the chroma data volume before destroying it | false | -| chroma_data_restore_from_snapshot_id | Restore the chroma data volume from a snapshot | null | -| chroma_port | The port that chroma listens on | 8000 | -| source_ranges | List of CIDR ranges to allow through the firewall | ["0.0.0.0/0"] | -| mgmt_source_ranges | List of CIDR ranges to allow for management of the Chroma instance. This is used for SSH incoming traffic filtering | ["0.0.0.0/0"] | - -> Note: All of the above variables can be exported via environment variables (`TF_=`) or set -> in a `.tfvars` file. - -Set up your Terraform variables and deploy your instance: - -```bash -export TF_VAR_AWS_ACCESS_KEY= -export TF_VAR_AWS_SECRET_ACCESS_KEY= -export TF_ssh_public_key="./chroma_id_rsa.pub" -export TF_ssh_private_key="./chroma_id_rsa" -export TF_VAR_chroma_release=0.4.20 -export TF_VAR_region="us-west-1" -export TF_VAR_public_access="true" -export TF_VAR_enable_auth="true" -export TF_VAR_auth_type="token" -export TF_VAR_chroma_data_volume_snapshot_before_destroy="true" -terraform apply -auto-approve -``` - -Verify that your instance is up and running: - -```bash -export instance_public_ip=$(terraform output instance_public_ip | sed 's/"//g') -curl -v http://$instance_public_ip:8000/api/v1/heartbeat -``` - -> Note: Depending on your OS the `sed` command might not work. In that case, you can manually copy the public IP from -> the Terraform output. - -To get the auth token generated during the setup: - -```bash -terraform output chroma_auth_token -``` - -For more details check -our [Terraform AWS deployment blueprint](https://github.com/chroma-core/chroma/tree/main/examples/deployments/aws-terraform) - -### GCP (Google Cloud Platform) - -You will need to install [gcloud CLI](https://cloud.google.com/sdk/gcloud) and authenticate with your GCP account: - -```bash -gcloud auth application-default login -``` - -GCP Configuration parameters: - -| Parameter Name | Description | Default Value | -|-----------------------------------|----------------------------------------------------------------------------------------|------------------------| -| project_id | The project id to deploy to | N/A | -| chroma_release | The chroma release to deploy | 0.4.20 | -| zone | N/A | us-central1-a | -| image | The image to use for the instance | debian-cloud/debian-11 | -| vm_user | The user to use for connecting to the instance. This is usually the default image user | debian | -| machine_type | N/A | e2-small | -| public_access | Enable public ingress on port 8000 | true | -| enable_auth | Enable authentication | true | -| auth_type | Authentication type | token | -| ssh_public_key | SSH Public Key | ./chroma_id_rsa.pub | -| ssh_private_key | SSH Private Key | ./chroma_id_rsa | -| chroma_instance_volume_size | The size of the instance volume - the root volume | 30 | -| chroma_data_volume_size | Volume Size of the attached data volume where your chroma data is stored | 20 | -| chroma_data_volume_device_name | The device name of the chroma data volume | chroma-disk-0 | -| prevent_chroma_data_volume_delete | Prevent the chroma data volume from being deleted when the instance is terminated | false | -| disk_type | The type of disk to use for the instance. Can be either pd-standard or pd-ssd | pd-ssd | -| labels | Labels to apply to all resources in this example | {environment: 'dev'} | -| chroma_port | The port that chroma listens on | 8000 | -| source_ranges | List of CIDR ranges to allow through the firewall | ["0.0.0.0/0"] | - -> Note: All of the above variables can be exported via environment variables (`TF_=`) or set -> in a `.tfvars` file. - -Set up your Terraform variables and deploy your instance: - -```bash -export TF_VAR_project_id= -export TF_ssh_public_key="./chroma_id_rsa.pub" -export TF_ssh_private_key="./chroma_id_rsa" -export TF_VAR_chroma_release="0.4.20" -export TF_VAR_zone="us-central1-a" -export TF_VAR_public_access="true" -export TF_VAR_enable_auth="true" -export TF_VAR_auth_type="token" -terraform apply -auto-approve -``` - -Verify that your instance is up and running: - -```bash -export instance_public_ip=$(terraform output instance_public_ip | sed 's/"//g') -curl -v http://$instance_public_ip:8000/api/v1/heartbeat -``` - -> Note: Depending on your OS the `sed` command might not work. In that case, you can manually copy the public IP from -> the Terraform output. - -To get the auth token generated during the setup: - -```bash -terraform output chroma_auth_token -``` - -For more details check -our [Terraform GCP deployment blueprint](https://github.com/chroma-core/chroma/tree/main/examples/deployments/google-cloud-compute) - -### Digital Ocean - -Digital Ocean Configuration parameters: - -| Parameter Name | Description | Default Value | -|-------------------------|---------------------------------------------------------------------------------------------------------------------|-----------------------| -| instance_image | The image to use for the instance | ubuntu-22-04-x64 | -| chroma_release | The chroma release to deploy | 0.4.20 | -| region | DO Region | nyc2 | -| instance_type | Droplet size | s-2vcpu-4gb | -| public_access | Enable public ingress on port 8000 | true | -| enable_auth | Enable authentication | true | -| auth_type | Authentication type | token | -| ssh_public_key | SSH Public Key | ./chroma-do.pub | -| ssh_private_key | SSH Private Key | ./chroma-do | -| chroma_data_volume_size | EBS Volume Size of the attached data volume where your chroma data is stored | 20 | -| chroma_port | The port that chroma listens on | 8000 | -| source_ranges | List of CIDR ranges to allow through the firewall | ["0.0.0.0/0", "::/0"] | -| mgmt_source_ranges | List of CIDR ranges to allow for management of the Chroma instance. This is used for SSH incoming traffic filtering | ["0.0.0.0/0", "::/0"] | - -> Note: All of the above variables can be exported via environment variables (`TF_=`) or set -> in a `.tfvars` file. - -Set up your Terraform variables and deploy your instance: - -```bash -export TF_VAR_do_token= -export TF_ssh_public_key="./chroma_id_rsa.pub" -export TF_ssh_private_key="./chroma_id_rsa" -export TF_VAR_chroma_release="0.4.20" -export TF_VAR_region="ams2" -export TF_VAR_public_access="true" -export TF_VAR_enable_auth="true" -export TF_VAR_auth_type="token" -terraform apply -auto-approve -``` - -Verify that your instance is up and running: - -```bash -export instance_public_ip=$(terraform output instance_public_ip | sed 's/"//g') -curl -v http://$instance_public_ip:8000/api/v1/heartbeat -``` - -> Note: Depending on your OS the `sed` command might not work. In that case, you can manually copy the public IP from -> the Terraform output. - -To get the auth token generated during the setup: - -```bash -terraform output chroma_auth_token -``` - -For more details check -our [Terraform Digital Ocean deployment blueprint](https://github.com/chroma-core/chroma/tree/main/examples/deployments/do-terraform) - -### Need help or have questions? - -Reach out to us on [Discord](https://discord.gg/MMeYNTmh3x) \ No newline at end of file diff --git a/docs/deployment/aws.md b/docs/deployment/aws.md new file mode 100644 index 00000000..468cfeb3 --- /dev/null +++ b/docs/deployment/aws.md @@ -0,0 +1,67 @@ +--- +slug: /deployment/aws +title: Amazon Web Services +--- + +# Amazon Web Services (AWS) + + +AWS Configuration variables + +| Parameter Name | Description | Default Value | +|--------------------------------------------|---------------------------------------------------------------------------------------------------------------------|---------------------| +| chroma_release | The chroma release to deploy | 0.4.20 | +| region | AWS Region | us-west-1 | +| instance_type | AWS EC2 Instance Type | t3.medium | +| public_access | Enable public ingress on port 8000 | true | +| enable_auth | Enable authentication | true | +| auth_type | Authentication type | token | +| import_keypair | Whether to import the keypair in AWS. Set this to `false` if your keypair is already available in AWS. | true | +| keypair_name | The name of the keypair to import | chroma_keypair | +| ssh_public_key | SSH Public Key | ./chroma_id_rsa.pub | +| ssh_private_key | SSH Private Key | ./chroma_id_rsa | +| chroma_instance_volume_size | The size of the instance volume - the root volume | 30 | +| chroma_data_volume_size | EBS Volume Size of the attached data volume where your chroma data is stored | 20 | +| chroma_data_volume_snapshot_before_destroy | Take a snapshot of the chroma data volume before destroying it | false | +| chroma_data_restore_from_snapshot_id | Restore the chroma data volume from a snapshot | null | +| chroma_port | The port that chroma listens on | 8000 | +| source_ranges | List of CIDR ranges to allow through the firewall | ["0.0.0.0/0"] | +| mgmt_source_ranges | List of CIDR ranges to allow for management of the Chroma instance. This is used for SSH incoming traffic filtering | ["0.0.0.0/0"] | + +> Note: All of the above variables can be exported via environment variables (`TF_=`) or set +> in a `.tfvars` file. + +Set up your Terraform variables and deploy your instance: + +```bash +export TF_VAR_AWS_ACCESS_KEY= +export TF_VAR_AWS_SECRET_ACCESS_KEY= +export TF_ssh_public_key="./chroma_id_rsa.pub" +export TF_ssh_private_key="./chroma_id_rsa" +export TF_VAR_chroma_release=0.4.20 +export TF_VAR_region="us-west-1" +export TF_VAR_public_access="true" +export TF_VAR_enable_auth="true" +export TF_VAR_auth_type="token" +export TF_VAR_chroma_data_volume_snapshot_before_destroy="true" +terraform apply -auto-approve +``` + +Verify that your instance is up and running: + +```bash +export instance_public_ip=$(terraform output instance_public_ip | sed 's/"//g') +curl -v http://$instance_public_ip:8000/api/v1/heartbeat +``` + +> Note: Depending on your OS the `sed` command might not work. In that case, you can manually copy the public IP from +> the Terraform output. + +To get the auth token generated during the setup: + +```bash +terraform output chroma_auth_token +``` + +For more details check +our [Terraform AWS deployment blueprint](https://github.com/chroma-core/chroma/tree/main/examples/deployments/aws-terraform) diff --git a/docs/deployment/digital-ocean.md b/docs/deployment/digital-ocean.md new file mode 100644 index 00000000..8581875a --- /dev/null +++ b/docs/deployment/digital-ocean.md @@ -0,0 +1,61 @@ +--- +slug: /deployment/digital-ocean +title: Digital Ocean +--- + +# Digital Ocean (DO) + + +Digital Ocean Configuration parameters: + +| Parameter Name | Description | Default Value | +|-------------------------|---------------------------------------------------------------------------------------------------------------------|-----------------------| +| instance_image | The image to use for the instance | ubuntu-22-04-x64 | +| chroma_release | The chroma release to deploy | 0.4.20 | +| region | DO Region | nyc2 | +| instance_type | Droplet size | s-2vcpu-4gb | +| public_access | Enable public ingress on port 8000 | true | +| enable_auth | Enable authentication | true | +| auth_type | Authentication type | token | +| ssh_public_key | SSH Public Key | ./chroma-do.pub | +| ssh_private_key | SSH Private Key | ./chroma-do | +| chroma_data_volume_size | EBS Volume Size of the attached data volume where your chroma data is stored | 20 | +| chroma_port | The port that chroma listens on | 8000 | +| source_ranges | List of CIDR ranges to allow through the firewall | ["0.0.0.0/0", "::/0"] | +| mgmt_source_ranges | List of CIDR ranges to allow for management of the Chroma instance. This is used for SSH incoming traffic filtering | ["0.0.0.0/0", "::/0"] | + +> Note: All of the above variables can be exported via environment variables (`TF_=`) or set +> in a `.tfvars` file. + +Set up your Terraform variables and deploy your instance: + +```bash +export TF_VAR_do_token= +export TF_ssh_public_key="./chroma_id_rsa.pub" +export TF_ssh_private_key="./chroma_id_rsa" +export TF_VAR_chroma_release="0.4.20" +export TF_VAR_region="ams2" +export TF_VAR_public_access="true" +export TF_VAR_enable_auth="true" +export TF_VAR_auth_type="token" +terraform apply -auto-approve +``` + +Verify that your instance is up and running: + +```bash +export instance_public_ip=$(terraform output instance_public_ip | sed 's/"//g') +curl -v http://$instance_public_ip:8000/api/v1/heartbeat +``` + +> Note: Depending on your OS the `sed` command might not work. In that case, you can manually copy the public IP from +> the Terraform output. + +To get the auth token generated during the setup: + +```bash +terraform output chroma_auth_token +``` + +For more details check +our [Terraform Digital Ocean deployment blueprint](https://github.com/chroma-core/chroma/tree/main/examples/deployments/do-terraform) \ No newline at end of file diff --git a/docs/deployment/gcp.md b/docs/deployment/gcp.md new file mode 100644 index 00000000..72cc7dad --- /dev/null +++ b/docs/deployment/gcp.md @@ -0,0 +1,73 @@ +--- +slug: /deployment/gcp +title: Google Cloud Platform +--- + +# Google Cloud Platform (GCP) + + +You will need to install [gcloud CLI](https://cloud.google.com/sdk/gcloud) and authenticate with your GCP account: + +```bash +gcloud auth application-default login +``` + +GCP Configuration parameters: + +| Parameter Name | Description | Default Value | +|-----------------------------------|----------------------------------------------------------------------------------------|------------------------| +| project_id | The project id to deploy to | N/A | +| chroma_release | The chroma release to deploy | 0.4.20 | +| zone | N/A | us-central1-a | +| image | The image to use for the instance | debian-cloud/debian-11 | +| vm_user | The user to use for connecting to the instance. This is usually the default image user | debian | +| machine_type | N/A | e2-small | +| public_access | Enable public ingress on port 8000 | true | +| enable_auth | Enable authentication | true | +| auth_type | Authentication type | token | +| ssh_public_key | SSH Public Key | ./chroma_id_rsa.pub | +| ssh_private_key | SSH Private Key | ./chroma_id_rsa | +| chroma_instance_volume_size | The size of the instance volume - the root volume | 30 | +| chroma_data_volume_size | Volume Size of the attached data volume where your chroma data is stored | 20 | +| chroma_data_volume_device_name | The device name of the chroma data volume | chroma-disk-0 | +| prevent_chroma_data_volume_delete | Prevent the chroma data volume from being deleted when the instance is terminated | false | +| disk_type | The type of disk to use for the instance. Can be either pd-standard or pd-ssd | pd-ssd | +| labels | Labels to apply to all resources in this example | {environment: 'dev'} | +| chroma_port | The port that chroma listens on | 8000 | +| source_ranges | List of CIDR ranges to allow through the firewall | ["0.0.0.0/0"] | + +> Note: All of the above variables can be exported via environment variables (`TF_=`) or set +> in a `.tfvars` file. + +Set up your Terraform variables and deploy your instance: + +```bash +export TF_VAR_project_id= +export TF_ssh_public_key="./chroma_id_rsa.pub" +export TF_ssh_private_key="./chroma_id_rsa" +export TF_VAR_chroma_release="0.4.20" +export TF_VAR_zone="us-central1-a" +export TF_VAR_public_access="true" +export TF_VAR_enable_auth="true" +export TF_VAR_auth_type="token" +terraform apply -auto-approve +``` + +Verify that your instance is up and running: + +```bash +export instance_public_ip=$(terraform output instance_public_ip | sed 's/"//g') +curl -v http://$instance_public_ip:8000/api/v1/heartbeat +``` + +> Note: Depending on your OS the `sed` command might not work. In that case, you can manually copy the public IP from +> the Terraform output. + +To get the auth token generated during the setup: + +```bash +terraform output chroma_auth_token +``` + +For more details check +our [Terraform GCP deployment blueprint](https://github.com/chroma-core/chroma/tree/main/examples/deployments/google-cloud-compute) \ No newline at end of file diff --git a/docs/deployment/index.md b/docs/deployment/index.md new file mode 100644 index 00000000..9172978b --- /dev/null +++ b/docs/deployment/index.md @@ -0,0 +1,104 @@ +--- +slug: /deployment +title: "☁️ Deployment" +hide_title: true +--- + +:::caution Alpha Status +Chroma Server is currently in Alpha. We are working hard to move Chroma from an in-memory single-process oriented library to a distributed production-grade DB! + +- [x] Alpha <- Currently +- [ ] Technical Preview - ~1 month away, powered by a completely new backend +- [ ] Full production +- [ ] GA - General Availability + +::: + +# ☁️ Deployment + +You can also deploy Chroma on a long-running server, and connect to it +remotely. + +There are many possible configurations, but for convenience we have +provided a very simple AWS CloudFormation template to experiment with +deploying Chroma to EC2 on AWS. + +## Hosted Chroma + +We want to offer hosted Chroma, and we need your help. + +Fill out the survey to jump the wait-list. Coming Q3 2023. + +[📝 30 second survey](https://airtable.com/shrOAiDUtS2ILy5vZ) + +## Docker + +You can run a Chroma server in a Docker container. + +You can get the Chroma Docker image from [Docker Hub](https://hub.docker.com/r/chromadb/chroma), or from +the [Chroma GitHub Container Registry](https://github.com/chroma-core/chroma/pkgs/container/chroma) + +```sh +docker pull chromadb/chroma +docker run -p 8000:8000 chromadb/chroma +``` + +You can also build the Docker image yourself from the Dockerfile in +the [Chroma GitHub repository](https://github.com/chroma-core/chroma) + +```sh +git clone git@github.com:chroma-core/chroma.git +cd chroma +docker-compose up -d --build +``` + +The Chroma client can then be configured to connect to the server running in the Docker container. + +```python +import chromadb + +chroma_client = chromadb.HttpClient(host='localhost', port=8000) +``` + +### Authentication with Docker + +By default, the Docker image will run with no authentication. Follow the [Authentication](./usage-guide#authentication) +section of the Usage Guide to configure authentication in the Docker container. + +You can also create a `.chroma_env` file setting the required environment variables and pass it to the Docker container +with the `--env-file` flag when running the container. + +```sh +docker run --env-file ./.chroma_env -p 8000:8000 chromadb/chroma +``` + +## Cloud Deployments + + +### Prerequisites + +- [Terraform CLI v1.3.4+](https://developer.hashicorp.com/terraform/tutorials/gcp-get-started/install-cli) + +Generate a key: + +```bash +ssh-keygen -t RSA -b 4096 -C "Chroma SSH Keypair" -N "" -f ./chroma_id_rsa && \ +chmod 400 ./chroma_id_rsa +``` + +The key pair will allow you to connect to your Chroma instance via SSH (applicable only for AWS, GCP and DO +deployments). In addition, the keypair is used for post provisioning steps such as formatting the Chroma data volume ( +applicable only for AWS deployments). + +### Providers + +Below are instructions for deploying Chroma with Terraform on the following public cloud providers: + +- [AWS](/deployment/aws) +- [Google Cloud Platform](/deployment/gcp) +- [Digital Ocean](/deployment/digital-ocean) + + +### Need help or have questions? + +Reach out to us on [Discord](https://discord.gg/MMeYNTmh3x) \ No newline at end of file diff --git a/sidebars.js b/sidebars.js index 62c6fd88..b5fbe556 100644 --- a/sidebars.js +++ b/sidebars.js @@ -36,7 +36,12 @@ const sidebars = { label: '🔌 Integrations', className: 'category-link', }, - 'deployment', + { + type: 'link', + href: '/deployment', + label: '☁️ Deployment', + className: 'category-link', + }, 'migration', 'observability', 'telemetry', @@ -72,6 +77,25 @@ const sidebars = { ], }, ], + deployment: [ + { + type: 'link', + label: '← Home', + href: '/' + }, + 'deployment/index', + { + type: 'category', + label: 'Deployments', + collapsed: false, + className: 'category-header', + items: [ + 'deployment/aws', + 'deployment/gcp', + 'deployment/digital-ocean', + ], + }, + ], embeddings: [ { type: 'link',