diff --git a/docs/deployment.md b/docs/deployment.md deleted file mode 100644 index 6ea29758..00000000 --- a/docs/deployment.md +++ /dev/null @@ -1,211 +0,0 @@ ---- -sidebar_position: 9 -title: "☁️ Deployment" ---- - -:::caution Alpha Status -Chroma Server is currently in Alpha. We are working hard to move Chroma from an in-memory single-process oriented library to a distributed production-grade DB! - -- [x] Alpha <- Currently -- [ ] Technical Preview - ~1 month away, powered by a completely new backend -- [ ] Full production -- [ ] GA - General Availability - -::: - -# ☁️ Deployment - -You can also deploy Chroma on a long-running server, and connect to it -remotely. - -There are many possible configurations, but for convenience we have -provided a very simple AWS CloudFormation template to experiment with -deploying Chroma to EC2 on AWS. - -## Hosted Chroma - -We want to offer hosted Chroma, and we need your help. - -Fill out the survey to jump the wait-list. Coming Q3 2023. - -[📝 30 second survey](https://airtable.com/shrOAiDUtS2ILy5vZ) - -## Docker - -You can run a Chroma server in a Docker container. - -You can get the Chroma Docker image from [Docker Hub](https://hub.docker.com/r/chromadb/chroma), or from the [Chroma GitHub Container Registry](https://github.com/chroma-core/chroma/pkgs/container/chroma) - -```sh -docker pull chromadb/chroma -docker run -p 8000:8000 chromadb/chroma -``` - -You can also build the Docker image yourself from the Dockerfile in the [Chroma GitHub repository](https://github.com/chroma-core/chroma) - -```sh -git clone git@github.com:chroma-core/chroma.git -cd chroma -docker-compose up -d --build -``` - -The Chroma client can then be configured to connect to the server running in the Docker container. - -```python -import chromadb -chroma_client = chromadb.HttpClient(host='localhost', port=8000) -``` - -### Authentication with Docker - -By default, the Docker image will run with no authentication. Follow the [Authentication](./usage-guide#authentication) section of the Usage Guide to configure authentication in the Docker container. - -You can also create a `.chroma_env` file setting the required environment variables and pass it to the Docker container with the `--env-file` flag when running the container. - -```sh -docker run --env-file ./.chroma_env -p 8000:8000 chromadb/chroma -``` - -## Simple AWS Deployment - -:warning: Chroma and its underlying database need at least 2gb of RAM, -which means it won't fit on the 1gb instances provided as part of the -AWS Free Tier. This template uses a `t3.small` EC2 instance, which -costs about two cents an hour, or $15 for a full month. If you follow these -instructions, AWS will bill you accordingly. - -:warning: This basic stack doesn't support any kind of authentication; -anyone who knows your server IP will be able to add and query for -embeddings. To secure this endpoint, you'll need to put it behind -[AWS API Gateway](https://aws.amazon.com/api-gateway/) or add your own -authenticating proxy. - -:warning: By default, this template saves all data on a single -volume. When you delete or replace it, the data will disappear. For -serious production use (with high availability, backups, etc.) please -read and understand the CloudFormation template and use it as a basis -for what you need, or reach out to the Chroma team for assistance. - -### Step 1: Get an AWS Account - -You will need an AWS Account. You can use one you already have, or -[create a new one](https://aws.amazon.com). - -### Step 2: Get credentials - -For this example, we will be using the AWS command line -interface. There are -[several ways](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-prereqs.html) -to configure the AWS CLI, but for the purposes of these examples we -will presume that you have -[obtained an AWS access key](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html) -and will be using environment variables to configure AWS. - -Export the `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` environment variables in your shell: - -``` - -export AWS_ACCESS_KEY_ID=**\*\***\*\*\*\***\*\*** -export AWS_SECRET_ACCESS_KEY=****\*\*****\*\*****\*\***** - -``` - -You can also configure AWS to use a region of your choice using the -`AWS_REGION` environment variable: - -``` - -export AWS_REGION=us-east-1 - -``` - -### Step 3: Run CloudFormation - -Chroma publishes Cloudformation templates to S3 for each release. - -To launch the template using AWS CloudFormation, run the following command line invocation: - -``` - -aws cloudformation create-stack --stack-name my-chroma-stack --template-url https://s3.amazonaws.com/public.trychroma.com/cloudformation/latest/chroma.cf.json - -``` - -Replace `--stack-name my-chroma-stack` with a different stack name, if you wish. - -Wait a few minutes for the server to boot up, and Chroma will be -available! You can get the public IP address of your new Chroma server using the AWS console, or using the following command: - -``` - -aws cloudformation describe-stacks --stack-name my-chroma-stack --query 'Stacks[0].Outputs' - -``` - -### Step 4: Customize the Stack (optional) - -The CloudFormation template allows you to pass particular key/value -pairs to override aspects of the stack. Available keys are: - -- `InstanceType` - the AWS instance type to run (default: `t3.small`) -- `KeyName` - the AWS EC2 KeyPair to use, allowing to access the instance via SSH (default: none) - -To set a CloudFormation stack's parameters using the AWS CLI, use the -`--parameters` command line option. Parameters must be specified using -the format `ParameterName={parameter},ParameterValue={value}`. - -For example, the following command launches a new stack similar to the -above, but on a `m5.4xlarge` EC2 instance, and adding a KeyPair named -`mykey` so anyone with the associated private key can SSH into the -machine: - -``` - -aws cloudformation create-stack --stack-name my-chroma-stack --template-url https://s3.amazonaws.com/public.trychroma.com/cloudformation/latest/chroma.cf.json \ - --parameters ParameterKey=KeyName,ParameterValue=mykey \ - ParameterKey=InstanceType,ParameterValue=m5.4xlarge - -``` - -### Step 5: Configure the Chroma Library - -When you launch the Chroma client library to actually use Chroma, all -you need to do is configure it to use the server's IP address and port -`8000`. You can do this in two ways: - -###### Using Environment Variables - -``` - -export CHROMA_API_IMPL=rest -export CHROMA_SERVER_HOST= -export CHROMA_SERVER_HTTP_PORT=8000 - -``` - -###### In Code - -``` - -import chromadb -from chromadb.config import Settings -chroma = chromadb.HttpClient(host=, port=8000) - -``` - -### Step 6: Clean Up (optional). - -To destroy the stack and remove all AWS resources, use the AWS CLI `delete-stack` command. - -``` - -aws cloudformation delete-stack --stack-name my-chroma-stack - -``` - -:warning: This will destroy all the data in your Chroma database, -unless you've taken a snapshot or otherwise backed it up. - -### Troubleshooting - -If you get an error saying `No default VPC for this user` when creating `ChromaInstanceSecurityGroup`, head to [AWS VPC section](https://us-east-1.console.aws.amazon.com/vpc/home?region=us-east-1#vpcs) and create a default VPC for your user. diff --git a/docs/deployment/aws.md b/docs/deployment/aws.md new file mode 100644 index 00000000..468cfeb3 --- /dev/null +++ b/docs/deployment/aws.md @@ -0,0 +1,67 @@ +--- +slug: /deployment/aws +title: Amazon Web Services +--- + +# Amazon Web Services (AWS) + + +AWS Configuration variables + +| Parameter Name | Description | Default Value | +|--------------------------------------------|---------------------------------------------------------------------------------------------------------------------|---------------------| +| chroma_release | The chroma release to deploy | 0.4.20 | +| region | AWS Region | us-west-1 | +| instance_type | AWS EC2 Instance Type | t3.medium | +| public_access | Enable public ingress on port 8000 | true | +| enable_auth | Enable authentication | true | +| auth_type | Authentication type | token | +| import_keypair | Whether to import the keypair in AWS. Set this to `false` if your keypair is already available in AWS. | true | +| keypair_name | The name of the keypair to import | chroma_keypair | +| ssh_public_key | SSH Public Key | ./chroma_id_rsa.pub | +| ssh_private_key | SSH Private Key | ./chroma_id_rsa | +| chroma_instance_volume_size | The size of the instance volume - the root volume | 30 | +| chroma_data_volume_size | EBS Volume Size of the attached data volume where your chroma data is stored | 20 | +| chroma_data_volume_snapshot_before_destroy | Take a snapshot of the chroma data volume before destroying it | false | +| chroma_data_restore_from_snapshot_id | Restore the chroma data volume from a snapshot | null | +| chroma_port | The port that chroma listens on | 8000 | +| source_ranges | List of CIDR ranges to allow through the firewall | ["0.0.0.0/0"] | +| mgmt_source_ranges | List of CIDR ranges to allow for management of the Chroma instance. This is used for SSH incoming traffic filtering | ["0.0.0.0/0"] | + +> Note: All of the above variables can be exported via environment variables (`TF_=`) or set +> in a `.tfvars` file. + +Set up your Terraform variables and deploy your instance: + +```bash +export TF_VAR_AWS_ACCESS_KEY= +export TF_VAR_AWS_SECRET_ACCESS_KEY= +export TF_ssh_public_key="./chroma_id_rsa.pub" +export TF_ssh_private_key="./chroma_id_rsa" +export TF_VAR_chroma_release=0.4.20 +export TF_VAR_region="us-west-1" +export TF_VAR_public_access="true" +export TF_VAR_enable_auth="true" +export TF_VAR_auth_type="token" +export TF_VAR_chroma_data_volume_snapshot_before_destroy="true" +terraform apply -auto-approve +``` + +Verify that your instance is up and running: + +```bash +export instance_public_ip=$(terraform output instance_public_ip | sed 's/"//g') +curl -v http://$instance_public_ip:8000/api/v1/heartbeat +``` + +> Note: Depending on your OS the `sed` command might not work. In that case, you can manually copy the public IP from +> the Terraform output. + +To get the auth token generated during the setup: + +```bash +terraform output chroma_auth_token +``` + +For more details check +our [Terraform AWS deployment blueprint](https://github.com/chroma-core/chroma/tree/main/examples/deployments/aws-terraform) diff --git a/docs/deployment/digital-ocean.md b/docs/deployment/digital-ocean.md new file mode 100644 index 00000000..8581875a --- /dev/null +++ b/docs/deployment/digital-ocean.md @@ -0,0 +1,61 @@ +--- +slug: /deployment/digital-ocean +title: Digital Ocean +--- + +# Digital Ocean (DO) + + +Digital Ocean Configuration parameters: + +| Parameter Name | Description | Default Value | +|-------------------------|---------------------------------------------------------------------------------------------------------------------|-----------------------| +| instance_image | The image to use for the instance | ubuntu-22-04-x64 | +| chroma_release | The chroma release to deploy | 0.4.20 | +| region | DO Region | nyc2 | +| instance_type | Droplet size | s-2vcpu-4gb | +| public_access | Enable public ingress on port 8000 | true | +| enable_auth | Enable authentication | true | +| auth_type | Authentication type | token | +| ssh_public_key | SSH Public Key | ./chroma-do.pub | +| ssh_private_key | SSH Private Key | ./chroma-do | +| chroma_data_volume_size | EBS Volume Size of the attached data volume where your chroma data is stored | 20 | +| chroma_port | The port that chroma listens on | 8000 | +| source_ranges | List of CIDR ranges to allow through the firewall | ["0.0.0.0/0", "::/0"] | +| mgmt_source_ranges | List of CIDR ranges to allow for management of the Chroma instance. This is used for SSH incoming traffic filtering | ["0.0.0.0/0", "::/0"] | + +> Note: All of the above variables can be exported via environment variables (`TF_=`) or set +> in a `.tfvars` file. + +Set up your Terraform variables and deploy your instance: + +```bash +export TF_VAR_do_token= +export TF_ssh_public_key="./chroma_id_rsa.pub" +export TF_ssh_private_key="./chroma_id_rsa" +export TF_VAR_chroma_release="0.4.20" +export TF_VAR_region="ams2" +export TF_VAR_public_access="true" +export TF_VAR_enable_auth="true" +export TF_VAR_auth_type="token" +terraform apply -auto-approve +``` + +Verify that your instance is up and running: + +```bash +export instance_public_ip=$(terraform output instance_public_ip | sed 's/"//g') +curl -v http://$instance_public_ip:8000/api/v1/heartbeat +``` + +> Note: Depending on your OS the `sed` command might not work. In that case, you can manually copy the public IP from +> the Terraform output. + +To get the auth token generated during the setup: + +```bash +terraform output chroma_auth_token +``` + +For more details check +our [Terraform Digital Ocean deployment blueprint](https://github.com/chroma-core/chroma/tree/main/examples/deployments/do-terraform) \ No newline at end of file diff --git a/docs/deployment/gcp.md b/docs/deployment/gcp.md new file mode 100644 index 00000000..72cc7dad --- /dev/null +++ b/docs/deployment/gcp.md @@ -0,0 +1,73 @@ +--- +slug: /deployment/gcp +title: Google Cloud Platform +--- + +# Google Cloud Platform (GCP) + + +You will need to install [gcloud CLI](https://cloud.google.com/sdk/gcloud) and authenticate with your GCP account: + +```bash +gcloud auth application-default login +``` + +GCP Configuration parameters: + +| Parameter Name | Description | Default Value | +|-----------------------------------|----------------------------------------------------------------------------------------|------------------------| +| project_id | The project id to deploy to | N/A | +| chroma_release | The chroma release to deploy | 0.4.20 | +| zone | N/A | us-central1-a | +| image | The image to use for the instance | debian-cloud/debian-11 | +| vm_user | The user to use for connecting to the instance. This is usually the default image user | debian | +| machine_type | N/A | e2-small | +| public_access | Enable public ingress on port 8000 | true | +| enable_auth | Enable authentication | true | +| auth_type | Authentication type | token | +| ssh_public_key | SSH Public Key | ./chroma_id_rsa.pub | +| ssh_private_key | SSH Private Key | ./chroma_id_rsa | +| chroma_instance_volume_size | The size of the instance volume - the root volume | 30 | +| chroma_data_volume_size | Volume Size of the attached data volume where your chroma data is stored | 20 | +| chroma_data_volume_device_name | The device name of the chroma data volume | chroma-disk-0 | +| prevent_chroma_data_volume_delete | Prevent the chroma data volume from being deleted when the instance is terminated | false | +| disk_type | The type of disk to use for the instance. Can be either pd-standard or pd-ssd | pd-ssd | +| labels | Labels to apply to all resources in this example | {environment: 'dev'} | +| chroma_port | The port that chroma listens on | 8000 | +| source_ranges | List of CIDR ranges to allow through the firewall | ["0.0.0.0/0"] | + +> Note: All of the above variables can be exported via environment variables (`TF_=`) or set +> in a `.tfvars` file. + +Set up your Terraform variables and deploy your instance: + +```bash +export TF_VAR_project_id= +export TF_ssh_public_key="./chroma_id_rsa.pub" +export TF_ssh_private_key="./chroma_id_rsa" +export TF_VAR_chroma_release="0.4.20" +export TF_VAR_zone="us-central1-a" +export TF_VAR_public_access="true" +export TF_VAR_enable_auth="true" +export TF_VAR_auth_type="token" +terraform apply -auto-approve +``` + +Verify that your instance is up and running: + +```bash +export instance_public_ip=$(terraform output instance_public_ip | sed 's/"//g') +curl -v http://$instance_public_ip:8000/api/v1/heartbeat +``` + +> Note: Depending on your OS the `sed` command might not work. In that case, you can manually copy the public IP from +> the Terraform output. + +To get the auth token generated during the setup: + +```bash +terraform output chroma_auth_token +``` + +For more details check +our [Terraform GCP deployment blueprint](https://github.com/chroma-core/chroma/tree/main/examples/deployments/google-cloud-compute) \ No newline at end of file diff --git a/docs/deployment/index.md b/docs/deployment/index.md new file mode 100644 index 00000000..9172978b --- /dev/null +++ b/docs/deployment/index.md @@ -0,0 +1,104 @@ +--- +slug: /deployment +title: "☁️ Deployment" +hide_title: true +--- + +:::caution Alpha Status +Chroma Server is currently in Alpha. We are working hard to move Chroma from an in-memory single-process oriented library to a distributed production-grade DB! + +- [x] Alpha <- Currently +- [ ] Technical Preview - ~1 month away, powered by a completely new backend +- [ ] Full production +- [ ] GA - General Availability + +::: + +# ☁️ Deployment + +You can also deploy Chroma on a long-running server, and connect to it +remotely. + +There are many possible configurations, but for convenience we have +provided a very simple AWS CloudFormation template to experiment with +deploying Chroma to EC2 on AWS. + +## Hosted Chroma + +We want to offer hosted Chroma, and we need your help. + +Fill out the survey to jump the wait-list. Coming Q3 2023. + +[📝 30 second survey](https://airtable.com/shrOAiDUtS2ILy5vZ) + +## Docker + +You can run a Chroma server in a Docker container. + +You can get the Chroma Docker image from [Docker Hub](https://hub.docker.com/r/chromadb/chroma), or from +the [Chroma GitHub Container Registry](https://github.com/chroma-core/chroma/pkgs/container/chroma) + +```sh +docker pull chromadb/chroma +docker run -p 8000:8000 chromadb/chroma +``` + +You can also build the Docker image yourself from the Dockerfile in +the [Chroma GitHub repository](https://github.com/chroma-core/chroma) + +```sh +git clone git@github.com:chroma-core/chroma.git +cd chroma +docker-compose up -d --build +``` + +The Chroma client can then be configured to connect to the server running in the Docker container. + +```python +import chromadb + +chroma_client = chromadb.HttpClient(host='localhost', port=8000) +``` + +### Authentication with Docker + +By default, the Docker image will run with no authentication. Follow the [Authentication](./usage-guide#authentication) +section of the Usage Guide to configure authentication in the Docker container. + +You can also create a `.chroma_env` file setting the required environment variables and pass it to the Docker container +with the `--env-file` flag when running the container. + +```sh +docker run --env-file ./.chroma_env -p 8000:8000 chromadb/chroma +``` + +## Cloud Deployments + + +### Prerequisites + +- [Terraform CLI v1.3.4+](https://developer.hashicorp.com/terraform/tutorials/gcp-get-started/install-cli) + +Generate a key: + +```bash +ssh-keygen -t RSA -b 4096 -C "Chroma SSH Keypair" -N "" -f ./chroma_id_rsa && \ +chmod 400 ./chroma_id_rsa +``` + +The key pair will allow you to connect to your Chroma instance via SSH (applicable only for AWS, GCP and DO +deployments). In addition, the keypair is used for post provisioning steps such as formatting the Chroma data volume ( +applicable only for AWS deployments). + +### Providers + +Below are instructions for deploying Chroma with Terraform on the following public cloud providers: + +- [AWS](/deployment/aws) +- [Google Cloud Platform](/deployment/gcp) +- [Digital Ocean](/deployment/digital-ocean) + + +### Need help or have questions? + +Reach out to us on [Discord](https://discord.gg/MMeYNTmh3x) \ No newline at end of file diff --git a/sidebars.js b/sidebars.js index 62c6fd88..b5fbe556 100644 --- a/sidebars.js +++ b/sidebars.js @@ -36,7 +36,12 @@ const sidebars = { label: '🔌 Integrations', className: 'category-link', }, - 'deployment', + { + type: 'link', + href: '/deployment', + label: '☁️ Deployment', + className: 'category-link', + }, 'migration', 'observability', 'telemetry', @@ -72,6 +77,25 @@ const sidebars = { ], }, ], + deployment: [ + { + type: 'link', + label: '← Home', + href: '/' + }, + 'deployment/index', + { + type: 'category', + label: 'Deployments', + collapsed: false, + className: 'category-header', + items: [ + 'deployment/aws', + 'deployment/gcp', + 'deployment/digital-ocean', + ], + }, + ], embeddings: [ { type: 'link',