diff --git a/TOC-tidb-cloud.md b/TOC-tidb-cloud.md index 719bb82134b27..3d78464e59b5c 100644 --- a/TOC-tidb-cloud.md +++ b/TOC-tidb-cloud.md @@ -252,6 +252,9 @@ - TiDB Node Group - [Overview of TiDB Node Group](/tidb-cloud/tidb-node-group-overview.md) - [Manage TiDB Node Groups](/tidb-cloud/tidb-node-group-management.md) + - Manage Connections by TiProxy + - [Overview of TiProxy](/tidb-cloud/tiproxy-overview-for-cloud.md) + - [Manage TiProxy](/tidb-cloud/tiproxy-management.md) - [Upgrade a TiDB Cluster](/tidb-cloud/upgrade-tidb-cluster.md) - [Delete a TiDB Cluster](/tidb-cloud/delete-tidb-cluster.md) - Migrate or Import Data diff --git a/media/tidb-cloud/tiproxy-billing.png b/media/tidb-cloud/tiproxy-billing.png new file mode 100644 index 0000000000000..e5d74bc76f463 Binary files /dev/null and b/media/tidb-cloud/tiproxy-billing.png differ diff --git a/media/tidb-cloud/tiproxy-disable-tiproxy.png b/media/tidb-cloud/tiproxy-disable-tiproxy.png new file mode 100644 index 0000000000000..b901578259f4c Binary files /dev/null and b/media/tidb-cloud/tiproxy-disable-tiproxy.png differ diff --git a/media/tidb-cloud/tiproxy-enable-tiproxy.png b/media/tidb-cloud/tiproxy-enable-tiproxy.png new file mode 100644 index 0000000000000..662de1527b506 Binary files /dev/null and b/media/tidb-cloud/tiproxy-enable-tiproxy.png differ diff --git a/media/tidb-cloud/tiproxy-topology.png b/media/tidb-cloud/tiproxy-topology.png new file mode 100644 index 0000000000000..4a014e603b339 Binary files /dev/null and b/media/tidb-cloud/tiproxy-topology.png differ diff --git a/tidb-cloud/built-in-monitoring.md b/tidb-cloud/built-in-monitoring.md index d7dedb2887189..91b6b2b86799f 100644 --- a/tidb-cloud/built-in-monitoring.md +++ b/tidb-cloud/built-in-monitoring.md @@ -78,6 +78,10 @@ The following sections illustrate the metrics on the **Metrics** page for TiDB C | TiFlash Memory Usage | node, limit | The memory usage statistics or upper limit of each TiFlash node. | | TiFlash IO MBps | node-write, node-read | The total bytes of read and write in each TiFlash node. | | TiFlash Storage Usage | node, limit | The storage usage statistics or upper limit of each TiFlash node. | +| TiProxy CPU Usage | node | The CPU usage statistics of each TiProxy node. The upper limit is 100%. | +| TiProxy Connections | node | The connection number on each TiProxy node. | +| TiProxy Throughput | node | The transferred bytes per second on each TiProxy node. | +| TiProxy Sessions Migration Reasons | reason | The number of session migrations that happen every minute and the reason for them. | ## Metrics for TiDB Cloud Serverless clusters diff --git a/tidb-cloud/limitations-and-quotas.md b/tidb-cloud/limitations-and-quotas.md index 04b52ccaff92b..65ed630dc61bb 100644 --- a/tidb-cloud/limitations-and-quotas.md +++ b/tidb-cloud/limitations-and-quotas.md @@ -29,6 +29,7 @@ TiDB Cloud limits how many of each kind of component you can create in a [TiDB C | Maximum number of total TiDB nodes for all clusters in your organization | 10 | | Maximum number of total TiKV nodes for all clusters in your organization | 15 | | Maximum number of total TiFlash nodes for all clusters in your organization | 5 | +| Maximum number of total TiProxy nodes for all clusters in your organization | 10 | > **Note:** > diff --git a/tidb-cloud/monitor-tidb-cluster.md b/tidb-cloud/monitor-tidb-cluster.md index aa941f6165acf..0dc5499ab4a82 100644 --- a/tidb-cloud/monitor-tidb-cluster.md +++ b/tidb-cloud/monitor-tidb-cluster.md @@ -32,6 +32,8 @@ You can see the current status of each running cluster on the cluster page. > > The TiDB node status is only available for TiDB Cloud Dedicated clusters. +The node names starting with `tidb` are TiDB nodes, and those starting with `tiproxy` are TiProxy nodes. + | TiDB node status | Description | |:--|:--| | **Available** | The TiDB node is healthy and available. | diff --git a/tidb-cloud/tidb-cloud-clinic.md b/tidb-cloud/tidb-cloud-clinic.md index f1f3511bf36e5..12885991e8381 100644 --- a/tidb-cloud/tidb-cloud-clinic.md +++ b/tidb-cloud/tidb-cloud-clinic.md @@ -56,6 +56,7 @@ The dashboards and their content are subject to change. Currently, the following - TiDB-Resource-Control - TiFlash-Summary - TiKV-Details +- TiProxy-Summary - User-Node-Info ## Analyze top slow queries diff --git a/tidb-cloud/tiproxy-management.md b/tidb-cloud/tiproxy-management.md new file mode 100644 index 0000000000000..ba6feec2ba106 --- /dev/null +++ b/tidb-cloud/tiproxy-management.md @@ -0,0 +1,124 @@ +--- +title: Manage TiProxy +summary: Learn about how to enable, disable, view and modify TiProxy.. +--- + +# Manage TiProxy + +This document describes how to enable, disable, view and modify TiProxy. + +## Enable TiProxy + +You can enable TiProxy for either a new cluster or an existing cluster in any TiDB node group. + +### Decide size and number of TiProxy instances + +The size and number of TiProxy instances depend on both the QPS and network bandwidth of your cluster. Network bandwidth is the sum of the client request and TiDB response bandwidth. + +The following tables shows the maximum QPS and network bandwidth of each TiProxy size. + +| Size | Maximum QPS | Maximum network bandwidth | +| :--| :--| +| Small | 30K | 93 MiB/s | +| Large | 120K | 312 MiB/s | + +The optional TiProxy sizes are `small` and `large`. The optional TiProxy instance numbers are 2, 3, 6, 9, 12, 15, 18, 21, and 24. The default two small-sized TiProxy instances can provide 60K QPS and 186 MiB/s network bandwidth. It is recommended that you reserve 20% of the QPS to prevent high latency. + +For example, if your cluster's maximum QPS is 100K and the maximum network bandwidth is 100 MiB/s, the size and number of TiProxy instances mainly depend on the QPS. In this case, you can select 6 small-sized TiProxy instances. + +### Enable TiProxy for a new cluster + +To enable TiProxy when creating a new cluster, click the TiProxy toggle and choose the TiProxy size and number. + +![Enable TiProxy](/media/tidb-cloud/tiproxy-enable-tiproxy.png) + +### Enable TiProxy for an existing cluster + +To enable TiProxy for an existing cluster, perform the following steps: + +1. In the [TiDB Cloud console](https://tidbcloud.com/), navigate to the [**Clusters**](https://tidbcloud.com/project/clusters) page of your project, and then click the name of your target cluster to go to its overview page. +2. Click **...** in the upper-right corner, and click **Modify** in the drop-down menu. The **Modify Cluster** page is displayed. +3. On the **Modify Cluster** page, click the TiProxy toggle and choose the TiProxy size and number. + +![Enable TiProxy](/media/tidb-cloud/tiproxy-enable-tiproxy.png) + +> **Note:** +> +> Enabling TiProxy will cause a rolling restart of TiDB in this TiDB node group, which causes connections to disconnect. In addition, creating new connections might hang for up to 30 seconds. Make sure that you enable TiProxy in the maintenance window. + +### Limitations and quotas + +- There must be at least two TiDB instance number in a TiDB node group. +- The TiDB instance size must be at least 4 vCPUs. +- The default maximum TiProxy instance number of an organization is 10. For more information, see [Limitations and Quotas](/tidb-cloud/limitations-and-quotas.md). +- The TiDB cluster version must be v6.5.0 or later. + +## Disable TiProxy + +To disable TiProxy, perform the following steps: + +1. In the [TiDB Cloud console](https://tidbcloud.com/), navigate to the [**Clusters**](https://tidbcloud.com/project/clusters) page of your project, and then click the name of your target cluster to go to its overview page. +2. Click **...** in the upper-right corner, and click **Modify** in the drop-down menu. The **Modify Cluster** page is displayed. +3. On the **Modify Cluster** page, click the TiProxy toggle to disable TiProxy. + +![Disable TiProxy](/media/tidb-cloud/tiproxy-disable-tiproxy.png) + +> **Note:** +> +> Disabling TiProxy will cause connections to disconnect. In addition, creating new connections might hang for up to 10 seconds. Make sure that you disable TiProxy in the maintenance window. + +## View TiProxy + +### View TiProxy topology + +To view the TiProxy topology, perform the following steps: + +1. In the [TiDB Cloud console](https://tidbcloud.com/), navigate to the [**Clusters**](https://tidbcloud.com/project/clusters) page of your project, and then click the name of your target cluster to go to its overview page. +2. In the left navigation pane, click **Monitoring > Nodes**. The **Node Map** page is displayed. +3. In the **Node Map** page, TiProxy topology is displayed in the **TiDB** panel. + +![TiProxy Topology](/media/tidb-cloud/tiproxy-topology.png) + +### View TiProxy metrics + +To view TiProxy metrics, perform the following steps: + +1. In the [TiDB Cloud console](https://tidbcloud.com/), navigate to the [**Clusters**](https://tidbcloud.com/project/clusters) page of your project, and then click the name of your target cluster to go to its overview page. +2. In the left navigation pane, click **Monitoring > Metrics**. The **Metrics** page is displayed. +3. On the **Metrics** page, click **Server** tab and scroll down to TiProxy-related metrics. To view TiProxy metrics in one TiDB node group, click **TiDB Node Group View** tab, select your TiDB node group, and scroll down to TiProxy-related metrics. + +The metrics include: + +- **TiProxy CPU Usage**: the CPU usage statistics of each TiProxy node. The upper limit is 100%. If the maximum CPU usage exceeds 80%, it is recommended that you scale out TiProxy. +- **TiProxy Connections**: the connection number on each TiProxy node. +- **TiProxy Throughput**: the transferred bytes per second on each TiProxy node. If the maximum throughput reaches the maximum network bandwidth, it is recommended that you scale out TiProxy. To learn about the maximum network bandwidth, see [Decide TiProxy instance size and number](#decide-tiproxy-instance-size-and-number). +- **TiProxy Sessions Migration Reasons**: the number of session migrations that happen every minute and the reason for them. For example, when TiDB scales in and TiProxy migrates sessions to other TiDB instances, the reason is `status`. For more migration reasons, see [TiProxy Monitoring Metrics](https://docs.pingcap.com/tidb/stable/tiproxy-grafana#balance). + +### View TiProxy bills + +To view TiProxy bills, perform the following steps: + +1. In the [TiDB Cloud console](https://tidbcloud.com), switch to your target organization using the combo box in the upper-left corner. +2. In the left navigation pane, click **Billing**. On the **Billing** page, the **Bills** tab is displayed by default. +3. In the **Summary by Service** section, TiProxy node cost is displayed under **TiDB Dedicated**, while TiProxy data transfer cost is included in **Data Transfer > Same Region**. + +![TiProxy Billing](/media/tidb-cloud/tiproxy-billing.png) + +## Modify TiProxy + +To scale in or scale out TiProxy, perform the following steps: + +1. In the [TiDB Cloud console](https://tidbcloud.com/), navigate to the [**Clusters**](https://tidbcloud.com/project/clusters) page of your project, and then click the name of your target cluster to go to its overview page. +2. Click **...** in the upper-right corner, and click **Modify** in the drop-down menu. The **Modify Cluster** page is displayed. +3. On the **Modify Cluster** page, modify the TiProxy number. + +![Modify TiProxy](/media/tidb-cloud/tiproxy-enable-tiproxy.png) + +> **Note** +> +> - Modifying the TiProxy size directly is not supported. It is recommended that you modify the number of TiProxy instances. If you have to modify the TiProxy size, you need to disable TiProxy in all the TiDB node groups and then enable it again to choose a different size. +> - Scaling in TiProxy will cause connections to disconnect. Make sure that you scale in TiProxy in the maintenance window. + +## Manage TiProxy in multiple TiDB node groups + +When you have multiple TiDB node groups, each TiDB node group has its dedicated TiProxy group. TiProxy routes traffic to the TiDB instances in the same TiDB node group to isolate computing resources. You can enable, disable, or modify TiProxy in each TiDB node group. However, the TiProxy size in all the TiDB node groups must be the same. diff --git a/tidb-cloud/tiproxy-overview-for-cloud.md b/tidb-cloud/tiproxy-overview-for-cloud.md new file mode 100644 index 0000000000000..9382dc8dbc610 --- /dev/null +++ b/tidb-cloud/tiproxy-overview-for-cloud.md @@ -0,0 +1,43 @@ +--- +title: Overview of TiProxy for TiDB Cloud +summary: Learn about the usage scenarios of TiProxy for TiDB Cloud. +--- + +# Overview of TiProxy for TiDB Cloud + +TiProxy is the official proxy component of PingCAP. It is placed between the client and the TiDB server to provide load balancing, connection persistence, and other features for TiDB. + +For more information, see [TiProxy Overview](https://docs.pingcap.com/tidb/stable/tiproxy-overview). + +## Scenarios + +TiProxy is suitable for the following scenarios: + +- Connection persistence: When a TiDB server performs scaling in, rolling upgrade, or rolling restart, the client connection breaks, resulting in an error. If the client does not have an idempotent error retry mechanism, you need to manually check and fix the error, which greatly increases the operational overhead. TiProxy can keep the client connection, so that the client does not report an error. +- Frequent scaling in and scaling out: The workload of an application might change periodically. To save costs, you can deploy TiDB on the cloud and automatically scale in and scale out TiDB servers according to the workload. However, scaling in might cause the client to disconnect, and scaling out might result in an unbalanced load. TiProxy can keep the client connection and achieve load balancing. +- CPU load imbalance: When background tasks consume a significant amount of CPU resources, or workloads across connections vary significantly, leading to an imbalanced CPU load, TiProxy can migrate connections based on CPU usage to achieve load balancing. For more details, see [CPU-based load balancing](https://docs.pingcap.com/tidb/stable/tiproxy-load-balance#cpu-based-load-balancing). + +For more scenarios, see [TiProxy User Scenarios](https://docs.pingcap.com/tidb/stable/tiproxy-overview#user-scenarios). + +## Limitations + +TiProxy cannot preserve client connections in the following scenarios: + +- Upgrading AWS EKS, Azure AKS, Google Cloud GKE, or Alibaba Cloud ACK. +- Disabling, scaling in, upgrading, or restarting TiProxy. +- A single statement or transaction that runs for more than 20 seconds. If your application needs a longer timeout, contact [TiDB Cloud Support](/tidb-cloud/tidb-cloud-support.md). + +For more scenarios, see [TiProxy Limitations](https://docs.pingcap.com/tidb/stable/tiproxy-overview#limitations). + +## Billing + +TiProxy introduces two types of costs: + +- Node costs. For more information, see [Node Cost](https://www.pingcap.com/tidb-dedicated-pricing-details/#node-cost) +- Data transfer costs. For more information, see [Data Transfer Cost](https://www.pingcap.com/tidb-dedicated-pricing-details/#data-transfer-cost). TiProxy will prioritize routing traffic to the TiDB instances in the same availability zone (AZ). However, if the TiDB workloads are uneven, it will also route traffic to other AZs, resulting in additional data transfer costs. + +You can view the TiProxy bill on the billing page. For more information, see [View TiProxy bills](/tidb-cloud/tiproxy-management.md#view-tiproxy-bills). + +## SLA impact + +TiProxy has no impact on SLA.