Skip to content

tiproxy: add tiproxy usage in the tidb cloud doc #21483

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 11 commits into
base: release-8.5
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions TOC-tidb-cloud.md
Original file line number Diff line number Diff line change
Expand Up @@ -252,6 +252,9 @@
- TiDB Node Group
- [Overview of TiDB Node Group](/tidb-cloud/tidb-node-group-overview.md)
- [Manage TiDB Node Groups](/tidb-cloud/tidb-node-group-management.md)
- Manage Connections by TiProxy
- [Overview of TiProxy](/tidb-cloud/tiproxy-overview-for-cloud.md)
- [Manage TiProxy](/tidb-cloud/tiproxy-management.md)
- [Upgrade a TiDB Cluster](/tidb-cloud/upgrade-tidb-cluster.md)
- [Delete a TiDB Cluster](/tidb-cloud/delete-tidb-cluster.md)
- Migrate or Import Data
Expand Down
Binary file added media/tidb-cloud/tiproxy-billing.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added media/tidb-cloud/tiproxy-disable-tiproxy.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added media/tidb-cloud/tiproxy-enable-tiproxy.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added media/tidb-cloud/tiproxy-topology.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 4 additions & 0 deletions tidb-cloud/built-in-monitoring.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,10 @@ The following sections illustrate the metrics on the **Metrics** page for TiDB C
| TiFlash Memory Usage | node, limit | The memory usage statistics or upper limit of each TiFlash node. |
| TiFlash IO MBps | node-write, node-read | The total bytes of read and write in each TiFlash node. |
| TiFlash Storage Usage | node, limit | The storage usage statistics or upper limit of each TiFlash node. |
| TiProxy CPU Usage | node | The CPU usage statistics of each TiProxy node. The upper limit is 100%. |
| TiProxy Connections | node | The connection number on each TiProxy node. |
| TiProxy Throughput | node | The transferred bytes per second on each TiProxy node. |
| TiProxy Sessions Migration Reasons | reason | The number of session migrations that happen every minute and the reason for them. |

## Metrics for TiDB Cloud Serverless clusters

Expand Down
1 change: 1 addition & 0 deletions tidb-cloud/limitations-and-quotas.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ TiDB Cloud limits how many of each kind of component you can create in a [TiDB C
| Maximum number of total TiDB nodes for all clusters in your organization | 10 |
| Maximum number of total TiKV nodes for all clusters in your organization | 15 |
| Maximum number of total TiFlash nodes for all clusters in your organization | 5 |
| Maximum number of total TiProxy nodes for all clusters in your organization | 10 |

> **Note:**
>
Expand Down
2 changes: 2 additions & 0 deletions tidb-cloud/monitor-tidb-cluster.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@ You can see the current status of each running cluster on the cluster page.
>
> The TiDB node status is only available for TiDB Cloud Dedicated clusters.

The node names starting with `tidb` are TiDB nodes, and those starting with `tiproxy` are TiProxy nodes.

| TiDB node status | Description |
|:--|:--|
| **Available** | The TiDB node is healthy and available. |
Expand Down
1 change: 1 addition & 0 deletions tidb-cloud/tidb-cloud-clinic.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ The dashboards and their content are subject to change. Currently, the following
- TiDB-Resource-Control
- TiFlash-Summary
- TiKV-Details
- TiProxy-Summary
- User-Node-Info

## Analyze top slow queries
Expand Down
124 changes: 124 additions & 0 deletions tidb-cloud/tiproxy-management.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
---
title: Manage TiProxy
summary: Learn about how to enable, disable, view and modify TiProxy..
---

# Manage TiProxy

This document describes how to enable, disable, view and modify TiProxy.

## Enable TiProxy

You can enable TiProxy for either a new cluster or an existing cluster in any TiDB node group.

### Decide size and number of TiProxy instances

The size and number of TiProxy instances depend on both the QPS and network bandwidth of your cluster. Network bandwidth is the sum of the client request and TiDB response bandwidth.

The following tables shows the maximum QPS and network bandwidth of each TiProxy size.

| Size | Maximum QPS | Maximum network bandwidth |
| :--| :--|
| Small | 30K | 93 MiB/s |
| Large | 120K | 312 MiB/s |

The optional TiProxy sizes are `small` and `large`. The optional TiProxy instance numbers are 2, 3, 6, 9, 12, 15, 18, 21, and 24. The default two small-sized TiProxy instances can provide 60K QPS and 186 MiB/s network bandwidth. It is recommended that you reserve 20% of the QPS to prevent high latency.

For example, if your cluster's maximum QPS is 100K and the maximum network bandwidth is 100 MiB/s, the size and number of TiProxy instances mainly depend on the QPS. In this case, you can select 6 small-sized TiProxy instances.

### Enable TiProxy for a new cluster

To enable TiProxy when creating a new cluster, click the TiProxy toggle and choose the TiProxy size and number.

![Enable TiProxy](/media/tidb-cloud/tiproxy-enable-tiproxy.png)

### Enable TiProxy for an existing cluster

To enable TiProxy for an existing cluster, perform the following steps:

1. In the [TiDB Cloud console](https://tidbcloud.com/), navigate to the [**Clusters**](https://tidbcloud.com/project/clusters) page of your project, and then click the name of your target cluster to go to its overview page.
2. Click **...** in the upper-right corner, and click **Modify** in the drop-down menu. The **Modify Cluster** page is displayed.
3. On the **Modify Cluster** page, click the TiProxy toggle and choose the TiProxy size and number.

![Enable TiProxy](/media/tidb-cloud/tiproxy-enable-tiproxy.png)

> **Note:**
>
> Enabling TiProxy will cause a rolling restart of TiDB in this TiDB node group, which causes connections to disconnect. In addition, creating new connections might hang for up to 30 seconds. Make sure that you enable TiProxy in the maintenance window.

### Limitations and quotas

- There must be at least two TiDB instance number in a TiDB node group.
- The TiDB instance size must be at least 4 vCPUs.
- The default maximum TiProxy instance number of an organization is 10. For more information, see [Limitations and Quotas](/tidb-cloud/limitations-and-quotas.md).
- The TiDB cluster version must be v6.5.0 or later.

## Disable TiProxy

To disable TiProxy, perform the following steps:

1. In the [TiDB Cloud console](https://tidbcloud.com/), navigate to the [**Clusters**](https://tidbcloud.com/project/clusters) page of your project, and then click the name of your target cluster to go to its overview page.
2. Click **...** in the upper-right corner, and click **Modify** in the drop-down menu. The **Modify Cluster** page is displayed.
3. On the **Modify Cluster** page, click the TiProxy toggle to disable TiProxy.

![Disable TiProxy](/media/tidb-cloud/tiproxy-disable-tiproxy.png)

> **Note:**
>
> Disabling TiProxy will cause connections to disconnect. In addition, creating new connections might hang for up to 10 seconds. Make sure that you disable TiProxy in the maintenance window.

## View TiProxy

### View TiProxy topology

To view the TiProxy topology, perform the following steps:

1. In the [TiDB Cloud console](https://tidbcloud.com/), navigate to the [**Clusters**](https://tidbcloud.com/project/clusters) page of your project, and then click the name of your target cluster to go to its overview page.
2. In the left navigation pane, click **Monitoring > Nodes**. The **Node Map** page is displayed.
3. In the **Node Map** page, TiProxy topology is displayed in the **TiDB** panel.

![TiProxy Topology](/media/tidb-cloud/tiproxy-topology.png)

### View TiProxy metrics

To view TiProxy metrics, perform the following steps:

1. In the [TiDB Cloud console](https://tidbcloud.com/), navigate to the [**Clusters**](https://tidbcloud.com/project/clusters) page of your project, and then click the name of your target cluster to go to its overview page.
2. In the left navigation pane, click **Monitoring > Metrics**. The **Metrics** page is displayed.
3. On the **Metrics** page, click **Server** tab and scroll down to TiProxy-related metrics. To view TiProxy metrics in one TiDB node group, click **TiDB Node Group View** tab, select your TiDB node group, and scroll down to TiProxy-related metrics.

The metrics include:

- **TiProxy CPU Usage**: the CPU usage statistics of each TiProxy node. The upper limit is 100%. If the maximum CPU usage exceeds 80%, it is recommended that you scale out TiProxy.
- **TiProxy Connections**: the connection number on each TiProxy node.
- **TiProxy Throughput**: the transferred bytes per second on each TiProxy node. If the maximum throughput reaches the maximum network bandwidth, it is recommended that you scale out TiProxy. To learn about the maximum network bandwidth, see [Decide TiProxy instance size and number](#decide-tiproxy-instance-size-and-number).
- **TiProxy Sessions Migration Reasons**: the number of session migrations that happen every minute and the reason for them. For example, when TiDB scales in and TiProxy migrates sessions to other TiDB instances, the reason is `status`. For more migration reasons, see [TiProxy Monitoring Metrics](https://docs.pingcap.com/tidb/stable/tiproxy-grafana#balance).

### View TiProxy bills

To view TiProxy bills, perform the following steps:

1. In the [TiDB Cloud console](https://tidbcloud.com), switch to your target organization using the combo box in the upper-left corner.
2. In the left navigation pane, click **Billing**. On the **Billing** page, the **Bills** tab is displayed by default.
3. In the **Summary by Service** section, TiProxy node cost is displayed under **TiDB Dedicated**, while TiProxy data transfer cost is included in **Data Transfer > Same Region**.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need add description for tiproxy is charge free for the Private Preview stage @joey-yez

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The doc is public from the public preview. It won't be committed during private preview.


![TiProxy Billing](/media/tidb-cloud/tiproxy-billing.png)

## Modify TiProxy

To scale in or scale out TiProxy, perform the following steps:

1. In the [TiDB Cloud console](https://tidbcloud.com/), navigate to the [**Clusters**](https://tidbcloud.com/project/clusters) page of your project, and then click the name of your target cluster to go to its overview page.
2. Click **...** in the upper-right corner, and click **Modify** in the drop-down menu. The **Modify Cluster** page is displayed.
3. On the **Modify Cluster** page, modify the TiProxy number.

![Modify TiProxy](/media/tidb-cloud/tiproxy-enable-tiproxy.png)

> **Note**
>
> - Modifying the TiProxy size directly is not supported. It is recommended that you modify the number of TiProxy instances. If you have to modify the TiProxy size, you need to disable TiProxy in all the TiDB node groups and then enable it again to choose a different size.
> - Scaling in TiProxy will cause connections to disconnect. Make sure that you scale in TiProxy in the maintenance window.

## Manage TiProxy in multiple TiDB node groups

When you have multiple TiDB node groups, each TiDB node group has its dedicated TiProxy group. TiProxy routes traffic to the TiDB instances in the same TiDB node group to isolate computing resources. You can enable, disable, or modify TiProxy in each TiDB node group. However, the TiProxy size in all the TiDB node groups must be the same.
43 changes: 43 additions & 0 deletions tidb-cloud/tiproxy-overview-for-cloud.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
---
title: Overview of TiProxy for TiDB Cloud
summary: Learn about the usage scenarios of TiProxy for TiDB Cloud.
---

# Overview of TiProxy for TiDB Cloud

TiProxy is the official proxy component of PingCAP. It is placed between the client and the TiDB server to provide load balancing, connection persistence, and other features for TiDB.

For more information, see [TiProxy Overview](https://docs.pingcap.com/tidb/stable/tiproxy-overview).

## Scenarios

TiProxy is suitable for the following scenarios:

- Connection persistence: When a TiDB server performs scaling in, rolling upgrade, or rolling restart, the client connection breaks, resulting in an error. If the client does not have an idempotent error retry mechanism, you need to manually check and fix the error, which greatly increases the operational overhead. TiProxy can keep the client connection, so that the client does not report an error.
- Frequent scaling in and scaling out: The workload of an application might change periodically. To save costs, you can deploy TiDB on the cloud and automatically scale in and scale out TiDB servers according to the workload. However, scaling in might cause the client to disconnect, and scaling out might result in an unbalanced load. TiProxy can keep the client connection and achieve load balancing.
- CPU load imbalance: When background tasks consume a significant amount of CPU resources, or workloads across connections vary significantly, leading to an imbalanced CPU load, TiProxy can migrate connections based on CPU usage to achieve load balancing. For more details, see [CPU-based load balancing](https://docs.pingcap.com/tidb/stable/tiproxy-load-balance#cpu-based-load-balancing).

For more scenarios, see [TiProxy User Scenarios](https://docs.pingcap.com/tidb/stable/tiproxy-overview#user-scenarios).

## Limitations

TiProxy cannot preserve client connections in the following scenarios:

- Upgrading AWS EKS, Azure AKS, Google Cloud GKE, or Alibaba Cloud ACK.
- Disabling, scaling in, upgrading, or restarting TiProxy.
- A single statement or transaction that runs for more than 20 seconds. If your application needs a longer timeout, contact [TiDB Cloud Support](/tidb-cloud/tidb-cloud-support.md).

For more scenarios, see [TiProxy Limitations](https://docs.pingcap.com/tidb/stable/tiproxy-overview#limitations).

## Billing

TiProxy introduces two types of costs:

- Node costs. For more information, see [Node Cost](https://www.pingcap.com/tidb-dedicated-pricing-details/#node-cost)
- Data transfer costs. For more information, see [Data Transfer Cost](https://www.pingcap.com/tidb-dedicated-pricing-details/#data-transfer-cost). TiProxy will prioritize routing traffic to the TiDB instances in the same availability zone (AZ). However, if the TiDB workloads are uneven, it will also route traffic to other AZs, resulting in additional data transfer costs.

You can view the TiProxy bill on the billing page. For more information, see [View TiProxy bills](/tidb-cloud/tiproxy-management.md#view-tiproxy-bills).

## SLA impact

TiProxy has no impact on SLA.
Loading