Support TiDB

[Torch3333]

Description

This adds support for TiDB versions 6.5, 7.5, and 8.5, a fully MySQL-compliant database. Connection to TiDB is done through the MySQL Connector/J JDBC driver.

A sample ScalarDB configuration to use TiDB is as follows.

scalar.db.storage=jdbc
scalar.db.contact_points=jdbc:mysql://localhost:4000/
scalar.db.username=root
scalar.db.password=tidb

Related issues and/or PRs

N/A

Changes made

• Adds TiDB integration tests to the CI
• Adds RdbEngine for TiDB

Checklist

The following is a best-effort checklist. If any items in this checklist are not applicable to this PR or are dependent on other, unmerged PRs, please still mark the checkboxes after you have read and understood each item.

• I have commented my code, particularly in hard-to-understand areas.
• I have updated the documentation to reflect the changes.
• I have considered whether similar issues could occur in other products, components, or modules if this PR is for bug fixes.
• Any remaining open issues linked to this PR are documented and up-to-date (Jira, GitHub, etc.).
• Tests (unit, integration, etc.) have been added for the changes.
• My changes generate no new warnings.
• Any dependent changes in other PRs have been merged and published.

Additional notes (optional)

N/A

Release notes

TiDB versions 6.5, 7.5, and 8.5 are now supported as underlying storage by using MySQL Connector/J JDBC driver.

[Copilot]

Pull Request Overview

This PR adds TiDB support to ScalarDB as an underlying storage by enabling connection through MySQL Connector/J JDBC driver. It supports TiDB versions 6.5, 7.5, and 8.5, with configuration to override the default SERIALIZABLE isolation level to REPEATABLE_READ or READ_COMMITTED since TiDB doesn't support SERIALIZABLE.

• Added isolation level configuration support in JDBC environment setup
• Implemented TiDB-specific handling for unsupported data types in import tests
• Added comprehensive CI integration tests for TiDB versions 6.5, 7.5, and 8.5

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File	Description
core/src/integration-test/java/com/scalar/db/storage/jdbc/JdbcEnv.java	Adds isolation level configuration property support for test environment
core/src/integration-test/java/com/scalar/db/storage/jdbc/JdbcAdminImportTestUtils.java	Implements TiDB detection and specific data type filtering for import tests
.github/workflows/ci.yaml	Adds CI integration test jobs for TiDB versions 6.5, 7.5, and 8.5

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[komamitsu]

LGTM! 👍

[brfrn169]

Since the ScalarDB transaction API sets a SERIALIZABLE isolation level by default not supported by TiDB

@Torch3333 I don’t think we set the SERIALIZABLE isolation level by default. We only set the isolation level when users specify the scalar.db.jdbc.isolation_level property. What do you think?

[feeblefakie]

@Torch3333 What would happen if users don't specify scalardb.jdbc.isolation_level=REPEATABLE_READ or READ-COMMITTED?

BTW, TiDB seems to be linearizable, independent of isolation levels. So, any isolation levels can probaly be fine for correctness. (cc: @brfrn169 )
https://docs.pingcap.com/tidb/stable/system-variables/#tidb_guarantee_linearizability-new-in-v50

[Torch3333]

@brfrn169 @feeblefakie @komamitsu

@Torch3333 I don’t think we set the SERIALIZABLE isolation level by default. We only set the isolation level when users specify the scalar.db.jdbc.isolation_level property. What do you think?

You are right. I mistakenly thought ScalarDB was setting the JDBC transaction isolation level for all the transaction managers, but in fact, it is only set for the JDBC transaction manager, not for Consensus Commit.
As discussed with @brfrn169 and @feeblefakie, we decided that setting it for the JDBC transaction manager was not required. From now on, the storage default isolation level will be used. For better clarity for our users, I made these changes in a separate PR #3076

What would happen if users don't specify scalardb.jdbc.isolation_level=REPEATABLE_READ or READ-COMMITTED?

TiDB uses the REPEATABLE-READ as the default level. READ-COMMITED is also an accepted value. Setting SERIALIZABLE will cause the driver to throw this error. The documentation explains it here.

The isolation level 'SERIALIZABLE' is not supported. Set tidb_skip_isolation_level_check=1 to skip this error

Other than that, I finally found a document listing the features of the MySQL Connector/J driver that TiDB does not support. As far as my understanding goes, I don't think ScalarDB uses any of the incompatibilities, so this confirms that using this driver instead of the TiDB modified version of Connector/J is fine.

[Torch3333]

I will put this PR in draft until this blocking dependent PR is merged.
#3076

[Torch3333]

@komamitsu You raised a point during the demo meeting more than 2 weeks ago about how the driver handles load balancing. There seems to be a library to handle just that, but I will investigate it once other higher-priority tasks are completed.

[feeblefakie]

@Torch3333 This can be opened again now?

[Torch3333]

The PR is ready for review again.
Please review everything from scratch once again, since there are numerous new files since your last review.
Please take a look at my explanation below about the changes.

[feeblefakie]

LGTM! Thank you!

[brfrn169]

LGTM! Thank you!

[komamitsu]

LGTM! 👍