ScalarDB Cluster with Cassandra

[yito88]

Description

Provide a brief description about why this PR is necessary. Be sure to provide context.

Related issues and/or PRs

If this PR addresses or references any issues and/or other PRs, list them here.

Changes made

Outline the specific changes made in this pull request. Include relevant details, such as added features, bug fixes, code refactoring, or improvements.

Checklist

The following is a best-effort checklist. If any items in this checklist are not applicable to this PR or are dependent on other, unmerged PRs, please still mark the checkboxes after you have read and understood each item.

• I have commented my code, particularly in hard-to-understand areas.
• I have updated the documentation to reflect the changes.
• Any remaining open issues linked to this PR are documented and up-to-date (Jira, GitHub, etc.).
• Tests (unit, integration, etc.) have been added for the changes.
• My changes generate no new warnings.
• Any dependent changes in other PRs have been merged and published.

Additional notes (optional)

Provide any additional information or notes that may be relevant to the reviewers or stakeholders.

[gemini-code-assist]

Summary of Changes

Hello @yito88, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

I've implemented significant changes to enable ScalarDB Cluster to operate with Apache Cassandra as its persistent storage layer. Previously, ScalarDB Cluster was primarily configured for PostgreSQL. This enhancement introduces the capability to dynamically provision and manage a Cassandra cluster alongside ScalarDB Cluster, allowing users to leverage Cassandra's distributed and highly available characteristics. The changes span across deployment scripts, configuration logic, and test infrastructure to ensure seamless integration and validation of this new storage option.

Highlights

• Added Cassandra Support for ScalarDB Cluster: I've extended the ScalarDB Cluster deployment to support Apache Cassandra as an underlying storage backend, in addition to the existing PostgreSQL support. This allows for greater flexibility in choosing the database for ScalarDB Cluster deployments.
• Dynamic Database Configuration and Deployment: The deployment logic now dynamically configures ScalarDB Cluster properties and deploys the appropriate database (PostgreSQL or Cassandra) using Helm charts based on the db-type specified. This includes handling connection details, user credentials, and cleanup procedures specific to each database.
• Test Runner Integration for Cassandra: I've updated the test runner to recognize and utilize the new cluster-cassandra database type, ensuring that tests can be executed against ScalarDB Cluster backed by Cassandra. This involved refactoring how database generation functions are loaded and how test options are parsed.
• Generalized Fault Injection: Fault injection mechanisms have been generalized to apply to any cluster node, rather than being specifically tied to PostgreSQL. This improves the robustness and applicability of chaos engineering experiments across different storage backends.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments or fill out our survey to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request effectively adds support for Cassandra as a backend for ScalarDB Cluster tests. The changes are well-structured, introducing dynamic configuration based on the database backend. I've identified a couple of areas for improvement: a duplicate configuration in a Helm command and an opportunity to make the test client's connection to Cassandra more robust by using all available contact points instead of just one. Overall, this is a solid contribution that extends the testing capabilities.

[gemini-code-assist]

This function currently returns only one IP address for Cassandra, which can be a single point of failure for the test client. It would be more robust to return all available Cassandra LoadBalancer IPs. The docstring should also be updated to reflect this change.

(defn get-cassandra-ip
  "Get all IPs of the Cassandra nodes"
  []
  (->> (c/exec :kubectl :get :svc)
       str/split-lines
       (filter #(str/includes? % "cassandra-scalardb-cluster"))
       (filter #(str/includes? % "LoadBalancer"))
       (map #(nth (str/split % #"\s+") 3))))

[gemini-code-assist]

To make the client connection to Cassandra more robust, get-cassandra-ip should return a list of all available IPs (as suggested in another comment). This call should then be updated to join those IPs into a comma-separated string for the contact_points property.

                       :cluster-cassandra ["cassandra"
                                           (str/join "," (get-cassandra-ip))
                                           "cassandra"]

[gemini-code-assist]

The db-key variable has been renamed to db-type in most places, which is a great improvement for clarity. This one was missed. Renaming it here will improve consistency across the codebase.

  [db-type]