Skip to content

quickstart.md

Latest commit

 

History

History
139 lines (100 loc) · 3.7 KB

quickstart.md

File metadata and controls

139 lines (100 loc) · 3.7 KB

Quickstart

This walkthrough takes about five minutes. By the end, you'll have Hoptimator running against a local Kubernetes cluster, two demo databases registered, and a materialized view turned into a real pipeline.

Prerequisites

You need:

  • A local Kubernetes cluster. The simplest option is Docker Desktop: install it, then enable Kubernetes under Settings → Kubernetes. Alternately use kind.
  • kubectl pointed at that cluster.
  • JDK 17+ and Make. The build uses the bundled Gradle wrapper.

Verify your context:

kubectl config current-context
# should print docker-desktop, or your kind cluster's name

1. Build and install the CLI

From the repo root:

make build install

This produces the hoptimator script (a wrapper around the SQL CLI) and the container images used by the operator.

2. Deploy the demo

make deploy-demo

This installs the Hoptimator CRDs into your cluster and applies two demo databases — ads and profiles — backed by an in-memory demodb source. It also registers the table and job templates needed to deploy pipelines locally.

You should now see Hoptimator's resources in the cluster:

kubectl get databases
# NAME              SCHEMA    URL
# ads-database      ADS       jdbc:demodb://names=ads
# profile-database  PROFILE   jdbc:demodb://names=profile

kubectl get tabletemplates
kubectl get jobtemplates

3. Open the SQL CLI

./hoptimator

This launches a sqlline shell connected to jdbc:hoptimator://. Type !intro at the prompt for a tour, or !quit to leave. For the full set of Hoptimator-specific commands (!resolve, !pipeline, !specify, …) see SQL CLI.

Try a quick query:

SELECT * FROM ADS.AD_CLICKS LIMIT 5;

4. Create a materialized view

Now do the interesting thing — declare a view that joins the two demo databases:

CREATE MATERIALIZED VIEW ADS.AUDIENCE AS
  SELECT FIRST_NAME, LAST_NAME
  FROM ADS.PAGE_VIEWS
  NATURAL JOIN PROFILE.MEMBERS;

Hoptimator plans the join, picks the connectors, and asks the deployer to materialize the resulting pipeline.

While you're iterating, use CREATE OR REPLACE MATERIALIZED VIEW ... to update the same view in place. Without OR REPLACE, re-running a CREATE for an existing name fails. Most of the development loop is faster if you treat the view as mutable.

5. See what happened

In another terminal:

kubectl get views
# NAME          CATALOG   SCHEMA   VIEW       SQL
# ads-audience            ADS      AUDIENCE   SELECT FIRST_NAME, LAST_NAME ...

kubectl get pipelines
# NAME          SQL                                STATUS
# ads-audience   CREATE DATABASE IF NOT EXISTS `ADS` WITH ();...   Ready.

Each Pipeline is a complete, standalone Kubernetes resource. You can describe it, watch its events, or kubectl get -o yaml it to see exactly what was deployed.

If you want to see what Hoptimator would have deployed without actually deploying it, the CLI has a dry-run mode for that:

!specify CREATE MATERIALIZED VIEW ADS.AUDIENCE AS
  SELECT FIRST_NAME, LAST_NAME
  FROM ADS.PAGE_VIEWS NATURAL JOIN PROFILE.MEMBERS

6. Clean up

DROP MATERIALIZED VIEW ADS.AUDIENCE;

Or tear down the whole demo:

make undeploy-demo

What's next

  • Read Concepts to put names to what you just saw.
  • Read the Architecture overview for the bigger picture of how SQL turns into running infrastructure.
  • Try a richer setup with real Kafka, Flink, MySQL, and Venice clusters via make deploy-dev-environment (see the development guide when it lands).