diff --git a/docs/connect/cli.md b/docs/connect/cli.md new file mode 100644 index 00000000..7053efc9 --- /dev/null +++ b/docs/connect/cli.md @@ -0,0 +1,183 @@ +(cli)= +(connect-cli)= +# Using command-line programs with CrateDB + +This section provides a quick overview about a few CLI programs, and how to +use them for connecting to CrateDB clusters. We recommend to use crash, +psql, http ([HTTPie]), or curl. + +You can use them to quickly validate HTTP and PostgreSQL connectivity to your +database cluster, or to conduct basic scripting. + +Before running the command-line snippets outlined below, please use the correct +settings instead of the placeholder tokens ``, `` and +``. + +When using CrateDB Cloud, `` will be something like +`.{aks1,eks1}.region.{azure,aws}.cratedb.net`. + + +(crash)= +## crash + +```{div} +:style: "float: right" +![image](https://cratedb.com/docs/crate/crash/en/latest/_images/query.png){w=240px} +``` + +The **CrateDB Shell** is an interactive command-line interface (CLI) tool for +working with CrateDB. For more information, see the documentation about [crash]. + +```{div} +:style: "clear: both" +``` + +::::{tab-set} + +:::{tab-item} CrateDB and CrateDB Cloud +:sync: server + +```{code-block} shell +CRATEPW= \ + crash --hosts 'https://:4200' --username '' \ + --command "SELECT 42.42;" +``` +::: + +:::{tab-item} CrateDB on localhost +:sync: localhost + +```{code-block} shell +# No authentication. +crash --command "SELECT 42.42;" + +``` +::: + +:::: + + +(psql)= +## psql + +```{div} +:style: "float: right" +![image](https://github.com/crate/crate-clients-tools/assets/453543/8f0a0e06-87f6-467f-be2d-b38121afbafa){w=240px} +``` + +**psql** is a terminal-based front-end to PostgreSQL. It enables you to type in +queries interactively, issue them to PostgreSQL, and see the query results. +For more information, see the documentation about [psql]. + +```{div} +:style: "clear: both" +``` + +::::{tab-set} + +:::{tab-item} CrateDB and CrateDB Cloud +:sync: server + +```{code-block} shell +PGUSER= PGPASSWORD= \ + psql postgresql:///crate --command "SELECT 42.42;" +``` +::: + +:::{tab-item} CrateDB on localhost +:sync: localhost + +```{code-block} shell +# No authentication. +psql postgresql://crate@localhost:5432/crate --command "SELECT 42.42;" +``` +::: + +:::: + + +(httpie)= +## HTTPie + +```{div} +:style: "float: right" +![image](https://github.com/crate/crate-clients-tools/assets/453543/f5a2916d-3730-4901-99cf-b88b9af03329){w=240px} +``` + +The **HTTPie CLI** is a modern, user-friendly command-line HTTP client with +JSON support, colors, sessions, downloads, plugins & more. +For more information, see the documentation about [HTTPie]. + +```{div} +:style: "clear: both" +``` + +::::{tab-set} + +:::{tab-item} CrateDB and CrateDB Cloud +:sync: server + +```{code-block} shell +http "https://:@:4200/_sql?pretty" \ + stmt="SELECT 42.42;" +``` +::: + +:::{tab-item} CrateDB on localhost +:sync: localhost + +```{code-block} shell +http "localhost:4200/_sql?pretty" \ + stmt="SELECT 42.42;" +``` +::: + +:::: + + +(curl)= +## curl + +```{div} +:style: "float: right" +![image](https://github.com/crate/crate-clients-tools/assets/453543/318b0819-a0d4-4112-a320-23852263362c){w=240px} +``` + +The venerable **curl** is the ubiquitous command line tool and library for transferring +data with URLs. For more information, see the documentation about [curl]. + +This example combines it with [jq], a lightweight and flexible command-line JSON processor. + +```{div} +:style: "clear: both" +``` + +::::{tab-set} + +:::{tab-item} CrateDB and CrateDB Cloud +:sync: server + +```{code-block} shell +echo '{"stmt": "SELECT 42.42;"}' \ + | curl "https://:@:4200/_sql?pretty" --silent --data @- | jq +``` +::: + +:::{tab-item} CrateDB on localhost +:sync: localhost + +```{code-block} shell +echo '{"stmt": "SELECT 42.42;"}' \ + | curl "localhost:4200/_sql?pretty" --silent --data @- | jq +``` +::: + +:::: + + + +[curl]: https://curl.se/ +[crash]: inv:crate-crash:*:label#index +[HTTPie]: https://httpie.io/ +[jq]: https://jqlang.github.io/jq/ +[psql]: https://www.postgresql.org/docs/current/app-psql.html diff --git a/docs/connect/configure.md b/docs/connect/configure.md new file mode 100644 index 00000000..741bbb70 --- /dev/null +++ b/docs/connect/configure.md @@ -0,0 +1,158 @@ +(connect-configure)= + +# Configure + +In order to connect to CrateDB, your application or driver needs to be +configured with corresponding connection properties. Please note that different +applications and drivers may obtain connection properties in different formats. + + + +::::::{tab-set} + +:::::{tab-item} CrateDB and CrateDB Cloud + +::::{grid} +:margin: 0 +:padding: 0 + +:::{grid-item} +:columns: 4 +:margin: 0 +:padding: 0 + +**Connection properties** + +:Host: ``.cratedb.net +:Port: 5432 (PostgreSQL) or
4200 (HTTP) +:User: `` +:Pass: `` + +::: + +:::{grid-item} +:columns: 8 +:margin: 0 +:padding: 0 +:class: driver-slim + +**Connection-string examples** +

+ +**Native PostgreSQL, psql** +```text +postgresql://:@.cratedb.net:5432/doc +``` + +**JDBC: PostgreSQL pgJDBC** +```text +jdbc:postgresql://.cratedb.net:5432/doc?user=&password= +``` + +**JDBC: CrateDB JDBC, e.g. Apache Flink** +```text +jdbc:crate://.cratedb.net:5432/doc?user=&password= +``` + +**HTTP: Admin UI, CLI, CrateDB drivers** +```text +https://:@.cratedb.net:4200/ +``` + +**SQLAlchemy** +```text +crate://:@.cratedb.net:4200/?schema=doc&ssl=true +``` + +::: + +:::: + +::::: + +:::::{tab-item} CrateDB on localhost + +::::{grid} +:margin: 0 +:padding: 0 + +:::{grid-item} +:columns: 4 +:margin: 0 +:padding: 0 + +**Connection properties** + +:Host: localhost +:Port: 5432 (PostgreSQL) or
4200 (HTTP) +:User: `crate` +:Pass: (empty) + +::: + +:::{grid-item} +:columns: 8 +:margin: 0 +:padding: 0 +:class: driver-slim + +**Connection-string examples** +

+ +**Native PostgreSQL, psql** +``` +postgresql://crate@localhost:5432/doc +``` + +**JDBC: PostgreSQL pgJDBC** +```text +jdbc:postgresql://localhost:5432/doc?user=crate +``` + +**JDBC: CrateDB JDBC, e.g. Apache Flink** +```text +jdbc:crate://localhost:5432/doc?user=&password= +``` + +**HTTP: Admin UI, CLI, CrateDB drivers** +```text +http://crate@localhost:4200/ +``` + +**SQLAlchemy** +```text +crate://crate@localhost:4200/?schema=doc +``` + +::: + +:::: +::::: + +:::::: + + +```{tip} +- CrateDB's fixed catalog name is `crate`, the default schema name is `doc`. +- CrateDB does not implement the notion of a database, + however tables can be created in different [schemas]. +- When asked for a *database name*, specifying a schema name (any), + or the fixed catalog name `crate` may be applicable. +- If a database-/schema-name is omitted while connecting, + the PostgreSQL drivers may default to the "username". +- The predefined [superuser] on an unconfigured CrateDB cluster is + called `crate`, defined without a password. +- For authenticating properly, please learn about the available + [authentication] options. +``` diff --git a/docs/connect/df/index.md b/docs/connect/df/index.md new file mode 100644 index 00000000..9ee05087 --- /dev/null +++ b/docs/connect/df/index.md @@ -0,0 +1,164 @@ +(df)= +(dataframe)= +(dataframes)= +# CrateDB and DataFrame libraries + +Data frame libraries and frameworks which can +be used together with CrateDB. + + +:::::{grid} 1 2 2 2 +:margin: 4 4 0 0 +:padding: 0 +:gutter: 2 + +::::{grid-item-card} {material-outlined}`lightbulb;2em` Tutorials +:link: guide:dataframes +:link-type: ref +Learn how to use CrateDB together with popular open-source data frame +libraries, on behalf of hands-on tutorials and code examples. ++++ +{tag-info}`Dask` {tag-info}`pandas` {tag-info}`Polars` +:::: + +::::{grid-item-card} {material-outlined}`read_more;2em` SQLAlchemy +CrateDB's SQLAlchemy dialect implementation provides fundamental infrastructure +to integrations with Dask, pandas, and Polars. ++++ +[ORM Guides](inv:guide#orm) • +{ref}`ORM Catalog ` +:::: + +::::: + + +(dask)= +## Dask + +[Dask] is a parallel computing library for analytics with task scheduling. +It is built on top of the Python programming language, making it easy to scale +the Python libraries that you know and love, like NumPy, pandas, and scikit-learn. + +```{div} +:style: "float: right" +[![](https://github.com/crate/crate-clients-tools/assets/453543/99bd2234-c501-479b-ade7-bcc2bfc1f288){w=180px}](https://www.dask.org/) +``` + +- [Dask DataFrames] help you process large tabular data by parallelizing pandas, + either on your laptop for larger-than-memory computing, or on a distributed + cluster of computers. + +- [Dask Futures], implementing a real-time task framework, allow you to scale + generic Python workflows across a Dask cluster with minimal code changes, + by extending Python's `concurrent.futures` interface. + +```{div} +:style: "clear: both" +``` + + +(pandas)= +## pandas + +```{div} +:style: "float: right" +[![](https://pandas.pydata.org/static/img/pandas.svg){w=180px}](https://pandas.pydata.org/) +``` + +[pandas] is a fast, powerful, flexible, and easy-to-use open-source data analysis +and manipulation tool, built on top of the Python programming language. + +Pandas (stylized as pandas) is a software library written for the Python programming +language for data manipulation and analysis. In particular, it offers data structures +and operations for manipulating numerical tables and time series. + +:::{rubric} Data Model +::: +- Pandas is built around data structures called Series and DataFrames. Data for these + collections can be imported from various file formats such as comma-separated values, + JSON, Parquet, SQL database tables or queries, and Microsoft Excel. +- A Series is a 1-dimensional data structure built on top of NumPy's array. +- Pandas includes support for time series, such as the ability to interpolate values + and filter using a range of timestamps. +- By default, a Pandas index is a series of integers ascending from 0, similar to the + indices of Python arrays. However, indices can use any NumPy data type, including + floating point, timestamps, or strings. +- Pandas supports hierarchical indices with multiple values per data point. An index + with this structure, called a "MultiIndex", allows a single DataFrame to represent + multiple dimensions, similar to a pivot table in Microsoft Excel. Each level of a + MultiIndex can be given a unique name. + +```{div} +:style: "clear: both" +``` + + +(polars)= +## Polars + +```{div} +:style: "float: right; margin-left: 0.5em" +[![](https://github.com/pola-rs/polars-static/raw/master/logos/polars-logo-dark.svg){w=180px}](https://pola.rs/) +``` + +[Polars] is a blazingly fast DataFrames library with language bindings for +Rust, Python, Node.js, R, and SQL. Polars is powered by a multithreaded, +vectorized query engine, it is open source, and written in Rust. + +- **Fast:** Written from scratch in Rust and with performance in mind, + designed close to the machine, and without external dependencies. + +- **I/O:** First class support for all common data storage layers: local, + cloud storage & databases. + +- **Intuitive API:** Write your queries the way they were intended. Polars, + internally, will determine the most efficient way to execute using its query + optimizer. Polars' expressions are intuitive and empower you to write + readable and performant code at the same time. + +- **Out of Core:** The streaming API allows you to process your results without + requiring all your data to be in memory at the same time. + +- **Parallel:** Polars' multi-threaded query engine utilises the power of your + machine by dividing the workload among the available CPU cores without any + additional configuration. + +- **Vectorized Query Engine:** Uses [Apache Arrow], a columnar data format, to + process your queries in a vectorized manner and SIMD to optimize CPU usage. + This enables cache-coherent algorithms and high performance on modern processors. + +- **Open Source:** Polars is and always will be open source. Driven by an active + community of developers. Everyone is encouraged to add new features and contribute. + It is free to use under the MIT license. + +:::{rubric} Data formats +::: + +Polars supports reading and writing to many common data formats. +This allows you to easily integrate Polars into your existing data stack. + +- Text: CSV & JSON +- Binary: Parquet, Delta Lake, AVRO & Excel +- IPC: Feather, Arrow +- Databases: MySQL, Postgres, SQL Server, Sqlite, Redshift & Oracle +- Cloud Storage: S3, Azure Blob & Azure File + +```{div} +:style: "clear: both" +``` + + +```{toctree} +:maxdepth: 1 +:hidden: + +learn +``` + + +[Apache Arrow]: https://arrow.apache.org/ +[Dask]: https://www.dask.org/ +[Dask DataFrames]: https://docs.dask.org/en/latest/dataframe.html +[Dask Futures]: https://docs.dask.org/en/latest/futures.html +[pandas]: https://pandas.pydata.org/ +[Polars]: https://pola.rs/ diff --git a/docs/integrate/df.md b/docs/connect/df/learn.md similarity index 97% rename from docs/integrate/df.md rename to docs/connect/df/learn.md index eda79aec..6eb0c722 100644 --- a/docs/integrate/df.md +++ b/docs/connect/df/learn.md @@ -1,6 +1,6 @@ -(df)= -(dataframes)= -# Dataframe Libraries +(dataframe-examples)= + +# DataFrame Examples How to use CrateDB together with popular open-source dataframe libraries. diff --git a/docs/connect/drivers.md b/docs/connect/drivers.md new file mode 100644 index 00000000..3456113f --- /dev/null +++ b/docs/connect/drivers.md @@ -0,0 +1,362 @@ +(connect-drivers)= + +# Drivers and Client Libraries + +This section lists drivers and adapters for supported programming languages, +frameworks, and environments. + +## PostgreSQL +The drivers listed in this section all use the [CrateDB PostgreSQL interface]. + +::::{sd-table} +:widths: 2 3 5 2 +:row-class: top-border + +:::{sd-row} +```{sd-item} +``` +```{sd-item} **Driver/Adapter** +``` +```{sd-item} **Description** +``` +```{sd-item} **Info** +``` +::: + +:::{sd-row} +```{sd-item} \- +``` +```{sd-item} +[PostgreSQL ODBC](https://odbc.postgresql.org/) +``` +```{sd-item} +The official PostgreSQL ODBC Driver. +For connecting to CrateDB from any environment that supports it. +``` +```{sd-item} +``` +::: + +:::{sd-row} +```{sd-item} .NET +``` +```{sd-item} +[Npgsql](https://www.npgsql.org/) +``` +```{sd-item} +An open source ADO.NET Data Provider for PostgreSQL, for program written in C#, +Visual Basic, and F#. +``` +```{sd-item} +[![](https://img.shields.io/github/v/tag/npgsql/npgsql?label=latest)](https://github.com/npgsql/npgsql) +[![](https://img.shields.io/badge/example-runnable-darkcyan)](https://github.com/crate/cratedb-examples/tree/main/by-language/csharp-npgsql) +``` +::: + +:::{sd-row} +```{sd-item} .NET +``` +```{sd-item} +[CrateDB Npgsql fork](https://cratedb.com/docs/npgsql/) +``` +```{sd-item} +This fork of the official driver was needed prior to CrateDB 4.2. +``` +```{sd-item} +[![](https://img.shields.io/github/v/tag/crate/npgsql?label=latest)](https://github.com/crate/npgsql) +``` +::: + +:::{sd-row} +```{sd-item} Golang +``` +```{sd-item} +[pgx](https://github.com/jackc/pgx) +``` +```{sd-item} +A pure Go driver and toolkit for PostgreSQL. +``` +```{sd-item} +[![](https://img.shields.io/github/v/tag/jackc/pgx?label=latest)](https://github.com/jackc/pgx) +``` +::: + +:::{sd-row} +```{sd-item} Java +``` +```{sd-item} +[PostgreSQL JDBC](https://jdbc.postgresql.org/) +``` +```{sd-item} +The official PostgreSQL JDBC Driver. +For connecting to CrateDB from any environment that supports it. +``` +```{sd-item} +[![](https://img.shields.io/github/v/tag/pgjdbc/pgjdbc?label=latest)](https://github.com/pgjdbc/pgjdbc) +[![](https://img.shields.io/badge/example-snippet-darkcyan)](#java) +[![](https://img.shields.io/badge/example-runnable-darkcyan)](https://github.com/crate/cratedb-examples/tree/main/by-language/java-jdbc) +``` +::: + + +:::{sd-row} +```{sd-item} Java +``` +```{sd-item} +[CrateDB PgJDBC fork](https://cratedb.com/docs/jdbc/) +``` +```{sd-item} +For connecting to CrateDB with specialized type system support and +other tweaks. Ignores the `ROLLBACK` statement and the `hstore` and +`jsonb` extensions. +``` +```{sd-item} +[![](https://img.shields.io/maven-central/v/io.crate/crate-jdbc?label=latest)](https://github.com/crate/pgjdbc) +``` +::: + +:::{sd-row} +```{sd-item} Node.js +``` +```{sd-item} +[node-postgres](https://node-postgres.com/) +``` +```{sd-item} +A collection of Node.js modules for interfacing with a PostgreSQL database using +JavaScript or TypeScript. + +Has support for callbacks, promises, async/await, connection pooling, prepared +statements, cursors, streaming results, C/C++ bindings, rich type parsing, and more. +``` +```{sd-item} +[![](https://img.shields.io/npm/v/pg?label=latest&color=blue)](https://github.com/brianc/node-postgres) +[![](https://img.shields.io/badge/example-snippet-darkcyan)](#javascript) +``` +::: + +:::{sd-row} +```{sd-item} PHP +``` +```{sd-item} +[PDO_PGSQL](https://www.php.net/manual/en/ref.pdo-pgsql.php) +``` +```{sd-item} +For connecting to CrateDB from PHP, supporting its PDO interface. +``` +```{sd-item} +[![](https://img.shields.io/github/v/tag/php/php-src?label=latest)](https://github.com/php/php-src/tree/master/ext/pdo_pgsql) +[![](https://img.shields.io/badge/example-runnable-darkcyan)](https://github.com/crate/cratedb-examples/tree/main/by-language/php-pdo) +``` +::: + +:::{sd-row} +```{sd-item} PHP +``` +```{sd-item} +[AMPHP](https://amphp.org/) +``` +```{sd-item} +For connecting to CrateDB using AMPHP, an Async PostgreSQL client for PHP. +AMPHP is a collection of high-quality, event-driven libraries for PHP +designed with fibers and concurrency in mind. +``` +```{sd-item} +[![](https://img.shields.io/github/v/tag/amphp/postgres?label=latest)](https://github.com/amphp/postgres) +[![](https://img.shields.io/badge/example-runnable-darkcyan)](https://github.com/crate/cratedb-examples/tree/main/by-language/php-amphp) +``` +::: + +:::{sd-row} +```{sd-item} Python +``` +```{sd-item} +[aiopg](https://github.com/aio-libs/aiopg) +``` +```{sd-item} +For connecting to CrateDB from Python, supporting Python's `asyncio` (PEP-3156/tulip) framework. +``` +```{sd-item} +[![](https://img.shields.io/github/v/tag/aio-libs/aiopg?label=latest)](https://github.com/aio-libs/aiopg) +[![](https://img.shields.io/badge/example-snippet-darkcyan)](#aiopg) +``` +::: + +:::{sd-row} +```{sd-item} Python +``` +```{sd-item} +[asyncpg](https://github.com/MagicStack/asyncpg) +``` +```{sd-item} +For connecting to CrateDB from Python, supporting Python's `asyncio`. +``` +```{sd-item} +[![](https://img.shields.io/github/v/tag/MagicStack/asyncpg?label=latest)](https://github.com/MagicStack/asyncpg) +[![](https://img.shields.io/badge/example-snippet-darkcyan)](#asyncpg) +``` +::: + +:::{sd-row} +```{sd-item} Python +``` +```{sd-item} +[psycopg3](https://www.psycopg.org/psycopg3/docs/) +``` +```{sd-item} +For connecting to CrateDB from Python, supporting Python's `asyncio`. +``` +```{sd-item} +[![](https://img.shields.io/github/v/tag/psycopg/psycopg?label=latest)](https://github.com/psycopg/psycopg) +[![](https://img.shields.io/badge/example-snippet-darkcyan)](#psycopg3) +``` +::: + +:::: + + +## HTTP +The drivers listed in this section all use the [CrateDB HTTP interface]. + +::::{sd-table} +:widths: 2 3 5 2 +:row-class: top-border + +:::{sd-row} +```{sd-item} +``` +```{sd-item} **Driver/Adapter** +``` +```{sd-item} **Description** +``` +```{sd-item} **Info** +``` +::: + +:::{sd-row} +```{sd-item} MicroPython +``` +```{sd-item} +[micropython-cratedb](https://github.com/crate/micropython-cratedb) +``` +```{sd-item} +A MicroPython library connecting to the CrateDB HTTP API. +``` +```{sd-item} +[![](https://img.shields.io/github/v/tag/crate/micropython-cratedb?label=latest)](https://github.com/crate/micropython-cratedb) +``` +::: + +:::{sd-row} +```{sd-item} Node.js +``` +```{sd-item} +[node-crate](https://www.npmjs.com/package/node-crate) +``` +```{sd-item} +A JavaScript library connecting to the CrateDB HTTP API. +``` +```{sd-item} +[![](https://img.shields.io/github/v/tag/megastef/node-crate?label=latest)](https://github.com/megastef/node-crate) +[![](https://img.shields.io/badge/example-application-darkcyan)](https://github.com/crate/devrel-shipping-forecast-geo-demo) +``` +::: + +:::{sd-row} +```{sd-item} PHP +``` +```{sd-item} +[CrateDB PDO driver](https://cratedb.com/docs/pdo/) +``` +```{sd-item} +For connecting to CrateDB from PHP. +``` +```{sd-item} +[![](https://img.shields.io/github/v/tag/crate/crate-pdo?label=latest)](https://github.com/crate/crate-pdo) +[![](https://img.shields.io/badge/example-snippet-darkcyan)](#php) +``` +::: + +:::{sd-row} +```{sd-item} PHP +``` +```{sd-item} +[CrateDB DBAL adapter](https://cratedb.com/docs/dbal/) +``` +```{sd-item} +For connecting to CrateDB from PHP, using DBAL and Doctrine. +``` +```{sd-item} +[![](https://img.shields.io/github/v/tag/crate/crate-dbal?label=latest)](https://github.com/crate/crate-dbal) +[![](https://img.shields.io/badge/example-snippet-darkcyan)](#php) +``` +::: + +:::{sd-row} +```{sd-item} Python +``` +```{sd-item} +[CrateDB Python driver](https://cratedb.com/docs/python/) +``` +```{sd-item} +For connecting to CrateDB from Python. Has support for [CrateDB BLOBs]. +``` +```{sd-item} +[![](https://img.shields.io/github/v/tag/crate/crate-python?label=latest)](https://github.com/crate/crate-python) +[![](https://img.shields.io/badge/docs-by%20example-darkgreen)][python-dbapi-by-example] +[![](https://img.shields.io/badge/example-snippet-darkcyan)](#crate-python) +``` +::: + +:::{sd-row} +```{sd-item} Python +``` +```{sd-item} +[SQLAlchemy dialect](https://cratedb.com/docs/sqlalchemy-cratedb/) +``` +```{sd-item} +For connecting to CrateDB from Python, using SQLAlchemy. +``` +```{sd-item} +[![](https://img.shields.io/github/v/tag/crate/sqlalchemy-cratedb?label=latest)](https://github.com/crate/sqlalchemy-cratedb) +[![](https://img.shields.io/badge/docs-by%20example-darkgreen)][python-sqlalchemy-by-example] +[![](https://img.shields.io/badge/example-snippet-darkcyan)](#sqlalchemy-cratedb) +``` +::: + +:::{sd-row} +```{sd-item} Ruby +``` +```{sd-item} +[CrateDB Ruby driver](https://github.com/crate/crate_ruby) +``` +```{sd-item} +A Ruby client library for CrateDB. +``` +```{sd-item} +[![](https://img.shields.io/github/v/tag/crate/crate_ruby?label=latest)](https://github.com/crate/crate_ruby) +[![](https://img.shields.io/badge/example-snippet-darkcyan)](#ruby) +[![](https://img.shields.io/badge/example-runnable-darkcyan)](https://github.com/crate/cratedb-examples/tree/main/by-language/ruby) +``` +::: + +:::{sd-row} +```{sd-item} Ruby +``` +```{sd-item} +[CrateDB ActiveRecord adapter](https://github.com/crate/activerecord-crate-adapter) +``` +```{sd-item} +Ruby on Rails ActiveRecord adapter for CrateDB. +``` +```{sd-item} +[![](https://img.shields.io/github/v/tag/crate/activerecord-crate-adapter?label=latest)](https://github.com/crate/activerecord-crate-adapter) +``` +::: + +:::: + + +```{tip} +Please visit the [](#build-status) page for an overview about the integration +status of the client drivers listed above, and more. +``` diff --git a/docs/connect/index.md b/docs/connect/index.md new file mode 100644 index 00000000..f05713bf --- /dev/null +++ b/docs/connect/index.md @@ -0,0 +1,190 @@ +(connect)= + +# Connect + +This documentation section is about connecting your applications to CrateDB +and CrateDB Cloud, using database drivers, compatibility adapters and dialects. + +:::{include} /_include/links.md +::: + +:::::{grid} +:padding: 0 + +::::{grid-item} +:class: rubric-slimmer +:columns: auto 9 9 9 + + +:::{rubric} Overview +::: +CrateDB connectivity options at a glance. + +You have a variety of options to connect to CrateDB, and to integrate it with +off-the-shelf, 3rd-party, open-source, and proprietary applications. + +:::{rubric} Protocol Support +::: +CrateDB supports both the HTTP protocol and the PostgreSQL wire protocol, +which ensures that many clients that work with PostgreSQL, will also work with +CrateDB. Through corresponding drivers, CrateDB is compatible with ODBC, +JDBC, and other database API specifications. By supporting SQL, CrateDB is +compatible with many standard database environments out of the box. + +While it is generally recommended to use the PostgreSQL interface (PG) for maximum +compatibility in PostgreSQL environments, the HTTP interface supports CrateDB +bulk operations and CrateDB BLOBs, which are not supported by the PostgreSQL +protocol. + +The HTTP protocol can also be used to connect from environments where +PostgreSQL-based communication is not applicable. + +:::{rubric} Details +::: + +CrateDB provides popular connectivity options with database drivers, +applications, and frameworks. + +To learn more, please refer to the documentation sections and hands-on +tutorials about supported client drivers, libraries, and frameworks, +and how to configure and use them with CrateDB optimally. +:::: + + +::::{grid-item} +:class: rubric-slim +:columns: auto 3 3 3 + +```{rubric} Reference Manual +``` +- [HTTP interface] +- [PostgreSQL interface] +- [SQL query syntax] +- [Bulk operations] +- [BLOBs] + +```{rubric} Protocols and API Standards +``` +- [HTTP protocol] +- [PostgreSQL wire protocol] +- [JDBC] +- [ODBC] +- [SQL] +:::: + +::::: + + +::::{grid} 2 2 2 4 +:padding: 0 + +:::{grid-item-card} +:columns: 12 +:link: connect-configure +:link-type: ref +:link-alt: Configure CrateDB connection string +:padding: 3 +:class-card: sd-pt-1 +:class-footer: text-smaller + +Configure database URI ++++ +In order to connect to CrateDB, your application or driver needs to be configured +with corresponding connection properties. Please note that different applications +and drivers may obtain connection properties in different formats. +::: + +:::{grid-item-card} +:columns: 12 +:link: connect-drivers +:link-type: ref +:link-alt: All drivers suitable for CrateDB +:padding: 3 +:class-card: sd-pt-1 +:class-footer: text-smaller + +All drivers ++++ +Drivers and adapters for supported programming languages, +frameworks, and environments. +::: + +:::{grid-item-card} +:link: connect-java +:link-type: ref +:link-alt: Connect using Java +:padding: 3 +:text-align: center +:class-card: sd-pt-1 + +Java +::: + +:::{grid-item-card} +:link: connect-javascript +:link-type: ref +:link-alt: Connect using JavaScript +:padding: 3 +:text-align: center +:class-card: sd-pt-1 + +JavaScript +::: + +:::{grid-item-card} +:link: connect-python +:link-type: ref +:link-alt: Connect using Python +:padding: 3 +:text-align: center +:class-card: sd-pt-1 + +Python +::: + +:::: + + +```{toctree} +:maxdepth: 1 +:hidden: + +configure +Drivers +CLI +DataFrame +ORM +``` + +```{toctree} +:maxdepth: 1 +:hidden: + +java +javascript +php +python +ruby +``` + + +[ADBC]: https://arrow.apache.org/docs/format/ADBC.html +[Authentication]: inv:crate-reference:*:label#admin_auth +[BLOBs]: inv:crate-reference:*:label#blob_support +[Bulk operations]: inv:crate-reference:*:label#http-bulk-ops +[CrateDB Examples]: https://github.com/crate/cratedb-examples +[CrateDB HTTP interface]: inv:crate-reference:*:label#interface-http +[CrateDB PostgreSQL interface]: inv:crate-reference:*:label#interface-postgresql +[HTTP interface]: inv:crate-reference:*:label#interface-http +[HTTP protocol]: https://en.wikipedia.org/wiki/HTTP +[JDBC]: https://en.wikipedia.org/wiki/Java_Database_Connectivity +[ODBC]: https://en.wikipedia.org/wiki/Open_Database_Connectivity +[PostgreSQL interface]: inv:crate-reference:*:label#interface-postgresql +[PostgreSQL wire protocol]: https://www.postgresql.org/docs/current/protocol.html +[python-dbapi-by-example]: inv:crate-python:*:label#by-example +[python-sqlalchemy-by-example]: inv:sqlalchemy-cratedb:*:label#by-example +[schema]: inv:crate-reference:*:label#ddl-create-table-schemas +[schemas]: inv:crate-reference:*:label#ddl-create-table-schemas +[SQL]: https://en.wikipedia.org/wiki/Sql +[SQL query syntax]: inv:crate-reference:*:label#sql +[superuser]: inv:crate-reference:*:label#administration_user_management diff --git a/docs/connect/java.rst b/docs/connect/java.rst new file mode 100644 index 00000000..f1854dc5 --- /dev/null +++ b/docs/connect/java.rst @@ -0,0 +1,41 @@ +.. _connect-java: + +==== +Java +==== + +JDBC is a standard Java API that provides a common interface for accessing +databases in Java. + +Example method used in implementation: + +.. code-block:: java + + import java.sql.*; + import java.util.Properties; + + public class Main { + public static void main(String[] args) { + try { + Properties properties = new Properties(); + properties.put("user", "admin"); + properties.put("password", ""); + properties.put("ssl", true); + Connection conn = DriverManager.getConnection( + "jdbc:crate://.cratedb.net:5432/", + properties + ); + + Statement statement = conn.createStatement(); + ResultSet resultSet = statement.executeQuery("SELECT name FROM sys.cluster"); + resultSet.next(); + String name = resultSet.getString("name"); + + System.out.println(name); + } catch (SQLException e) { + e.printStackTrace(); + } + } + } + +See full documentation :ref:`here `. \ No newline at end of file diff --git a/docs/connect/javascript.rst b/docs/connect/javascript.rst new file mode 100644 index 00000000..e4d9129d --- /dev/null +++ b/docs/connect/javascript.rst @@ -0,0 +1,61 @@ +.. _connect-javascript: + +========== +JavaScript +========== + +This section provides a quick overview of available Node.js modules and +drivers for CrateDB Cloud. + +node-postgres +------------- + +node-postgres is a collection of Node.js modules for interfacing with a CrateDB +Cloud database. + +Example implementation will look like this: + +.. code-block:: javascript + + const { Client } = require("pg"); + + const crateClient = new Client({ + host: ".cratedb.net", + port: 5432, + user: "admin", + password: "", + ssl: true, + }); + + (async () => { + await crateClient.connect(); + const result = await crateClient.query("SELECT name FROM sys.cluster"); + console.log(result.rows[0]); + })(); + +For more information see `node-postgres documentation`_. + +node-crate +---------- + +node-crate is an independent Node.js driver driver for CrateDB that communicates via +the `_sql` HTTP endpoint. + +Example implementation will look like this: + +.. code-block:: javascript + + const crate = require("node-crate"); + + crate.connect(`https://admin:${encodeURIComponent("")}@.cratedb.net:4200`); + + (async () => { + const result = await crate.execute("SELECT name FROM sys.cluster"); + console.log(result.rows[0]); + })(); + + +For more information see `node-crate documentation`_. + +.. _node-postgres documentation: https://www.npmjs.com/package/pg +.. _node-crate documentation: https://www.npmjs.com/package/node-crate diff --git a/docs/connect/orm.md b/docs/connect/orm.md new file mode 100644 index 00000000..6096e099 --- /dev/null +++ b/docs/connect/orm.md @@ -0,0 +1,31 @@ +(orm)= +# CrateDB and ORM libraries + +CrateDB provides dialect support for the following ORM libraries and +frameworks. + +## SQLAlchemy + +[SQLAlchemy] is the Python SQL toolkit and Object Relational Mapper that +gives application developers the full power and flexibility of SQL. + +Python-based [DataFrame](df/index.md) +and [ML](#machine-learning) libraries, and a few [ETL](#etl) +frameworks, are using SQLAlchemy as database adapter library when connecting to +[RDBMS]. + +```{div} +:style: "clear: both" +``` + +- [The CrateDB SQLAlchemy Dialect] +- [Working with SQLAlchemy and CrateDB] +- [SQLAlchemy Code Examples] + + + +[RDBMS]: https://en.wikipedia.org/wiki/RDBMS +[SQLAlchemy]: https://www.sqlalchemy.org/ +[SQLAlchemy Code Examples]: https://github.com/crate/cratedb-examples/tree/main/by-language/python-sqlalchemy +[The CrateDB SQLAlchemy Dialect]: inv:sqlalchemy-cratedb:*:label#index +[Working with SQLAlchemy and CrateDB]: inv:sqlalchemy-cratedb:*:label#by-example diff --git a/docs/connect/php.rst b/docs/connect/php.rst new file mode 100644 index 00000000..31f8acbb --- /dev/null +++ b/docs/connect/php.rst @@ -0,0 +1,70 @@ +.. _connect-php: + +=== +PHP +=== + +This section provides a quick overview of available PHP extensions for CrateDB +Cloud. + +PDO +--- + +The PHP Data Objects (PDO) is a standard PHP extension that defines a common +interface for accessing databases in PHP. + +Example implementation will look like this: + +.. code-block:: php + + .cratedb.net:4200', + 'admin', + '' + ); + + $stm = $pdo->query('SELECT name FROM sys.cluster'); + $name = $stm->fetch(); + print $name[0]; + + ?> + +See full documentation :ref:`here `. + +DBAL +---- + +DBAL is a PHP database abstraction layer that comes with database schema +introspection, schema management, and PDO support. + +Example implementation will look like this: + +.. code-block:: php + + 'Crate\DBAL\Driver\PDOCrate\Driver', + 'user' => 'admin', + 'password' => '', + 'host' => '.cratedb.net', + 'port' => 4200 + ); + + $connection = \Doctrine\DBAL\DriverManager::getConnection($params); + $sql = 'SELECT name FROM sys.cluster'; + $name = $connection->query($sql)->fetch(); + + print $name['name']; + + ?> + +See full documentation :ref:`here `. diff --git a/docs/connect/python.rst b/docs/connect/python.rst new file mode 100644 index 00000000..2255691c --- /dev/null +++ b/docs/connect/python.rst @@ -0,0 +1,163 @@ +.. _connect-python: + +====== +Python +====== + +This guide demonstrates how to connect to a CrateDB Cloud cluster using different +kinds of Python drivers. Individual drivers offer specific features for specific +needs of your application, so consider reading this enumeration carefully. + +.. _crate-python: + +crate-python +------------ + +The ``crate`` Python package offers a database client implementation compatible +with the Python Database API 2.0 specification, and also includes the CrateDB +SQLAlchemy dialect. See the full documentation :ref:`here `. +The package can be installed using ``pip install crate``. + +.. code-block:: python + + from crate import client + + conn = client.connect("https://.cratedb.net:4200", username="admin", password="", verify_ssl_cert=True) + + with conn: + cursor = conn.cursor() + cursor.execute("SELECT * FROM sys.summits") + result = cursor.fetchone() + print(result) + +.. _sqlalchemy-cratedb: + +sqlalchemy-cratedb +------------------ + +The `SQLAlchemy`_ dialect for CrateDB, based on the HTTP-based DBAPI client +library `crate-python`_. +See the full documentation :ref:`here `. +The package can be installed using ``pip install sqlalchemy-cratedb``. + +.. code-block:: python + + import sqlalchemy as sa + + engine = sa.create_engine("crate://localhost:4200", echo=True) + connection = engine.connect() + + result = connection.execute(sa.text("SELECT * FROM sys.summits;")) + for record in result.all(): + print(record) + +.. _psycopg2: + +psycopg2 +-------- + +Psycopg is a popular PostgreSQL database adapter for Python. Its main features +are the complete implementation of the Python DB API 2.0 specification and the +thread safety (several threads can share the same connection). +For more information, see the `psycopg documentation`_. + +.. code-block:: python + + import psycopg2 + + conn = psycopg2.connect(host=".cratedb.net", port=5432, user="admin", password="", sslmode="require") + + with conn: + with conn.cursor() as cursor: + cursor.execute("SELECT * FROM sys.summits") + result = cursor.fetchone() + print(result) + +.. _psycopg3: + +psycopg3 +-------- + +`Psycopg 3`_ is a newly designed PostgreSQL database adapter for the Python +programming language. Psycopg 3 presents a familiar interface for everyone who +has used Psycopg 2 or any other DB-API 2.0 database adapter, but allows to use +more modern PostgreSQL and Python features, such as: + +- Asynchronous support +- COPY support from Python objects +- A redesigned connection pool +- Support for static typing +- Server-side parameters binding +- Prepared statements +- Statements pipeline +- Binary communication +- Direct access to the libpq functionalities + +.. code-block:: python + + import psycopg + + with psycopg.connect("postgres://crate@localhost:5432/doc") as conn: + with conn.cursor() as cursor: + cursor.execute("SELECT * FROM sys.summits") + for record in cursor: + print(record) + +.. _aiopg: + +aiopg +----- + +aiopg is a python library for accessing a PostgreSQL database from the asyncio +(PEP-3156/tulip) framework. It wraps asynchronous features of the Psycopg +database driver. +For more information, see the `aiopg documentation`_. + +.. code-block:: python + + import asyncio + import aiopg + + async def run(): + async with aiopg.create_pool(host=".cratedb.net", port=5432, user="admin", password="", sslmode="require") as pool: + async with pool.acquire() as conn: + async with conn.cursor() as cursor: + await cursor.execute("SELECT * FROM sys.summits") + result = await cursor.fetchone() + print(result) + + loop = asyncio.get_event_loop() + loop.run_until_complete(run()) + +.. _asyncpg: + +asyncpg +------- + +asyncpg is a database interface library designed specifically for PostgreSQL +and Python/asyncio. asyncpg is an efficient, clean implementation of the +PostgreSQL server binary protocol for use with Python's asyncio framework. +For more information, see the `asyncpg documentation`_. + +.. code-block:: python + + import asyncio + import asyncpg + + async def run(): + conn = await asyncpg.connect(host=".cratedb.net", port=5432, user="admin", password="", ssl=True) + try: + result = await conn.fetch("SELECT * FROM sys.summits") + finally: + await conn.close() + print(result) + + loop = asyncio.get_event_loop() + loop.run_until_complete(run()) + + +.. _aiopg documentation: https://aiopg.readthedocs.io/ +.. _asyncpg documentation: https://magicstack.github.io/asyncpg/current/ +.. _psycopg documentation: https://www.psycopg.org/docs/ +.. _Psycopg 3: https://www.psycopg.org/psycopg3/docs/ +.. _SQLAlchemy: https://www.sqlalchemy.org/ diff --git a/docs/connect/ruby.rst b/docs/connect/ruby.rst new file mode 100644 index 00000000..0e8bb66d --- /dev/null +++ b/docs/connect/ruby.rst @@ -0,0 +1,26 @@ +.. _connect-ruby: + +==== +Ruby +==== + +This section shows an example implementation of Ruby client library for CrateDB +Cloud: + +.. code-block:: ruby + + require 'crate_ruby' + + + client = CrateRuby::Client.new( + servers: [".cratedb.net:4200"], + username: "admin", + password: "", + ssl: true + ) + result = client.execute("SELECT name FROM sys.cluster") + p result.to_a + +For additional information see `our GitHub documentation`_. + +.. _our GitHub documentation: https://github.com/crate/crate_ruby/blob/main/README.rst diff --git a/docs/feature/connectivity/index.md b/docs/feature/connectivity/index.md deleted file mode 100644 index ad7a7698..00000000 --- a/docs/feature/connectivity/index.md +++ /dev/null @@ -1,149 +0,0 @@ -(connect)= -(connectivity)= - -# Connectivity - -:::{include} /_include/links.md -::: - -:::::{grid} -:padding: 0 - -::::{grid-item} -:class: rubric-slimmer -:columns: auto 9 9 9 - - -:::{rubric} Overview -::: -CrateDB connectivity options at a glance. - -You have a variety of options to connect to CrateDB, and to integrate it with -off-the-shelve, 3rd-party, open-source, and proprietary applications. - -:::{rubric} About -::: -CrateDB supports both the HTTP protocol and the PostgreSQL wire protocol, -which ensures that many clients that work with PostgreSQL, will also work with -CrateDB. - -Through corresponding drivers, CrateDB is compatible with JDBC, ODBC, -and other database API specifications. -By supporting SQL, CrateDB is compatible with many standard database -environments out of the box. - -:::{rubric} Details -::: - -CrateDB provides plenty of connectivity options with database drivers, -applications, and frameworks, in order to get time series data in and -out of CrateDB, and to connect to other applications. - -To learn more, please refer to the documentation sections and hands-on -tutorials about supported client drivers, libraries, and frameworks, -and how to configure and use them with CrateDB optimally. -:::: - - -::::{grid-item} -:class: rubric-slim -:columns: auto 3 3 3 - -```{rubric} Reference Manual -``` -- [HTTP interface] -- [PostgreSQL interface] -- [SQL query syntax] - -```{rubric} Protocols and API Standards -``` -- [HTTP protocol] -- [PostgreSQL wire protocol] -- [JDBC] -- [ODBC] -- [SQL] -:::: - -::::: - - -## Synopsis - -In order to provide a CrateDB instance for testing purposes, use, for -example, Docker. -```shell -docker run --rm -it --publish=4200:4200 --publish=5432:5432 crate/crate:nightly -``` -:::{tip} -The [CrateDB Examples] repository also includes a collection of -clear and concise examples how to connect to and work with CrateDB, -using different environments, applications, or frameworks. -::: - - -## Learn - -::::{grid} 2 3 3 3 -:padding: 0 - -:::{grid-item-card} Ecosystem Catalog -:link: catalog -:link-type: ref -:link-alt: Ecosystem Catalog -:padding: 3 -:class-card: sd-pt-3 -:class-title: sd-fs-5 -:class-body: sd-text-center -:class-footer: text-smaller -{material-outlined}`category;3.5em` -+++ -Explore all open-source and partner applications and solutions. -::: - -:::{grid-item-card} Driver Support -:link: crate-clients-tools:connect -:link-type: ref -:link-alt: Driver Support -:padding: 3 -:class-card: sd-pt-3 -:class-title: sd-fs-5 -:class-body: sd-text-center -:class-footer: text-smaller -{material-outlined}`link;3.5em` -+++ -List of HTTP and PostgreSQL client drivers, and tutorials. -::: - -:::{grid-item-card} Integration Tutorials -:link: integrate -:link-type: ref -:link-alt: Integration Tutorials -:padding: 3 -:class-card: sd-pt-3 -:class-title: sd-fs-5 -:class-body: sd-text-center -:class-footer: text-smaller -{material-outlined}`local_library;3.5em` -+++ -Learn how to integrate CrateDB with applications and tools. -::: - -:::: - - - -[CrateDB Examples]: https://github.com/crate/cratedb-examples -[Driver Support]: inv:crate-clients-tools:*:label#connect -[Ecosystem Catalog]: inv:crate-clients-tools:*:label#index -[HTTP interface]: inv:crate-reference:*:label#interface-http -[HTTP protocol]: https://en.wikipedia.org/wiki/HTTP -[JDBC]: https://en.wikipedia.org/wiki/Java_Database_Connectivity -[ODBC]: https://en.wikipedia.org/wiki/Open_Database_Connectivity -[PostgreSQL interface]: inv:crate-reference:*:label#interface-postgresql -[PostgreSQL wire protocol]: https://www.postgresql.org/docs/current/protocol.html -[SQL]: https://en.wikipedia.org/wiki/Sql -[SQL query syntax]: inv:crate-reference:*:label#sql - - -```{include} /_include/styles.html -``` diff --git a/docs/feature/index.md b/docs/feature/index.md index e00232d0..e9fc0117 100644 --- a/docs/feature/index.md +++ b/docs/feature/index.md @@ -18,7 +18,6 @@ All features of CrateDB at a glance. :maxdepth: 1 sql/index -connectivity/index document/index relational/index Search: FTS, Geo, Vector, Hybrid @@ -75,7 +74,7 @@ landscapes. :gutter: 2 :::{grid-item-card} {material-outlined}`link;2em` Connectivity -:link: connectivity +:link: connect :link-type: ref :link-alt: About connection options with CrateDB diff --git a/docs/index.md b/docs/index.md index 9d6b01cf..cbad059a 100644 --- a/docs/index.md +++ b/docs/index.md @@ -305,6 +305,7 @@ use-cases. :hidden: install/index +connect/index getting-started feature/index admin/index diff --git a/docs/install/index.rst b/docs/install/index.rst index b9d02d77..8a160563 100644 --- a/docs/install/index.rst +++ b/docs/install/index.rst @@ -1,8 +1,8 @@ .. _install: -############ -Installation -############ +####### +Install +####### This section of the documentation covers the installation of CrateDB on different kinds of operating systems and environments, both suitable for on-premises and diff --git a/docs/integrate/index.md b/docs/integrate/index.md index 1564e96a..7b5179c9 100644 --- a/docs/integrate/index.md +++ b/docs/integrate/index.md @@ -13,8 +13,6 @@ optimally. :maxdepth: 2 ide -orm -df etl/index cdc/index mcp/index diff --git a/docs/integrate/orm.md b/docs/integrate/orm.md deleted file mode 100644 index 8a81de9e..00000000 --- a/docs/integrate/orm.md +++ /dev/null @@ -1,14 +0,0 @@ -(orm)= -# ORM Libraries - -How to use CrateDB together with popular open-source ORM libraries. - -## SQLAlchemy -- [The CrateDB SQLAlchemy Dialect] -- [Working with SQLAlchemy and CrateDB] -- [SQLAlchemy Code Examples] - - -[SQLAlchemy Code Examples]: https://github.com/crate/cratedb-examples/tree/main/by-language/python-sqlalchemy -[The CrateDB SQLAlchemy Dialect]: inv:sqlalchemy-cratedb:*:label#index -[Working with SQLAlchemy and CrateDB]: inv:sqlalchemy-cratedb:*:label#by-example