Integrate/AMQP: Add entry point page, micro tutorial, and category item

Author: amotl

docs/ingest/etl/index.md

@@ -132,6 +132,14 @@ Load data from database systems.
 
 
 ::::{grid-item-card} {material-outlined}`fast_forward;2em` Streams
+
+- {ref}`amqp`
+
+  The AMQP protocol is an open standard application layer protocol for
+  message-oriented middleware. The defining features of AMQP are message
+  orientation, queuing, routing (including point-to-point and
+  publish-and-subscribe), reliability, and security.
+
 - {ref}`kafka`
 
   Apache Kafka is an open-source distributed event streaming platform

docs/integrate/amqp/index.md

@@ -0,0 +1,53 @@
+(amqp)=
+# AMQP
+
+```{div} .float-right
+[![AMQP logo](https://www.cleo.com/sites/default/files/styles/desktop_664_270_scale/public/2023-12/amqp-logo.png.webp){width=180px loading=lazy}][AMQP]
+```
+```{div} .clearfix
+```
+
+:::{rubric} About
+:::
+
+The [AMQP] protocol is an open standard application layer protocol for
+message-oriented middleware. The defining features of AMQP are message
+orientation, queuing, routing (including point-to-point and
+publish-and-subscribe), reliability, and security.
+
+:::{rubric} Synopsis
+:::
+
+Use LorryStream to receive JSON data from an AMQP queue, continuously loading
+records into CrateDB.
+```shell
+uvx --from=lorrystream lorry relay \
+    "amqp://guest:guest@localhost:5672/%2F?queue=testdrive&content-type=json" \
+    "crate://localhost/?table=testdrive"
+```
+
+:::{rubric} Learn
+:::
+
+[LorryStream] is a lightweight and polyglot stream-processing library, used as a
+data backplane, message relay, or pipeline subsystem.
+
+::::{grid}
+
+:::{grid-item-card} Tutorial: Use LorryStream
+:link: amqp-tutorial
+:link-type: ref
+How to load data from AMQP into CrateDB using LorryStream.
+:::
+
+::::
+
+:::{toctree}
+:maxdepth: 1
+:hidden:
+Tutorial <tutorial>
+:::
+
+
+[LorryStream]: https://lorrystream.readthedocs.io/
+[AMQP]: https://www.amqp.org/

docs/integrate/amqp/tutorial.md

@@ -0,0 +1,96 @@
+(amqp-tutorial)=
+
+# Load data from an AMQP queue into CrateDB
+
+The tutorial will walk you through starting the [RabbitMQ] AMQP broker
+and CrateDB, publishing JSON data to an AMQP queue, consuming and relaying
+it into a CrateDB table continuously, and validating that the data has
+been stored successfully.
+The data transfer is supported by the [LorryStream AMQP source] data
+pipeline element.
+
+## Prerequisites
+
+Docker is used for running all components. This approach works consistently
+across Linux, macOS, and Windows.
+
+Alternatively, you can use Podman. You can also use a different AMQP broker such as
+Apache Qpid, Apache ActiveMQ, IBM MQ, or Solace. Azure Event Hubs and Azure Service
+Bus speak AMQP as well, but with protocol and authentication specifics; adjust
+settings accordingly.
+
+Create a shared network.
+```shell
+docker network create cratedb-demo
+```
+
+Start CrateDB.
+```shell
+docker run --name=cratedb --rm --network=cratedb-demo \
+  --publish=4200:4200 --publish=5432:5432 \
+  --env=CRATE_HEAP_SIZE=2g docker.io/crate -Cdiscovery.type=single-node
+```
+
+Start RabbitMQ.
+```shell
+docker run --name=rabbitmq --rm --network=cratedb-demo \
+  --publish=5672:5672 docker.io/rabbitmq:3
+```
+> Note: This broker configuration allows anonymous access for demonstration purposes only.
+> Do not expose it to untrusted networks. For production, configure authentication/TLS.
+
+Prepare shortcuts for the CrateDB shell, LorryStream, and the AMQP client
+programs.
+
+::::{tab-set}
+
+:::{tab-item} Linux and macOS
+To make the settings persistent, add them to your shell profile (e.g., `~/.profile` or `~/.zshrc`).
+```shell
+alias crash="docker run --rm -it --network=cratedb-demo ghcr.io/crate/cratedb-toolkit crash"
+alias lorry="docker run --rm -i --network=cratedb-demo ghcr.io/daq-tools/lorrystream lorry"
+alias amqpcat="docker run --rm -i --network=cratedb-demo docker.io/cloudamqp/amqpcat amqpcat"
+```
+:::
+:::{tab-item} Windows PowerShell
+To make the settings persistent, add them to your PowerShell profile (`$PROFILE`).
+```powershell
+function crash { docker run --rm -it --network=cratedb-demo ghcr.io/crate/cratedb-toolkit crash @args }
+function lorry { docker run --rm -i --network=cratedb-demo ghcr.io/daq-tools/lorrystream lorry @args }
+function amqpcat { docker run --rm -i --network=cratedb-demo docker.io/cloudamqp/amqpcat amqpcat @args }
+```
+:::
+:::{tab-item} Windows Command
+```shell
+doskey crash=docker run --rm -it --network=cratedb-demo ghcr.io/crate/cratedb-toolkit crash $*
+doskey lorry=docker run --rm -i --network=cratedb-demo ghcr.io/daq-tools/lorrystream lorry $*
+doskey amqpcat=docker run --rm -i --network=cratedb-demo docker.io/cloudamqp/amqpcat amqpcat $*
+```
+:::
+
+::::
+
+## Usage
+
+Invoke the data transfer pipeline.
+```shell
+lorry relay \
+  "amqp://guest:guest@rabbitmq:5672/%2F?exchange=default&queue=default&routing-key=testdrive&setup=exchange,queue,bind&content-type=json" \
+  "crate://cratedb/?table=testdrive"
+```
+
+Publish a JSON message to AMQP.
+```shell
+echo '{"temperature": 42.84, "humidity": 83.1}' | \
+  amqpcat --producer --uri='amqp://guest:guest@rabbitmq:5672/%2F' \
+    --exchange=default --queue=default --routing-key=testdrive
+```
+
+Inspect data stored in CrateDB.
+```shell
+crash --hosts cratedb -c "SELECT * FROM testdrive"
+```
+
+
+[LorryStream AMQP source]: https://lorrystream.readthedocs.io/source/amqp.html
+[RabbitMQ]: https://www.rabbitmq.com/

docs/integrate/index.md

@@ -17,6 +17,7 @@ Please also visit the [Overview of CrateDB integration tutorials].
 :maxdepth: 1
 
 airflow/index
+amqp/index
 aws-lambda/index
 azure-functions/index
 cluvio/index