Skip to content

Commit b8b144e

Browse files
committed
ETL/CDC: Implement suggestions by CodeRabbit
1 parent e07f378 commit b8b144e

File tree

51 files changed

+453
-518
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

51 files changed

+453
-518
lines changed

docs/_include/links.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
<!-- markdownlint-disable MD034 -->
12
<!-- markdownlint-disable MD053 -->
23

34
[Amazon DynamoDB Streams]: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Streams.html

docs/_static/css/custom.css

Lines changed: 19 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,15 @@ TODO: Upstream to crate-docs-theme.
55
*/
66

77
/* General */
8+
.float-right {
9+
float: right;
10+
}
11+
.clearfix {
12+
clear: both;
13+
}
14+
.text-right {
15+
text-align: right;
16+
}
817
/*
918
.sd-card-body {
1019
line-height: 1.1em;
@@ -37,7 +46,7 @@ TODO: Upstream to crate-docs-theme.
3746
}
3847
.no-margin > * {
3948
margin-top: 0 !important;
40-
margin-bottom: 0 !important;;
49+
margin-bottom: 0 !important;
4150
}
4251

4352

@@ -52,6 +61,14 @@ TODO: Upstream to crate-docs-theme.
5261
}
5362

5463
/* When a logo wants to be displayed next to the title */
64+
5565
#main-content h1 {
56-
display: inline-block;
66+
display: inline-flex;
67+
align-items: center;
68+
gap: 0.5rem; /* space between logo and title */
69+
}
70+
/* Optional: ensure logos inside h1 scale and align nicely */
71+
#main-content h1 img {
72+
vertical-align: middle;
73+
max-height: 2rem; /* adjust as needed per design */
5774
}

docs/ingest/cdc/index.md

Lines changed: 7 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -5,15 +5,13 @@
55
:::
66

77
:::{div}
8-
Options to connect and integrate CrateDB with third-party
9-
CDC applications, mostly using [CrateDB's PostgreSQL interface].
10-
CrateDB also provides native adapter components to leverage advanced
11-
features.
12-
13-
This documentation section lists CDC applications,
14-
frameworks, and solutions, which can be used together with CrateDB,
15-
and outlines how to use them optimally.
16-
Additionally, see support for {ref}`generic ETL <etl>` solutions.
8+
Options to integrate CrateDB with third‑party CDC applications, typically via
9+
[CrateDB's PostgreSQL interface]. CrateDB also provides native adapter components
10+
to leverage advanced features.
11+
12+
This section lists ETL applications and frameworks that work with CrateDB, and
13+
outlines how to use them effectively. Additionally, see support for
14+
{ref}`generic ETL <etl>` solutions.
1715
:::
1816

1917

docs/ingest/etl/index.md

Lines changed: 8 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -7,15 +7,12 @@
77
:::
88

99
:::{div}
10-
Options to connect and integrate CrateDB with third-party
11-
ETL applications, mostly using [CrateDB's PostgreSQL interface].
12-
CrateDB also provides native adapter components to leverage advanced
13-
features.
14-
15-
This documentation section lists ETL applications and
16-
frameworks which can be used together with CrateDB, and outlines how
17-
to use them optimally.
18-
Additionally, see support for {ref}`cdc` solutions.
10+
Options to integrate CrateDB with third‑party ETL applications, typically via
11+
[CrateDB's PostgreSQL interface]. CrateDB also provides native adapter components
12+
to leverage advanced features.
13+
14+
This section lists ETL applications and frameworks that work with CrateDB, and
15+
outlines how to use them effectively. Additionally, see support for {ref}`cdc` solutions.
1916
:::
2017

2118

@@ -67,7 +64,7 @@ Additionally, see support for {ref}`cdc` solutions.
6764
specification, including powerful data extraction and consolidation elements.
6865

6966
+++
70-
Data pipeline programming frameworks and platforms.
67+
Use data pipeline programming frameworks and platforms.
7168
::::
7269

7370

@@ -95,7 +92,7 @@ Data pipeline programming frameworks and platforms.
9592
applications.
9693

9794
+++
98-
Visual data flow and integration frameworks and platforms.
95+
Use visual data flow and integration frameworks and platforms.
9996
::::
10097

10198

docs/integrate/apache-airflow/index.md

Lines changed: 11 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -6,12 +6,10 @@
66
:::{include} /_include/links.md
77
:::
88

9-
```{div}
10-
:style: "float: right"
11-
[![Apache Airflow logo](https://19927462.fs1.hubspotusercontent-na1.net/hub/19927462/hubfs/Partner%20Logos/392x140/Apache-Airflow-Logo-392x140.png?width=784&height=280&name=Apache-Airflow-Logo-392x140.png){h=60px}][Apache Airflow]
9+
```{div} .float-right
10+
[![Apache Airflow logo](https://19927462.fs1.hubspotusercontent-na1.net/hub/19927462/hubfs/Partner%20Logos/392x140/Apache-Airflow-Logo-392x140.png?width=784&height=280&name=Apache-Airflow-Logo-392x140.png){height=60px loading=lazy}][Apache Airflow]
1211
```
13-
```{div}
14-
:style: "clear: both"
12+
```{div} .clearfix
1513
```
1614

1715
:::{rubric} About
@@ -20,16 +18,16 @@
2018
:::{div}
2119
[Apache Airflow] is an open source software platform to programmatically author,
2220
schedule, and monitor workflows, written in Python.
23-
[Astronomer] offers managed Airflow services on the cloud of your choice, in
24-
order to run Airflow with less overhead.
21+
[Astronomer] offers managed Airflow services on the cloud of your choice, to
22+
run Airflow with less overhead.
2523
:::
2624

2725
:::{dropdown} **Details**
2826
Airflow has a modular architecture and uses a message queue to orchestrate an
2927
arbitrary number of workers. Pipelines are defined in Python, allowing for
3028
dynamic pipeline generation and on-demand, code-driven pipeline invocation.
3129

32-
Pipeline parametrization is using the powerful Jinja templating engine.
30+
Pipeline parameterization is using the powerful Jinja templating engine.
3331
To extend the system, you can define your own operators and extend libraries
3432
to fit the level of abstraction that suits your environment.
3533
:::
@@ -55,16 +53,16 @@ journey. Spend time where it counts.
5553
behind the project. Get world-class support, fast-tracked bug fixes, and same-day
5654
access to new Airflow versions.
5755

58-
```{div}
59-
:style: "clear: both"
56+
```{div} .clearfix
6057
```
58+
6159
:::
6260

6361

6462
:::{rubric} Learn: Starter Tutorials
6563
:::
6664

67-
::::{grid}
65+
::::{grid} 2
6866

6967
:::{grid-item-card} Tutorial: Import Parquet files
7068
:link: https://community.cratedb.com/t/automating-the-import-of-parquet-files-with-apache-airflow/1247
@@ -85,7 +83,7 @@ into CrateDB.
8583
:::{rubric} Learn: Advanced Tutorials
8684
:::
8785

88-
::::{grid}
86+
::::{grid} 3
8987

9088
:::{grid-item-card} Tutorial: Export to S3
9189
:link: https://community.cratedb.com/t/cratedb-and-apache-airflow-automating-data-export-to-s3/901
@@ -112,7 +110,7 @@ and cost-effectiveness.
112110

113111

114112
```{seealso}
115-
**Repository:** https://github.com/crate/cratedb-airflow-tutorial
113+
**Repository:** <https://github.com/crate/cratedb-airflow-tutorial>
116114
<br>
117115
**Product:** [CrateDB and Apache Airflow]
118116
<br>

docs/integrate/apache-flink/index.md

Lines changed: 6 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -5,15 +5,13 @@
55
:::{include} /_include/links.md
66
:::
77

8-
```{div}
9-
:style: "float: right; text-align: right"
10-
[![Apache Flink logo](https://flink.apache.org/flink-header-logo.svg){h=60px}][Apache Flink]
8+
```{div} .float-right .text-right
9+
[![Apache Flink logo](https://flink.apache.org/flink-header-logo.svg){height=60px loading=lazy}][Apache Flink]
1110
<br>
12-
<a href="https://github.com/crate/cratedb-examples/actions/workflows/framework-flink-kafka-java.yml" target="_blank">
13-
<img src="https://img.shields.io/github/actions/workflow/status/crate/cratedb-examples/framework-flink-kafka-java.yml?branch=main&label=Apache%20Kafka,%20Apache%20Flink" loading="lazy"></a>
11+
<a href="https://github.com/crate/cratedb-examples/actions/workflows/framework-flink-kafka-java.yml" target="_blank" rel="noopener noreferrer">
12+
<img src="https://img.shields.io/github/actions/workflow/status/crate/cratedb-examples/framework-flink-kafka-java.yml?branch=main&label=Apache%20Kafka,%20Apache%20Flink" loading="lazy" alt="CI status: Apache Kafka, Apache Flink"></a>
1413
```
15-
```{div}
16-
:style: "clear: both"
14+
```{div} .clearfix
1715
```
1816

1917
[Apache Flink] is a programming framework and distributed processing engine
@@ -33,7 +31,7 @@ Systems Award].
3331
![](https://flink.apache.org/img/flink-home-graphic.png){h=200px}
3432

3533
:::{dropdown} **Managed Flink**
36-
A few companies are specializing in offering managed Flink services.
34+
A few companies are specializing in offering managed Flink services.
3735

3836
- [Aiven] offers their managed [Aiven for Apache Flink] solution.
3937
- [Immerok Cloud]'s offering is being converged into [Flink managed by Confluent],

docs/integrate/apache-hop/index.md

Lines changed: 6 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,10 @@
11
(apache-hop)=
22
# Apache Hop
33

4-
```{div}
5-
:style: "float: right"
6-
[![Apache Hop logo](https://hop.apache.org/img/hop-logo.svg){h=60px}][Apache Hop]
4+
```{div} .float-right
5+
[![Apache Hop logo](https://hop.apache.org/img/hop-logo.svg){height=60px loading=lazy}][Apache Hop]
76
```
8-
```{div}
9-
:style: "clear: both"
7+
```{div} .clearfix
108
```
119

1210
:::{rubric} About
@@ -28,9 +26,9 @@ orchestration.
2826
- **Lifecycle Management** enables developers and administrators to switch between
2927
projects, environments and purposes without leaving your train of thought.
3028

31-
![](https://github.com/crate/crate-clients-tools/assets/453543/da6baf11-8430-4a0f-b2df-55717ce02802){h=120px}
32-
![](https://github.com/crate/crate-clients-tools/assets/453543/60cfc82a-db0a-49f1-8e26-a37b774b3614){h=120px}
33-
![](https://github.com/crate/crate-clients-tools/assets/453543/2bd59577-b664-45ae-a71e-36a130d36739){h=120px}
29+
![Apache Hop pipeline editor screenshot](https://github.com/crate/crate-clients-tools/assets/453543/da6baf11-8430-4a0f-b2df-55717ce02802){h=120px}
30+
![Apache Hop workflow view screenshot](https://github.com/crate/crate-clients-tools/assets/453543/60cfc82a-db0a-49f1-8e26-a37b774b3614){h=120px}
31+
![Apache Hop execution monitor screenshot](https://github.com/crate/crate-clients-tools/assets/453543/2bd59577-b664-45ae-a71e-36a130d36739){h=120px}
3432

3533
:::
3634

docs/integrate/apache-iceberg/index.md

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,10 @@
11
(apache-iceberg)=
22
# Apache Iceberg
33

4-
```{div}
5-
:style: "float: right"
6-
[![Apache Iceberg logo](https://iceberg.apache.org/assets/images/Iceberg-logo.svg){h=60px}][Apache Iceberg]
4+
```{div} .float-right
5+
[![Apache Iceberg logo](https://iceberg.apache.org/assets/images/Iceberg-logo.svg){height=60px loading=lazy}][Apache Iceberg]
76
```
8-
```{div}
9-
:style: "clear: both"
7+
```{div} .clearfix
108
```
119

1210
:::{rubric} About

docs/integrate/apache-kafka/index.md

Lines changed: 5 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,13 @@
11
(apache-kafka)=
22
# Apache Kafka
33

4-
```{div}
5-
:style: "float: right; text-align: right"
6-
[![Apache Kafka logo](https://kafka.apache.org/logos/kafka_logo--simple.png){h=60px}][Apache Kafka]
4+
```{div} .float-right .text-right
5+
[![Apache Kafka logo](https://kafka.apache.org/logos/kafka_logo--simple.png){height=60px loading=lazy}][Apache Kafka]
76
<br>
8-
<a href="https://github.com/crate/cratedb-examples/actions/workflows/framework-flink-kafka-java.yml" target="_blank">
9-
<img src="https://img.shields.io/github/actions/workflow/status/crate/cratedb-examples/framework-flink-kafka-java.yml?branch=main&label=Apache%20Kafka,%20Apache%20Flink" loading="lazy"></a>
7+
<a href="https://github.com/crate/cratedb-examples/actions/workflows/framework-flink-kafka-java.yml" target="_blank" rel="noopener noreferrer">
8+
<img src="https://img.shields.io/github/actions/workflow/status/crate/cratedb-examples/framework-flink-kafka-java.yml?branch=main&label=Apache%20Kafka,%20Apache%20Flink" loading="lazy" alt="CI status: Apache Kafka, Apache Flink"></a>
109
```
11-
```{div}
12-
:style: "clear: both"
10+
```{div} .clearfix
1311
```
1412

1513
:::{include} /_include/links.md

docs/integrate/apache-nifi/index.md

Lines changed: 8 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,10 @@
11
(apache-nifi)=
22
# Apache NiFi
33

4-
```{div}
5-
:style: "float: right"
6-
[![Apache NiFi logo](https://nifi.apache.org/images/apache-nifi-drop-logo.svg){h=60px}][Apache NiFi]
4+
```{div} .float-right
5+
[![Apache NiFi logo](https://nifi.apache.org/images/apache-nifi-drop-logo.svg){height=60px loading=lazy}][Apache NiFi]
76
```
8-
```{div}
9-
:style: "clear: both"
7+
```{div} .clearfix
108
```
119

1210
:::{rubric} About
@@ -20,18 +18,18 @@ worldwide across every industry.
2018
:::{dropdown} **Details**
2119

2220
- **Data provenance tracking**: Complete lineage of information from beginning to end.
23-
- **Extensive configuration**: Loss-tolerant and guaranteed delivery, Low latency and
21+
- **Extensive configuration**: Loss-tolerant and guaranteed delivery, low latency and
2422
high throughput, Dynamic prioritization, Runtime modification of flow configuration,
2523
Back pressure control.
2624
- **Browser-based user interface**: Seamless experience for design, control, feedback,
2725
and monitoring
2826
- **Secure communication**: HTTPS with configurable authentication strategies,
29-
Multi-tenant authorization and policy management, Standard protocols for encrypted
27+
multi-tenant authorization and policy management, standard protocols for encrypted
3028
communication including TLS and SSH.
3129

32-
![](https://github.com/crate/crate-clients-tools/assets/453543/ba6973dd-2eec-4f1f-a436-96aac7eb9892){h=120px}
33-
![](https://github.com/crate/crate-clients-tools/assets/453543/7fd4d2e7-98bc-44ee-b441-e1835016ab4d){h=120px}
34-
![](https://github.com/crate/crate-clients-tools/assets/453543/ccfa4ac7-0d60-432f-b952-2b50789cd325){h=120px}
30+
![NiFi flow canvas screenshot 1](https://github.com/crate/crate-clients-tools/assets/453543/ba6973dd-2eec-4f1f-a436-96aac7eb9892){height=120px loading=lazy}
31+
![NiFi flow canvas screenshot 2](https://github.com/crate/crate-clients-tools/assets/453543/7fd4d2e7-98bc-44ee-b441-e1835016ab4d){height=120px loading=lazy}
32+
![NiFi processors screenshot](https://github.com/crate/crate-clients-tools/assets/453543/ccfa4ac7-0d60-432f-b952-2b50789cd325){height=120px loading=lazy}
3533

3634
:::
3735

0 commit comments

Comments
 (0)