From 235500b1db79de3ab525b7bc4a3cc3a0d48de3b9 Mon Sep 17 00:00:00 2001 From: Andreas Motl Date: Wed, 22 Oct 2025 04:57:40 +0200 Subject: [PATCH 1/2] Topics: Absorb "Migrations" from "Solutions" --- docs/solution/index.md | 1 - docs/topic/index.md | 1 + docs/{solution => topic}/migrate/index.md | 0 docs/{solution => topic}/migrate/rockset/index.md | 0 docs/{solution => topic}/migrate/rockset/query.md | 0 5 files changed, 1 insertion(+), 1 deletion(-) rename docs/{solution => topic}/migrate/index.md (100%) rename docs/{solution => topic}/migrate/rockset/index.md (100%) rename docs/{solution => topic}/migrate/rockset/query.md (100%) diff --git a/docs/solution/index.md b/docs/solution/index.md index 93d6dfef..fe5af96e 100644 --- a/docs/solution/index.md +++ b/docs/solution/index.md @@ -97,5 +97,4 @@ or TensorFlow. analytics/index industrial/index Machine learning -migrate/index ``` diff --git a/docs/topic/index.md b/docs/topic/index.md index c06b81ee..3d002ab5 100644 --- a/docs/topic/index.md +++ b/docs/topic/index.md @@ -106,4 +106,5 @@ Data Visualization Programming Frameworks Software Testing Time Series Data +migrate/index ``` diff --git a/docs/solution/migrate/index.md b/docs/topic/migrate/index.md similarity index 100% rename from docs/solution/migrate/index.md rename to docs/topic/migrate/index.md diff --git a/docs/solution/migrate/rockset/index.md b/docs/topic/migrate/rockset/index.md similarity index 100% rename from docs/solution/migrate/rockset/index.md rename to docs/topic/migrate/rockset/index.md diff --git a/docs/solution/migrate/rockset/query.md b/docs/topic/migrate/rockset/query.md similarity index 100% rename from docs/solution/migrate/rockset/query.md rename to docs/topic/migrate/rockset/query.md From b73c68120e2c64ba639481362f0f37b290e1a991 Mon Sep 17 00:00:00 2001 From: Andreas Motl Date: Wed, 22 Oct 2025 05:08:55 +0200 Subject: [PATCH 2/2] Solutions: Refurbish whole section --- docs/_include/card/timeseries-dask.md | 2 +- docs/_include/card/timeseries-explore.md | 2 +- docs/_include/card/timeseries-intro.md | 27 +- docs/admin/sharding-partitioning.md | 2 +- docs/feature/query/index.md | 2 +- docs/handbook/index.md | 21 +- docs/integrate/pandas/tutorial-jupyter.md | 15 +- docs/solution/analytics/bitmovin.md | 127 ++++++ docs/solution/analytics/index.md | 107 ++--- docs/solution/index.md | 130 +++--- docs/solution/industrial/abb.md | 77 ++++ docs/solution/industrial/index.md | 383 +++--------------- docs/solution/industrial/rauch.md | 79 ++++ docs/solution/industrial/spgo.md | 98 +++++ docs/solution/industrial/tgw.md | 192 +++++++++ docs/solution/machine-learning/index.md | 47 +-- .../analysis.md} | 9 +- docs/solution/time-series/fundamentals.md | 110 +++++ .../time-series}/generate/cli.md | 0 .../time-series}/generate/go.md | 0 .../time-series}/generate/index.md | 0 .../time-series}/generate/node.md | 0 .../time-series}/generate/python.md | 0 .../time-series}/index.md | 37 +- .../time-series}/learn/normalize-pandas.md | 0 .../time-series}/learn/query.md | 3 +- .../time-series}/learn/with-metadata.md | 2 +- .../time-series}/longterm.md | 12 +- .../time-series}/video.md | 28 +- docs/topic/index.md | 48 ++- docs/topic/migrate/rockset/index.md | 4 +- docs/topic/migrate/rockset/query.md | 14 +- docs/topic/timeseries/fundamentals.md | 79 ---- 33 files changed, 994 insertions(+), 663 deletions(-) create mode 100644 docs/solution/analytics/bitmovin.md create mode 100644 docs/solution/industrial/abb.md create mode 100644 docs/solution/industrial/rauch.md create mode 100644 docs/solution/industrial/spgo.md create mode 100644 docs/solution/industrial/tgw.md rename docs/solution/{machine-learning/time-series.md => time-series/analysis.md} (97%) create mode 100644 docs/solution/time-series/fundamentals.md rename docs/{topic/timeseries => solution/time-series}/generate/cli.md (100%) rename docs/{topic/timeseries => solution/time-series}/generate/go.md (100%) rename docs/{topic/timeseries => solution/time-series}/generate/index.md (100%) rename docs/{topic/timeseries => solution/time-series}/generate/node.md (100%) rename docs/{topic/timeseries => solution/time-series}/generate/python.md (100%) rename docs/{topic/timeseries => solution/time-series}/index.md (81%) rename docs/{topic/timeseries => solution/time-series}/learn/normalize-pandas.md (100%) rename docs/{topic/timeseries => solution/time-series}/learn/query.md (98%) rename docs/{topic/timeseries => solution/time-series}/learn/with-metadata.md (99%) rename docs/{topic/timeseries => solution/time-series}/longterm.md (91%) rename docs/{topic/timeseries => solution/time-series}/video.md (90%) delete mode 100644 docs/topic/timeseries/fundamentals.md diff --git a/docs/_include/card/timeseries-dask.md b/docs/_include/card/timeseries-dask.md index e24f4a69..84f2de85 100644 --- a/docs/_include/card/timeseries-dask.md +++ b/docs/_include/card/timeseries-dask.md @@ -2,7 +2,7 @@ :::{grid-item} :columns: auto 9 9 9 -**Notebook: How to Build Time Series Applications with CrateDB** +**Notebook: How to build time series applications with CrateDB** This notebook illustrates how to import and work with time series data using CrateDB and [Dask DataFrame]s. diff --git a/docs/_include/card/timeseries-explore.md b/docs/_include/card/timeseries-explore.md index 166de1a5..95e10e9e 100644 --- a/docs/_include/card/timeseries-explore.md +++ b/docs/_include/card/timeseries-explore.md @@ -2,7 +2,7 @@ :::{grid-item} :columns: auto 9 9 9 -**CrateDB for Time Series Modeling, Exploration, and Visualization** +**CrateDB for time series modeling, exploration, and visualization** Access time series data from CrateDB via SQL, load it into pandas DataFrames, and visualize it using Plotly. diff --git a/docs/_include/card/timeseries-intro.md b/docs/_include/card/timeseries-intro.md index 6068e23a..454c658b 100644 --- a/docs/_include/card/timeseries-intro.md +++ b/docs/_include/card/timeseries-intro.md @@ -3,8 +3,8 @@ :padding: 0 :gutter: 2 -::::{grid-item-card} {material-outlined}`topic;2em` Time Series: Device Readings with Metadata -:link: guide:timeseries-objects +::::{grid-item-card} {material-outlined}`topic;2em` Time series: Device readings with metadata +:link: timeseries-objects :link-type: ref :class-footer: text-smaller @@ -19,11 +19,11 @@ for fast aggregations. - Relational JOIN operations. - Common table expressions (CTEs). +++ -Combine time series data with document data: CrateDB is all you need. +Combine time series with document data: CrateDB is all you need. :::: -::::{grid-item-card} {material-outlined}`lightbulb;2em` Time Series: Analyzing Weather Data -:link: guide:timeseries-analysis-weather +::::{grid-item-card} {material-outlined}`lightbulb;2em` Time series: Analyzing weather data +:link: timeseries-analysis-weather :link-type: ref :class-footer: text-smaller CrateDB provides advanced SQL features for querying time series data. @@ -40,5 +40,22 @@ CrateDB provides advanced SQL features for querying time series data. Advanced queries on time series data: CrateDB is all you need. :::: +::::{grid-item-card} {material-outlined}`area_chart;2em` Time series: Process financial data +:link: pandas-tutorial-jupyter +:link-type: ref +:class-footer: text-smaller +Acquire and store historical data from S&P-500 companies into CrateDB +using Python. + +:::{rubric} What's Inside +::: +- Acquire historical stock ticker data from the Yahoo! Finance API. + +- Store data into CrateDB. + +- Query back data from CrateDB. ++++ +Custom ETL tasks: CrateDB is all you need. +:::: ::::: diff --git a/docs/admin/sharding-partitioning.md b/docs/admin/sharding-partitioning.md index ce0a542d..1696b6cf 100644 --- a/docs/admin/sharding-partitioning.md +++ b/docs/admin/sharding-partitioning.md @@ -1,6 +1,6 @@ (sharding-partitioning)= -# Sharding and Partitioning 101 +# Sharding and partitioning 101 ## Introduction diff --git a/docs/feature/query/index.md b/docs/feature/query/index.md index 0f158b55..462db883 100644 --- a/docs/feature/query/index.md +++ b/docs/feature/query/index.md @@ -53,7 +53,7 @@ FROM OrderedData ORDER BY location, timestamp; ::: -{{ '{}(#timeseries-analysis-advanced)'.format(tutorial) }} +{{ '{}(#timeseries-analysis-weather)'.format(tutorial) }} :::: ::::{grid-item} diff --git a/docs/handbook/index.md b/docs/handbook/index.md index a199005e..5aea1ea7 100644 --- a/docs/handbook/index.md +++ b/docs/handbook/index.md @@ -225,26 +225,29 @@ Load data from many sources into CrateDB. :link: solutions :link-type: ref :link-alt: Solutions built with CrateDB -Learn about solutions built with CrateDB and -how others are using CrateDB successfully. +Learn how to use CrateDB for time series use-cases, +about industry solutions built with CrateDB and +how others are using CrateDB successfully with +both standard software components and in +proprietary system landscapes. +++ **What's inside:** -Full-text and semantic search, real-time raw-data analytics, -industrial data, machine learning, data migrations. +Time series data. Industrial big data. +Real-time raw-data analytics. Machine learning. ::: -:::{grid-item-card} {material-outlined}`numbers;2em` Topics +:::{grid-item-card} {material-outlined}`numbers;2em` Categories / Topics :link: topics :link-type: ref :link-alt: CrateDB topics overview -Learn how to apply CrateDB's features to optimally cover use-cases -across different application and topic domains. +Learn how to apply CrateDB's features to optimally cover +different application categories and topic domains. For example, connect CrateDB with third-party software applications, libraries, and frameworks. +++ **What's inside:** -Business intelligence, data lineage, data visualization, -programming frameworks, software testing, time series data. +Business intelligence, data lineage, data migrations, data visualization, +programming frameworks, software testing. ::: :::: diff --git a/docs/integrate/pandas/tutorial-jupyter.md b/docs/integrate/pandas/tutorial-jupyter.md index b1990131..ca7eb1e3 100644 --- a/docs/integrate/pandas/tutorial-jupyter.md +++ b/docs/integrate/pandas/tutorial-jupyter.md @@ -1,5 +1,5 @@ (pandas-tutorial-jupyter)= -# Automating financial data collection and storage in CrateDB with Python and pandas +# Process financial data using CrateDB, Jupyter, and pandas :::{article-info} --- @@ -27,11 +27,14 @@ Before anything else, I must make sure I have my setup ready. So, let’s get started. -## Setting up CrateDB, Jupyter, and Python +## Prerequisites + +You will need access to a CrateDB cluster and a Jupyter environment with +pandas and the psycopg2 packages installed. ### CrateDB -If you’re new to CrateDB and want to get started quickly and easily, a great option is to try the **Free Tier** in CrateDB Cloud. With the **Free Tier**, you have a limited Cluster that is free forever; no payment method is required. Now, if you are ready to experience the full power of CrateDB Cloud, take advantage of the 200$ in free credits to try the cluster of your dreams. +If you’re new to CrateDB and want to get started quickly and easily, a great option is to try the **Free Tier** in CrateDB Cloud. With the **Free Tier**, you have a limited Cluster that is free forever; no payment method is required. Now, if you are ready to experience the full power of CrateDB Cloud, take advantage of $200 in free credits to explore CrateDB Cloud's full capabilities. To start with CrateDB Cloud, [navigate to the CrateDB website](https://cratedb.com/download?hsCtaTracking=caa20047-f2b6-4e8c-b7f9-63fbf818b17f%7Cf1ad6eaa-39ac-49cd-8115-ed7d5dac4d63) and follow the steps to create your CrateDB Cloud account. Once you log in to the CrateDB Cloud UI, select **Deploy Cluster** to create your free cluster, and you are ready to go! @@ -55,9 +58,9 @@ The [Jupyter Notebook](https://jupyter.org/) is an open-source web application t A Jupyter Notebook is an excellent environment for this project. It contains executable documents (the code) and human-readable documents (tables, figures, etc.) in the same place! -I follow the [Jupiter Installation tutorial](https://jupyter.org/install.html) for the Notebook, which is quickly done with Python and the terminal command +I follow the [Jupyter Installation tutorial](https://jupyter.org/install.html) for the Notebook, which is quickly done with Python and the terminal command `pip3 install notebook` -and now I run the Notebook (using Jupyter 1.0.0) with the command +and now I run the Notebook with the command `jupyter notebook` Setup done! @@ -204,7 +207,7 @@ and it looks like this: ## Connecting to CrateDB -In the **Overview** tab of my CrateDB Cloud Cluster I find several ways to connect to CrateDB with CLI, Python, JavaScript, among others. So I select the **Python** option and choose one of the variants, such as **psycopg2**(version 2.9.1). +In the **Overview** tab of my CrateDB Cloud Cluster I find several ways to connect to CrateDB with CLI, Python, JavaScript, among others. So I select the **Python** option and choose one of the variants, such as **psycopg2**. ![connections-for-cratedb-cloud|690x386](https://us1.discourse-cdn.com/flex020/uploads/crate/original/1X/2891e21d7ad9cd34eed068153285530badb0dc66.png){w=800px} diff --git a/docs/solution/analytics/bitmovin.md b/docs/solution/analytics/bitmovin.md new file mode 100644 index 00000000..ac8f14f6 --- /dev/null +++ b/docs/solution/analytics/bitmovin.md @@ -0,0 +1,127 @@ +(bitmovin)= +# Bitmovin insights + +:::{div} sd-text-muted +Multi-tenant data analytics on top of billions of records. +::: + +:::{rubric} About +::: + +Bitmovin is a leading video streaming company that built the world’s +first commercial adaptive streaming player and deployed the first +software-defined encoding service that runs on any cloud platform. + +The use-case of Bitmovin illustrates why traditional databases are +incapable of handling so many data records while keeping them all +available for querying in real time. + +> CrateDB enables use cases we couldn't satisfy with other +> database systems, also with databases which are even stronger +> focused on the time series domain. +> +> CrateDB is not your normal database! +> +> -- Daniel Hölbling‑Inzko, Director of Engineering Analytics, Bitmovin + +:::{rubric} See also +::: + +:::{card} Bitmovin: Analyzing large volumes of video streaming events while reducing the cost of analytics +:link: https://cratedb.com/stories/bitmovin +:link-type: url +CrateDB forms the backbone of Bitmovin's real-time video analytics platform. + +Bitmovin's database cluster includes 14 nodes, storing 140 terabytes worth +of structured data such as user events and user interactions. +The video analytics application adds around 2 billion new events per day, +with the largest tables comprising around 60 billion playback events in total. +::: + + +:::::{info-card} + +::::{grid-item} +:columns: 6 + +{material-outlined}`analytics;2em`   **Real-time analytics on user events** + + + +-- [Bitmovin: Improving the streaming experience with real-time analytics] +:::: + +::::{grid-item} +:columns: 6 + +Bitmovin, as a leader in video codec algorithms and as a web-based video +stream broadcasting provider, produces billions of rows of data and stores +them in CrateDB, allowing their customers to do analytics on it. + +One of their product's subsystems, a video analytics component, required to +serve real-time analytics on massive, fast-moving data, so they needed +to find a performing database at the right cost. + +:::{article-info} +--- +author: Daniel Hölbling‑Inzko, Georg Traar +date: October 14, 2022 +read-time: 50 min watch +class-container: sd-p-2 sd-outline-muted sd-rounded-1 +--- +::: +:::: + +::::: + + +:::::{info-card} + +::::{grid-item} +:columns: 6 + +{material-outlined}`video_camera_back;2em`   **Live video broadcasting campaigns** + + + +-- [How Bitmovin uses CrateDB to monitor the biggest live video events] +:::: + +::::{grid-item} +:columns: 6 + +Bitmovin produces billions of rows of data and stores it in CrateDB. +In this talk, Daniel explains how Bitmovin uses CrateDB to monitor +the most significant live video events and especially which features +they are using to address their monitoring and scalability challenges. + +Learn also about their typical queries and how the support from Crate\.io +helps them in their day-to-day data operations. + +:::{article-info} +--- +author: Daniel Hölbling‑Inzko +date: Nov 15, 2022 +read-time: 35 min watch +class-container: sd-p-2 sd-outline-muted sd-rounded-1 +--- +::: +:::: + +::::: + + +:Industry: + {tags-secondary}`Broadcasting` + {tags-secondary}`Media Transcoding` + {tags-secondary}`Streaming Media` + +:Tags: + {tags-primary}`Event Tracking` + {tags-primary}`Real-Time Analytics` + {tags-primary}`Multi Tenancy` + {tags-primary}`SaaS` + + +[Bitmovin: Improving the streaming experience with real-time analytics]: https://youtu.be/4BPApD0Piyc?feature=shared +[How Bitmovin uses CrateDB to monitor the biggest live video events]: https://youtu.be/IR6hokaYv5g?feature=shared diff --git a/docs/solution/analytics/index.md b/docs/solution/analytics/index.md index 648fc6ef..2519aff8 100644 --- a/docs/solution/analytics/index.md +++ b/docs/solution/analytics/index.md @@ -1,7 +1,9 @@ (analytics)= # Real-time raw-data analytics -**CrateDB provides real-time analytics on raw data stored for the long term** +:::{div} sd-text-muted +CrateDB provides real-time analytics on raw data stored for the long term. +::: In all domains of real-time analytics where you absolutely must have access to all the records, and can't live with any down-sampled variants, because records are @@ -18,80 +20,53 @@ Other than integrating well with commodity systems using standard database access interfaces like ODBC or JDBC, it provides a proprietary HTTP interface on top. -:Tags: - {tags-primary}`Analytics` - {tags-primary}`Long Term Storage` - -:Related: - [](#timeseries) • - [](#timeseries-longterm) • - [](#machine-learning) - -:Product: - [Real-time Analytics Database] - - -(bitmovin)= -## Bitmovin Insights - -Multi tenant data analytics on top of billions of records. - -> CrateDB enables use cases we couldn't satisfy with other -database systems, also with databases which are even stronger -focused on the time series domain. -> -> CrateDB is not your normal database! -> -> -- Daniel Hölbling-Inzko, Director of Engineering Analytics, Bitmovin - -:Industry: - {tags-secondary}`Broadcasting` - {tags-secondary}`Media Transcoding` - {tags-secondary}`Streaming Media` - -:Tags: - {tags-primary}`Event Tracking` - {tags-primary}`Real-Time Analytics` - {tags-primary}`Multi Tenancy` - {tags-primary}`SaaS` - -:Related: - [CrateDB provides the backbone of Bitmovin's real-time video analytics platform] \ - [How Bitmovin uses CrateDB to monitor the biggest live video events] - +:::{rubric} See also +::: -::::{info-card} +:::::{grid} +:padding: 0 +:gutter: 2 -:::{grid-item} -:columns: 8 +::::{grid-item-card} {material-outlined}`link;1.5em` Related +:columns: 12 6 3 3 -{material-outlined}`analytics;2em`   **Bitmovin: Real-Time Analytics** +- {ref}`timeseries` +- {ref}`machine-learning` +- {ref}`industrial` ++++ +Related topics in the same area. +:::: -Bitmovin, as a leader in video codec algorithms and as a web-based video -stream broadcasting provider, produces billions of rows of data and stores -them in CrateDB, allowing their customers to do analytics on it. +::::{grid-item-card} {material-outlined}`group;1.5em` Customer insights +:columns: 12 6 4 4 -One of their product's subsystems, a video analytics component, required to -serve real-time analytics on very large and fast-moving data, so they needed -to find a performing database at the right cost. +:::{toctree} +:maxdepth: 1 +bitmovin +::: ++++ +Companies that are successfully using CrateDB in their technology stack. +:::: -- [Bitmovin: Improving the Streaming Experience with Real-Time Analytics] +::::{grid-item-card} {material-outlined}`factory;1.5em` Product +:columns: 12 12 5 5 -The use-case of Bitmovin illustrates why traditional databases weren't capable -to deal with so many data records and keep them all available for querying in -real time. -::: +- [Media & entertainment] +- [Real-time analytics database] +- [Streaming Analytics] ++++ +Real-time analytics on large volumes of data from click event streams and +similar applications. +:::: -:::{grid-item} -:columns: 4 +::::: - -::: -:::: +:Tags: + {tags-primary}`Analytics` + {tags-primary}`Long Term Storage` -[Bitmovin: Improving the Streaming Experience with Real-Time Analytics]: https://youtu.be/4BPApD0Piyc?feature=shared -[CrateDB provides the backbone of Bitmovin's real-time video analytics platform]: https://cratedb.com/customers/bitmovin -[How Bitmovin uses CrateDB to monitor the biggest live video events]: https://youtu.be/IR6hokaYv5g?feature=shared -[Real-time Analytics Database]: https://cratedb.com/solutions/real-time-analytics-database +[Media & entertainment]: https://cratedb.com/media-entertainment +[Real-time analytics database]: https://cratedb.com/solutions/real-time-analytics-database +[Streaming Analytics]: https://cratedb.com/use-cases/streaming-analytics diff --git a/docs/solution/index.md b/docs/solution/index.md index fe5af96e..c2978188 100644 --- a/docs/solution/index.md +++ b/docs/solution/index.md @@ -1,9 +1,63 @@ -(use-cases)= (solutions)= +(use-cases)= + # Solutions and use cases +:::{toctree} +:hidden: +time-series/index +industrial/index +analytics/index +machine-learning/index +::: + + +## Explanations + :::{div} sd-text-muted -Learn about solutions built with CrateDB and +About time series data storage and analytics, and machine learning. +::: + +::::{grid} 1 2 2 2 +:gutter: 2 +:padding: 0 + +:::{grid-item-card} {material-outlined}`stacked_line_chart;2em` Time series data +:link: timeseries +:link-type: ref +:link-alt: About CrateDB for time series data analysis +Enhance your understanding of how to use CrateDB for time series use-cases, +and how to apply time series modeling and analysis procedures to your data. ++++ +**What's inside:** +- Advanced statistical analysis +- Data visualization +- Machine learning +- Scientific computing +::: + +:::{grid-item-card} {material-outlined}`model_training;2em` Machine learning +:link: machine-learning +:link-type: ref +:link-alt: About CrateDB for machine learning applications +Get an overview of how CrateDB provides support for different kinds of +machine learning tasks, and learn how to integrate CrateDB with machine +learning frameworks and tools. ++++ +**What's inside:** +- Vector store: Vector search, Hybrid search, LangChain +- Text-to-SQL: LlamaIndex, MCP, MindsDB +- Time series analysis: R, TensorFlow +- MLOps and model training: MLflow, PyCaret, scikit-learn +::: + +:::: + + +## Case studies + +:::{div} sd-text-muted +About solutions built with CrateDB and how others are using CrateDB successfully. ::: @@ -17,53 +71,59 @@ build real-time data management and analytics solutions and platforms. :padding: 0 :gutter: 2 - -:::{grid-item-card} {material-outlined}`search;2em` Full-text and semantic search -:link: search-overview +:::{grid-item-card} {material-outlined}`precision_manufacturing;2em` Industrial big data +:link: industrial :link-type: ref -:link-alt: About CrateDB's search features - -CrateDB enables you to build powerful search experiences for websites, -applications, and enterprise data. +:link-alt: Use CrateDB in industrial data platforms +Learn how others are successfully using CrateDB within industrial, +engineering, manufacturing, production, and logistics domains. +++ **What's inside:** -Full-text search, Geo search, Vector search, Hybrid search. +About the unique challenges and complexities of industrial big data. ::: - :::{grid-item-card} {material-outlined}`analytics;2em` Real-time analytics on raw data :link: analytics :link-type: ref :link-alt: About CrateDB's analytics features - CrateDB provides real-time analytics on raw data. Learn how others are successfully running real-time multi-tenant data analytics applications on top of billions of records. +++ **What's inside:** -If you absolutely must keep the records, because they are unique, -downsampling is not an option. +For scenarios where all records must be retained due +to their unique value, downsampling is not applicable. ::: +:::: -:::{grid-item-card} {material-outlined}`precision_manufacturing;2em` Industrial data -:link: industrial -:link-type: ref -:link-alt: Use CrateDB in industrial data platforms +## See also -Learn how others are successfully using CrateDB within industrial, -engineering, manufacturing, production, and logistics domains. +:::{div} sd-text-muted +Other notable features of CrateDB. +::: + +::::{grid} 1 2 2 2 +:margin: 4 4 0 0 +:padding: 0 +:gutter: 2 + +:::{grid-item-card} {material-outlined}`search;2em` Full-text and semantic search +:link: search-overview +:link-type: ref +:link-alt: About CrateDB's search features +CrateDB enables you to build powerful search experiences for websites, +applications, and enterprise data. +++ **What's inside:** -About the unique challenges and complexities of industrial big data. +Learn how to leverage full-text, geospatial-, +vector-, and hybrid-search capabilities. ::: - :::{grid-item-card} {material-outlined}`manage_history;2em` Metrics and telemetry data store :link: metrics-store :link-type: ref :link-alt: Using CrateDB as a long-term metrics store - Store metrics and telemetry data for the long term, with the benefits of using standard database interfaces, SQL query language, and horizontal scalability through clustering as you go. @@ -73,28 +133,4 @@ Never retire old records to cold storage, always have them ready for historical analysis. ::: - -:::{grid-item-card} {material-outlined}`model_training;2em` Machine Learning -:link: machine-learning -:link-type: ref -:link-alt: About CrateDB for machine learning applications - -Learn how to integrate CrateDB with machine learning frameworks and tools. -+++ -**What's inside:** -Use CrateDB with LangChain, LlamaIndex, MLflow, PyCaret, scikit-learn, -or TensorFlow. -::: - - :::: - - -```{toctree} -:maxdepth: 1 -:hidden: - -analytics/index -industrial/index -Machine learning -``` diff --git a/docs/solution/industrial/abb.md b/docs/solution/industrial/abb.md new file mode 100644 index 00000000..285f6b1c --- /dev/null +++ b/docs/solution/industrial/abb.md @@ -0,0 +1,77 @@ +(abb)= +# ABB insights + +:::{article-info} +--- +author: Marko Sommarberg and Christian Lutz +date: May 23, 2023 +read-time: 5 min read, 20 min watch +class-container: sd-p-2 sd-outline-muted sd-rounded-1 +--- +::: + +:::{div} sd-text-muted +ABB Ability™ Genix is an advanced analytics platform for industrial data. +::: + +:::{rubric} About +::: + +The ABB Ability™ Genix platform applies AI and analytics to unlock the +value of industrial data using CrateDB. + +With high scalability and easy integration, CrateDB is a strategic database +component of ABB Ability™ Genix across industries. + +> After exploring the market, we found out that CrateDB was the +> best fit, considering our requirements and use-cases. +> +> -- Marko Sommarberg, Digital Strategy and Business Development Lead, ABB + +:::{rubric} See also +::: + +:::{card} ABB Genix is improving industrial productivity and operational excellence +:link: https://cratedb.com/customers/abb +:link-type: url +ABB's flagship digital solution transforms this by delivering real-time +analytics across multiple sectors. Launched in 2020, ABB Ability Genix +combines data-centric approaches with AI/ML and domain knowledge to +deliver contextually rich data. + +The platform expands beyond sensors and devices +to include engineering, design, and IT data, integrating them into +the platform with pre-made adapters and ABB's domain knowledge. +::: + +::::{grid} 1 1 2 2 +:gutter: 2 +:padding: 0 + +:::{grid-item-card} Data Innovation Summit » ABB: AI and analytics applied to industrial data +:link: https://www.youtube.com/watch?v=45fZYJLh2Qg +:link-type: url +Learn how ABB recommends tangible actions to optimize operations and increase +asset availability in industrial use cases by analyzing vast amounts of data +in real time with CrateDB, in the range of 30k to 120k events/sec, boiling down +to a processing rate of roughly 1 million values per second. ++++ +ABB Ability™ Genix Industrial Analytics and AI Suite is an advanced analytics +platform and application portfolio that unlocks value from industrial data and +drives better business results. +::: + +:::{grid-item} + +::: + +:::: + +:Industry: + {tags-secondary}`Engineering` + {tags-secondary}`Manufacturing` + {tags-secondary}`Production` + +:Tags: + {tags-primary}`Multi Tenancy` + {tags-primary}`Data Platform` diff --git a/docs/solution/industrial/index.md b/docs/solution/industrial/index.md index a5dcb575..865eced0 100644 --- a/docs/solution/industrial/index.md +++ b/docs/solution/industrial/index.md @@ -2,7 +2,11 @@ (iiot)= (industrial)= (industry-40)= -# Industrial data +# Industrial big data + +:::{div} sd-text-muted +Industry 4.0, but cost-effective: CrateDB is the backbone for all the analytics. +::: Today's industrial and logistics data acquisition and processing systems are complex and distributed, with a very high degree of automation. @@ -14,367 +18,82 @@ The complexities of industrial big data are characterized by its high variety, unstructured features, different data sampling rates, and how these attributes influence data storage, retention, and integration. -Learn how to use CrateDB in long term storage and analytics scenarios for -industrial / IIoT / Industry 4.0 application scenarios within +Learn how to use CrateDB in long-term storage and analytics scenarios for +industrial / IIoT / Industry 4.0 environments within engineering, manufacturing, production, and logistics, as well as other operational domains, or within similar environments where billions of data records from any kinds of machines or devices need to be processed, stored, and queried. -With CrateDB, compatible to PostgreSQL, you can do all of that using plain SQL. +With CrateDB, compatible with PostgreSQL, you can do all of that using plain SQL. Other than integrating well with commodity systems using standard database access interfaces like ODBC or JDBC, it provides a proprietary HTTP interface on top. -:Tags: - {tags-primary}`Data Historian` - {tags-primary}`Industrial IoT` - {tags-primary}`SCADA` - {tags-primary}`MDE` - -:Technologies: - {tags-info}`DAQ` - {tags-info}`PLC` - {tags-info}`SPS` - -:Product: - [IoT database] - -:Related: - [](#document) • - [](#geospatial) • - [](#fulltext) • - [](#analytics) • - [](#timeseries) • - [](#machine-learning) - - +(industrial-customer-insights)= (reference-architectures)= -## Reference Architectures +:::{rubric} See also +::: -Reference architectures showing how CrateDB can be used in various use cases. +:::::{grid} +:padding: 0 +:gutter: 2 + +::::{grid-item-card} {material-outlined}`apartment;1.5em` Reference architectures +:columns: 12 6 4 4 :::{toctree} :maxdepth: 1 - Azure IoT Machine Learning ::: - - -(abb)= -## ABB Insights - -Advanced Analytics Platform for Industrial Data. - -:Industry: - {tags-secondary}`Engineering` - {tags-secondary}`Manufacturing` - {tags-secondary}`Production` - -:Tags: - {tags-primary}`Multi Tenancy` - {tags-primary}`Data Platform` - -:Related: - [ABB Genix is improving industrial productivity and operational excellence] - - -::::{info-card} - -:::{grid-item} -:columns: 8 - -{material-outlined}`analytics;2em`   **ABB Genix: Advanced Analytics Platform** - -Learn how ABB recommends tangible actions to optimize operations and increase -asset availability in industrial use cases by analyzing vast amounts of data -in real time with CrateDB, in the range of 30k to 120k events/sec, boiling down -to a processing rate of roughly 1 million values per second. - -- [ABB: AI and Analytics applied to Industrial Data] - -ABB Ability™ Genix Industrial Analytics and AI Suite is an advanced analytics -platform and application portfolio that unlocks value from industrial data and -drives better business results. - -With high scalability and easy integration, CrateDB is a strategic database -component of ABB Ability™ Genix across industries. -::: - -:::{grid-item} -:columns: 4 - - - ---- -{tags-info}`2023` -::: - ++++ +Reference architectures about how CrateDB can be used in various environments. :::: +::::{grid-item-card} {material-outlined}`group;1.5em` Customer insights +:columns: 12 6 3 3 -(rauch)= -## Rauch Insights - -Scaling a high-speed production environment with CrateDB, to -ensure real-time production quality control. - -:Industry: - {tags-secondary}`Beverage` - {tags-secondary}`Food` - {tags-secondary}`FMCG` - {tags-secondary}`Packaging` - {tags-secondary}`Production` - -:Tags: - {tags-primary}`Unstructured Data` - {tags-primary}`PostgreSQL compatibility` - {tags-primary}`Excellency in Support` - -:Related: - [Scalable, High-Performance Database for FMCG] - - -::::{info-card} - -:::{grid-item} -:columns: 8 - -{material-outlined}`data_exploration;2em`   **Rauch: High-Speed Production Lines** - -CrateDB helps Rauch to identify and predict production issues and gives access -to production data for many years, with no need for additional infrastructure. - -Rauch fills 33 cans per second, which adds up to 400 data records per second -that are processed, stored, and analyzed. In total, they are within the range -of one to ten billion records persisted in CrateDB. - -- [Rauch: High-Speed Production Lines] - -The use-case of Rauch demonstrates why traditional databases weren't capable to -deal with so many data records and unstructured data. The benefits of CrateDB -made Rauch choose it over other databases, such as PostgreSQL compatibility, -the support for unstructured data, and its excellent customer support. -::: - -:::{grid-item} -:columns: 4 - - - ---- -{tags-info}`2022` +:::{toctree} +:maxdepth: 1 +abb +rauch +spgo +tgw ::: - ++++ +Companies that are successfully using CrateDB in their technology stack. :::: +::::{grid-item-card} {material-outlined}`factory;1.5em` Product +:columns: 12 12 5 5 -(spgo)= -## SPGo! Insights - -Monitoring conveyor belts in the mining industry. - -:Industry: - {tags-secondary}`Engineering` - {tags-secondary}`Mining` - {tags-secondary}`Production` - -:Tags: - {tags-primary}`Sensor Data Acquisition` - {tags-primary}`Machine Monitoring` - {tags-primary}`Predictive Maintenance` - -:Technologies: - {tags-info}`BLE Wireless` - {tags-info}`MQTT` - -:Related: - [SPGo!'s digital inspection of conveyor belts revolutionizes the market with CrateDB] - - -::::{info-card} - -:::{grid-item} -:columns: 8 - -{material-outlined}`engineering;2em`   **SPGo!: Monitoring and Predictive Maintenance** - -SPGo!, by PETROMIN, has developed a system that allows monitoring mining -material conveyor belts with more than 40,000 sensors in real-time and -760 million records per day. SPGo! trusts CrateDB as a partner on this journey. - -120 billion records each years - -- [SPGo!: Monitoring and Predictive Maintenance] - -You will learn about many details of this solution, including real-time -component monitoring, predictive failure analysis, management of operations, -data-driven predictive maintenance, large daily data intakes, and KPIs that -help the mining businesses save resources, workforce, and losses, due to -decreased downtime. -::: - -:::{grid-item} -:columns: 4 - - - ---- -{tags-info}`2023` -::: - +- [IoT database] +- [FMCG (Fast-moving consumer goods)] +- [Logistics] +- [Manufacturing] ++++ +Real-time analytics on large volumes of data from IoT devices, sensors, and +production systems in manufacturing, shipping, fulfillment, and logistics. :::: - -(tgw)= -## TGW Insights - -Today's warehouses are complex systems with a very high degree of automation. -TGW simplifies aggregating massive volumes of diverse data with CrateDB. - -:Industry: - {tags-secondary}`Logistics` - {tags-secondary}`Shipping` - {tags-secondary}`Warehouse Intelligence` +::::: :Tags: - {tags-primary}`Sensor Data Acquisition` - {tags-primary}`Digital Twin` - -:Related: - [TGW Logistics redefines warehouse intelligence] - - -::::{info-card} - -:::{grid-item} -:columns: 8 - -{material-outlined}`hub;2em`   **TGW: Connected Warehouses** - -_CrateDB's support for unstructured data, its fast query engine, -scalability, and excellent support, is unparalleled._ - -TGW Logistics Group implements advanced analytics for automated warehouses -they are operating across the globe for customers like Amazon, Coop, and -Zalando. Their systems collect a vast amount of data, apply AI to them, -and support all kinds of data-driven applications. - -- [TGW: Connected Warehouses] - -TGW removed data silos with all different kinds of data formats, data -structures from PLCs, databases, sensor information, etc. - -NoSQL databases weren't a sustainable solution for their use case. On the -migration path, it was easy to start with CrateDB, and now it is at the -heart of everything they are doing, and gives them peace of mind. -::: - -:::{grid-item} -:columns: 4 - - - ---- -{tags-info}`2023` -::: - -:::: - - -::::{info-card} - -:::{grid-item} -:columns: 8 - -{material-outlined}`inventory;2em`   **TGW: Data acquisition in high-speed logistics** - -_Storing, querying, and analyzing industrial IoT data and metadata without -much hassle._ - -TGW Logistics Group implements key factors to the successful operation of these -warehouses, by having a holistic view on the entire system acquiring data from -various components like sensors, PLCs, embedded controllers, and software -systems. - -- [TGW: Fixing data silos in a high-speed logistics environment] - -TGW states that all these components can be seen as "data silos", -distributed across the entire site, each of them storing just some pieces of -information in various data structures and different ways to access it. - -After trying multiple database systems, TGW Logistics moved to CrateDB for -its ability to aggregate different data formats and the ability to query this -information without further ado. -::: - -:::{grid-item} -:columns: 4 - - - ---- -{tags-info}`2022` -::: - -:::: - - - -::::{info-card} - -:::{grid-item} -:columns: 8 - -{material-outlined}`dashboard;2em`   **TGW: Challenges in storing and analyzing industrial data** - -_Not All Time Series Are Equal: Challenges in Storing and Analyzing Industrial Data._ - -In the second presentation, you will learn how TGW leverages CrateDB to build -digital twins of physical warehouses around the world, by using its unique set -of features suitable for storing and querying complex industrial big data with -high variety, unstructured features, and at different data frequencies. - -- [TGW: Storing and analyzing real-world industrial data] - -**What's inside** - -- The Complexity of IoT Data: An examination of the unique properties of - industrial IoT data, including slow-moving structured information and - high-frequency measurements. - -- Challenges and Solutions: Discussion of the difficulties in data storage, - retention, and integration posed by this complexity, and how CrateDB - provides a targeted solution. - -- Real-World Applications: Exploration of actual customer use cases to - illustrate how CrateDB can be applied in various industrial scenarios. - -::: - -:::{grid-item} -:columns: 4 - - - ---- -{tags-info}`2023` -::: - -:::: + {tags-primary}`Data Historian` + {tags-primary}`Industrial IoT` + {tags-primary}`Industry 4.0` + {tags-primary}`SCADA` + {tags-primary}`MDE` +:Technologies: + {tags-info}`DAQ` + {tags-info}`PLC` + {tags-info}`SPS` -[ABB: AI and Analytics applied to Industrial Data]: https://youtu.be/45fZYJLh2Qg?feature=shared -[ABB Genix is improving industrial productivity and operational excellence]: https://cratedb.com/customers/abb -[Scalable, High-Performance Database for FMCG]: https://cratedb.com/fmcg-database +[FMCG (Fast-moving consumer goods)]: https://cratedb.com/industries/fmcg [IoT database]: https://cratedb.com/use-cases/iot-database -[Rauch: High-Speed Production Lines]: https://youtu.be/gJPmJ0uXeVs?feature=shared -[SPGo!: Monitoring and Predictive Maintenance]: https://youtu.be/eRqn7GhFO-s?feature=shared -[SPGo!'s digital inspection of conveyor belts revolutionizes the market with CrateDB]: https://cratedb.com/customers/spgo -[TGW Logistics redefines warehouse intelligence]: https://cratedb.com/customers/tgw-logistics -[TGW: Connected Warehouses]: https://youtu.be/X2o0-W8-mCM?feature=shared -[TGW: Fixing data silos in a high-speed logistics environment]: https://youtu.be/6dgjVQJtSKI?feature=shared -[TGW: Storing and analyzing real-world industrial data]: https://youtu.be/ugQvihToY0k?feature=shared +[Logistics]: https://cratedb.com/industries/logistics +[Manufacturing]: https://cratedb.com/industries/manufacturing diff --git a/docs/solution/industrial/rauch.md b/docs/solution/industrial/rauch.md new file mode 100644 index 00000000..18a15235 --- /dev/null +++ b/docs/solution/industrial/rauch.md @@ -0,0 +1,79 @@ +(rauch)= +# Rauch insights + +:::{article-info} +--- +author: Arno Breuss +date: June 28, 2022 +read-time: 5 min read, 5 min watch +class-container: sd-p-2 sd-outline-muted sd-rounded-1 +--- +::: + +:::{div} sd-text-muted +Improving OEE: Real-time monitoring of 120,000 cans per hour per production line. +::: + +:::{rubric} About +::: + +Rauch is scaling a high-speed production environment with CrateDB, to +ensure real-time production quality control. + +> We needed a database that could watch, record and analyze production in +> real time. CrateDB gives us the freedom to be cumulative and scale +> limitless - we found no alternative database with such simplicity and efficiency. +> +> -- Arno Breuss, CIO of Rauch Fruchtsäfte + +:::{rubric} See also +::: + +:::{card} Scalable, high-performance database for FMCG +:link: https://cratedb.com/fmcg-database +:link-type: url +CrateDB helps Rauch to identify and predict production issues and enables access +to production data for many years, with no need for additional infrastructure +while storing one to ten billion records in CrateDB at the same time. + +CrateDB enhances FMCG operations by optimizing shop floor efficiency through +real-time equipment monitoring and workflow optimization. It ensures product +quality with integrated quality control measures, reducing defects and waste. +Additionally, CrateDB enhances compliance and traceability by enabling batch +tracking, regulatory reporting, and swift recall capabilities. +::: + +::::{grid} 1 1 2 2 +:gutter: 2 +:padding: 0 + +:::{grid-item-card} Rauch: High-speed production lines +:link: https://www.youtube.com/watch?v=gJPmJ0uXeVs +:link-type: url +In this interview, Arno Breuss, CIO of Rauch Fruchtsäfte, talks about how Rauch +is filling 33 cans per second and how that adds up to 400 data records per second +which are being processed, stored, and analyzed. He explains why their traditional +databases weren't capable of dealing with so many data records and unstructured data. ++++ +The presentation outlines the benefits of CrateDB that made Rauch choose CrateDB +over other databases such as PostgreSQL compatibility, support for unstructured +data, and customer support. +::: + +:::{grid-item} + +::: + +:::: + +:Industry: + {tags-secondary}`Beverage` + {tags-secondary}`Food` + {tags-secondary}`FMCG` + {tags-secondary}`Packaging` + {tags-secondary}`Production` + +:Tags: + {tags-primary}`Unstructured Data` + {tags-primary}`PostgreSQL compatibility` + {tags-primary}`Excellency in Support` diff --git a/docs/solution/industrial/spgo.md b/docs/solution/industrial/spgo.md new file mode 100644 index 00000000..583b41db --- /dev/null +++ b/docs/solution/industrial/spgo.md @@ -0,0 +1,98 @@ +(spgo)= +# SPGo! insights + +:::{article-info} +--- +author: Michael Mella Navarro, Nixon David Monge Calle, Hernán Lionel Cianfagna +date: January 31, 2023 +read-time: 5 min read, 55 min watch +class-container: sd-p-2 sd-outline-muted sd-rounded-1 +--- +::: + +:::{div} sd-text-muted +SPGo! builds systems for the mining industry +for monitoring material conveyor belt idlers. +::: + +:::{rubric} About +::: + +The objective of SPGo! is to improve the profitability of their clients' +businesses in the mining industry by applying IIoT technology with a +predictive maintenance model to equipment inspection through the +management of big data. + +> CrateDB is a trustful partner that supports us on this journey. +> +> -- Patricio Quezada, General Manager and Founder of PETROMIN + +:::{rubric} See also +::: + +:::{card} SPGo!'s digital inspection of conveyor belts revolutionizes the market with CrateDB +:link: https://cratedb.com/customers/spgo +:link-type: url +SPGo!, by PETROMIN, has developed a system that allows monitoring mining +material conveyor belts with more than 40,000 sensors in real-time, +processing 760 million records per day, or 120 billion records each year. + +By using a predictive algorithm, SPGo! alerts their clients when to conduct +maintenance work on specific equipment (that is set to fail). +This helps the mining customer save resources, costs, and losses due to downtime. +::: + + +:::::{info-card} + +::::{grid-item} +:columns: 6 + +{material-outlined}`dashboard;2em`   **SPGo!: Monitoring and predictive maintenance** + +:::{div} sd-text-muted +SPGo! switched from MongoDB to CrateDB for performance and interoperability reasons. +::: + + + +-- [SPGo!: Monitoring and predictive maintenance] +:::: + +::::{grid-item} +:columns: 6 + +The speakers will share many details of this solution, including real-time +component monitoring, predictive failure analysis, management of operations, +data-driven predictive maintenance, large daily data intakes, and KPIs that +help the mining businesses save resources, workforce, and losses, due to +decreased downtime. + +:::{rubric} You will learn +::: +- How the mining industry business struggles to predict potential issues with critical equipment. +- How SPGo! planned to improve the profitability of their client's business by applying IIoT technology with a predictive maintenance model. +- How SPGo! verified a 30%-increase in data-driven predictive maintenance with CrateDB. +- How SPGo! manages large data intakes daily. + +:::: + +::::: + + +:Industry: + {tags-secondary}`Engineering` + {tags-secondary}`Mining` + {tags-secondary}`Production` + +:Tags: + {tags-primary}`Sensor Data Acquisition` + {tags-primary}`Machine Monitoring` + {tags-primary}`Predictive Maintenance` + +:Technologies: + {tags-info}`BLE Wireless` + {tags-info}`MQTT` + + +[SPGo!: Monitoring and predictive maintenance]: https://www.youtube.com/watch?v=eRqn7GhFO-s diff --git a/docs/solution/industrial/tgw.md b/docs/solution/industrial/tgw.md new file mode 100644 index 00000000..d4c374ef --- /dev/null +++ b/docs/solution/industrial/tgw.md @@ -0,0 +1,192 @@ +(tgw)= +# TGW insights + +:::{div} sd-text-muted +Use CrateDB to optimize distribution centers in warehouse logistics. +::: + +:::{rubric} About +::: + +TGW Logistics Group implements advanced analytics for automated warehouses +operating across the globe for clients like Amazon, Coop, and +Zalando. Their systems collect a vast amount of data, apply AI to them, +and support all kinds of data-driven applications. + +Today's warehouses are complex systems with a very high degree of automation. +TGW simplifies aggregating massive volumes of diverse data with CrateDB. + +CrateDB's support for unstructured data, its fast query engine, +scalability, and excellent support, is unparalleled. + +> CrateDB is a highly scalable database for time series and event data +> with a very fast query engine using standard SQL. +> +> -- Alexander Mann, Connected Warehouse Architecture TGW Logistics Group + +:::::{info-card} + +::::{grid-item} +:columns: 6 + +{material-outlined}`hub;2em`   **TGW: Connected Warehouses** + +:::{div} sd-text-muted +::: + + + +-- [TGW Logistics: Connected Warehouses] +:::: + +::::{grid-item} +:columns: 6 + +TGW removed data silos with all different kinds of data formats, data +structures from PLCs, databases, sensor information, etc. + +NoSQL databases weren't a sustainable solution for their use case. +After trying multiple database systems, TGW Logistics selected CrateDB for +its ability to aggregate different data formats and the ability to query this +information instantly. + +On the migration path, it was easy to start with CrateDB, and now it is at the +heart of everything they are doing, and gives them peace of mind. + +:::{article-info} +--- +author: Alexander Mann +date: March 20, 2023 +read-time: 5 min watch +class-container: sd-p-2 sd-outline-muted sd-rounded-1 +--- +::: +:::: + +::::: + +:::{rubric} See also +::: + +:::{card} TGW Logistics redefines warehouse intelligence with CrateDB +:link: https://cratedb.com/customers/tgw-logistics +:link-type: url + +**Use case:** +- Accelerate aggregation and access to large volumes of diverse data collected in real-time from warehouse systems around the world. + +**Challenges:** +- Data is in many different formats spread across diverse systems worldwide +- Classical and NoSQL databases cannot handle the volume of unstructured and object data +- Data silos are difficult to combine and query, limiting analytics and modeling capabilities +::: + + +:::::{info-card} + +::::{grid-item} +:columns: 6 + +{material-outlined}`inventory;2em`   **TGW: Data acquisition in high-speed logistics** + +:::{div} sd-text-muted +Storing, querying, and analyzing industrial IoT data and metadata without +any hassle. +::: + + + +-- [TGW: Fixing data silos in a high-speed logistics environment] +:::: + +::::{grid-item} +:columns: 6 + +TGW Logistics Group implements key factors to the successful operation of these +warehouses, by having a holistic view on the entire system acquiring data from +various components like sensors, PLCs, embedded controllers, and software +systems. + +All the components can effectively be seen as individual "data silos", +distributed across the entire site. Each of them stores just some pieces of +information in various data structures and different ways to access it. + +:::{article-info} +--- +author: Alexander Mann, Jan Weber +date: June 28, 2022 +read-time: 35 min watch +class-container: sd-p-2 sd-outline-muted sd-rounded-1 +--- +::: +:::: + +::::: + + + +:::::{info-card} + +::::{grid-item} +:columns: 6 + +{material-outlined}`dashboard;2em`   **TGW: Challenges in storing and analyzing industrial data** + +:::{div} sd-text-muted +Not all time series are equal: Challenges in storing and analyzing industrial data. +::: + + + +-- [TGW: Storing and analyzing real-world industrial data] +:::: + +::::{grid-item} +:columns: 6 + +In the second presentation, you will learn how TGW leverages CrateDB to build +digital twins of physical warehouses around the world. The unique set +of features is suitable for storing and querying complex industrial big data with +high variety, unstructured features, and at different data sampling rates. + +:::{rubric} What's inside +::: + +- The Complexity of IoT Data: An examination of the unique properties of + industrial IoT data, including slow-moving structured information and + high-frequency measurements. + +- Challenges and Solutions: Discussion of the difficulties in data storage, + retention, and integration posed by this complexity, and how CrateDB + provides a targeted solution. + +- Real-World Applications: Exploration of actual customer use cases to + illustrate how CrateDB can be applied in various industrial scenarios. + +:::{article-info} +--- +author: Alexander Mann, Georg Traar +date: October 5, 2023 +read-time: 20 min watch +class-container: sd-p-2 sd-outline-muted sd-rounded-1 +--- +::: + +:::: + +::::: + + +:Industry: + {tags-secondary}`Logistics` + {tags-secondary}`Shipping` + {tags-secondary}`Warehouse Intelligence` + +:Tags: + {tags-primary}`Sensor Data Acquisition` + {tags-primary}`Digital Twin` + + +[TGW Logistics: Connected Warehouses]: https://youtu.be/X2o0-W8-mCM?feature=shared +[TGW: Fixing data silos in a high-speed logistics environment]: https://youtu.be/6dgjVQJtSKI?feature=shared +[TGW: Storing and analyzing real-world industrial data]: https://youtu.be/ugQvihToY0k?feature=shared diff --git a/docs/solution/machine-learning/index.md b/docs/solution/machine-learning/index.md index 8e95a0b5..936521e7 100644 --- a/docs/solution/machine-learning/index.md +++ b/docs/solution/machine-learning/index.md @@ -1,13 +1,13 @@ (ml)= (ml-tools)= (machine-learning)= -# Machine learning with CrateDB +# Machine learning :::{include} /_include/links.md ::: :::{div} sd-text-muted -CrateDB provides a vector store natively, and adapters for integrating +CrateDB provides a vector type natively, and adapters for integrating with machine learning frameworks. ::: @@ -107,39 +107,15 @@ Load and analyze data from database systems for time series anomaly detection and forecasting. ::: -::::{grid} 2 -:gutter: 3 - -:::{grid-item-card} Statistical analysis and visualization on huge datasets -:link: r-tutorial -:link-type: ref -Learn how to create a machine learning pipeline using R and CrateDB. -::: - -:::{grid-item-card} Regression analysis with pandas and scikit-learn -:link: scikit-learn -:link-type: ref -Use pandas and scikit-learn to run a regression analysis within a Jupyter Notebook. -::: - -:::{grid-item-card} Build model for predictive maintenance with TensorFlow -:link: tensorflow-tutorial -:link-type: ref -Learn how to build a machine learning model that will predict whether -a machine will fail within a specified time window in the future. -::: - -:::{grid-item-card} Advanced time series analysis with MLflow and PyCaret -:link: ml-timeseries +:::{card} Time series analysis using ML +:link: timeseries-analysis-ml :link-type: ref -Learn how to conduct advanced data analysis on large time series datasets -with CrateDB, MLflow, and PyCaret: -Anomaly detection and forecasting, time series decomposition, -Exploratory data analysis (EDA). +- **End-to-end:** Statistical analysis and visualization on huge datasets. +- **Traditional:** Regression analysis within a Jupyter Notebook. +- **Predictive maintenance:** Build a machine learning model to predict machine failures. +- **Advanced time series analysis:** Conduct advanced data analysis on large time series datasets. ::: -:::: - ## MLOps and model training @@ -193,10 +169,3 @@ exploratory data analysis (EDA). ::: :::: - - -:::{toctree} -:maxdepth: 1 -:hidden: -time-series -::: diff --git a/docs/solution/machine-learning/time-series.md b/docs/solution/time-series/analysis.md similarity index 97% rename from docs/solution/machine-learning/time-series.md rename to docs/solution/time-series/analysis.md index e3bdcb72..2f9f3d4e 100644 --- a/docs/solution/machine-learning/time-series.md +++ b/docs/solution/time-series/analysis.md @@ -1,7 +1,6 @@ (ml-timeseries)= -(timeseries-advanced)= -(timeseries-analysis)= -# Advanced Time Series Analysis +(timeseries-analysis-advanced)= +# Advanced time series analysis :::{div} sd-text-muted Learn how to conduct advanced data analysis on large time series datasets @@ -17,7 +16,7 @@ with CrateDB. ::: (timeseries-anomaly-forecasting)= -## Anomaly Detection and Forecasting +## Anomaly detection and forecasting To gain insights from your data in a one-shot or recurring way, based on machine learning techniques, you may want to look into applying [anomaly] @@ -77,7 +76,7 @@ to train various time series forecasting models. (timeseries-decomposition)= -## Time Series Decomposition +## Time series decomposition [Decomposition of time series] is a statistical task that deconstructs a [time series] into several components, each representing one of the underlying diff --git a/docs/solution/time-series/fundamentals.md b/docs/solution/time-series/fundamentals.md new file mode 100644 index 00000000..4ec0414d --- /dev/null +++ b/docs/solution/time-series/fundamentals.md @@ -0,0 +1,110 @@ +(timeseries-basics)= +(timeseries-fundamentals)= +# Time series fundamentals with CrateDB + +:::{div} sd-text-muted +Learn how to conduct fundamental data analysis on large time series datasets +with CrateDB. +::: + +{tags-primary}`Metadata integration` +{tags-primary}`Advanced SQL for time series` + +:::{include} /_include/links.md +::: + +## Getting started + +After evaluating {ref}`connectivity options `, you would like to get +hands-on with CrateDB. We prepared a few introductory tutorials, some of +them in executable forms, to demonstrate CrateDB's features to work with +time series data on the spot. You may want to use them as starting points +for your own explorations. + +:::{include} /_include/card/timeseries-intro.md +::: + +:::{include} /_include/card/timeseries-explore.md +::: + +:::{include} /_include/card/timeseries-datashader.md +::: + +:::{include} /_include/card/timeseries-dask.md +::: + + +## Special features +Working with time series data often requires special feature support to enable +fluent data workflows. + +- {ref}`downsampling-timestamp-binning` +- {ref}`downsampling-lttb` +- {ref}`ni-interpolate` +- [Interpolating missing time series values] +- {ref}`crate-reference:aggregation-percentile` + + +(timeseries-analysis-ml)= +## Time series analysis + +Analyze time series data with statistical and machine learning techniques, +for time series anomaly detection and forecasting. + +::::{grid} 2 +:gutter: 3 + +:::{grid-item-card} Statistical analysis and visualization on huge datasets +:link: r-tutorial +:link-type: ref +Learn how to create a machine learning pipeline using R and CrateDB. +::: + +:::{grid-item-card} Regression analysis with pandas and scikit-learn +:link: scikit-learn +:link-type: ref +Use pandas and scikit-learn to run a regression analysis within a Jupyter Notebook. +::: + +:::{grid-item-card} Build model for predictive maintenance with TensorFlow +:link: tensorflow-tutorial +:link-type: ref +Learn how to build a machine learning model that will predict whether +a machine will fail within a specified time window in the future. +::: + +:::{grid-item-card} Advanced time series analysis +:link: ml-timeseries +:link-type: ref +Learn how to conduct advanced data analysis on large time series datasets +with CrateDB, MLflow, and PyCaret: +Anomaly detection and forecasting, time series decomposition, +Exploratory data analysis (EDA). +::: + +:::: + +## Big data operations + +CrateDB clusters can elastically scale to store and query large time +series data efficiently. CrateDB provides corresponding operational support. +- {ref}`sharding-partitioning` +- [CrateDB partitioned table vs. TimescaleDB Hypertable] + +## See also + +- {ref}`timeseries-analysis-advanced` +- {ref}`timeseries-video` + + +:::{toctree} +:hidden: +generate/index +learn/normalize-pandas +learn/query +learn/with-metadata +::: + + +[CrateDB partitioned table vs. TimescaleDB Hypertable]: https://community.cratedb.com/t/cratedb-partitioned-table-vs-timescaledb-hypertable/1713 +[Interpolating missing time series values]: https://community.cratedb.com/t/interpolating-missing-time-series-values/1010 diff --git a/docs/topic/timeseries/generate/cli.md b/docs/solution/time-series/generate/cli.md similarity index 100% rename from docs/topic/timeseries/generate/cli.md rename to docs/solution/time-series/generate/cli.md diff --git a/docs/topic/timeseries/generate/go.md b/docs/solution/time-series/generate/go.md similarity index 100% rename from docs/topic/timeseries/generate/go.md rename to docs/solution/time-series/generate/go.md diff --git a/docs/topic/timeseries/generate/index.md b/docs/solution/time-series/generate/index.md similarity index 100% rename from docs/topic/timeseries/generate/index.md rename to docs/solution/time-series/generate/index.md diff --git a/docs/topic/timeseries/generate/node.md b/docs/solution/time-series/generate/node.md similarity index 100% rename from docs/topic/timeseries/generate/node.md rename to docs/solution/time-series/generate/node.md diff --git a/docs/topic/timeseries/generate/python.md b/docs/solution/time-series/generate/python.md similarity index 100% rename from docs/topic/timeseries/generate/python.md rename to docs/solution/time-series/generate/python.md diff --git a/docs/topic/timeseries/index.md b/docs/solution/time-series/index.md similarity index 81% rename from docs/topic/timeseries/index.md rename to docs/solution/time-series/index.md index 70bf2043..4a4fd5ea 100644 --- a/docs/topic/timeseries/index.md +++ b/docs/solution/time-series/index.md @@ -1,13 +1,13 @@ (timeseries)= -# Time Series Data +# Time series data :::{div} sd-text-muted -Learn how to use CrateDB for time series use cases. +Use CrateDB to store and query massive amounts of time series data. ::: CrateDB is a distributed and scalable SQL database for storing and analyzing massive amounts of data in near real-time, even with complex queries. It is -PostgreSQL-compatible, and based on Lucene. +PostgreSQL-compatible, and based on Lucene. ::::{grid} 1 2 2 2 :margin: 4 4 0 0 @@ -28,8 +28,8 @@ Operations: Sharding and Partitioning. ::: -:::{grid-item-card} {material-outlined}`analytics;2em` Advanced -:link: timeseries-analysis +:::{grid-item-card} {material-outlined}`analytics;2em` Advanced analysis +:link: timeseries-analysis-advanced :link-type: ref :link-alt: About time series analysis @@ -41,7 +41,7 @@ anomaly detection, forecasting. ::: -:::{grid-item-card} {material-outlined}`smart_display;2em` Video Tutorials +:::{grid-item-card} {material-outlined}`smart_display;2em` Video tutorials :link: timeseries-video :link-type: ref :link-alt: Video tutorials about time series with CrateDB @@ -54,7 +54,7 @@ and analyzing. Industrial applications. ::: -:::{grid-item-card} {material-outlined}`school;2em` Academy » Advanced Time Series +:::{grid-item-card} {material-outlined}`school;2em` Academy » Advanced time series :link: https://cratedb.com/academy/time-series/ :link-type: url :link-alt: Academy Resources: Advanced Time Series @@ -69,7 +69,7 @@ Machine Learning on Time Series Data: EDA, Decomposition, AutoML. ::: -:::{grid-item-card} {material-outlined}`manage_history;2em` Long Term Storage +:::{grid-item-card} {material-outlined}`manage_history;2em` Long-term storage :link: timeseries-longterm :link-type: ref :link-alt: About storing time series data for the long term @@ -90,17 +90,17 @@ Optimizing storage for historic time series data. :::{seealso} **Domains:** -[](#metrics-store) • -[](#analytics) • -[](#industrial) • -[](#machine-learning) +{ref}`analytics` • +{ref}`industrial` • +{ref}`machine-learning` • +{ref}`metrics-store` **Features:** -[](#connect) • -[](#querying) • -[](#document) • -[](#fulltext) • -[](#geospatial) +{ref}`connect` • +{ref}`querying` • +{ref}`document` • +{ref}`fulltext` • +{ref}`geospatial` **Product:** [Time Series Data] • @@ -112,8 +112,9 @@ Optimizing storage for historic time series data. :hidden: Fundamentals +Advanced analysis video -Long Term Store +Long-term store ::: diff --git a/docs/topic/timeseries/learn/normalize-pandas.md b/docs/solution/time-series/learn/normalize-pandas.md similarity index 100% rename from docs/topic/timeseries/learn/normalize-pandas.md rename to docs/solution/time-series/learn/normalize-pandas.md diff --git a/docs/topic/timeseries/learn/query.md b/docs/solution/time-series/learn/query.md similarity index 98% rename from docs/topic/timeseries/learn/query.md rename to docs/solution/time-series/learn/query.md index 07464174..c258fea0 100644 --- a/docs/topic/timeseries/learn/query.md +++ b/docs/solution/time-series/learn/query.md @@ -1,9 +1,8 @@ -(timeseries-analysis-advanced)= (timeseries-analysis-weather)= (timeseries-querying)= (timeseries-tutorial-weather)= -# Analyzing Weather Data +# Analyzing weather data :::{article-info} --- diff --git a/docs/topic/timeseries/learn/with-metadata.md b/docs/solution/time-series/learn/with-metadata.md similarity index 99% rename from docs/topic/timeseries/learn/with-metadata.md rename to docs/solution/time-series/learn/with-metadata.md index 7c855132..e87cc4e4 100644 --- a/docs/topic/timeseries/learn/with-metadata.md +++ b/docs/solution/time-series/learn/with-metadata.md @@ -2,7 +2,7 @@ (timeseries-with-metadata)= (timeseries-tutorial-metadata)= -# Analyzing Device Readings with Metadata Integration +# Analyzing device readings with metadata integration :::{article-info} --- diff --git a/docs/topic/timeseries/longterm.md b/docs/solution/time-series/longterm.md similarity index 91% rename from docs/topic/timeseries/longterm.md rename to docs/solution/time-series/longterm.md index 0194bff3..f04633bb 100644 --- a/docs/topic/timeseries/longterm.md +++ b/docs/solution/time-series/longterm.md @@ -1,6 +1,6 @@ (timeseries-longterm)= (timeseries-long-term-storage)= -# Time Series Long Term Storage +# Time series long-term storage CrateDB stores large volumes of data, keeping it accessible for querying and insightful analysis, even considering historic data records. @@ -45,12 +45,12 @@ a dedicated layout, and querying using the UNNEST table function. **CrateDB as metrics and log data store for the long term** Store and analyze high volumes of system monitoring information. -Read more about using CrateDB as [](#metrics-store). +Read more about using CrateDB as {ref}`metrics-store`. ::: :::{grid-item} :columns: 3 -{tags-primary}`Long Term Storage` +{tags-primary}`Long-term Storage` {tags-primary}`Metrics` {tags-primary}`Logging` ::: @@ -65,13 +65,13 @@ Read more about using CrateDB as [](#metrics-store). **CrateDB provides real-time analytics on raw data stored for the long term** Keep massive amounts of data ready in the hot zone for analytics purposes. -Read more about using CrateDB for [](#analytics). +Read more about using CrateDB for {ref}`analytics`. ::: :::{grid-item} :columns: 3 -{tags-primary}`Long Term Storage` -{tags-primary}`Real-Time Analytics` +{tags-primary}`Long-term storage` +{tags-primary}`Real-time analytics` ::: :::: diff --git a/docs/topic/timeseries/video.md b/docs/solution/time-series/video.md similarity index 90% rename from docs/topic/timeseries/video.md rename to docs/solution/time-series/video.md index 0efa728a..e41742a1 100644 --- a/docs/topic/timeseries/video.md +++ b/docs/solution/time-series/video.md @@ -1,10 +1,10 @@ (timeseries-video)= -# Video Tutorials +# Video tutorials Video tutorials about time series with CrateDB. -## Time Series Data and CrateDB +## Time series data and CrateDB ::::{info-card} @@ -20,7 +20,7 @@ CrateDB simplifies the complexity of managing time-series data. It provides a comprehensive solution for storing, querying, and extracting insights from large-scale and high-volume time-series datasets. -Learn more about CrateDB and [](#timeseries). +Learn more about CrateDB and {ref}`timeseries`. ::: :::{grid-item} @@ -35,7 +35,7 @@ Learn more about CrateDB and [](#timeseries). :::: -## Importing and Exporting Data with CrateDB +## Importing and exporting data with CrateDB ::::{info-card} @@ -54,7 +54,7 @@ Afterwards, it will demonstrate how to export data from CrateDB to a local file system, using the [`COPY TO`] statement. For more information about how to import and export -data from/into CrateDB, please refer to [](#import-export). +data from/into CrateDB, please refer to {ref}`import-export`. ::: :::{grid-item} @@ -70,7 +70,7 @@ data from/into CrateDB, please refer to [](#import-export). -## Analyzing Time Series Data with CrateDB +## Analyzing time series data with CrateDB ::::{info-card} @@ -109,7 +109,7 @@ strategy for your time series data in CrateDB. :::: -## CrateDB in Analytics Applications +## CrateDB in analytics applications ::::{info-card} @@ -120,7 +120,7 @@ strategy for your time series data in CrateDB. Learn how Bitmovin leverages CrateDB to support real-time analytics on top of tracking data from their video streaming broadcasting system. -- [](#bitmovin) +- {ref}`bitmovin` ::: :::{grid-item} @@ -136,7 +136,7 @@ top of tracking data from their video streaming broadcasting system. :::: -## CrateDB in Industrial Applications +## CrateDB in industrial applications ::::{info-card} @@ -146,14 +146,14 @@ top of tracking data from their video streaming broadcasting system. **Industrial Analytics Platform, High-Speed Production Lines, and Logistics** Learn how ABB, Rauch, SPGo!, and TGW leverage CrateDB to support their -application platforms, high-speed shop-floor production lines, machine +application platforms, high-speed shop floor production lines, machine monitoring solutions, and logistics databases for warehouses around the world. -- [](#abb) -- [](#rauch) -- [](#spgo) -- [](#tgw) +- {ref}`abb` +- {ref}`rauch` +- {ref}`spgo` +- {ref}`tgw` ::: :::{grid-item} diff --git a/docs/topic/index.md b/docs/topic/index.md index 3d002ab5..b7602916 100644 --- a/docs/topic/index.md +++ b/docs/topic/index.md @@ -15,8 +15,8 @@ with third-party software applications, libraries, and frameworks. ::::{grid} 1 2 2 2 :margin: 4 4 0 0 -:padding: 0 :gutter: 2 +:padding: 0 :::{grid-item-card} {material-outlined}`analytics;2em` Business Intelligence :link: bi @@ -49,16 +49,6 @@ Learn how to use CrateDB for data visualization. Cluvio, Explo, Grafana, Metabase, Plotly, PyViz, Superset. ::: -:::{grid-item-card} {material-outlined}`model_training;2em` Machine Learning -:link: machine-learning -:link-type: ref -:link-alt: Machine Learning with CrateDB -Learn how to use CrateDB for machine learning. -+++ -**What's inside:** -CrateDB vector store, Text-to-SQL, Time series analysis, MLOps and model training. -::: - :::{grid-item-card} {material-outlined}`grid_goldenratio;2em` Programming Frameworks :link: framework :link-type: ref @@ -79,18 +69,35 @@ Learn how to do software testing with CrateDB. Java, Python, Testcontainers. ::: -:::{grid-item-card} {material-outlined}`stacked_line_chart;2em` Time Series Data -:link: timeseries +:::{grid-item-card} {material-outlined}`drive_file_move;2em` Warehouse Migrations +:link: migrations +:link-type: ref +:link-alt: Migrate workloads to CrateDB +Guidelines, tools, and ETL recipes, to support migrating data +warehouse workloads to CrateDB. ++++ +**What's inside:** +Java, Python, Testcontainers. +::: + +:::: + +:::{rubric} See also +::: + +::::{grid} 1 2 2 2 +:margin: 4 4 0 0 +:gutter: 2 +:padding: 0 + +:::{grid-item-card} {material-outlined}`model_training;2em` Machine Learning +:link: machine-learning :link-type: ref -:link-alt: About CrateDB for time series data analysis -Learn how to use CrateDB for time series use-cases, -and how to apply time series modeling and analysis procedures -to your data. +:link-alt: Machine Learning with CrateDB +Learn how to use CrateDB for machine learning. +++ **What's inside:** -About data-import and -export, advanced statistical -analysis, scientific computing, data visualization, -and machine learning. +CrateDB vector store, Text-to-SQL, Time series analysis, MLOps and model training. ::: :::: @@ -105,6 +112,5 @@ Data Lineage Data Visualization Programming Frameworks Software Testing -Time Series Data migrate/index ``` diff --git a/docs/topic/migrate/rockset/index.md b/docs/topic/migrate/rockset/index.md index 660eb550..b77da19d 100644 --- a/docs/topic/migrate/rockset/index.md +++ b/docs/topic/migrate/rockset/index.md @@ -288,8 +288,8 @@ Learn how to migrate your database use cases and workloads from Rockset to Crate - [MongoDB CDC Relay] :::{rubric} General I/O ::: -- [Data loading](#etl) with CrateDB. -- [](#cdc) with CrateDB. +- {ref}`Data loading ` with CrateDB. +- {ref}`cdc` with CrateDB. :::: ::::{grid-item-card} diff --git a/docs/topic/migrate/rockset/query.md b/docs/topic/migrate/rockset/query.md index 1b016cd1..812bacb1 100644 --- a/docs/topic/migrate/rockset/query.md +++ b/docs/topic/migrate/rockset/query.md @@ -70,16 +70,16 @@ under a different name, or there may be a simple workaround for your use cases. | APPROX_DISTINCT(x[, e])| `hyperloglog_distinct` | | ARRAY_CONCAT(array1, array2, ...)| `array_cat` | | ARRAY_CONTAINS(array, element)| `element = ANY (array)` | -| ARRAY_CREATE(val1, val2, ...)| `[val1, val2, ...]` or `_array(val1,val2, ...)` | +| ARRAY_CREATE(val1, val2, ...)| `[val1, val2, ...]` or `_array(val1,val2, ...)` | | ARRAY_DISTINCT(array)| `array_unique` | | ARRAY_EXCEPT(array1, array2)| `array_unique(array_difference(array1, array2))` | | ARRAY_FLATTEN(array)| `array_unnest` | -| ARRAY_INTERSECT(array1, array2)| ` array(select DISTINCT a FROM UNNEST(array1) a WHERE a IN (SELECT UNNEST(array2)))` | +| ARRAY_INTERSECT(array1, array2)| `array(select DISTINCT a FROM UNNEST(array1) a WHERE a IN (SELECT UNNEST(array2)))` | | ARRAY_JOIN(array, delimiter, nullReplacement)| [`array_to_string`] | -| ARRAY_MAP(function_name, array)| `(select array_agg(function_name(unnest)) from unnest(array))` | +| ARRAY_MAP(function_name, array)| `(select array_agg(function_name(unnest)) from unnest(array))` | | ARRAY_REMOVE(array, val)| `array_difference(array,[val])` | -| ARRAY_SHUFFLE(array)| `array(select unnest from unnest(array) ORDER BY random())` | -| ARRAY_SORT(array)| `array(select unnest from unnest(array) ORDER BY unnest)` | +| ARRAY_SHUFFLE(array)| `array(select unnest from unnest(array) ORDER BY random())` | +| ARRAY_SORT(array)| `array(select unnest from unnest(array) ORDER BY unnest)` | | ARRAY_UNION(array1, array2)| `array_unique` | | ASINH(x)| `LN(x + SQRT((x * x) + 1))` | | ATANH(x)| `0.5*ln((1+x)/(1-x))` | @@ -112,7 +112,7 @@ under a different name, or there may be a simple workaround for your use cases. | MILLISECONDS(n)| `AGE(n::LONG,0)` | | MINUTES(n)| `'n MINUTES'::INTERVAL` | | MONTHS(n)| `'n MONTHS'::INTERVAL` | -| PARSE_DATE_ISO8601(string)| `date_trunc('day',string::TIMESTAMP)` | +| PARSE_DATE_ISO8601(string)| `date_trunc('day',string::TIMESTAMP)` | | PARSE_DATETIME_ISO8601(string)| `string::TIMESTAMP` | | POSITION(substring IN string)| `strpos(string , substring)` | | POW(x, y)| `power(x,y)` | @@ -121,7 +121,7 @@ under a different name, or there may be a simple workaround for your use cases. | SEQUENCE(start, stop[, step])| `generate_series` | | SIGN(x)| See [^sign] for CrateDB <5.8 | | SPLIT(string, delimiter)[index]| `split_part(string, delimiter, index)` | -| ST_ASTEXT(geography)| See [](#ST_ASTEXT) for `POLYGON`s | +| ST_ASTEXT(geography)| See {ref}`ST_ASTEXT` for `POLYGON`s | | ST_GEOGFROMTEXT(well_known_text)| `well_known_text::geo_shape` | | ST_GEOGPOINT(longitude, latitude)| `[longitude, latitude]::geo_point` | | ST_INTERSECTS(geography_a, geography_b)| `intersects(geo_shape, geo_shape)` | diff --git a/docs/topic/timeseries/fundamentals.md b/docs/topic/timeseries/fundamentals.md deleted file mode 100644 index a9ec4d36..00000000 --- a/docs/topic/timeseries/fundamentals.md +++ /dev/null @@ -1,79 +0,0 @@ -(timeseries-basics)= -(timeseries-fundamentals)= -# Time Series Fundamentals with CrateDB - -:::{div} sd-text-muted -Learn how to conduct fundamental data analysis on large time series datasets -with CrateDB. -::: - -{tags-primary}`Metadata integration` -{tags-primary}`Advanced SQL for time series` - -:::{include} /_include/links.md -::: - -## Getting Started - -After evaluating [connectivity options](#connect), you would like to get -hands-on with CrateDB. We prepared a few introductory tutorials, some of -them in executable forms, to demonstrate CrateDB's features to work with -time series data on the spot. You may want to use them as starting points -for your own explorations. - -:::{include} /_include/card/timeseries-intro.md -::: - -:::{include} /_include/card/timeseries-explore.md -::: - -:::{include} /_include/card/timeseries-datashader.md -::: - -:::{include} /_include/card/timeseries-dask.md -::: - - -## Special Features -Working with time series data needs special feature support to enable -fluent data workflows. - -:::{rubric} Downsampling and Interpolation -::: -- [](#downsampling-timestamp-binning) -- [](#downsampling-lttb) -- [](#ni-interpolate) -- [Interpolating missing time series values] -- [](inv:crate-reference#aggregation-percentile) - -## Operations -CrateDB provides operational support to store and query large time series data -efficiently. -- [](#sharding-partitioning) -- [CrateDB partitioned table vs. TimescaleDB Hypertable] - - -:::{tip} -For more in-depth information, please visit the documentation pages about -[](#timeseries-advanced). Alternatively, you -may prefer the [](#timeseries-video). -::: - - -:::{toctree} -:hidden: - -generate/index -learn/normalize-pandas -learn/query -learn/with-metadata -::: - -:::{todo} -Outdated: [](#timeseries-generate), [](#timeseries-normalize), [Financial data collection and processing using pandas] -::: - - -[CrateDB partitioned table vs. TimescaleDB Hypertable]: https://community.cratedb.com/t/cratedb-partitioned-table-vs-timescaledb-hypertable/1713 -[Financial data collection and processing using pandas]: https://community.cratedb.com/t/automating-financial-data-collection-and-storage-in-cratedb-with-python-and-pandas-2-0-0/916 -[Interpolating missing time series values]: https://community.cratedb.com/t/interpolating-missing-time-series-values/1010