You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/integrate/rsyslog/tutorial.md
+28-23Lines changed: 28 additions & 23 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,30 +1,35 @@
1
1
(rsyslog-tutorial)=
2
-
# Storing server logs on CrateDB for fast search and aggregations
2
+
# Store server logs on CrateDB for fast search and aggregations
3
3
4
4
## Introduction
5
5
6
-
Did you know that CrateDB can be a great store for your server logs?
6
+
CrateDB stores server logs efficiently and makes them easy to query.
7
7
8
-
If you have been using log aggregation tools or even some of the most advanced commercial SIEM systems, you have probably experienced the same frustrations I have:
8
+
Common pain points with traditional log stacks and SIEMs include:
9
9
10
-
* timeouts when searching logs over long periods of time
11
-
*a complex and proprietary query syntax
12
-
*difficulties integrating queries on logs data into application monitoring dashboards
10
+
* timeouts when searching across long time ranges
11
+
*proprietary, complex query syntaxes
12
+
*awkward integrations with application monitoring dashboards
13
13
14
-
Storing server logs on CrateDB solves these problems, it allows to query the logs with standard SQL and from any tool supporting the PostgreSQL protocol; its unique indexing also makes full-text queries and aggregations super fast.
15
-
Let me show you an example.
14
+
CrateDB addresses these issues: query logs with standard SQL from any
15
+
PostgreSQL‑compatible tool, and use full‑text search and aggregations
16
+
backed by efficient indexes. The sections below walk through a minimal
17
+
setup.
16
18
17
19
## Setup
18
20
19
21
### CrateDB
20
22
21
-
First, we will need an instance of CrateDB, it may be best to have a dedicated cluster for this purpose, to separate the monitoring system from the systems being monitored, but for the purpose of this demo we can just have a singlenode cluster on a docker container:
23
+
First, start CrateDB. For production, use a dedicated cluster. For this demo, run a single‑node container:
Next, we need a table to store the logs, let's connect to`http://localhost:4200/#!/console` and run:
32
+
Next, create a table for logs. Open`http://localhost:4200/#!/console` and run:
28
33
29
34
```sql
30
35
CREATETABLEdoc.systemevents (
@@ -39,7 +44,7 @@ CREATE TABLE doc.systemevents (
39
44
SysLogTag TEXT
40
45
);
41
46
```
42
-
Tip: if you are on a headless system you can also run queries with {ref}`command-line tools <connect-cli>`.
47
+
Tip: On headless systems, run queries with the {ref}`command-line tools <connect-cli>`.
43
48
44
49
Then we need an account for the logging system:
45
50
@@ -59,11 +64,10 @@ GRANT DML ON TABLE doc.systemevents TO rsyslog;
59
64
We will use [rsyslog](https://github.com/rsyslog/rsyslog) to send the logs to CrateDB, for this setup we need `rsyslog` v8.2202 or higher and the `ompgsql` module:
Let's now configure it to use the account we created earlier:
@@ -79,20 +83,20 @@ If you are interested in more advanced setups involving queuing for additional r
79
83
80
84
### MediaWiki
81
85
82
-
Now let's imagine that we want to run a container with [MediaWiki](https://www.mediawiki.org/wiki/MediaWiki)to host an intranet and we want all logs to go to CrateDB, we can just deploy this with:
86
+
To generate logs, run a [MediaWiki](https://www.mediawiki.org/wiki/MediaWiki)container and forward its logs to rsyslog:
0 commit comments