From c9491a62eca85077feb6e56a7efcf7035b286e25 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Volkan=20Yaz=C4=B1c=C4=B1?= Date: Wed, 3 Jul 2024 14:08:52 +0200 Subject: [PATCH 1/3] Improve Log4j 2 example - Switch from deprecated JSON Layout to its successor JSON Template Layout - Use variable names in capitals (the recommended convention by Log4j) - Explain configuration using AsciiDoc callouts --- docs/index.asciidoc | 69 +++++++++++++++++++++++---------------------- 1 file changed, 35 insertions(+), 34 deletions(-) diff --git a/docs/index.asciidoc b/docs/index.asciidoc index 3d8756c..a51406f 100644 --- a/docs/index.asciidoc +++ b/docs/index.asciidoc @@ -28,31 +28,41 @@ Like stdin and file inputs, each event is assumed to be one line of text. Can either accept connections from clients or connect to a server, depending on `mode`. -===== Accepting log4j2 logs - -Log4j2 can send JSON over a socket, and we can use that combined with our tcp -input to accept the logs. - -First, we need to configure your application to send logs in JSON over a -socket. The following log4j2.xml accomplishes this task. - -Note, you will want to change the `host` and `port` settings in this -configuration to match your needs. - - - - - - - - - - - - - - -To accept this in Logstash, you will want tcp input and a date filter: +===== Accepting Log4j 2 logs + +Log4j 2 can write ECS-compliant JSON-formatted log events to a TCP socket. +We can combine with our TCP input to accept the logs from applications using Log4j 2. + +First, we need to configure your application to write JSON-formatted logs to a TCP socket: + +.Example `log4j2.xml` configuration for writing JSON-formatted logs to Logstash TCP input +[source,xml] +---- + + + + + eventTemplateUri="classpath:EcsLayout.json" + nullEventDelimiterEnabled="true"/> + + + + + + + + +---- +<1> Using Socket Appender to write logs to a TCP socket – make sure to *change the `host` attribute* to match your setup +<2> Using https://logging.apache.org/log4j/2.x/manual/json-template-layout.html[JSON Template Layout] to encode log events in JSON +<3> Using the ECS (Elastic Common Schema) layout bundled with JSON Template Layout +<4> Configuring that written log events should be terminated with a null (i.e., `\0`) character + +To accept this in Logstash, you will want a TCP input: input { tcp { @@ -61,15 +71,6 @@ To accept this in Logstash, you will want tcp input and a date filter: } } -and add a date filter to take log4j2's `timeMillis` field and use it as the -event timestamp - - filter { - date { - match => [ "timeMillis", "UNIX_MS" ] - } - } - [id="plugins-{type}s-{plugin}-ecs_metadata"] ==== Event Metadata and the Elastic Common Schema (ECS) From 83ca9988c3c357c75aada35b91c96d6bf1026722 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Volkan=20Yaz=C4=B1c=C4=B1?= Date: Tue, 18 Feb 2025 20:42:43 +0100 Subject: [PATCH 2/3] Mention that `port` needs to be adapted Co-authored-by: Ry Biesemeyer --- docs/index.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/index.asciidoc b/docs/index.asciidoc index a51406f..918d50b 100644 --- a/docs/index.asciidoc +++ b/docs/index.asciidoc @@ -57,7 +57,7 @@ First, we need to configure your application to write JSON-formatted logs to a T ---- -<1> Using Socket Appender to write logs to a TCP socket – make sure to *change the `host` attribute* to match your setup +<1> Using Socket Appender to write logs to a TCP socket – make sure to *change the `host` and `port` attributes* to match your setup <2> Using https://logging.apache.org/log4j/2.x/manual/json-template-layout.html[JSON Template Layout] to encode log events in JSON <3> Using the ECS (Elastic Common Schema) layout bundled with JSON Template Layout <4> Configuring that written log events should be terminated with a null (i.e., `\0`) character From 5116d6826e3525ed792282aea01ff8b10e232c59 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Volkan=20Yaz=C4=B1c=C4=B1?= Date: Tue, 18 Feb 2025 22:12:50 +0100 Subject: [PATCH 3/3] Remove null delimiter from JTL --- docs/index.asciidoc | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/docs/index.asciidoc b/docs/index.asciidoc index 918d50b..09c41c8 100644 --- a/docs/index.asciidoc +++ b/docs/index.asciidoc @@ -46,8 +46,7 @@ First, we need to configure your application to write JSON-formatted logs to a T - eventTemplateUri="classpath:EcsLayout.json" - nullEventDelimiterEnabled="true"/> + eventTemplateUri="classpath:EcsLayout.json"/> @@ -60,7 +59,6 @@ First, we need to configure your application to write JSON-formatted logs to a T <1> Using Socket Appender to write logs to a TCP socket – make sure to *change the `host` and `port` attributes* to match your setup <2> Using https://logging.apache.org/log4j/2.x/manual/json-template-layout.html[JSON Template Layout] to encode log events in JSON <3> Using the ECS (Elastic Common Schema) layout bundled with JSON Template Layout -<4> Configuring that written log events should be terminated with a null (i.e., `\0`) character To accept this in Logstash, you will want a TCP input: