Skip to content

Commit ce44e3e

Browse files
committed
pipeline: input: tail: convert property keys to lowercase
Signed-off-by: Eduardo Silva <edsiper@gmail.com>
1 parent 18fa318 commit ce44e3e

File tree

1 file changed

+37
-37
lines changed

1 file changed

+37
-37
lines changed

pipeline/inputs/tail.md

Lines changed: 37 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -10,33 +10,33 @@ The plugin supports the following configuration parameters:
1010

1111
| Key | Description | Default |
1212
| :-- | :---------- | :------ |
13-
| `Buffer_Chunk_Size` | Set the initial buffer size to read file data. This value is used to increase buffer size. The value must be according to the [Unit Size](../../administration/configuring-fluent-bit/unit-sizes.md) specification. | `32k` |
14-
| `Buffer_Max_Size` | Set the limit of the buffer size per monitored file. When a buffer needs to be increased, this value is used to restrict the memory buffer growth. If reading a file exceeds this limit, the file is removed from the monitored file list. The value must be according to the [Unit Size](../../administration/configuring-fluent-bit/unit-sizes.md) specification. | `32k` |
15-
| `Path` | Pattern specifying a specific log file or multiple ones through the use of common wildcards. Allows multiple patterns separated by commas. | _none_ |
16-
| `Path_Key` | If enabled, it appends the name of the monitored file as part of the record. The value assigned becomes the key in the map. | _none_ |
17-
| `Exclude_Path` | Set one or multiple shell patterns separated by commas to exclude files matching certain criteria, For example, `Exclude_Path *.gz,*.zip`. | _none_ |
18-
| `Offset_Key` | If enabled, Fluent Bit appends the offset of the current monitored file as part of the record. The value assigned becomes the key in the map. | _none_ |
19-
| `Read_from_Head` | For new discovered files on start (without a database offset/position), read the content from the head of the file, not tail. | `False` |
20-
| `Refresh_Interval` | The interval of refreshing the list of watched files in seconds. | `60` |
21-
| `Rotate_Wait` | Specify the number of extra time in seconds to monitor a file once is rotated in case some pending data is flushed. | `5` |
22-
| `Ignore_Older` | Ignores files older than `ignore_older`. Supports `m`, `h`, `d` (minutes, hours, days) syntax. | Read all. |
23-
| `Skip_Long_Lines` | When a monitored file reaches its buffer capacity due to a very long line (`Buffer_Max_Size`), the default behavior is to stop monitoring that file. `Skip_Long_Lines` alter that behavior and instruct Fluent Bit to skip long lines and continue processing other lines that fit into the buffer size. | `Off` |
24-
| `Skip_Empty_Lines` | Skips empty lines in the log file from any further processing or output. | `Off` |
25-
| `DB` | Specify the database file to keep track of monitored files and offsets. | _none_ |
26-
| `DB.sync` | Set a default synchronization (I/O) method. This flag affects how the internal SQLite engine do synchronization to disk, for more details about each option see [the SQLite documentation](https://www.sqlite.org/pragma.html#pragma_synchronous). Most scenarios will be fine with `normal` mode. If you need full synchronization after every write operation set `full` mode. `full` has a high I/O performance cost. Values: `Extra`, `Full`, `Normal`, `Off`. | `normal` |
27-
| `DB.locking` | Specify that the database will be accessed only by Fluent Bit. Enabling this feature helps increase performance when accessing the database but restricts externals tool from querying the content. | `false` |
28-
| `DB.journal_mode` | Sets the journal mode for databases (`WAL`). Enabling `WAL` provides higher performance. `WAL` isn't compatible with shared network file systems. | `WAL` |
29-
| `DB.compare_filename` | This option determines whether to review both `inode` and `filename` when retrieving stored file information from the database. `true` verifies both `inode` and `filename`, while `false` checks only the `inode`. To review the `inode` and `filename` in the database, refer [see `keep_state`](#tailing-files-keeping-state). | `false` |
30-
| `Mem_Buf_Limit` | Set a memory limit that Tail plugin can use when appending data to the engine. If the limit is reached, it will be paused. When the data is flushed it resumes. | _none_ |
31-
| `Exit_On_Eof` | When reading a file will exit as soon as it reach the end of the file. Used for bulk load and tests. | `false` |
32-
| `Parser` | Specify the name of a parser to interpret the entry as a structured message. | _none_ |
33-
| `Key` | When a message is unstructured (no parser applied), it's appended as a string under the key name `log`. This option lets you define an alternative name for that key. | `log` |
34-
| `Inotify_Watcher` | Set to `false` to use file stat watcher instead of `inotify`. | `true` |
35-
| `Tag` | Set a tag with `regexextract` fields that will be placed on lines read. For example, `kube.<namespace_name>.<pod_name>.<container_name>.<container_id>`. Tag expansion is supported: if the tag includes an asterisk (`*`), that asterisk will be replaced with the absolute path of the monitored file, with slashes replaced by dots. See [Workflow of Tail + Kubernetes Filter](../filters/kubernetes.md#workflow-of-tail--kubernetes-filter). | _none_ |
36-
| `Tag_Regex` | Set a regular expression to extract fields from the filename. For example: `(?<pod_name>[a-z0-9](?:[-a-z0-9]*[a-z0-9])?(?:\\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*)_(?<namespace_name>[^_]+)_(?<container_name>.+)-(?<container_id>[a-z0-9]{64})\.log$`. | _none_ |
37-
| `Static_Batch_Size` | Set the maximum number of bytes to process per iteration for the monitored static files (files that already exist upon Fluent Bit start). | `50M` |
38-
| `File_Cache_Advise` | Set the `posix_fadvise` in `POSIX_FADV_DONTNEED` mode. This reduces the usage of the kernel file cache. This option is ignored if not running on Linux. | `On` |
39-
| `Threaded` | Indicates whether to run this input in its own [thread](../../administration/multithreading.md#inputs). | `false` |
13+
| `buffer_chunk_size` | Set the initial buffer size to read file data. This value is used to increase buffer size. The value must be according to the [Unit Size](../../administration/configuring-fluent-bit/unit-sizes.md) specification. | `32k` |
14+
| `buffer_max_size` | Set the limit of the buffer size per monitored file. When a buffer needs to be increased, this value is used to restrict the memory buffer growth. If reading a file exceeds this limit, the file is removed from the monitored file list. The value must be according to the [Unit Size](../../administration/configuring-fluent-bit/unit-sizes.md) specification. | `32k` |
15+
| `path` | Pattern specifying a specific log file or multiple ones through the use of common wildcards. Allows multiple patterns separated by commas. | _none_ |
16+
| `path_key` | If enabled, it appends the name of the monitored file as part of the record. The value assigned becomes the key in the map. | _none_ |
17+
| `exclude_path` | Set one or multiple shell patterns separated by commas to exclude files matching certain criteria, For example, `exclude_path *.gz,*.zip`. | _none_ |
18+
| `offset_key` | If enabled, Fluent Bit appends the offset of the current monitored file as part of the record. The value assigned becomes the key in the map. | _none_ |
19+
| `read_from_head` | For new discovered files on start (without a database offset/position), read the content from the head of the file, not tail. | `false` |
20+
| `refresh_interval` | The interval of refreshing the list of watched files in seconds. | `60` |
21+
| `rotate_wait` | Specify the number of extra time in seconds to monitor a file once is rotated in case some pending data is flushed. | `5` |
22+
| `ignore_older` | Ignores files older than `ignore_older`. Supports `m`, `h`, `d` (minutes, hours, days) syntax. | Read all. |
23+
| `skip_long_lines` | When a monitored file reaches its buffer capacity due to a very long line (`buffer_max_size`), the default behavior is to stop monitoring that file. `skip_long_lines` alter that behavior and instruct Fluent Bit to skip long lines and continue processing other lines that fit into the buffer size. | `off` |
24+
| `skip_empty_lines` | Skips empty lines in the log file from any further processing or output. | `off` |
25+
| `db` | Specify the database file to keep track of monitored files and offsets. | _none_ |
26+
| `db.sync` | Set a default synchronization (I/O) method. This flag affects how the internal SQLite engine do synchronization to disk, for more details about each option see [the SQLite documentation](https://www.sqlite.org/pragma.html#pragma_synchronous). Most scenarios will be fine with `normal` mode. If you need full synchronization after every write operation set `full` mode. `full` has a high I/O performance cost. Values: `extra`, `full`, `normal`, `off`. | `normal` |
27+
| `db.locking` | Specify that the database will be accessed only by Fluent Bit. Enabling this feature helps increase performance when accessing the database but restricts externals tool from querying the content. | `false` |
28+
| `db.journal_mode` | Sets the journal mode for databases (`wal`). Enabling `wal` provides higher performance. `wal` isn't compatible with shared network file systems. | `wal` |
29+
| `db.compare_filename` | This option determines whether to review both `inode` and `filename` when retrieving stored file information from the database. `true` verifies both `inode` and `filename`, while `false` checks only the `inode`. To review the `inode` and `filename` in the database, refer [see `keep_state`](#tailing-files-keeping-state). | `false` |
30+
| `mem_buf_limit` | Set a memory limit that Tail plugin can use when appending data to the engine. If the limit is reached, it will be paused. When the data is flushed it resumes. | _none_ |
31+
| `exit_on_eof` | When reading a file will exit as soon as it reach the end of the file. Used for bulk load and tests. | `false` |
32+
| `parser` | Specify the name of a parser to interpret the entry as a structured message. | _none_ |
33+
| `key` | When a message is unstructured (no parser applied), it's appended as a string under the key name `log`. This option lets you define an alternative name for that key. | `log` |
34+
| `inotify_watcher` | Set to `false` to use file stat watcher instead of `inotify`. | `true` |
35+
| `tag` | Set a tag with `regexextract` fields that will be placed on lines read. For example, `kube.<namespace_name>.<pod_name>.<container_name>.<container_id>`. Tag expansion is supported: if the tag includes an asterisk (`*`), that asterisk will be replaced with the absolute path of the monitored file, with slashes replaced by dots. See [Workflow of Tail + Kubernetes Filter](../filters/kubernetes.md#workflow-of-tail--kubernetes-filter). | _none_ |
36+
| `tag_regex` | Set a regular expression to extract fields from the filename. For example: `(?<pod_name>[a-z0-9](?:[-a-z0-9]*[a-z0-9])?(?:\\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*)_(?<namespace_name>[^_]+)_(?<container_name>.+)-(?<container_id>[a-z0-9]{64})\.log$`. | _none_ |
37+
| `static_batch_size` | Set the maximum number of bytes to process per iteration for the monitored static files (files that already exist upon Fluent Bit start). | `50M` |
38+
| `file_cache_advise` | Set the `posix_fadvise` in `POSIX_FADV_DONTNEED` mode. This reduces the usage of the kernel file cache. This option is ignored if not running on Linux. | `on` |
39+
| `threaded` | Indicates whether to run this input in its own [thread](../../administration/multithreading.md#inputs). | `false` |
4040

4141
## Buffers and memory management
4242

@@ -131,7 +131,7 @@ Fluent Bit 1.8 and later supports multiline core capabilities for the Tail input
131131
- Multiline Core
132132
- Old Multiline
133133

134-
### Multiline Core (v1.8)
134+
### Multiline Core
135135

136136
The new multiline core is exposed by the following configuration:
137137

@@ -148,7 +148,7 @@ The new multiline core is exposed by the following configuration:
148148
- `multiline_flush`
149149
- `docker_mode`
150150

151-
### Multiline and containers (v1.8)
151+
### Multiline and containers
152152

153153
If you are running Fluent Bit to process logs coming from containers like Docker or CRI, you can use the built-in modes. This helps reassemble multiline messages originally split by Docker or CRI:
154154

@@ -188,20 +188,20 @@ For the old multiline configuration, the following options exist to configure th
188188

189189
| Key | Description | Default |
190190
| :--- | :--- | :--- |
191-
| `Multiline` | If enabled, the plugin will try to discover multiline messages and use the proper parsers to compose the outgoing messages. When this option is enabled the Parser option isn't used. | `Off` |
192-
| `Multiline_Flush` | Wait period time in seconds to process queued multiline messages. | `4` |
193-
| `Parser_Firstline` | Name of the parser that matches the beginning of a multiline message. The regular expression defined in the parser must include a group name (named `capture`), and the value of the last match group must be a string. | _none_ |
194-
| `Parser_N` | Optional. Extra parser to interpret and structure multiline entries. This option can be used to define multiple parsers. For example, `Parser_1 ab1`, `Parser_2 ab2`, `Parser_N abN`. | _none_ |
191+
| `multiline` | If enabled, the plugin will try to discover multiline messages and use the proper parsers to compose the outgoing messages. When this option is enabled the Parser option isn't used. | `off` |
192+
| `multiline_flush` | Wait period time in seconds to process queued multiline messages. | `4` |
193+
| `parser_firstline` | Name of the parser that matches the beginning of a multiline message. The regular expression defined in the parser must include a group name (named `capture`), and the value of the last match group must be a string. | _none_ |
194+
| `parser_N` | Optional. Extra parser to interpret and structure multiline entries. This option can be used to define multiple parsers. For example, `parser_1 ab1`, `parser_2 ab2`, `parser_N abN`. | _none_ |
195195

196196
### Old Docker mode configuration parameters
197197

198198
Docker mode exists to recombine JSON log lines split by the Docker daemon due to its line length limit. To use this feature, configure the tail plugin with the corresponding parser and then enable Docker mode:
199199

200200
| Key | Description | Default |
201201
| :--- | :--- | :--- |
202-
| `Docker_Mode` | If enabled, the plugin will recombine split Docker log lines before passing them to any parser. This mode can't be used at the same time as Multiline. | `Off` |
203-
| `Docker_Mode_Flush` | Wait period time in seconds to flush queued unfinished split lines. | `4` |
204-
| `Docker_Mode_Parser` | Specify an optional parser for the first line of the Docker multiline mode. The parser name to be specified must be registered in the `parsers.conf` file. | _none_ |
202+
| `docker_mode` | If enabled, the plugin will recombine split Docker log lines before passing them to any parser. This mode can't be used at the same time as Multiline. | `Off` |
203+
| `docker_mode_flush` | Wait period time in seconds to flush queued unfinished split lines. | `4` |
204+
| `docker_mode_parser` | Specify an optional parser for the first line of the Docker multiline mode. The parser name to be specified must be registered in the `parsers.conf` file. | _none_ |
205205

206206
## Get started
207207

@@ -264,7 +264,7 @@ Dec 14 06:41:08 Exception in thread "main" java.lang.RuntimeException: Something
264264
at com.myproject.module.MyProject.main(MyProject.java:6)
265265
```
266266

267-
Specify a `Parser_Firstline` parameter that matches the first line of a multiline event. When a match is made, Fluent Bit reads all future lines until another match with `Parser_Firstline` is made.
267+
Specify a `parser_firstline` parameter that matches the first line of a multiline event. When a match is made, Fluent Bit reads all future lines until another match with `parser_firstline` is made.
268268

269269
In this case you can use the following parser, which extracts the time as `time` and the remaining portion of the multiline as `log`.
270270

0 commit comments

Comments
 (0)