v11

Fluentd v11 ideas

Syntax:

• must this feature will be included
• likely this feature will be included depending on implementation
• feedback this feature will not be included unless there're good feedback

Core

• SocketManager: Restarting Fluentd doesn't cause downtime by keeping listening sockets alive must
• Worker processes: One configuration file can include multiple Fluentd instances feedback
• Windows support: Fluentd runs on windows must
• Gemfile: Fluentd loads Gemfile to manage versions of plugins likely

Configuration file

• Ruby-based DSL:
• Labels: Add new <label> section feedback
• Error stream: Broken records go to a special stream to handle them separately must
  • implemented using label
• Log stream: Internal logs go to a special stream to handle them separately likely
  • implemented using label
• Nested-Match: <match> and <source> sections can have <match> section likely
• Filter-syntax: Add new <filter> syntax feedback
• Copy-syntax: Add new <copy> syntax feedback
• Variables in config: Users can embed environment variables to config files with #{...} syntax likely
• Literals in config: Values in configuration file has type information feedback
• Multi-line string literal feedback

Plugin API

• Per-plugin log level: feedback
• Namespace changed from Fluentd::* to Fluentd::Plugin::*

Plugins need to be ported

• High priority
  • in_http
  • in_unix (to support PHP loggers)
  • in_exec
  • in_tail
  • out_exec
  • out_exec_filter
  • buf_file
  • out_file
• Medium priority
  • out_roundrobin
  • in_syslog
  • Done: out_null
• Low priority, or unnecessary
  • out_unix
  • in_debug_agent
  • in_gc_stat
  • in_monitor_agent
  • in_object_space
  • in_status
  • out_test

Documents

• Fluentdプラグインのv10→v11移植ガイド
• Fluentdプラグインのv10→v11移植ガイド filterぷらぎん編

TODO & Need contribution!

• buf_file is not implemented yet!
• Done: We need list of standard plugins which is not ported to v11 yet (out_file, in_syslog, etc.) !
  • Then, we need to port them!
• SocketManager doesn't work on Windows
• Fluentd needs to be a service on Windows instead of daemon. But it's not implemented
• Done: Fluentd code needs more comments so that users can read code easily