Skip to content

feat(k8saudit): file watching#1167

Open
RichardoC wants to merge 5 commits intofalcosecurity:mainfrom
RichardoC:feature/k8saudit-file-watching
Open

feat(k8saudit): file watching#1167
RichardoC wants to merge 5 commits intofalcosecurity:mainfrom
RichardoC:feature/k8saudit-file-watching

Conversation

@RichardoC
Copy link
Contributor

What type of PR is this?

/kind feature

Any specific area of the project related to this PR?

/area plugins

What this PR does / why we need it:
Add continuous file watching capability for audit logs using the
tail:// URL scheme. This allows monitoring files for new entries
and handles log rotation via inode detection and file truncation.

Which issue(s) this PR fixes:

Fixes #191

Special notes for your reviewer: Generated with Claude Code. Full transcript attached
claude-log.txt

RichardoC and others added 3 commits January 22, 2026 16:59
Add continuous file watching capability for audit logs using the
tail:// URL scheme. This allows monitoring files for new entries
and handles log rotation via inode detection and file truncation.

Closes falcosecurity#191

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Richard Tweed <RichardoC@users.noreply.github.com>
Add documentation for the new tail:// URL scheme and
watchPollIntervalMs configuration option.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Richard Tweed <RichardoC@users.noreply.github.com>
Signed-off-by: Richard Tweed <RichardoC@users.noreply.github.com>
@poiana
Copy link
Contributor

poiana commented Jan 22, 2026

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: RichardoC
Once this PR has been reviewed and has the lgtm label, please assign leogr for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@poiana poiana requested review from irozzo-1A and leogr January 22, 2026 17:14
@poiana poiana added the size/L label Jan 22, 2026
@RichardoC RichardoC changed the title Feature/k8saudit file watching feat(k8saudit): file watching Jan 22, 2026
@github-actions
Copy link

Rules files suggestions

rules

Comparing b4696a614148482909f4e483c2906685abfe2151 with latest tag plugins/k8saudit/v0.16.0

No changes detected

Copy link
Contributor

@irozzo-1A irozzo-1A left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @RichardoC, thanks for your contribution. I suggest using file instead of tail to have a more standard URI, and using an event based approach instead of polling.

return k.OpenWebServer(u.Host, u.Path, false)
case "https":
return k.OpenWebServer(u.Host, u.Path, true)
case "tail":
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
case "tail":
case "file":

This plugin supports consuming Kubernetes Audit Events coming from the [Webhook backend](https://kubernetes.io/docs/tasks/debug/debug-cluster/audit/#webhook-backend) or from a file. For webhooks, the plugin embeds a web server that listens on a configurable port and accepts POST requests. The posted JSON object comprises one or more events. The web server of the plugin can be configured as part of the plugin's init configuration and open parameters. For files, the plugin expects content to be in [JSONL format](https://jsonlines.org/), where each line represents a JSON object, containing one or more audit events.

The expected way of using the plugin with Falco is through a Webhook. File reading support can be used with Stratoshark or testing and development.
The expected way of using the plugin with Falco is through a Webhook. File reading support can be used with Stratoshark or testing and development. The `tail://` scheme enables continuous file watching with log rotation support, useful for reading audit logs written to disk by the API server.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The expected way of using the plugin with Falco is through a Webhook. File reading support can be used with Stratoshark or testing and development. The `tail://` scheme enables continuous file watching with log rotation support, useful for reading audit logs written to disk by the API server.
The expected way of using the plugin with Falco is through a Webhook. File reading support can be used with Stratoshark or testing and development. The `file://` scheme enables continuous file watching with log rotation support, useful for reading audit logs written to disk by the API server.

**Open Parameters**:
- `http://<host>:<port>/<endpoint>`: Opens an event stream by listening on an HTTP web server
- `https://<host>:<port>/<endpoint>`: Opens an event stream by listening on an HTTPS web server
- `tail://<filepath>`: Opens an event stream by continuously watching a file for new audit events, similar to `tail -f`. Handles log rotation (inode changes) and file truncation automatically. Example: `tail:///var/log/kube-apiserver/audit.log`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- `tail://<filepath>`: Opens an event stream by continuously watching a file for new audit events, similar to `tail -f`. Handles log rotation (inode changes) and file truncation automatically. Example: `tail:///var/log/kube-apiserver/audit.log`
- `file://<filepath>`: Opens an event stream by continuously watching a file for new audit events, similar to `tail -f`. Handles log rotation (inode changes) and file truncation automatically. Example: `file:///var/log/kube-apiserver/audit.log`

UseAsync bool `json:"useAsync" jsonschema:"title=Use async extraction,description=If true then async extraction optimization is enabled (Default: true),default=true"`
MaxEventSize uint64 `json:"maxEventSize" jsonschema:"title=Maximum event size,description=Maximum size of single audit event (Default: 262144),default=262144"`
WebhookMaxBatchSize uint64 `json:"webhookMaxBatchSize" jsonschema:"title=Maximum webhook request size,description=Maximum size of incoming webhook POST request bodies (Default: 12582912),default=12582912"`
WatchPollIntervalMs uint64 `json:"watchPollIntervalMs" jsonschema:"title=Watch poll interval,description=Polling interval in milliseconds when watching a file with tail:// scheme (Default: 250),default=250"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can avoid this if we use an event based approach e.g. inotify

// OpenFileTail opens a source.Instance that continuously watches a file for
// new K8S Audit Events, similar to "tail -f". It handles log rotation by
// detecting file truncation or inode changes.
func (k *Plugin) OpenFileTail(path string) (source.Instance, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Polling and opening the file each time is not very efficient way of tailing the file content, what about using fsnotify. You can take inspiration from the snippet below:

package main

import (
	"fmt"
	"io"
	"os"

	"github.com/fsnotify/fsnotify"
)

func main() {
	filePath := "test.log"
	
	watcher, _ := fsnotify.NewWatcher()
	defer watcher.Close()

	// 1. Open the file and seek to the end
	file, _ := os.Open(filePath)
	file.Seek(0, io.SeekEnd)

	// 2. Start watching the file for changes
	watcher.Add(filePath)

	fmt.Println("Tailing test.log... (supports rotation/truncation)")

	for {
		select {
		case event, ok := <-watcher.Events:
			if !ok {
				return
			}

			// Handle Write: New data added
			if event.Op&fsnotify.Write == fsnotify.Write {
				io.Copy(os.Stdout, file)
			}

			// Handle Rename/Remove: This is Log Rotation
			if event.Op&(fsnotify.Rename|fsnotify.Remove) == fsnotify.Remove || event.Op&fsnotify.Rename == fsnotify.Rename {
				fmt.Println("\n[File rotated, re-opening...]")
				file.Close()
				
				// Wait/Retry until the new file exists
				for {
					newFile, err := os.Open(filePath)
					if err == nil {
						file = newFile
						watcher.Add(filePath) // Re-watch the new file
						break
					}
                     // backoff
				}
			}

		case err, ok := <-watcher.Errors:
			if !ok {
				return
			}
			fmt.Println("Error:", err)
		}
	}
}

Co-authored-by: Iacopo Rozzo <iacopo@sysdig.com>
Signed-off-by: Richard Tweed <RichardoC@users.noreply.github.com>
@RichardoC
Copy link
Contributor Author

Hi @RichardoC, thanks for your contribution. I suggest using file instead of tail to have a more standard URI, and using an event based approach instead of polling.

Thank you for the thorough review @irozzo-1A , making those changes now

@RichardoC
Copy link
Contributor Author

Claude log for the changes
Uploading claude-log-1.txt…

Replace polling-based file watching with fsnotify for better efficiency.

- Use fsnotify to watch parent directory (per maintainer recommendation)
- Rename scheme from tail:// to file://
- Remove watchPollIntervalMs config (no longer needed)
- Rename test package from tail to filewatch

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Richard Tweed <RichardoC@users.noreply.github.com>
@RichardoC RichardoC force-pushed the feature/k8saudit-file-watching branch from 0613d65 to c12e1e0 Compare January 26, 2026 17:26
@RichardoC
Copy link
Contributor Author

The cargo install issue in ci seems unrelated to my changes

require (
github.com/alecthomas/jsonschema v0.0.0-20220216202328-9eeeec9d044b
github.com/falcosecurity/plugin-sdk-go v0.8.3
github.com/fsnotify/fsnotify v1.9.0 // indirect
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why // indirect? 🤔

This should be a direct dependency since it's explicitly imported in source.go

limitations under the License.
*/

package filewatch_test
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since there's no filewatch package, why is this file here? 🤔

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature Request: Let K8saudit plugin watch/tail file and parse new lines.

4 participants