From 8403ac487f65607d692f55c4a3f267045a31f796 Mon Sep 17 00:00:00 2001 From: David Newhall II Date: Tue, 20 Feb 2024 07:22:26 -0800 Subject: [PATCH] update readme --- README.md | 72 +++++++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 68 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 313afc6..be9e6a1 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,71 @@ -# Log buffer +# Fog Willow -This app listens on a UDP port. Parses packets with a specific signature. Buffers the packet payloads and writes them to disk. +## Log Buffer -Allows us to efficiently transport logs out of the PHP app and directly to the system with the hard disk where we store log files. +This app listens on a UDP port. Parses packets with a specific format. Buffers the packet payloads and writes them to disk. -See [config file example](https://github.com/Notifiarr/fogwillow/blob/main/fog.conf). \ No newline at end of file +Allows us to efficiently transport logs out of the PHP app and +directly to the system with the hard disk where we store log files. +We use this because NFS was too slow, and we can trade a bit of reliability for faster log exfiltration. + + +## Packet Format + +The first line of the packet is an integer that signals how many settings to expect. +A minimum of 1 settings is required: `filepath=/path` +Other recognized [settings](https://github.com/Notifiarr/fogwillow/blob/main/pkg/fog/set.go#L8) are: +`flush=true`, `truncate=true`, `delete=true`, `password=p4ssw0rd` + +### Example Packets + +```text +1 +filepath=/path/to/file.log +[INFO] this is a log line +``` + +```text +2 +filepath=/path/to/file.log +password=option4lPassw0rd +[INFO] this is a log line +``` + +Create a packet with `echo` and `netcat`: +```bash +echo -n "1\nfilepath=/tmp/filename.txt\nfile content goes here\nline 2 in the file" | nc -uw0 127.0.0.1 9000 +``` + +## Metrics + +Has a Prometheus exporter built in with juicy [metrics](https://github.com/Notifiarr/fogwillow/blob/main/pkg/metrics/metrics.go)!
+[![grafana](https://github.com/Notifiarr/fogwillow/wiki/images/grafana-thumb.png "grafana images")](https://github.com/Notifiarr/fogwillow/wiki/images/grafana-thumb.png) + +## PHP Client + +Very simple procedure we use for testing. + +```php +$socket = socket_create(AF_INET, SOCK_DGRAM, SOL_UDP); + +function socket_put_contents($socket, $outputfile, $line, $host, $length = 8000) +{ + if (strlen($line) > $length) { + foreach (str_split($line, $length) as $piece) { + $loggerPayload = "1\nfilepath=".$outputfile."\n".$piece; + usleep(1); + socket_sendto($socket, $loggerPayload, strlen($loggerPayload), 0, $host, 9000); + } + } else { + $loggerPayload = "1\nfilepath=" . $outputfile . "\n" . $line; + socket_sendto($socket, $loggerPayload, strlen($loggerPayload), 0, $host, 9000); + } +} +``` + +# Usage + +We run this from a Docker container directly on a Synology NAS. Using this image:
+`ghcr.io/Notifiarr/fogwillow:main` + +Mount `/config` and give it a `/config/fog.conf` file that looks like the [example](https://github.com/Notifiarr/fogwillow/blob/main/fog.conf). You also want to mount a place to store files, and set that to `output_path` in the config file. \ No newline at end of file