How do I parse the logs daily and produce monthly reports?

[jheiselman]

We parse our logs offline at the end of each day and generate static HTML reports. I get that I can use --preserve and --restore, but when I did that it looks like the resulting HTML file only includes data from yesterday and not from the day before.
We are just starting to explore using GoAccess as a replacement for AWStats, so sorry if it's obvious, but we're coming from a very different setup.

[allinurl]

No worries. You’ll want to use --persist instead of --preserve, and then follow up with --restore. You can test it on a smaller subset first if that helps. Let me know how it goes.

[jheiselman]

when I looked at the generated reports today, I noticed that they said the correct time frame (the last two days), but all of the charts only seemed to show yesterday's data none from two days ago. I think I'll process logs going back to the beginning of the year and see what the reports look like.

Persist and restore seem to be working. The log format is definitely correct. I tested on a single day's file over and over until I got that worked out and the results for that single day look spot on.

[jheiselman]

I have found my error. The graphs show a fixed number of data points. If I set --date-spec min, then it simply cannot show more than a single day.
This does highlight one drawback to GoAccess. There does not seem to be a way to view the graphs for any given day, week, or month without simply running the tool multiple times to generate several different HTML outputs. And then I would need to build some sort of front-end to this pages for our end users. Is this assertion correct?

[allinurl]

Take a look, it may do what you're after:

--max-items=<num>
Maximum number of items to display per panel (1 to n). Only static HTML, CSV, and JSON output allow more than the default of 366 items (or 50 in real-time HTML).

Let me know if that helps.

[jheiselman]

I'm actually fine with keeping the graphs to a higher level. Am I correct in that I need separate graphs though for each month with my other options? Using --date-spec day and then wanting to get each month to be able to be displayed separately from each other.

[allinurl]

The max-items setting determines how many entries are displayed in the graphs and tables. If you're rendering data non-real-time, you may be able to cover more than a single day's worth of data and thus no need for different HTML reports.