plugins_history table filled with many "missing-in-last-scan" entries

[dsnz]

hello,

even with hourly db cleanup the plugins_history gets quickly new entries and in some cases has reached up to 30_000

I inspected the table with SQLite cli and found that there are 2 types of entries

"new" and
"missing-in-last-scan" (!)

the "new" ones are the real ones
and when a "new" is entered there are also entered many "missing-in-last-scan" before it ("before" in the numeric id order)
these entries can be many hundred often and I saw one time 5500 such entries !
what's extra strange is that many have older created-changed dates (even many dates before)

are these normal entries and needed for operation or something is wrong with my settings ?

ref discussion

[jokob-sk]

This behavior is expected and also depends on the plugin.

plugins_history records state transitions, not just successful detections.

• new = object detected in the current scan
• missing-in-last-scan = object previously known to that plugin but not returned in the current scan

If a specific plugin suddenly reports fewer results (config or scope change, temporary failure, first run after re-enable, etc.), all objects previously known to that plugin are marked as missing-in-last-scan. Depending on the plugin and how many objects it tracks, this can result in hundreds or thousands of entries in a single scan.

new entries are inserted after missing entries by design.

Older timestamps are normal: history rows preserve original object timestamps, so ID order ≠ time order.

Cleanup only removes entries outside the retention window and does not suppress valid state transitions.

[dsnz]

so, you say that these entries are definitely needed for the correct operation of netalertx ? (not just .. how to say .. cosmetic entries ..)

if yes then it's fine for me

[ though DB cleanup should be scheduled by default to be frequent; for me will need to get it down to every 30 minutes ]

[jokob-sk]

NAX gives the flexibility to fit many setups, so feel free to decrease the time between executions. See performance tips for details: https://jokob-sk.github.io/NetAlertX/PERFORMANCE/?h=perfor

[dsnz]

I have a few very fast scanners (1-3 sec each) , a DB of 5 MB and Synology SSD disks !!
can't see that it's a matter of speed
it's obvious that when the DB fills quickly with thousands new entries that will be cleaned up 30 minutes later by DB cleanup that most entries serve little purpose

[jokob-sk]

Feel free to open a PR to fix it if you know how to improve this. I didn't face such an issue before.