Paperless-NGX several auto tasks stopped working

[MarcvsTvllivs]

I am using Paperless-NGX in an LXC installed with the script, and I have a strange issue that nobody else seems to be having: Several (but not all) of the automated/recurring tasks that Paperless is supposed to be handling in the background have stopped working one day (Dec 22, to be precise). From what I have found so far this affects:

• Classifier training (note: a manual training started through the CLI works fine).
• Email tasks.

These worked perfectly fine until they suddenly stopped on Dec 22 (which I can see from the logs). No errors etc. - the log entires just stop on that day. I cannot seem to find any errors elsewhere that led to this either, with cron being present and running. Interestingly, the following automatic tasks work just fine:

• Consumption of new files from the consumer folder and otherwise.
• Classification of new files with the classifier.
• Triggering and running of workflows upon consumption.

The only other strange thing I can find is that the GUI's system status shows the Classifier as last trained on Dec 22 (even though it has been plenty updated since then, just by me triggering that training manually).

Has anybody seen something like this and might be able to point me in the right direction?

[MickLesk]

I had the Same. Whats the Output in your Services? Do you have updated the Services (Increase the Startup Sequenze) ? Do you have Python Issues?

Checkout the Install folder Here, paperless-install.sh and check If all of your Services exactly in this format

[MarcvsTvllivs]

Thanks for your response! I'm sorry but I'm not very experienced with Python. How do I get the output?

I didn't really change much other than install another OCR language and adjust some settings in the .conf.

[MickLesk]

Then Check the Install File of our Project. There are maybe some Updates in this (Special the service Files).

After this, Execute all your Services or reboot your lxc and Test again.

Ive struggel with this issue a few hours, But I can't remember the exact solution right now. Something in Python was faulty in connection with the service. But you can also run these manually using the services.

[knario]

Hi, I think I read some time ago in the community that with this command it is fixed.
I don't know if I'm wrong, to me it stopped sorting documents after an update, I found this command and now, every time I update, I put it to make it work.

I solved it by running python3 manage.py document_create_classifier inside the paperless/src folder.

[endbacher]

I'm facing a similar issue with automated tasks as @MarcvsTvllivs described above: my paperless is planned to be filled with documents by manual upload on paperless browser interface, mobile apps, e-mail and document scanner.

The paperless-ngx setup itself (latest version) and all manual operations to get documents into it work as expected, for example manual e-mail download if triggered via E-Mail / process e-mails is consuming new e-mails or sending documents to /opt/paperless/consume from document scanner make them show up in the folder. Unfortunately paperless neither processes e-mails or PDFs automatically, there are no updates recorded in both paperless.log and mail.log for any activity.

If I manually run from the paperless lxc console
systemctl enable -q --now paperless-webserver paperless-scheduler paperless-task-queue paperless-consumer
all works as it should, including auto consumption and classification but as soon as reboot takes place auto operations are not working again.

I assume that something is not working (correctly) with the services but unfortunately I lack knowledge on what to put where in which way for an automatic an (re)start of the services therefore some advice on how to set this correctly would be highly appreciated to point me into the right direction.

[MarcvsTvllivs]

Then Check the Install File of our Project. There are maybe some Updates in this (Special the service Files).

After this, Execute all your Services or reboot your lxc and Test again.

Ive struggel with this issue a few hours, But I can't remember the exact solution right now. Something in Python was faulty in connection with the service. But you can also run these manually using the services.

Apologies for taking a while to get to this. In short, I don't think the install file changed much since I set this up in December. I did execute all the Services parts of it again though to see whether it fixes things. It didn't. Also note that everything was working at first, so things seem to have been correctly installed in the beginning. Then something changed and I do not know what.

I solved it by running python3 manage.py document_create_classifier inside the paperless/src folder.

That's the manual command to update the classifier and it works. Similarly, the manual command to check mail works, too. The issue is that they (and who knows what else) no longer run automatically.

@endbacher yep, that does sound like my issue except that for some reason mine still processes PDFs. I've tried the command you recommend to see whether that at least manually re-enables things after reboot.

EDIT: Nope, the command did not work for me. Still no automatic training or mail processing even after I run it.

[MarcvsTvllivs]

For those struggling with this as well: I did a little troubleshooting session with ChatGPT and was suggested that this sounds like a Celery issue. After some poking around I confirmed that, and was then told to delete the schedule database "celerybeat-schedule.db" (in the /opt/paperless/data/ directory) and rebuild it. That appears to have solved it!

Interestingly, the database file was (and the new one continues to be) named "celerybeat-schedule.db.db", i.e., a redundant ".db". Not sure what that is about.

[endbacher]

Hi @MarcvsTvllivs, I'm also pretty sure that it is related somehow to the start of the services, unfortunately your approach does not help on my installation: celerybeat-schedule.db was properly deleted and rebuilt but as soon as reboot takes place it's the same as described above until I run the above mentioned command manually.

Hi @MickLesk.

is there any way to influence the "start timing" of the services, for example put in a delay of some minutes upon reboot to ensure system is up and running or to put the command above somewhere to have it executed with some minutes of delay? What I found during this week is that once services are started they also stay in this state and properly execute their tasks of consuming documents or fetching e-mails.