Question about real-time listing monitoring with Crawlee

[qmb874248n-code]

Hi, I’m exploring Crawlee (Python + Playwright) for a personal automation project.

I’m particularly interested in:

• Monitoring category or listing pages where product URLs are not known in advance
• Detecting newly appearing items in near real time
• Triggering a follow-up automation flow once a new item is detected

Before I go deeper, I wanted to ask:
Is Crawlee commonly used for this kind of real-time listing detection workflow, and are there any recommended patterns or examples for handling this cleanly?

Thanks in advance — the project looks very solid.

[janbuchar]

Hello and thank you for your interest in Crawlee! On its own, Crawlee is better suited for batch-style jobs, where you start the crawler, it goes through a website and terminates. Then you can export the results, re-run the crawl and check for changes.

In other words, there is no built-in infrastructure for this in Crawlee, but you can totally build it on top of Crawlee.