[SDBELGA-1056] Add get_all_batch_elastic to Eve resource service

[MarkLark86]

Purpose

We didn't have a way to iterate, in batches, items in Elasticsearch. The only way to do it was manually, and using the from and to params (which is not advisable on large indexes). The current workaround for the ProdAPI/Events endpoint is to use size=10_000 which is not ideal.

What has changed

• Add a new method get_all_batch_elastic to the Eve resource service

Resolves: SDBELGA-1056

[Copilot]

Pull request overview

Adds a new Elasticsearch-backed batch iterator to the Eve resource service to support efficient full-index iteration without using from/to pagination.

Changes:

• Introduced get_all_batch_elastic() on the resource service using search_after pagination.
• Added a raw-search path (search_raw) in the Eve backend to bypass resource-specific filters when needed.
• Added datalayer tests covering batch iteration and query filtering via Elasticsearch.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.

File	Description
tests/datalayer_tests.py	Adds tests for the new get_all_batch_elastic batch iterator (including lookup filtering).
superdesk/services.py	Implements get_all_batch_elastic() using Elasticsearch search_after pagination.
superdesk/eve_backend.py	Adds search_raw() to access the Elasticsearch backend without resource-specific filtering.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.