Request for Clarification on Sorting Mechanism in API Documentation

[massimobrivio]

Problem

When querying publications by category using the arXiv API and sorting them by relevance, it is unclear how the results are ordered. For example, the results from the following query are not sorted by publication or update date, and all should be equally relevant to the topic of AI, so what are the important parameters for the sorting?

http://export.arxiv.org/api/query?search_query=cat:cs.AI&sortBy=relevance&sortOrder=descending&start=0&max_results=10

Desired solution

A more detailed explanation in the API documentation about the sorting mechanism when querying by relevance. Specifically, the main parameters that determine the order of the results should be clarified. Specifically, how does sorting by relevance work when all the publications are equally relevant to the search query?

[jweiskoff]

Fixed in #1100

[jweiskoff]

Unfortunately this is just standard Lucene's RELEVANCE sort from an older version of the documentation. It's probably using some internal optimization that's not clearly explained on that page.

[massimobrivio]

Unfortunately this is just standard Lucene's RELEVANCE sort from an older version of the documentation. It's probably using some internal optimization that's not clearly explained on that page.

@jweiskoff

I opened a clarification request on Apache Lucene, they might be able to clarify the internal workings of their sorting algorithm.
Link to issue: apache/lucene#14295

[jweiskoff]

I suspect that you won't get much feedback from them, since this could be considered a trade secret, but maybe. 🤷🏼