Add Open Library ISBN search translator - #3640
Conversation
|
At least historically, I think we avoided relying on Open Library because the metadata quality/accuracy was not great. Has it improved? Any way to compare to WorldCat, etc., in terms of coverage and quality? Either way, I imagine we'd want to put OL above WorldCat but below national databases like LoC, even if the rate limit is higher. |
c850812 to
50f7a28
Compare
|
The open library api I think still contains some fairly messy user-uploaded data, however I know they also uploaded the Library of Congress Data dumps that were released in 2016 and those are pretty good. https://loc.gov/cds/products/MDSConnect-books_all.html (example record recording this as the source: https://openlibrary.org/books/OL3561303M.json) The idea for us to have it as first priority is to take the load off of LoC and leave it for ISBNs added from 2016 onward. We deployed this and the translation-server fix on Wednesday at ~11:15 UTC and it made a big difference in how many 404s we were returning (though hard to tell which did what, I guess!)
Totally understand that your use case is different though! We maintain a fork and can just keep the priority difference in the fork alone. |
Looks up book metadata by ISBN via the Open Library ISBN API (https://openlibrary.org/dev/docs/api/books). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
50f7a28 to
bd8d883
Compare
|
Changed to 100, but maybe priority should be even lower, like 110? Thoughts? |
|
I haven't tested the translator yet, but in my previous work with open Library, they never had subtitles of books, which of course are quite common in academic works. The two test cases just don't have subtitles, so impossible to tell. But did you test with works that did? Am I misremembering that or just use the wrong API? That's the biggest data quality issue that I've seen, and I think that should determine somewhat where we place this in the priority hierarchy. |

Looks up book metadata by ISBN via the Open Library ISBN API
(https://openlibrary.org/dev/docs/api/books).
Raises priority above LoC as it supports 3 rps vs LoC's 5 requests per minute