Skip to content

Add Open Library ISBN search translator - #3640

Open
mvolz wants to merge 1 commit into
zotero:masterfrom
mvolz:open-library-isbn
Open

Add Open Library ISBN search translator#3640
mvolz wants to merge 1 commit into
zotero:masterfrom
mvolz:open-library-isbn

Conversation

@mvolz

@mvolz mvolz commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Looks up book metadata by ISBN via the Open Library ISBN API
(https://openlibrary.org/dev/docs/api/books).

Raises priority above LoC as it supports 3 rps vs LoC's 5 requests per minute

@AbeJellinek

Copy link
Copy Markdown
Member

At least historically, I think we avoided relying on Open Library because the metadata quality/accuracy was not great. Has it improved? Any way to compare to WorldCat, etc., in terms of coverage and quality?

Either way, I imagine we'd want to put OL above WorldCat but below national databases like LoC, even if the rate limit is higher.

@mvolz
mvolz force-pushed the open-library-isbn branch from c850812 to 50f7a28 Compare August 28, 2026 15:04
@mvolz

mvolz commented Aug 28, 2026

Copy link
Copy Markdown
Contributor Author

The open library api I think still contains some fairly messy user-uploaded data, however I know they also uploaded the Library of Congress Data dumps that were released in 2016 and those are pretty good. https://loc.gov/cds/products/MDSConnect-books_all.html (example record recording this as the source: https://openlibrary.org/books/OL3561303M.json)

The idea for us to have it as first priority is to take the load off of LoC and leave it for ISBNs added from 2016 onward. We deployed this and the translation-server fix on Wednesday at ~11:15 UTC and it made a big difference in how many 404s we were returning (though hard to tell which did what, I guess!)

Screenshot From 2026-08-26 16-22-14

Totally understand that your use case is different though! We maintain a fork and can just keep the priority difference in the fork alone.

Looks up book metadata by ISBN via the Open Library
ISBN API
(https://openlibrary.org/dev/docs/api/books).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@mvolz
mvolz force-pushed the open-library-isbn branch from 50f7a28 to bd8d883 Compare August 28, 2026 15:06
@mvolz

mvolz commented Sep 1, 2026

Copy link
Copy Markdown
Contributor Author

Changed to 100, but maybe priority should be even lower, like 110? Thoughts?

@adam3smith

Copy link
Copy Markdown
Collaborator

I haven't tested the translator yet, but in my previous work with open Library, they never had subtitles of books, which of course are quite common in academic works. The two test cases just don't have subtitles, so impossible to tell. But did you test with works that did? Am I misremembering that or just use the wrong API?

That's the biggest data quality issue that I've seen, and I think that should determine somewhat where we place this in the priority hierarchy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants