Support Chndoi, CNKI's Chinese DOI registrar

[AbeJellinek]

https://forums.zotero.org/discussion/119667/chndoi-add-item-from-china-doi-database
https://doi.org/10.16418/j.issn.1000-3045.2019.09.003

Unclear if these all have a common prefix (10.16418?) or if there's a machine-readable API; DOI.org returns HTML with no regard for the Accept header.

[doubanchan]

@jiaojiaodubai has realized a relatively efficient translator in CNKI.js.

also take a look at issue 347 in translators_CN/issues/347 (in Chinese).

Besides CNKI, there's another DOI registrar jointly operated by WanFang and ISTIC (Institute of Scientific and Technical Information of China).

[jiaojiaodubai]

DOIs point to CNKI has been resolved in translators_CN/CNKI.js (support access outside China mainland as well), other sites still in progress. In translators_CN, I have made a radical refactor for CNKI.js to support all item types in CNKI and its subsites.

[jiaojiaodubai]

The issue is blocked because DOIs pointing to CNKI cannot be opened in the search translator with requestDocument(). However, these links can be opened successfully in a real browser, we may require further support from upstream (zotero/translate).

[dstillman]

@jiaojiaodubai: What exactly do you mean by "DOIs pointing to CNKI cannot be opened in the search translator with requestDocument()"?

[jiaojiaodubai]

China DOI is a website similar to CrossRef. Some Chinese DOIs will be resolved to the webpages of China DOI. Therefore, it is not enough to simply modify the DOI pattern in DOI.js. It is also necessary to write a dedicated Translator for it and add it to DOI Content Navigation.js.

Take the DOI mentioned in the issue as an example. 10.16418/j.issn.1000-3045.2019.09.003 can be concatenated into the URL https://doi.org/10.16418/j.issn.1000-3045.2019.09.003. If you visit this URL, you will enter the webpage of China DOI. The two links on the page point to the CNKI webpage for mainland China and the oversea respectively.

Usually, we can select one of the URLs in the search Translator (or web Translator) of China DOI and use requestDocument to obtain the target webpage of this URL. Finally, we call the CNKI Translator for parsing. However, now when using requestDocument to access these links, we are always blocked by CNKI's CAPTCHA.

[dstillman]

However, now when using requestDocument to access these links, we are always blocked by CNKI's CAPTCHA.

It depends on exactly what triggers the CAPTCHA and whether it goes away with appropriate cookies, but it's not impossible we could intercept these specific requests and display the CAPTCHA to the user. We have some code to do that for ScienceDirect. But if the page itself requires JS to run, that's not going to work from the translation framework.

[jiaojiaodubai]

It depends on exactly what triggers the CAPTCHA and whether it goes away with appropriate cookies

I haven't carefully investigated the meaning of every CNKI cookie, but I can basically confirm that it is authenticated through cookies.

• On a browser with cleaned cookies, CAPTCHA will pop up when visiting detail page of CNKI by URL directly.
• When calling CNKI.js on browser to scrape CNKI's multi-item page, it does not rely on the JS of detail page.

We have some code to do that for ScienceDirect.

browserDownload.js? What I need to do is to access CNKI in advance through Zotero's hidden browser and reuse cookies, right? Although it is a bit cumbersome, I am willing to promote this, which can help a large number of Chinese PDFs automatically retrieve item information through DOI when dragging into Zotero.