General questions

[OriolAbril]

Hi! Thanks for the library, it looks very promising and I can't wait to try it. I would like to use the extension with ArviZ docs. So far I haven't managed to get it to work so I'll try to go over my experience here so that you can hopefully guide me into opening proper issues for bug reports (if any) and enhancements as well as some PRs to improve documentation of the extension?

The first thing is that it took me a while to see if the extension would work at all in our case, as we don't really have any .. code:: blocks, we use matplotlib's, ipython's or bokeh's directives to get the code to autoexecute and include the output, as well as jupyter notebooks parsed with myst_parser. All three directives seem to also use highlight-python notranslate so it looks like they should be compatible out of the box and I did try using the matplotlib example on your homepage to use the plot directive and worked. But it might be worth it to explain what are the actual requirements for this to work or include some examples in the docs, perhaps with ipython and matplotlib ones that are quite popular to show they work too.

myst seems to use highlight-ipython3 notranslate which I'm not sure how easy it would be to cover too. This page for example is generated from a jupyter notebook: https://arviz-devs.github.io/arviz/getting_started/Introduction.html.

I naively added the highlight-ipython3 notranslate to https://github.com/felix-hilden/sphinx-codeautolink/blob/master/src/sphinx_codeautolink/extension/block.py#L193 to see what happened and added a jupyter notebook file to the example docs in my fork. The code in the notebook was not hyperlinked, but at least it did not break the rendering nor links for the rest of the pages, which for now is good enough.

I eventually moved to try it on in ArviZ docs, installed the extension, added it to conf.py and set codeautolink_autodoc_inject = False as we don't want the auto tables there, they would quickly become quite large and we'd rather users focus on the examples in the docstring for quick reference or on the dedicated plotting, labeling... guides for specific tasks which are also useful even if the exact function is not being used as many arguments are common between most plotting functions for example. But I still haven't managed to get the docs to build, I get the following error:

Extension error (sphinx_codeautolink.extension):
Handler <bound method SphinxCodeAutoLink.generate_backref_tables of <sphinx_codeautolink.extension.SphinxCodeAutoLink object at 0x7f8d6a43b2e0>> for event 'doctree-resolved' threw an exception (exception: 'builtin_function_or_method' object has no attribute '__annotations__')

I am trying to build only part of the docs to see if I can identify what triggers that error, but so far all the subsets I have tried end up just like this. I was also wondering if this step is always necessary or if it's only necessary when planning to generate example tables either automatically or with the directive. In our case, even a hacky workaround that completely disables the feature and only adds links to code would be amazing, not sure if it's possible.

I also removed autodoc in one of the experiments when trying to remove the api section completely from the equation which triggered a whole different error. I didn't completely understood the error nor why was it triggered by autodoc being missing, but I think the extension would be also extremely useful in documentation that doesn't use autodoc at all. I am thinking for example about collections of tutorials and case studies like https://pymc-examples.readthedocs.io/en/latest/ that thanks to the integration with intersphinx could easily link from case studies to the api docs in the main documentation website.

[felix-hilden]

Hi and thanks! I moved this to a discussion to hopefully triage the individual issues a bit better. I'll try to go over it and maybe we'll get to a resolution 👍

Custom code blocks

Firstly, I was not aware of any third-party libraries using their own directives as code blocks, so it's no wonder that they don't work 😅 So we should figure out how the directives work. Different highlights don't really matter as long as the code is valid Python and we can ignore the violating parts similar to doctest prefixes. The CSS class highlight-python notranslate is only used to locate the HTML to insert links, not to parse the source. The classes will eventually be added there, but having extra candidates doesn't break anything. So yeah, if it's reasonably possibly doable, I'm very much in favor of supporting more block types!

Annotations error

Now with respect to your __annotations__ error, that could very well be a bug in our logic! Without any indication as to what breaks it it's hard to tell though. Could you try to narrow it down by building some dummy documentation? And what do you mean by "a part of the documentation"? This error occurs when following type hints in the documented library to link more complex access like lib.foo().attr to lib.Bar.attr.

Disregarding backreference tables

Implementing a hacky version of this extension entirely outside the build process is possible by just reading HTML and Sphinx's result object inventory, but I'm quite opposed to that, which I'm sure you'll understand.

Autodoc-free documentation

I haven't considered documentation without autodoc, but the use case you presented is solid. If you've specified an intersphinx entry for the library you're trying to link to, I see no reason for it not to work. Obviously it didn't though, so I'm keen on seeing the error that was raised!

Hopefully this clears things up a bit! Let's discuss more and I'll open up more focused issues as we go 🤗

[OriolAbril]

Thanks! I didn't realize discussions were open, sorry about that, this is a much better place. I'll try to answer in different "answers"/"posts" so we can then if necessary continue in comments and focus a bit on each point independently.

[OriolAbril]

Custom code blocks

I opened #45 which should serve as an example on how matplotlib's custom plot directive already seems to integrate with the extension seamlessly. You should only need to generate the docs from the PR locally or to set up rtd PR previews. Here are the main third party code directives I am aware of:

• https://matplotlib.org/stable/api/sphinxext_plot_directive_api.html?highlight=plot_directive#module-matplotlib.sphinxext.plot_directive
• https://ipython.readthedocs.io/en/stable/sphinxext.html
• https://docs.bokeh.org/en/latest/docs/reference/sphinxext.html#module-bokeh.sphinxext.bokeh_plot
• MyST/jupyter notebooks. Not sure which is the best resource to share as an explanation for that. AFAIK, the code-cell directive
  is used either to parse the code cells in jupyter notebook or used as a "regular" directive within markdown files. Maybe https://myst-nb.readthedocs.io/en/latest/use/markdown.html is a good starting point.
• https://jupyter-sphinx.readthedocs.io/en/latest/

[OriolAbril]

No, compare uses the ipython direcrive, autocorr uses the matplotlib plot directive

[felix-hilden]

Oh, so these are completely separate issues then, and the autocorrelation should surely work with the currently existing machinery. Could it have something to do with generating the files? How do you do it?

[OriolAbril]

Not sure, we use the directive in the docstrings and then use autosummary to include them in the docs

[felix-hilden]

Right, autosummary doesn't generate RST files, so that might be the key. This needs to be tested, but my current guess is that either the blocks aren't analysed at all, or they are not saved or cannot be parsed in HTML because the files haven't come from concrete sources. I'm on it!

Or wait, does it? Well anyways I'll investigate.

[OriolAbril]

it can do, and in our case it does. We have written https://github.com/arviz-devs/arviz/blob/main/doc/source/api/plots.rst which when building the documentation writes all the object specific rst files in the generated/ folder. I have never really checked what is actually the content of those files though

[OriolAbril]

Annotations error

I started trying to get a minimal build that reproduces the error, but I didn't have much time and wasn't able to get anywhere, a couple cases were very simplified and worked, a couple others had different errors (like removing the whole api section and consequently not using autodoc).

I will try to get a mwe and get back to you here.

[OriolAbril]

I tracked the error down to np.random.normal and similar calls, see #47 as a way to trigger it with two extra lines only in your intersphinx example.

[felix-hilden]

I had a look at np.random.normal. It's a method (bound to some underlying object) but despite that it doesn't have an __annotations__ dict. Even unannotated functions and lambdas should have them though (as an empty dict), so something odd is going on there. Perhaps C bindings or something, further investigation needed. As to why Bokeh breaks the linking.. no idea still. The type hints shouldn't be followed, because there isn't any attribute access after the call.

[OriolAbril]

Disregarding backreference tables

I completely understand, and my goal is to get it to work and to also be able to use the example tables, even if not automatically nor everywhere. As you can see from my hichlight-ipython3 approach, I am an avid sphinx user, but I have never written an extension nor I understand how they work nor how much of a challenge would supporting extra third party code directives be. A solution of that sort would be as a last resort which doesn't seem necessary after your thoughtful response

[felix-hilden]

I agree, and we can consider this resolved then!

[OriolAbril]

Autodoc-free documentation

I will share the error as soon as possible. This I do know how to reproduce in a mwe so it should not take too much time. Thanks for considering this!

[felix-hilden]

Was the error "unknown event name: autodoc-process-docstring"?

[OriolAbril]

Yes!

[felix-hilden]

Alright, we need a way of disabling our autodoc integration conditionally. I'll see what can be done. As a last resort we could require autodoc (can be done in code if the user hasn't enabled it which is nice) even though the documentation doesn't use it.

[felix-hilden]

Seems that Sphinx doesn't have a good way of doing it, so we'll go with activating autodoc anyway: #48