Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Latex not working in jupyter notebook v7. #4336

Closed
pfebrer opened this issue Aug 22, 2023 · 45 comments · Fixed by #4763
Closed

Latex not working in jupyter notebook v7. #4336

pfebrer opened this issue Aug 22, 2023 · 45 comments · Fixed by #4763
Assignees
Labels
bug something broken P3 backlog sev-2 serious problem

Comments

@pfebrer
Copy link

pfebrer commented Aug 22, 2023

I recently upgraded to jupyter notebook v7 and I noticed that some plotly plots that previously rendered flawlessly now fail to render. I get the following output:

Screenshot from 2023-08-22 12-27-21

After some exploration I found out that it comes from an error using MathJax in jupyterlab-plotly. It tries to access MathJax.config.startup.output, but MathJax.config.startup is undefined.

It is very easy to reproduce this error:

  • Create a new python environment (I tried with python 3.8).
  • pip install jupyter plotly
  • Open a jupyter notebook with jupyter notebook.
  • Run a cell with the following code:
from plotly.graph_objects import Figure

Figure().update_layout(title="$Title$")
@pfebrer
Copy link
Author

pfebrer commented Aug 22, 2023

Ok, I installed the jupyterlab-plotly extension with jupyter labextension install plotlywidget and it works, so I guess this will be fixed in the next python package release (?)

@pfebrer
Copy link
Author

pfebrer commented Aug 22, 2023

No, sorry it also doesn't work, the previous code snippet just literally shows "$Title$" as a title, but prepending the string with 'r' makes the issue show again:

from plotly.graph_objects import Figure

Figure().update_layout(title=r"$Title$")

@zhimin-z
Copy link

zhimin-z commented Aug 27, 2023

Hi, I faced the same issue, any update?

@pfebrer
Copy link
Author

pfebrer commented Aug 27, 2023

No, I still need some solution for this 😅

@newville
Copy link
Contributor

@zhimin-z @pfebrer I've been seeing this issue for a while too. Is there any understanding of the cause and what might be done to fix it?

@zhimin-z
Copy link

Yeah, still there... @newville
image

@newville
Copy link
Contributor

@zhimin-z Yes, the problem still exists. Is there any understanding of the cause of the problem? Has anyone investigated what is going on in the code? I'm afraid this is a bit outside of my area of expertise.

@zhimin-z
Copy link

I have no idea...anyone who is in charge of latex functionality in this repo?

@pfebrer
Copy link
Author

pfebrer commented Nov 14, 2023

I'm tagging @alexcjohnson since he seems to be the last to have committed something to jupyterlab-plotly other than version changes :)

It looks like this is an easy fix that just needs an update to use the latest MathJax version, or somehow forcing jupyterlab to use an oldest one, I don't know if that's possible.

@alexcjohnson alexcjohnson added bug something broken sev-2 serious problem labels Nov 14, 2023
@newville
Copy link
Contributor

@pfebrer @alexcjohnson Thanks for looking into this. Is there anything the rest of us can do to help further troubleshoot or fix this issue? I'll admit that I do not have a deep understanding of how plotply.py and plotly.js interact or which version of mathjax
is actually being used.

@pfebrer
Copy link
Author

pfebrer commented Nov 22, 2023

I don't know, everything I know is that jupyterlab uses MathJax 3.2 and plotly's docs use 2.7, but in principle version 3 should work as well according to plotly/plotly.js#6073

I don't even know how to modify jupyterlab-plotly and use a modified version of the extension. I have tried to pip install -e . the plotly repo to play a little bit with it but then jupyter doesn't render anything.

@newville
Copy link
Contributor

@pfebrer Thanks, sorry to be dense, but what precisely would one have to do to make it work? Could that be folded into the plotly-py code?

The issue is kind of 3 months old, and it seems that there has been no explanation or acknowledgment of this issue.
But, I don't really know what that means. Maybe the solution is obvious to someone?

@pfebrer
Copy link
Author

pfebrer commented Nov 27, 2023

I don't know, if I knew I would have submitted a PR already :)

I already tried to fix it and I couldn't even set up a dev environment. I'm not a plotly developer so I don't have more time to spend on it unfortunately.

They already marked it as a bug, so I guess they'll address it whenever they have time. The bug is very annoying and I guess plenty of people will find it since everyone uses jupyter notebooks, so eventually they will have to fix it if they don't want people to move to other packages because of this very small detail.

@pfebrer
Copy link
Author

pfebrer commented Nov 27, 2023

Could that be folded into the plotly-py code?

My impression for this was that plotlyjs should be modified (not just the javascript in jupyterlab-plotly), but I might be wrong.

@newville
Copy link
Contributor

@pfebrer @alexcjohnson it looks like maybe this is related to #3661, #3696, and #3706, first identified 18 months ago and related to plotly using MathJax 2 when other code (say, Jupyter) has already loaded MathJax 3.

FWIW, the last release of MathJax2 was 3 years ago. MathJax 3 has been actively developed, and MathJax 4 is now in beta.

@alexcjohnson
Copy link
Collaborator

As @pfebrer pointed out, Plotly.js has been able to use MathJax v3 or v2 since plotly/plotly.js#6073 which was released in v2.10 (March 2022), so the problem is something about how we're detecting or invoking MathJax in certain environments.

I have to admit my own attempts a few months ago to get multiple versions of Jupyter and Jupyterlab installed side-by-side so I could test and fix these issues left me with a borked environment that wouldn't run anything correctly, and I haven't tried to go back to it since then. So if someone else wants to give this a shot we'd welcome a PR - otherwise we'll get back to it at some point but I can't say when.

@newville thanks for the note about MathJax 4, we'll give it a look! We'll certainly want to get it working in Plotly.js and Dash, regardless of the issues we've had in the various Jupyter environments.

@newville
Copy link
Contributor

@alexcjohnson Thanks for the response. For me, running a standalone plot script, as in https://plotly.com/python/LaTeX/ from a plain Python shell works fine (Python 3.10.12 or 3.11.5 on MacOS) -- it opens a browser in Chrome and the plot correctly renders the LaTeX.

Running that same script in a Jupyter Notebook fails to render the LaTeX. To reproduce, one should be able to install Miniforge (https://github.com/conda-forge/miniforge), activate that environment, and then install with

mamba install -c conda-forge notebook pandas plotly

I might be willing to help, but I am having difficulty even navigating where to start here (not very fluent with the Jupyter code or Plotly). I cannot really even figure out where the

Javascript error: cannot read properties of undefined

is coming from.

@pfebrer
Copy link
Author

pfebrer commented Nov 28, 2023

I cannot really even figure out where the

Javascript error: cannot read properties of undefined
is coming from.

If you open the browser console (F12) you'll see some more information. But the JavaScript files are minified so it's really an impossible mission to debug from there.

That's why I tried to set up a dev environment, but I failed. @alexcjohnson maybe if you can point us to somewhere where it is explained how to set up the dev environment for the whole stack (plotly.py, jupyterlab-plotly, plotlyjs), we could try to help. Otherwise, at least in my case, it feels too difficult to try to contribute because I have to figure out everything myself and I don't know if I'm doing things right.

@newville
Copy link
Contributor

newville commented Dec 1, 2023

@alexcjohnson @pfebrer Just in case it wasn't obvious (or in case it helps with debugging), downgrading to jupyter notebook < 7 does avoid this problem:

pip install "notebook<7"

@newville
Copy link
Contributor

@alexcjohnson @pfebrer any thoughts on whether progress is going to be possible on this? Help (or an attempt to help, at least) was offered, but the very low response rate here is less than encouraging. I hope all is well.

@newville
Copy link
Contributor

@alexcjohnson @pfebrer any updates on this?

Perhaps a pull request on the documentation (https://github.com/plotly/plotly.py/blob/master/doc/python/LaTeX.md) to change that to state that latex formatting of strings is no longer supported in Jupyter would be in order? The current statement there is misleading at best.

@pfebrer
Copy link
Author

pfebrer commented Jan 25, 2024

I have no idea how to fix it and I'm not part of the plotly team, so I don't see how I can help, sorry :(

I would like to see this fixed as well, but there's nothing I can do.

@newville
Copy link
Contributor

@pfebrer Thanks - I am in the same position. It seems like the developers are either ignoring this or won't be able to fix it. I think that perhaps the best option is to admit that this is not working, and put in a PR to change to documentation to say that latex strings do not work with Jupyter Notebook 7.

@timoklein
Copy link

I hope this gets moved up the priority list because it's kind of a dealbreaker for using plotly in scientific publications. Just had to fall back to matplotlib because I was not able to properly name my axes on jupyterlab 4.2.0.

@LunarLanding
Copy link

Currently migrating my report from plotly/plotly.js#5374 to here.
On plotly 4.22, jupyter lab 4.2.3, python 3.11.9, firefox 127.0.2, OSX Sonoma 14.5:

import plotly.graph_objects as go
f=go.Figure()

f.update_yaxes(title=dict(text="$log_{10}(D)$", font_size=16))

f.show()
image

@newville
Copy link
Contributor

@LunarLanding @timoklein Yeah, the lack of attention and the lack of response to this issue is pretty disappointing. It was raised 10 months ago now. And as you say, it is a dealbreaker for using Plotly with JupyterLab.

Both Plotly and JupyterLab explicitly say in multiple places that they work together, and both explicitly say that Latex strings are supported and render correctly. The fact is -- and has been for a very long time -- that they do not. Using latex strings with Plotly in JupyterLab or Notebook>7 straight-up breaks any rendering by Plotly.

The Plotly developers continue to not acknowledge this bug and have shown no inclination to address it.

@marthacryan
Copy link
Collaborator

marthacryan commented Jul 2, 2024

Hey all! @newville @LunarLanding @timoklein I'm investigating how to fix this, but in the meantime I've had some success with changing the default renderer in plotly.io like this (before running your plotly code):

import plotly.io as pio
pio.renderers.default = 'iframe'

Let me know if that works for you! Sorry for the delay on this and thanks for posting this issue.

@newville
Copy link
Contributor

newville commented Jul 3, 2024

@marthacryan Thanks very much for responding and for the helpful suggestion. That does work for the couple of examples I've tried. So that is very promising.

I have not looked at the code in any detail. Is there some documentation or more information about these renderers?

When using 'iframe', a folder called "iframe_figures" is created in the working directory, which has non-trivial HTML files (it appears one per figure, though I am not certain of that). These seem to not be cleaned up when the notebook is closed. Would it be possible to either clean these on exit or perhaps write these files to a common temporary folder like /tmp?

FWIW, these HTML files appear to load https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js, which is 5+ years old.
Is that necessary, or could a more recent version be used?

Anyway, that seems very promising. Thanks!

@archmoj
Copy link
Contributor

archmoj commented Jul 3, 2024

Thanks very much @marthacryan.
It looks like the default renderer is not correctly set on plotly.py.
What do you think?

@pfebrer
Copy link
Author

pfebrer commented Jul 3, 2024

Hi @archmoj, if what you are proposing is to set iframe as the default renderer, in my opinion it doesn't make much sense as a definitive solution, specially if that creates an extra folder in your filesystem as @newville says.

Still, if an iframe can be displayed without writing files to your filesystem I think what would make sense is to temporarily make the notebook renderer use the iframe machinery until the mathjax bug is fixed.

But to me time seems better invested on understanding what goes on with mathjax than in making iframes not write files to the filesystem. Specially since we have finally caught the attention of someone who seems to have the knowledge to solve the issue, i.e. @marthacryan :)

@pfebrer
Copy link
Author

pfebrer commented Jul 3, 2024

In other words, if plotly creates extra folders on my filesystem by default it is more probable that I stop using plotly than if latex doesn't work by default 😅

@newville
Copy link
Contributor

newville commented Jul 3, 2024

@pfebrer @marthacryan Well, I might be OK with a renderer that creates a temporary file if they are cleaned up or can be centralized to make them easier to clean up (perhaps /tmp or $HOME/.plotly_iframe). Maybe add a method to clean temporary and/or old temporary plotly-generated files that can be registered to run in an "atexit" handler?

But, it seems like the observation that the "iframe" renderer is OK must help isolate the problem. For the "plotly_mimetype" renderer, I see "application/vnd.plotly.v1+json", and that is as far as I can go. That is basically "compiled, indecipherable code that plotly devs own". If that does not work and the developers are unable to fix the bug they caused, then yeah "don't use" is the only sane response.

Just to be clear: plotly does not work with Jupyter Notebook > 7. Workarounds that have unfortunate side effects point to solutions to this problem, but the problem is not "fixed".

For sure, responding to a bug report is appreciated, but taking multiple months to respond is kind of a problem in itself, and now the developers have 2 problems to solve: the technical "how to render plots with Jupyter Notebook" (seems like a pretty basic "fitness of purpose" sort of requirement) and the social "how to convince people who want to use your software and report problems in the forum you set up to believe your software is actually being supported".

@pfebrer
Copy link
Author

pfebrer commented Jul 3, 2024

But, it seems like the observation that the "iframe" renderer is OK must help isolate the problem.

Well, showing it in a separate tab with fig.show("browser") has always worked, so the iframe solution is a nice discovery to keep things in the notebook but I don't think it helps explain the bug, it is just being shown in an isolated iframe.

For sure, responding to a bug report is appreciated, but taking multiple months to respond is kind of a problem in itself, and now the developers have 2 problems to solve: the technical "how to render plots with Jupyter Notebook" (seems like a pretty basic "fitness of purpose" sort of requirement) and the social "how to convince people who want to use your software and report problems in the forum you set up to believe your software is actually being supported".

I don't pay for using plotly, so all I can do is be grateful that they have open sourced the library and hope that they'll find time to interact with issues and fix the bugs :)

@marthacryan
Copy link
Collaborator

marthacryan commented Jul 8, 2024

I have not looked at the code in any detail. Is there some documentation or more information about these renderers?

@newville Yep, this was already linked in the thread somewhere but the documentation is here: https://plotly.com/python/renderers/

When using 'iframe', a folder called "iframe_figures" is created in the working directory, which has non-trivial HTML files (it appears one per figure, though I am not certain of that). These seem to not be cleaned up when the notebook is closed. Would it be possible to either clean these on exit or perhaps write these files to a common temporary folder like /tmp?

I think rather than changing the location of those files, I'll just try to fix up the default renderer.

FWIW, these HTML files appear to load https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js, which is 5+ years old. Is that necessary, or could a more recent version be used?

Yep this is my theory right now! I still need to look into what's going on exactly but it looks to be related to the MathJax version.

Anyway, that seems very promising. Thanks!

Yay glad it's helping at least in a temporary way!

@marthacryan
Copy link
Collaborator

It looks like the default renderer is not correctly set on plotly.py. What do you think?

@archmoj I think the better solution will be to fix up that default renderer instead of changing to iframe.

@marthacryan
Copy link
Collaborator

Ah another quick solution here: try pip install jupyterlab_mathjax2. It seems to work locally for me, and that way you can avoid the extra files. Again, still trying to fix this default renderer, but hopefully it'll be a good solution while I'm looking into it

@newville
Copy link
Contributor

newville commented Jul 9, 2024

@marthacryan Thanks for the suggestion. When I do that, then I do not see any text labels at all (for the example at the start of this discussion). Can you give an example script that works for you?

@marthacryan
Copy link
Collaborator

@newville Oh interesting. This is what I'm seeing for that example in both JupyterLab and Jupyter Notebook (v7):
image
This is with python 3.12 and 3.8 in a conda env and in python venv. I know calling jupyter in the shell sometimes doesn't point to the environment's version of jupyter, do you want to try running which jupyter and see if it's pointing to new installation of jupyter in your env?

@newville
Copy link
Contributor

newville commented Jul 9, 2024

@marthacryan Thanks.
Running jupyter notebook with a Python 3.11 environment and with jupyterlab_mathjax2 installed, I get this:

Screenshot 2024-07-09 at 2 06 51 PM

@LunarLanding
Copy link

LunarLanding commented Jul 10, 2024

@marthacryan iframe mode works, thanks! I don't mind the folders much, although it would be better if I could choose the name to something like .iframes.
jupyterlab_mathjax2 also works.
The ideal solution for me would be for it to just work by loading plotly / mathjax rendering from the internet, and not saving it locally, either in files or in the cell output. Seems the second solution does just that.
For the second method, I also tested that on page reload, and on cold start (jupyter server started, notebook opened without running cells) everything works.
First a preview saved as png is shown, then both a svg + png are shown, then just the svg without the rendered latex, then the latex renders.

However, I remember that there were other issues that made plotly latex rendering subpar compared to matplotlib.
Namely, mixing math context '$$' and text results in the text being ignored (see screenshot).

@newville I think you forgot a rawstring marker (R'$\alpha...') but I noticed that even adding it, mixing normal text and latex in a string results in the normal text being ignored. But I get some latex rendered, so there is something different between my environment and yours. I think it is because I installed not only the notebook, but jupyter lab, and specifically jupyterlab_mathjax2 ( but the test below was run in the notebook).

import sys,notebook,plotly
print(sys.version,sys.platform)
for m in (plotly,notebook):
    print(m.__name__,m.__version__)
import plotly.graph_objects as go
f=go.Figure()
f.update_layout(title=dict(text=R'$\alpha$ Title'))
# f.update_yaxes(title=dict(text="a label $log_{10}(D)$", font_size=16))
f.show()
image

Reproducing:

micromamba create -n plotly_latex_bug jupyterlab jupyter plotly jupyterlab_mathjax2
micromamba run -n plotly_latex_bug jupyter notebook
micromamba run -n plotly_latex_bug jupyter lab

@archmoj
Copy link
Contributor

archmoj commented Jul 10, 2024

What is the version of Mathjax on window when using default renderer?

@newville
Copy link
Contributor

@marthacryan @LunarLanding The previous screen grab I did was on macOS. On Windows, with the jupyterlab-mathjax2 hack installed, I get

WithNotebook7_mathjax2

So, the problem is not tied to one system or platform.

@gvwilson gvwilson added the P3 backlog label Aug 12, 2024
@newville
Copy link
Contributor

@gvwilson Do I read the P3 label you added as meaning "not pressing"?

Can you verify that you understand that this problem is actually a show-stopper for using plotly with the latest version of jupyter and jupyterlab? The first paragraph of the readme advertising this project (https://github.com/plotly/plotly.py/blob/master/README.md) literally says:

Inside [Jupyter](https://jupyter.org/install) (installable with pip install "jupyterlab>=3" "ipywidgets>=7.6"):

import plotly.express as px
fig = px.bar(x=["a", "b", "c"], y=[1, 3, 2])
fig.show()

but that for anyone wanting to, you know, plot scientific data, that

import plotly.express as px
fig = px.bar(x=[r"$\alpha$", r"$\beta$", r"$\gamma$"], y=[1, 3, 2])
fig.show()

fails to render a plot.

Please verify that this issue is actually considered "not pressing".

@gvwilson
Copy link
Contributor

Hi @newville - "not pressing" means that no-one on my team is actively working on it in the current cycle. If other issues take less time to resolve than expected, we'll bump its priority and find someone to tackle it. PRs from the community are always welcome, and we'll review them as quickly as we can. Thanks - @gvwilson

@newville
Copy link
Contributor

@gvwilson Oh. "not pressing" being short-hand for "not being actively worked on" seems like an odd usage to me.
Is that "If we are working on it, it is urgent, if we are not working on it, it is not urgent"??

As with many open-source projects, it is very hard to know from GitHub alone who is on "the team"? It seems like you view "your team" and "the community" are different things. That could be self-fulfilling ;)

As it happens, this Issue was first raised almost exactly one year ago.

It sure seems like the Plotly developers will not be able to solve this. I hope that is not so, as it is unfortunately cripples using Plotly with Jupyter for many people with scientific data. I wish you all the best, but it is not clear to me who your target audience is. There are other graphics libraries out there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug something broken P3 backlog sev-2 serious problem
Projects
None yet
9 participants