Octave functions to display mimetypes

[rapgenic]

We should add functions to display Octave objects as specific output mimes

• Audio
• Images
• Videos?
• ...

[AntoinePrv]

From the IPython doc https://ipython.readthedocs.io/en/stable/config/integrating.html#rich-display
They define custom methods that get called to present the rich display, and have a generic _repr_mimebundle_ for generic mimetype.

Should we pick a convention for naming such methods in Octave?
It seems that one can customize string display by defining the display method, so perhaps somehting like display_xxx?

[AntoinePrv]

Also relating to #50, it seems that it is not XOctave but Octave itself that does the display.
So perhaps we have to play with displayformat?

[rapgenic]

From what I know Octave has two ways of showing output: the disp function and the display function. The former usually prints a textual more low-level output, the latter instead is used for rich (text, at least in native octave) display.

When the interpreter evaluates the parse-tree of some code, it automatically inserts a display call for every expression not terminated with a semicolon.

So the method for integrating rich output display is overriding the display function (unless we want to start poking with the parse-tree, which is possible, I think, but a lot messier and probably a lot of work). If one wants the textual version they would use the disp call instead (invoked manually).

In Octave the display method is already defined for built-in types (which xoctave deletes and overrides with the display.m function). To allow the user to customize how the built-in types are presented, I came up with the idea of displayformat function (which I don't particularly like tbh) that allows to store the preferences which are then retrieved by the overridden display function to decide what to do. (Note that at the moment displayformat is only used by the overridden display function used to show builtin types)

Classes can on the other hand easily provide a custom display method, so my idea was that class authors should check if they are running in a xoctave instance (e.g. by checking the global variable XOCTAVE) and if so call the display_data function to show the rich output, instead of just printing some rich text.

The display_data function however is not very user friendly, as one has to specify the mimetype and has to know how the objects are defined in the jupyter protocol.

For ease of use, then, I thought it might be helpful to provide a few prebaked functions (e.g. display_html, display_latex, display_audio, display_image etc.) that would take arguments with octave builtin types (e.g a string for html or latex, a matrix for audio or image, etc.) and automatically call display_data with the correct parameters.

Those functions would be then used by class authors in their own display function implementation to display rich output from the internal representation of their classes in a simpler way, without having to know the details of jupyter protocol.

[AntoinePrv]

That explains a lot, thanks for this valuable information.

In Octave the display method is already defined for built-in types (which xoctave deletes and overrides with the display.m function). To allow the user to customize how the built-in types are presented, I came up with the idea of displayformat function (which I don't particularly like tbh) that allows to store the preferences which are then retrieved by the overridden display function to decide what to do. (Note that at the moment displayformat is only used by the overridden display function used to show builtin types)

I didn't know that. I don't think it's so bad. Libraries like numpy and pandas in Python also have global styling options.
it would be nice if we could get it scoped though, like xoctave.displayformat, and have separate displayformat for different objects (tables, equations...).

For ease of use, then, I thought it might be helpful to provide a few prebaked functions (e.g. display_html, display_latex, display_audio, display_image etc.) that would take arguments with octave builtin types (e.g a string for html or latex, a matrix for audio or image, etc.) and automatically call display_data with the correct parameters.

Those functions would be then used by class authors in their own display function implementation to display rich output from the internal representation of their classes in a simpler way, without having to know the details of jupyter protocol.

I like that plan. The display_data would be an extension for xoctave?

[rapgenic]

it would be nice if we could get it scoped though, like xoctave.displayformat, and have separate displayformat for different objects (tables, equations...).

I totally agree with you, this was my plan too, but I didn't tackle it because it was almost as defining an API, and I didn't have enough information to make choices that I would have needed to support in the future even if they were the wrong ones.

Regarding scoping it's totally possible (as I have found developing the widgets) by simply placing the relevant m-file inside a +scope folder.

I like that plan. The display_data would be an extension for xoctave?

display_data is already a function available inside octave runtime, and it's used for displaying latex within symbolic and control packages (at the moment of course I have overridden manually their display methods)

See:

xeus-octave/share/xeus-octave/@sym/display.m

Lines 1 to 6 in 8150461

	function display(x)
	name = private_disp_name(x, inputname (1));
	out.("text/latex") = [ "$$" name, "=", latex(x) ,"$$"];
	display_data(out);
	end