implement actualtext, artifact from recent graphicx

[xworld21]

Fix #2679 more or less as we discussed, but with more care about how the keys interact with each other:

• if artifact is true, @description is set to empty unless actualtext, alt are present
• if actualtext and alt are both present, save both as respectively description and aria-description

The latter might be slightly controversial. I played quickly with VoiceOver and it reads both alt and aria-description in sequence, with a proper pause (feels like "actualtext: alt"), which seems reasonable.

We could add a further heuristic that if actualtext and alt are equal, then just @description is enough.

I have added a test with all 8 combinations to clarify the intended semantics.

[u-fischer]

ngpdf (which implements the derivation from pdf to html) uses a <span> and drops the graphic if an actualtext is present, and <img alt=...> if there is an alt (but no actualtext) and ignores graphics with artifact. You can test that at https://texlive.net/ngpdf

\DocumentMetadata{tagging=on}
\documentclass{article}
\usepackage{graphicx}
\begin{document}
\includegraphics[actualtext=A,width=10pt]{example-image-a}t the begin there was

\includegraphics[alt=A duck]{example-image-duck}

\includegraphics[alt=A duck,actualtext=D]{example-image-duck}uck

\includegraphics[artifact,width=1cm]{example-image}
\end{document}

[brucemiller]

Thanks @u-fischer for the clarifications; that really helps! It's perhaps quite fortuitous that we're also dealing with a related issue (ACM's \Description) in #2666. It would be good to have a consistent, flexible approach to these data that we can build upon (and in particular, noting that HTML is not the only target).

Both these systems potentially supply material for two distinct use cases: (1) replacement text to be spoken in place of a given image or construct or (2) descriptive text used when a reader asks for more information. Of course, that distinction will easily be confused, and the way that the material is best delivered to the reader will depend on the eventual medium or delivery device. The artifact flag presumably just signals that the replacement text used should be empty (and not otherwise inferred from any other data).

And, although some of the documentation for these encourage using concise language free of markup, authors will do what they want, so I'm more inclined to store this data in XML elements, rather than attributes; and in particular, I'd want to avoid diving directly into aria and defer that until postprocessing (or XSLT) when the target format is dealt with.
Normally, I'd use the ltx:note element, with appropriate @role attribute, to distinguish the use cases, but am open to introducing new element(s) if that's more appropriate. (As @dginev will know, the ltx:note element is already quite heavily overloaded!)

[dginev]

As much as I appreciate the broader discussion on related work (thank you for raising awareness!), I am a little worried we'll hurt the primary use case without good cause.

Citing the graphicx documentation:

actualtext ActualText text for accessibility uses. By default this key is not
used but users are encouraged to add text here that may be used in tagged PDF or as the text to use in formats that do not use the image.

artifact Boolean to mark the graphic as an artifact for accessibility uses. By default this key is not used but users are encouraged to flag decorative or other non structural images as an artifact for tagged PDF.

The ngpdf implementation seems to be in conflict with the actualtext description (to me), as HTML as a format uses images, very similarly to PDF. I think keeping the image and adding an accessible label fits more naturally. As to handling artifact, mapping that to aria-hidden="true" seems to be a best practice as well from an ARIA perspective.

I am quite happy with Vincenzo's approach - there is a lot of work already done by the ARIA group which is clearly documented and standard. It will be great if we can use that without reinventing the full breadth of their spec with alternatives in LaTeXML's schema.

We can also separately create XSLT rules mapping any attributes in the aria: namespace to reasonable JATS <annotation>, <alt-text> etc elements, where appropriate. A good question to ask is "which spec is leading the adoption momentum" and whether we want to be in synergy with them. Adding a new custom schema piece here doesn't seem warranted to me, as @brucemiller has heard me opine before.

[u-fischer]

@dginev

The ngpdf implementation seems to be in conflict with the actualtext description (to me),

No, the graphicx documentation doesn't try to explain when to use the keys, it only says that the keys exist and that their value will be passed to PDF keys (alt=> /Alt, actualtext=>/ActualText) if a PDF is tagged. More documentation about how and when to use them is in the documentation of the tagging project.

ngpdf then handles these keys and it does it as specified in the derivation algorithm.

[dginev]

No, the graphicx documentation doesn't try to explain when to use the keys,

Well, the text I read exactly explains the intent of when to use the keys. What am I missing? I understand there are direct mappings for accessible PDF, but that isn't a supported target of LaTeXML.

[u-fischer]

The description only says "ActualText text for accessibility uses", which doesn't say much unless you know what the PDF term ActualText means.

The actual documentation about the intent of the keys is in latex-lab-graphic.pdf

[dginev]

The key snippet from the description is "or as the text to use in formats that do not use the image."

That is also extremely standard - text alternatives are needed when the reader doesn't consume the image for some reason or another. But most of latexml's target formats can use the underlying image asset. So they should, in my opinion.

[u-fischer]

The key snippet from the description is "or as the text to use in formats that do not use the image."

Yes, but that too is not very specific as it doesn't say which format are used.
The key has by purpose not a definitive description in this documentation as different formats can use it in different ways.

How you handle the keys is really up to you, but do not take alternative text and actualtext as two words for the same thing. In PDF there is a clear distinction and PDF is here different to html. Users do not expect to hear "graphic" if actualtext is used, see e.g. nvaccess/nvda#18843.

[brucemiller]

Addressing several points....
I find it rather unfortunate for a project like LaTeXML, that attempts to target different formats, when enhancements to LaTeX with broad utility are defined in terms of a single target. That said, given the general intention of these enhancements, I see no need for LaTeXML to actually discard the image when ActualText is given, so long as it can arrange that the actualtext is what's spoken when the document is being spoken accessibly, irrespective of what any other implementation may do.

Next, aria is indeed quite complex, and I wouldn't want to reinvent a whole alternative representation for it within LaTeXML. On the other hand, exactly because of that, if we've already cast the data into aria I have zero expectation that we'll be able to deconstruct it afterwards to reassemble into some other representation such as JATS. Thus I'd prefer a more abstract representation at the earlier, XML, stage. Experience with too early insertion of too much CSS would seem to have been a lesson.
Finally, that both \includegraphics alt keyword and \Description should provide a "description", but are here getting two distinct representations in our XML bothers me.

[u-fischer]

when enhancements to LaTeX with broad utility are defined in terms of a single target.

PDF is a major output format of LaTeX, and so it is unavoidable that there are keys and commands that target mostly PDF specific needs, e.g. pdftitle or pdflang from hyperref, or pdfversion and pdfstandard in \DocumentMetadata, or as here the artifact and actualtext key.

For such keys other output formats will have to decide if they ignore them or map to something in their format.

Finally, that both \includegraphics alt keyword and \Description should provide a "description", but are here getting two distinct representations in our XML bothers me

well \Description is a acmart specific command. Currently it doesn't seem to be used anywhere. You could open an issue and discuss with the maintainer how he plans to use it.

[brucemiller]

when enhancements to LaTeX with broad utility are defined in terms of a single target.

PDF is a major output format of LaTeX, and so it is unavoidable

I completely understand & agree; it's unavoidable there's going to be pdf specific data in LaTeX. My point was that some things (like accessibility), the data are not pdf-specific (though the implementation will be); in those cases avoiding too much pdf specificity in the documentation would be good (for users, implementers, converters & latex longevity in general).

well \Description is a acmart specific command. Currently it doesn't seem to be used anywhere. You could open an issue and discuss with the maintainer how he plans to use it.

Apparently authors are using it on arXiv; I'd be surprised if the maintainer hasn't got some plan for tying it into your pdf-accessibility machinery if not already. But my reason for mentioning it here was purely from a LaTeXML POV. Its intention as an object "description" (rather than speech replacement) seems clear, but I'm trying to assure that "descriptions", whatever package/binding they come from, are represented & treated consistently within LaTeXML.

[xworld21]

This generated more discussion than I expected!

I see two sticking points:

• Unlike with ngpdf.com, the HTML output of LaTeXML is supposed to be a complete PDF replacement, so it needs to include the picture. However, HTML does not have a mechanism comparable /ActualText (whose behaviour I read from (summaries of) the PDF spec itself, not from the graphicx documentation). There are some crude workarounds with background-image, none of them particularly good.
• This PR follows the naive HTML model and merges the actualtext=, alt= options. This is a lossy choise and prevents other outputs and downstream usesrs from making better choices.

My intention was to introduce initial support that can be merged in 0.9 so I will still campaign for merging as is, but I agree that the XML should preserve the info. I'd rather use additional attributes and/or children of the <ltx:graphics> element though. /ActualText and /Alt are pure text in PDF, so they don't need to be elements; and even if they were elements like <ltx:note>, they should be descendants of <ltx:graphics> both for logical and practical XSLT reasons.

[brucemiller]

I'll take the view that the discussion only sounds more contentious than it really is.

I'm not sure I understand @xworld21 first sticking point; Don't the two pieces of information we're discussing just map to aria's description (for alt and \Description) or label (for actualtext)? [when aria is applicable].
Of course, historically html img's @alt represents both those cases and \includegraphics[alt] as well as LaTeXML's ltx:graphics@description have inherited that blur. It's kinda amusing that your code is forced to use two attributes, @description and @aria:description, for the 2 distinct quantities! So, it would be good (if attributes are the right choice) to have an attribute to use for the replacement text (but label is not a very good name in a LaTeX context).

So for me, the sticking point is whether I can convince yall that using elements for these two (which would need to be within the meta-class and be allowed content within an ltx:graphics), and the collect the conversion of these into a single place in postprocessing that would convert them to appropriate aria, whenever aria is the right choice.

[dginev]

Changing to using an element to hold the accessible text, giving it an xml:id, then pointing to it via an aria:labelledby or aria:describedby is fine by me. If the #2666 PR is merged we will have a simple ltx:note formulation that can do that.

Reinventing a custom LaTeXML technique to do the pointing just doesn't strike me as helpful. We'll increase the learning curve and confuse anyone external about what LaTeXML is trying to do differently. I am quite happy we are using the xml:id attribute and not a ltx:xmlid mirror, for example.

[brucemiller]

Shouldn't this be aria:label?
Perhaps you were thrown by the awkward naming of ltx:graphics@description (as I was), which is a stand-in for html's img@alt, whose specification seems to shift over time.
The graphics keywords alt and actualtext are apparently only very subtly different; there's not (yet) a key for a detailed description, which is what aria:description should be (as I understand it).