support touch input

[pkra]

Never moved here from krautzource/mathjax-sre-walker#9. To quote

On mobile, we lack proper exploration.

We seem to get only the benefit of being a progressive enhancement, e.g., Talkback users can switch to read-by-word and go through the description word by word. But we do not get full exploration and exhighlighting that way.

[pkra]

From krautzource/mathjax-sre-walker#9 (comment)

HammerJS seems inactive. Maybe https://github.com/AlloyTeam/AlloyFinger is worth a look (there are other libraries). The main question is if mobile AT has a way to pass inputs through.

[pkra]

https://github.com/john-doherty/swiped-events seems like a good source of inspiration.

[pkra]

I think I have a solution. Needs more testing because we'll have to listen (and preventDefault) for touch on the whole document.

[pkra]

Needs more testing because we'll have to listen (and preventDefault) for touch on the whole document.

Unfortunately, it looks like this idea won't work on mobile VoiceOver.

[pkra]

Unfortunately, it looks like this idea won't work on mobile VoiceOver.

There might be a chance after all. Needs investigating.

[pkra]

For future reference: https://patrickhlauke.github.io/touch/tracker/multi-touch-tracker-pointer-hud.html (double tap+hold+... should trigger this with VO)

[pkra]

Coming back to this after much pondering, I currently see two paths forward.

a) implement walking by swiping
b) implement a new approach (more aligned with "standard" tree exploration, i.e., what APG shows).

For a), the main challenge is whether it's good enough (especially VO with touch would not be great UX) and if we find reasonable resolution for the up/down swipe vs scrolling conflict.

For b), I don't see a way to make this work (well) for non AT users. It would also require a fairly different bit of logic.

[pkra]

For b), links will finally come to haunt me.

[pkra]

Here's another stray idea while talking with @zorkow: a mini UI of 4 arrows, sticky on top-left (or wherever dir dictates).

[pkra]

Coming back to this after much unsuccessful research in the previous years.

Some notes

There are two problems and I don't see a simultaneous solution: touch input exploration vs exploration with mobile screenreaders.

For touch exploration, I have some leftover changes (which I don't fully understand right now) which do a decent enough job for touch users (except that it's hard to drop out of focus/interaction on safari but perhaps that's not insurmountable).

In theory, talkback users could use that approach since talkback allows gesture pass through. However, "4-finger-double-tap-and-hold + swipe" isn't exactly highly usable.

However, I couldn't get VoiceOver to work with that at all - the double-tap-hold-wait-for-three-dings-interact never worked for me. And again, it's not exactly usable.

On the other hand, the approach from https://github.com/krautzource/granularity-walker works for talkback and VO users. Perhaps that approach could work for touch input as well. Perhaps both could be combined.