Question: Logical or Physical Pixels?

[mfisher31]

For these functions, what should users of the library expect: to use a logical or physical coordinate space? Should they all be the same?

PuglRect puglGetFrame(const PuglView* view);
PuglStatus puglSetSize(PuglView* view, unsigned width, unsigned height);
puglSetSizeHint(PuglView*    view, PuglSizeHint hint, PuglSpan     width,  PuglSpan     height);
// and so on...

I assume logical since that's the natural way to think about geometry in GUI's, but that isn't what Pugl does in reality. The reason I wanted a puglGetScaleFactor is because PuglEvents received in LVTK widgets are in Physical pixels and need scaled if != 1.0. This could have changed in pugl, as I haven't tested LVTK with new code in pugl:main. Snippet of that code.. this is scaling to convert everything to Logical.

static PuglStatus expose (View& view, const PuglExposeEvent& ev) {
        if (ev.flags & PUGL_IS_HINT) {
            VIEW_DBG ("expose: hint");
            return PUGL_SUCCESS;
        }

        auto x = (float) ev.x / view.scale_factor();  // "scale_factor" comes directly from puglGetScaleFactor
        auto y = (float) ev.y / view.scale_factor();
        auto w = (float) ev.width / view.scale_factor();
        auto h = (float) ev.height / view.scale_factor();
        auto r = Rectangle<float> { x, y, w, h }.as<int>();

        VIEW_DBG3 ("expose: " << r.str());
        // this initiates the drawing of the widget hierarchy which want Logical pixels.
        view.owner.expose (r.intersection (view.owner.bounds().at (0)));
        return PUGL_SUCCESS;
    }

But I think it's important to make it clear what we're dealing with. I always assumed the coordinates were supposed to be physical for some reason because of the above. Again, I haven't tried the main branch in LVTK so I don't know if it's relevant anymore. I'm honestly asking now, what are they supposed to be? physical or logical?

Especially for puglSetSizeHint. And I'm sure you are well aware: nobody wants calculate the scaling factor of monitors, backing drawing contexts, and what not just to set a default size. We just want to say "set the size to 640x360 and always manifest it 640x360 visually regardless of backend graphics setup and monitors used.

So, with all that: the Cairo demo in main is setting a size hint defaulting @ 512x512. But when I run this on a 2x/4x monitor it displays as 256x256 in logical pixels. Running the same program on a 1x, normal, monitor it fills about half the vertical space which is what is to be expected. That's my thought process anyway.

I'm on board for whatever, and maybe some of the details need documented, so it's clear the expectations about coordinate space. I'm not tied to puglGetScaleFactor either BTW, but if the public pugl API is operating in physical pixels, this information is essential to have early for the next layer in the GUI chain: the UI toolkit level.. maybe even before puglRealize... IDK.

[drobilla]

Pugl intentionally does not have any such distinction in the public API, and if it does, it's a bug. If you ask for a view of 512x512 pixels, then you get 512x512 pixels. There are only pixels. There are no logical pixels, or physical pixels, or quasi pixels, or pseudo pixels, or any other kind of pixels some deranged mind in Cupertino thought up because they need to sell "Retina" without actually making any of the software changes necessary to properly support different DPIs.

To be fair, Microsoft fucked this up egregiously too, if in a different way. They both have their reasons - well, reason singular (backwards compatibility for proprietary applications) - but that reason doesn't apply here, so there's no reason for pugl users to suffer that pain.

We just want to say "set the size to 640x360 and always manifest it 640x360 visually regardless of backend graphics setup and monitors used.

What exactly does "visually" mean? It sounds like you want it to mean, well, not 640x360 pixels? At the end of the day, you have to actually draw stuff, and I don't think asking for a window to be 640x360 pixels and actually getting 1280x720 is in any way simple. That's both confusing and error-prone, and in reality 99% of Pugl apps would simply be broken on MacOS if it were that way. A window that literally has 640x360 pixels in it is, well, 640x360 pixels big. That's the number of pixels you need to draw, that's the number of pixels that are visible on the screen, that's objectively the number of pixels. I guess that's "physical", but they're just... pixels.

If you want the window to be bigger than that because the DPI is very high, well, then yep, you want the window to be bigger than that, so you have to... make the window bigger than that. So, yeah, you need the scale factor for that, but you need the scale factor (better the DPI, but we all fucked that one up too, so oh well) regardless. I don't think pretending pixels are anything but, well, pixels actually makes anything easier here. It just makes it worse. Needing to size things based on the DPI is inherently a part of dealing with variable DPI, unless you provide a mode where old apps that assume a fixed low DPI are automatically scaled up. That's why MacOS does this, forever punishing DPI-aware code to achieve it. Pugl doesn't have any legacy like that to support, so it has no such mode, so there's no need to punish users with made-up pixels.

Whether the current API is actually adequate for this is another question. I'm certainly not attached to it and puglGetScaleFactor is questionable at best (it should probably (also?) be in PuglConfigureEvent for one), but I merged it from falkTX to make things basically work and reduce fork divergence. Feel free to improve it, although if it's not obvious, I will resist exposing "logical pixel" insanity in the API with every fibre of my being ;)

[mfisher31]

Just trying to help.... thank you that answers my question. It's all raw pixel sizing, perfect, now I know for sure. The docs aren't clear on that though.

Visually means exactly what you said it does... the thing GUI programmers actually care about: the perceived size of widgets and windows the screen we're looking at. And I use the terms logical and physical just so you know what I'm talking about.

It's impossible to consistently, from client code using pugl, to define a default size using setSizeHint and have it "visually" be the same on 1x/2x/4x monitors without scale readily available.

So in short, I don't care that pugl doesn't try to hold my hand with scaling. But I DO want LVTK to be able to do so for it's users... and not have jumpy windows initially because scale factor is queue'd up in an event somewhere.

[mfisher31]

I'll do some RND one of these days soon. Configure sounds right to me (or maybe even Realize) if scale has to come in an event. Will be interesting to see if that affects "jumpiness"

[mfisher31]

Oh, and also. sorry. I don't want pugl tap dancing around so called logical pixels publicly & internally either. We're in agreement on that. But scale factor IS absolutely needed waaay early from a toolkit developer's perspective.

[drobilla]

I'm not yelling at you, I'm yelling at Apple :D It annoys the hell out of me how much of a mess this stuff is, across every single platform. X11, for its part, adopted the "let's just pretend the DPI is always 96 regardless of what the DPI actually is". It's effectively impossible to render text at a proper size on all three, it's all so embarrassing. Not sure about Wayland, I haven't done any scale stuff there yet. Anyway...

Yeah, needing a view for puglGetScaleFactor is the main problem. IIRC it's actually accessible from the world on all the current platforms internally, the API is just wrong. Adding it to the configure event is necessary to support moving across displays and such (and just generally fits the idea that everything about the visual configuration of a view is in the configure event), but that would have the same problem, so it's not really an alternative so much as an addition.

The tricky part is there there can be multiple displays, and they can have different scales, so conceptually there is no "world" scale factor at all. Pugl currently has no notion of a "screen" though. One would also be necessary for really doing fullscreen or mode switching or things like that properly as well.

I fear the complexity of adding any sort of actual display API, but maybe just an integer display ID or something works. I haven't made it that far. Could also just ignore this part and add a puglGetDefaultScaleFactor(world) or something, which could not break the current API (not that I care much until it's released), but would probably mean having to break or augment it later, although it wouldn't be the end of the world if not because if the configure event stuff was there the initial display would just be another time you might need to change it, as if it were dragged to a new screen which could happen anyway, so the initial one being "wrong" wouldn't really matter.

[mfisher31]

Ok, good. I was worried there for a second :) I did actually set up a test case where the scale factor is delivered in a Configure event. I must admit, I can see your point now, puglGetScaleFactor is not an ideal solution by any means. I have changed sides on this one officially. In other words, handling it in a configure event works, and looks good. Not jumpy as I thought it would be.

This also helped me uncover areas in mac.m where over scaling or under scaling was happening depending on if TopLevel or Child view..... fuller report on that coming soon. My test apparatus lives inside Roboverb.

The pugl code is this one: https://gitlab.com/lvtk/pugl/-/compare/main...scale-as-event?from_project_id=11851529 . Not sure how correct any of it is yet as I've only experimented on a Mac too.

Based on what you're saying about the platforms having problems themselves.... Is worrying about scale + DPI just a non-issue? What about provide an assumed scale (what we have now) + a "quality" factor or something to hint that client code might need an adjustment scale THEY come up with? I've seen in more than one GUI toolkit where scaling is constantly adjusted to compensate for differing platforms immediately before initiating widget-tree rendering (in logical pixels :)) (not an expert in that area, but do actually read source code like a book some times).

[mfisher31]

I also saw this: PUGL_SHOW_PASSIVE this is one could certainly be used to keep "jumpiness" hidden until a configure comes in with the desired sizes.

[drobilla]

At least one configure event must always be delivered before actual exposure, so there shouldn't be any jumpiness at all as far as the window contents goes. I'm not sure about the window size itself, that can't be controlled so tightly. Dynamic resizing is certainly best avoided in general (I wish it didn't have to be in the API whatsoever, but that's a fantasy), so having both is still probably better, but... yeah, maybe just "you have to deal with this on configure" isn't so bad.

What about provide an assumed scale (what we have now) + a "quality" factor or something to hint that client code might need an adjustment scale THEY come up with?

The whole point of the scale factor is that the applications doesn't set it. If an application wants to have further internal notions of scale (or zoom, or font size, or rotation, or whatever else), that's fine, but that's their problem and has nothing to do with Pugl.

[drobilla]

Also I ran into some problem or another on some platform or another trying to put the scale in PuglConfigureEvent but I don't remember what it was.

[mfisher31]

Hmmm I'll keep playing around with it and see what I can break! Scale + "reconfigure" flag or return state so internal pugl can react. Code example on that one says it all. This is detecting a change in pugl's scale factor, then wanting a new size scaled up so widgets can work in user space.

    static PuglStatus configure (View& view, const PuglConfigureEvent& ev) {
        // ....
        auto& widget = view.widget;

        if (view.pugl_scale != ev.scale) {
            VIEW_DBG ("pugl: scale changed: " << ev.scale);
            view.pugl_scale = ev.scale;
            
            // The pugl frame needs adjusted to fit widget size in user coords. So
            // request one and wait for the next configure.
            VIEW_DBG ("pugl: resize frame for widget: " << widget.bounds().str() 
                << " to PuglCoord " << (widget.bounds() * view.scale_factor()).str());

            auto r = widget.bounds() * view.scale_factor();  // this correctly scales a rectangle.
            puglSetSize (view.view, r.width, r.height);
            return PUGL_BAD_CONFIGURATION;  // instead of bad config, reconfigure wanted.
        }
       // ..... no re-config needed scale down and initiate widget drawing.
 }

[mfisher31]

An explicit reconfig wanted lets pugl know whether or not it should do whatever is does next, or handle a reconfigure. bad config could work too I guess, but thats more of a "find a graceful fallback" deal rather than a "standard operating procedure" reaction.

[mfisher31]

Yeah, IDK about this one. I could also see a new puglReconfigure(...) that could be used to return an event back to the backend. a milliseconds worth of time went in to that one :)

[drobilla]

That's what puglSendEvent is for, although it doesn't support configure on MacOS at the moment.

[drobilla]

Returning to this on my Mac, what I currently see with pugl_cairo_demo on a MBP with a 2.0 scale factor:

The demo requests a default size of 512x512 (meaning physical pixels, or just pixels), and the window is initially that size, i.e., the cairo surface is 512x512 pixels. Expose events are as well, i.e., the full view expose events are 512x512. Mouse button, motion, and scroll events are reported relative to that, i.e., the center of the view is 256x256.

This is all as expected. Do you have a case where Pugl itself is accepting or reporting inconsistent units?

Of course, this demo isn't using the scale for anything, in the Pugl API 512 means 512 physical pixels on all platforms. If you want it to be twice that large, then you need to literally do that. Then it would be 1024x1024.

tl;dr: I don't see a problem here. Regarding initial jumpiness, don't oscillate, choose the size you want from the start (puglGetScaleFactor works before the view is realized).

[drobilla]

tl;dr: I don't see a problem here

... okay, well, not with the units exposed anyway.

I think the thing that can't work here is changing the scale, so dragging a window to a higher DPI display and expecting it to be twice as many physical pixels large. I have no idea how that can work (for one, a window can be on several displays at once), but it doesn't sound like fun. It might be possible to make it work to detect that case and reconfigure yourself when necessary, but (as you discovered) it will need special handling.

When using physical pixels everywhere, when a window is dragged to a different display, all Pugl can do is report what happened: the same old size in physical pixels, with a new position, and maybe a new scale factor because this display is different. If so, if the application knows the size it is at the current scale, it can somehow enqueue a request to enlarge/shrink, but it will have to be careful to track the scale it's adjusted to to not end up in disastrous loops.

If Pugl used logical pixels, that problem of needing to reconfigure yourself goes away, but (in addition to having this complexity) then it would be expected that your underlying surface has changed in physical pixels, and that is an event that needs to be communicated and handled correctly.

tl;dr: Adjusting scale for different displays dynamically doesn't work, and I'm not sure it can work with the current API, but it's probably a pretty involved thing to implement. I think it should be possible to, say, be twice as large on a system with a scale factor of 2 though. It's just a bit of a PITA to do, but it always is in some way or another 🤷