feat(capture/linux): add KWin direct screencast capture method#5009
feat(capture/linux): add KWin direct screencast capture method#5009Kishi85 wants to merge 20 commits intoLizardByte:masterfrom
Conversation
Kishi85
force-pushed
the
kwin-direct-screencast
branch
9 times, most recently
from
2ce02db to
b1dbab7
Compare
|
Tested on KDE Plasma 6.6.4, RX 9070 XT, Mesa 26.2 dev, CachyOS with Vulkan CBR encoding. Works great once you get past the setup — I had to set Once running, streaming is smooth. At one point Moonlight picked the external IP instead of the local one (sometimes It does), so the stream was going out through the internet and back without me even noticing — only realized when I turned on the debug overlay and saw higher the Internet network latency. |
Kishi85
changed the title
ReenigneArcher
changed the title
Kishi85
changed the title
Kishi85
changed the title
|
void-linux/void-packages#31274 (comment) Has had the permission issue before on xdg-portal-kde and fixed it by adding |
|
Adding a dummy desktop file like this as I'll have to test if we have to package and properly distribute the .desktop file or if it is sufficient to create it on-the-fly upon kwingrab init (in ~/.local/share/applications/app-dev.lizardbyte.app.Sunshine-kwin-screencast-helper.desktop before accessing zkde_screencast_unstable_v1). The latter would be preferable, we could also just remove the desktop file upon destruction to have a clean state. |
Kishi85
force-pushed
the
kwin-direct-screencast
branch
13 times, most recently
from
980f115 to
2e361c6
Compare
|
Running this with CAP_SYS_ADMIN breaks KWin's permission check (because it's not allowed to resolve the executable path for the PID with the capability). I'll have to check if the same workaround we do for portalgrab works here. |
|
@ReenigneArcher Do you think this should be enabled for FreeBSD and Flatpak as well (to be on-par with portalgrab)? Nothing should break, the worst case would be that kwingrab does not work (most likely due to missing permissions). Unfortunately I can't test those 2 options by myself atm, hence I've left them out in eba45e6. |
|
My thoughts: XDG Desktop Portal is always guaranteed to work for a Flatpak since the XDG standard is mandated by the package format, and the Desktop Portal is specifically targeted for sandboxed apps. KDE also (uniquely) has a secondary permission system - plumbed through flatpak - that allows bypassing of manual portal setup. For that reason, kwingrab is technically not necessary unless there is a measurable performance difference in the screencast flow through pipewire between the methods. But on the other hand, kwingrab has the same security requirements as portal, and can work without issue as long as your .desktop file is installed, correct? I don't see the harm in enabling it, as long as we can verify it works as expected. Having it enabled for all possible Linux packaging formats would simplify documentation. FreeBSD doesn't support Flatpak, and while I think they do support the XDG standard, Desktop Portal is not guaranteed to be present, and if it is, it's unclear how to leverage the permission bypass without a working flatpak command, so it seems that kwingrab makes sense for this environment. |
|
|
Technical sidenote: Looking at So we can assume that all pipewire stream features for portalgrab via xdg-desktop-portal-kde should also be directly available for kwingrab. |
|
That's the issue with kwingrab; there's extra maintenance burden in directly utilizing the (unstable) Wayland extension that would be transparently handled by XDG Portal via the KDE helper. Relevant PR that may impact us in future, if the "v1" protocol is obsoleted: https://invent.kde.org/plasma/kwin/-/merge_requests/9048 |
We're not the only ones in that boat then (OBS for example also utilizes it) and if it breaks we still have a working fallback with portalgrab (which Sunshine will always need for anything non-KWin). Also we have enough time to prepare so thanks for mentioning the drafts early. kwingrab is just a way to get a more direct stream for KWin-based systems with a few bonuses (like working screen priorities). The main streaming logic is already shared with portalgrab by utilizing pipewire_display_t so it's just the negotiation that might break until fixed (although I'm not seeing a huge difference for output streaming between v1 and v2 as it is right now. We should even be able to implement a fallback from v2 to v1 once the necessary definitions are in plasma-wayland-protocols). EDIT: I've got curious and have a preliminary implementation based on currently available draft protocol spec for kde_screencast_v2 ready for a future PR (after the spec hits master on plasma-wayland-protocols) here: Kishi85@281cc3a |
This commit is taken from LizardByte#4724 with all pipewire duplication removed and changed to pipewire.cpp Co-authored-by: Ramalama2 <6314556+Ramalama2@users.noreply.github.com>
…wayland output index This is just a first shot at working multi-monitor support for basic testing.
…in XDG_DATA_HOME/applications
Utilitzies kde_output_order_v1 wayland protocol
…s permission check KWin's Wayland permission check requires the wayland client processs pid's executable to be readable by KWin. While running with CAP_SYS_ADMIN this is not possible so KWin will fail the check.
This is necessary for running with CAP_SYS_ADMIN as the normal check for display_names being empty will not work as that will always return a dummy value to not drop capabilities early (before encoder probing).
…ermission Also extend the permission helper to check if system file exists
To be able to re-use existing wayland code
…s properly cleaned up on error
…l versions easier
…ssion system If the permission system is deactivated via environment variable KWIN_WAYLAND_NO_PERMISSION_CHECKS=1 dropping elevated privileges (CAP_SYS_ADMIN) is not necessary as KWin will not have to match the PID executable to a desktop file.
… Flatpak / FreeBSD It's always added right after ENABLE_PORTAL to be consistent with portalgrab.
|
A few minor things I've done/added in the last few days:
EDIT: I've also no further ideas on how to improve this so if anyone has any feel free to comment. The changes for upcoming kde-screencasting-v2 (which no longer has unstable in it's name and seems to be implemented as an overhaul in part due to changes to XDG portal to allow using pipewire-serials) are also ready for a future PR once plasma-wayland-protocols contains the necessary spec file and testing the feature is possible. |
|
6 participants
Description
This PR adapts the changes from #4724 to the changes from #5008. Here's an updated description (adapted from the original PR) of what this PR does and why it is implemented:
Why?
The portal path requires two additional D-Bus services (
xdg-desktop-portalandxdg-desktop-portal-kde) that act as pure pass-through layers with no added functionality for Sunshine's use case. In containerized environments (LXC/Docker), the portal frontend also has a known bug (EACCESon/proc/<pid>/rootin user namespaces) that requires patching.By speaking the KWin screencast Wayland protocol directly, Sunshine:
dbus-update-activation-environmenthackspipewire.cpp(pipewire_display_tclass from refactor(linux/xdgportal): Split pipewire code into it's own file so it can be reused #5008)How it works
New file
kwingrab.cppimplements a standalone Wayland client that:zkde_screencast_unstable_v1from the registrywl_outputobjects and selects the target outputstream_output()to request a screencast stream from KWincreatedcallbackEnabled via
capture = kwinin the Sunshine config and the CMake flag-DSUNSHINE_ENABLE_KWIN=ON.NOTE: While KWin's permission system is enabled (which it is by default for security reasons) a desktop file granting the necessary interface permission for
zkde_screencast_unstable_v1to Sunshine's process executable (as given byreadlink /proc/<pid-of-sunshine>/exe) and KWin being able to determine the executable is required for kwingrab to work. kwingrab tries to set up permissions as best as possible by:/usr/share/applications.~/.local/share/applicationsshould the system-wide file not suffice/be available.If setting up permission via desktop file does not work or the use-case does not support working with KWin's permission system (e.g. due to not being able to drop CAP_SYS_ADMIN) it is possible to disable KWin permission checks by setting
KWIN_WAYLAND_NO_PERMISSION_CHECKS=1in (global) environment to make kwingrab work.Real-world testing
Tested extensively over WAN (VDSL and LTE/5G) with Moonlight at 1080p/4K, 60/120fps. The KWin direct path delivers noticeably smoother streaming compared to NvFBC + Xorg, particularly over WAN where consistent frame cadence matters most. KWin's vsync-aligned, compositor-driven frame delivery produces a more even packet flow than NvFBC's software-timer-based polling.
ToDo-List:
using systemdwith CAP_SYS_ADMINScreenshot
Issues Fixed or Closed
Roadmap Issues
Type of Change
Checklist
AI Usage