refactor(VFTCapture): modernize and fix vive face tracker on linux #173

Naraenda · 2025-12-06T00:23:29Z

Closes #146.

This PR contains the following set of changes:

Refactors ViveFaceTracker.cs.
- The main change is the refactor from static members to a full wrapper around the file descriptor.
- De-duplicated code.
- Added more comments.
- Removed "unsafe" code.
- Slightly better logging.
Fixes the VFT device on Linux.
- The original issue stems from incorrect assumption about available control buffer size.
- New code handles both 64 and 384 byte buffer.
- Main logic should resemble reference implementation much more closer. See https://github.com/LordOfDragons/vivefacialtracker

While working on this, I realized Windows could be fixed really easy by using Window's IOCTL libraries. That's out of scope for this PR though (and I don't have a Windows install anymore).

CLAassistant · 2025-12-06T00:23:35Z

All committers have signed the CLA.

dfgHiatus · 2025-12-07T01:08:12Z

Thanks for your PR! We don't have the Vive Facial Tracker on hand here, but if we can get some people to test this we can get this merged in. Maybe some people from the LVRA discord could help?

hypevhs · 2025-12-10T11:08:25Z

I have a Vive Facial Tracker and an Arch Linux install. I'd like to try testing this soon.

I don't know if there are any prerequisites other than building this branch and trying to run the desktop application with the VFT connected.

Naraenda · 2025-12-10T16:06:28Z

I have a Vive Facial Tracker and an Arch Linux install. I'd like to try testing this soon.

I don't know if there are any prerequisites other than building this branch and trying to run the desktop application with the VFT connected.

Dependencies haven't changed, so you can just use the same build process as before.

… magic packet

feilen · 2025-12-21T06:11:12Z

Heya! Apologies if I'm doing something wrong here, quite new to this project. I built this module in a separate source tree and placed it in the Modules for the upstream built app, I'm getting the following:

./Baballonia.Desktop
info: Baballonia.Views.MainWindow[0]
      Baballonia version 1.1.0.9 starting...
info: Microsoft.Hosting.Lifetime[0]
      Application started. Press Ctrl+C to shut down.
info: Microsoft.Hosting.Lifetime[0]
      Hosting environment: Production
info: Microsoft.Hosting.Lifetime[0]
      Content root path: /home/feilen/Apps/Baballonia/Baballonia.x64.v1.1.0.9rc4
info: DefaultInferenceRunner.eyeModel.onnx[0]
      /home/feilen/Apps/Baballonia/Baballonia.x64.v1.1.0.9rc4/eyeModel.onnx initialization finished
fail: Baballonia.VFTCapture.VftCapture[0]
      Error in ioctl uvc_xu_control_query request:
        {q:GET_LEN,sz:2}
        {r:-1,e:2}: No such file or directory
info: DefaultInferenceRunner.faceModel.onnx[0]
      /home/feilen/Apps/Baballonia/Baballonia.x64.v1.1.0.9rc4/faceModel.onnx initialization finished
info: Baballonia.Services.ActivationService[0]
      Configuring OpenVR...
warn: Baballonia.Services.OpenVRService[0]
      Failed to Enable SteamVR AutoStart: Init_NoServerForBackgroundApp
warn: Baballonia.Services.ActivationService[0]
      Failed to configure OpenVR during ActivationService startup. Skipping.
info: Baballonia.Services.LocalSettingsService[0]
      Saving settings
fail: Baballonia.Services.Inference.SingleCameraSourceFactory[0]
      No data was received from Bigscreen Bigeye (video0), with VFTCapture, closing... Maybe the camera is opened somewhere else?
fail: Baballonia.VFTCapture.VftCapture[0]
      Error in ioctl uvc_xu_control_query request:
        {q:GET_LEN,sz:2}
        {r:-1,e:2}: No such file or directory
fail: Baballonia.VFTCapture.VftCapture[0]
      Error in ioctl uvc_xu_control_query request:
        {q:GET_LEN,sz:2}
        {r:-1,e:2}: No such file or directory
fail: Baballonia.Services.Inference.SingleCameraSourceFactory[0]
      No data was received from Bigscreen Bigeye (video0), with VFTCapture, closing... Maybe the camera is opened somewhere else?
fail: Baballonia.VFTCapture.VftCapture[0]
      Error in ioctl uvc_xu_control_query request:
        {q:GET_LEN,sz:2}
        {r:-1,e:2}: No such file or directory

Let me know if you need anything else! I'm running Ubuntu 24.04.3 LTS if that matters. I've not added any additional udev rules specific to the VFT either.

feilen · 2025-12-21T06:13:22Z

Should note that this is after removing the LibUVCCapture and OpenCV dlls as it was trying to open with those previously.

Trying again in case it was a fluke, got the following:

info: Microsoft.Hosting.Lifetime[0]
      Hosting environment: Production
info: Microsoft.Hosting.Lifetime[0]
      Content root path: /home/feilen/Apps/Baballonia/Baballonia.x64.v1.1.0.9rc4
info: DefaultInferenceRunner.eyeModel.onnx[0]
      /home/feilen/Apps/Baballonia/Baballonia.x64.v1.1.0.9rc4/eyeModel.onnx initialization finished
info: DefaultInferenceRunner.faceModel.onnx[0]
      /home/feilen/Apps/Baballonia/Baballonia.x64.v1.1.0.9rc4/faceModel.onnx initialization finished
fail: Baballonia.ViewModels.SplitViewPane.HomePageViewModel[0]
      System.ArgumentNullException: Value cannot be null. (Parameter 'No provider for Bigscreen Bigeye (video0) not found')
         at Baballonia.Services.Inference.SingleCameraSourceFactory.CreateStart(String address) in /home/runner/work/Baballonia/Baballonia/src/Baballonia/Factories/SingleCameraSourceFactory.cs:line 64
         at Baballonia.Services.EyePipelineManager.StartLeftVideoSource(String cameraAddress, String preferredBackend) in /home/runner/work/Baballonia/Baballonia/src/Baballonia/Services/EyePipelineManager.cs:line 131
         at Baballonia.Services.EyePipelineManager.TryStartLeftIfNotRunning(String cameraAddress, String preferredBackend) in /home/runner/work/Baballonia/Baballonia/src/Baballonia/Services/EyePipelineManager.cs:line 268
         at Baballonia.ViewModels.SplitViewPane.HomePageViewModel.StartCamera(CameraControllerModel model) in /home/runner/work/Baballonia/Baballonia/src/Baballonia/ViewModels/SplitViewPane/HomePageViewModel.cs:line 627
fail: Baballonia.ViewModels.SplitViewPane.HomePageViewModel[0]
      System.ArgumentNullException: Value cannot be null. (Parameter 'No provider for Bigscreen Bigeye (video0) not found')
         at Baballonia.Services.Inference.SingleCameraSourceFactory.CreateStart(String address) in /home/runner/work/Baballonia/Baballonia/src/Baballonia/Factories/SingleCameraSourceFactory.cs:line 64
         at Baballonia.Services.EyePipelineManager.StartRightVideoSource(String cameraAddress, String preferredBackend) in /home/runner/work/Baballonia/Baballonia/src/Baballonia/Services/EyePipelineManager.cs:line 200
         at Baballonia.Services.EyePipelineManager.TryStartRightIfNotRunning(String cameraAddress, String preferredBackend) in /home/runner/work/Baballonia/Baballonia/src/Baballonia/Services/EyePipelineManager.cs:line 279
         at Baballonia.ViewModels.SplitViewPane.HomePageViewModel.StartCamera(CameraControllerModel model) in /home/runner/work/Baballonia/Baballonia/src/Baballonia/ViewModels/SplitViewPane/HomePageViewModel.cs:line 631
info: Baballonia.Services.ActivationService[0]
      Configuring OpenVR...
warn: Baballonia.Services.OpenVRService[0]
      Failed to Enable SteamVR AutoStart: Init_NoServerForBackgroundApp
warn: Baballonia.Services.ActivationService[0]
      Failed to configure OpenVR during ActivationService startup. Skipping.
info: Baballonia.Services.LocalSettingsService[0]
      Saving settings
fail: Baballonia.Services.Inference.SingleCameraSourceFactory[0]
      No data was received from HTC_Multimedia_Camera HTC_Multimedia_Camera (video2), with VFTCapture, closing... Maybe the camera is opened somewhere else?
fail: Baballonia.Services.Inference.SingleCameraSourceFactory[0]
      No data was received from HTC_Multimedia_Camera HTC_Multimedia_Camera (video2), with VFTCapture, closing... Maybe the camera is opened somewhere else?

dfgHiatus · 2025-12-21T06:16:00Z

No data was received from Bigscreen Bigeye (video0), with VFTCapture, closing... Maybe the camera is opened somewhere else?

If I were you, enable 'advanced settings' and make sure only the face camera has the VFTCapture set as the backend. The Bigscreen can be whatever else

feilen · 2025-12-21T06:24:09Z

Hmm, I have Advanced Settings enabled but I don't see where I can set the backend?

dfgHiatus · 2025-12-21T06:25:42Z

It might be a thing too where you need to type out the device path, IE /dev/videoX instead of the camera from the dropdown

feilen · 2025-12-21T06:29:12Z

Similar again, with the backend explicitly selected:

fail: Baballonia.Services.Inference.SingleCameraSourceFactory[0]
      No data was received from /dev/video0, with OpenCvCapture, closing... Maybe the camera is opened somewhere else?
fail: Baballonia.Services.Inference.SingleCameraSourceFactory[0]
      No data was received from /dev/video0, with VFTCapture, closing... Maybe the camera is opened somewhere else?

dfgHiatus · 2025-12-21T06:30:50Z

Hm, back to the drawing board here. Thanks for your info so far!

feilen · 2025-12-21T06:31:51Z

All good! I'll take a few more swings at it myself, not impossible that the issue is simply on my end.

feilen · 2025-12-21T06:49:21Z

Copied the udevadm rules from the BSB eyetracking README and adjusted them for the vendor/product of the VFT, no dice. So it's not likely a permission issue

Naraenda · 2025-12-21T10:25:30Z

Hmm, it might be related to OpenCVSharp. I'm building mine seperately. I can try rebasing this onto the newer capture method PRs.

At least the camera ioctl calls work. That's what the PR modifies.

feilen · 2025-12-29T05:43:10Z

Oh, suppose I should ask before I delve to deep - is there a way to even train a model for the VFT on linux? I don't think Baballonia on Windows can capture from it raw either, unless I'm mistaken.

dfgHiatus · 2025-12-29T05:46:05Z

While we natively support training eye models on Linux, we do not do the same for the face. We have a strong enough general model here.

And you are correct. Currently we cannot open the VFT camera on Windows or Linux on main or v1109rc1.

refactor(VFTCapture): modernize and fix vive face tracker on linux

69d8130

Naraenda mentioned this pull request Dec 7, 2025

[Bug]: Vive Facial Tracker on linux fails to fetch data #146

Open

feat(VFTCapture.cs): improve logging, introduce sleeps and send extra…

3058744

… magic packet

refactor(VFTCapture): modernize and fix vive face tracker on linux #173

Are you sure you want to change the base?

refactor(VFTCapture): modernize and fix vive face tracker on linux #173

Uh oh!

Conversation

Naraenda commented Dec 6, 2025

Uh oh!

CLAassistant commented Dec 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dfgHiatus commented Dec 7, 2025

Uh oh!

hypevhs commented Dec 10, 2025

Uh oh!

Naraenda commented Dec 10, 2025

Uh oh!

feilen commented Dec 21, 2025

Uh oh!

feilen commented Dec 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dfgHiatus commented Dec 21, 2025

Uh oh!

feilen commented Dec 21, 2025

Uh oh!

dfgHiatus commented Dec 21, 2025

Uh oh!

feilen commented Dec 21, 2025

Uh oh!

dfgHiatus commented Dec 21, 2025

Uh oh!

feilen commented Dec 21, 2025

Uh oh!

feilen commented Dec 21, 2025

Uh oh!

Naraenda commented Dec 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

feilen commented Dec 29, 2025

Uh oh!

dfgHiatus commented Dec 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

CLAassistant commented Dec 6, 2025 •

edited

Loading

feilen commented Dec 21, 2025 •

edited

Loading

Naraenda commented Dec 21, 2025 •

edited

Loading

dfgHiatus commented Dec 29, 2025 •

edited

Loading