Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrating VR into simulate #1452

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open

Conversation

nishbo
Copy link

@nishbo nishbo commented Feb 26, 2024

The current branch uses OpenVR (very close to the original mjvive project) to render the scenes in VR. It is turned on via MUJOCO_BUILD_SIMULATE_VR cmake option. If the VR could not be initialized, the simulate runs normally. I aimed at minimizing the number of changes/additions to simulate.cc. Supports definition of VR transformation in the model (instead of hardcoded default) - see balloons model.

Tested on Windows 10/11, using Steam VR:
Valve Index - works
HTC Vive XR Elite - works, streaming using Vive Streaming Hub
Meta Quest 3 - technically works (through Steam Link), but the projections are wrong, will need fixing.

Current problems:

  1. Not enough documentation. Suggestions on where to add docs are welcome.
  2. I need to manually add openvr_api.lib to the linker include directories in Visual Studio and copy openvr.dll into the output directory. Probably can be fixed at the cmake level, not sure how, suggestions welcome.
  3. Local copy of glad.

Directions before merging:

  1. Switching between VR transformations (partially complete).
  2. Fixing Quest 3.
  3. Adding UI bars to VR.
  4. Adding controllers.

Copy link

google-cla bot commented Feb 26, 2024

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

simulate/cmake/SimulateDependencies.cmake Outdated Show resolved Hide resolved
simulate/simulate.cc Outdated Show resolved Hide resolved
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't auto-generated, right? Could this be taken from the checked out repository, instead of checking in our own copy?

simulate/simulate_vr/simulate_vr.cc Outdated Show resolved Hide resolved
CMakeLists.txt Outdated Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants