Fix depth picking against splats #13118
Conversation
|
Thank you for the pull request, @ggetz! ✅ We can confirm we have a CLA on file for you. |
| const renderStateOptions = renderResources.renderStateOptions; | ||
| renderStateOptions.cull.enabled = false; | ||
| renderStateOptions.depthMask = false; | ||
| renderStateOptions.depthMask = true; |
There was a problem hiding this comment.
I wonder if this would cause any self-occlusion issues... like spats occluding other splats.
It shouldn't be super noticeable since they're sorted back to front, but I could imagine a portion of a previously drawn ellipse poking through the currently drawn ellipse.
Might be good to verify the rendering before and after this change.
There was a problem hiding this comment.
Lacking some context about what exactly the RenderState is or how depthMask is used or how it affects picking: There certainly are a few things where it's hard to talk about a "right" or "wrong" when it comes to details of rendering splats.
An example: Two spikey splats, poking through a "disc-ey" splat, with the magenta one being "in front of" the disc, and the yellow one being "behind" the disc. BabylonJS shows some "discontinuity" here as well, depending on the sort order....
I think that's "anticipated".
The PLY (Binary LE) and GLB+SPZ attached here...
|
I might be mistaken, but enabling depthMask here seems problematic for Gaussian splats. 20260107_100423.mp4 |
|
Approximating Gaussian splats as point-based depth contributors for picking (center depth only, in a separate pass) might be a more stable way to support pickPosition than writing splat depth in the main render pass. |
|
@BlockCnFuture Was this video taken on this branch? I wonder whether it would significantly different on the current main branch. At least, I've seen ~"similar artifacts" during camera rotations, because the sort order is updated asynchronously. |
I pulled the latest code from the main branch and then set depthMask to true. After enabling it, I noticed very obvious Gaussian flickering, even with very slight rotations. This issue disappears when it is turned off. |
|
Thanks for input @BlockCnFuture, @javagl, and @lilleyse! It doesn't sound like this isn't a viable fix in it's current state. I'm going to close this PR as the picking support is documented in #12712. |
Description
I'm retargeting an update from a previous PR, #12542, as it appears to still be relevant.
The update fixes depth picking (i.e.,
Scene.pickPosition) with splats. I could not find any regressions when switching thedepthMaskproperty of the render state totrue—but please let me know if I'm overlooking anything.Issue number and link
Fixes #12712
Testing plan
Reproduction example
Notice how in main the picked position does not pick the splat primitive, but rather falls back to picking the terrain.
Heads up that I'm seeing a unit test failure in this branch, but also in
main. It does not appear to be relevant to this change, but I've opened an issue to track, #13119.5) Check Spherical Harmonic specular on a Gaussian splats tileset Scene/GaussianSplatPrimitiveAuthor checklist
CONTRIBUTORS.mdCHANGES.mdwith a short summary of my changeI have updated the inline documentation, and included code examples where relevant