Confused about deformation field

[ricshaw]

Hi, thanks for the great work!
I'm trying to get a textured mesh from my nerfie capture but I'm confused about how the deformation field works.
I can get a canonical mesh (no warp) by feeding in a voxel grid of points, predicting alphas, and doing marching cubes to get a mesh of vertices and triangles. Now I want to warp this mesh using the deformation field into each of my training images. But when I feed in these mesh vertices as nerf points, the resulting warped points (out['warped_points']) don't align with my training images when I project them into each camera using camera.project(). However, if I feed in a voxel grid and do marching cubes over the warped alphas for each image, then the resulting vertices line up perfectly with my training images.
I don't understand why it works with the voxel grid, but doesn't work with the mesh vertices. It seems like deforming the points is not the same as getting the deformed alpha and doing marching cubes on that.