Unbiased Gradient Estimation for Differentiable Surface Splatting via Poisson Sampling

We propose an efficient and GPU-accelerated sampling framework which enables unbiased gradient approximation for differentiable point cloud rendering based on surface splatting. Our framework models the contribution of a point to the rendered image as a probability distribution. We derive an unbiased approximative gradient for the rendering function within this model. To efficiently evaluate the proposed sample estimate, we introduce a tree-based data-structure which employs multipole methods to draw samples in near linear time. Our gradient estimator allows us to avoid regularization required by previous methods, leading to a more faithful shape recovery from images. Furthermore, we validate that these improvements are applicable to real-world applications by refining the camera poses and point cloud obtained from a real-time SLAM system. Finally, employing our framework in a neural rendering setting optimizes both the point cloud and network parameters, highlighting the framework’s ability to enhance data driven approaches.

@InProceedings { MWK22,
  author       = "M{\"u}ller, Jan U. and Weinmann, Michael and Klein, Reinhard",
  title        = "Unbiased Gradient Estimation for Differentiable Surface Splatting via Poisson Sampling",
  booktitle    = "Computer Vision -- ECCV 2022",
  pages        = "281--299",
  year         = "2022",
  publisher    = "Springer Nature Switzerland",
  key          = "mueller-2022-unbiased",
  keywords     = "differentiable rendering, point cloud, multipole method, shape recovery, scene reconstruction",
  url          = "http://graphics.tudelft.nl/Publications-new/2022/MWK22"
}