We present a method for reconstructing 3D shape of arbitrary Lambertian objects based on measurements by miniature, energy-efficient, low-cost single-photon cameras. These cameras, operating as time resolved image sensors, illuminate the scene with a very fast pulse of diffuse light and record the shape of that pulse as it returns back from the scene at a high temporal resolution. We propose to model this image formation process, account for its non-idealities, and adapt neural rendering to reconstruct 3D geometry from a set of spatially distributed sensors with known poses. We show that our approach can successfully recover complex 3D shapes from simulated data. We further demonstrate 3D object reconstruction from real-world captures, utilizing measurements from a commodity proximity sensor. Our work draws a connection between image based modeling and active range scanning, and offers a step towards 3D vision with single-photon cameras.

@InProceedings{MSJLHGGL24,
  author       = "Mu, Fangzhou and Sifferman, Carter and Jungerman, Sacha and Li, Yiquan and Han, Mark and Gleicher, Michael and Gupta, Mohit and Li, Yin",
  title        = "Towards 3D Vision with Low-Cost Single Photon Cameras",
  booktitle    = "Computer Vision and Pattern Recognition Conference (CVPR)",
  pages        = "5302--5311",
  year         = "2024",
  doi          = "https://doi.org/10.1109/CVPR52733.2024.00507",
  url          = "http://graphics.cs.wisc.edu/Papers/2024/MSJLHGGL24"
}