4D reconstruction

In computer vision and computer graphics, 4D reconstruction is the process of capturing the shape and appearance of real objects along a temporal dimension.[1][2][3][4] This process can be accomplished by methods such as depth camera imaging,[1] photometric stereo, or structure from motion,[5] and is also referred to as spatio-temporal reconstruction.[4]

4D Gaussian splatting

This section is an excerpt from Gaussian splatting § 3D Temporal Gaussian splatting.[edit]

Extending 3D Gaussian splatting to dynamic scenes, 3D Temporal Gaussian splatting incorporates a time component, allowing for real-time rendering of dynamic scenes with high resolutions.[6] It represents and renders dynamic scenes by modeling complex motions while maintaining efficiency. The method uses a HexPlane to connect adjacent Gaussians, providing an accurate representation of position and shape deformations. By utilizing only a single set of canonical 3D Gaussians and predictive analytics, it models how they move over different timestamps.[7]

It is sometimes referred to as "4D Gaussian splatting"; however, this naming convention implies the use of 4D Gaussian primitives (parameterized by a 4×4 mean and a 4×4 covariance matrix). Most work in this area still employs 3D Gaussian primitives, applying temporal constraints as an extra parameter of optimization.

Achievements of this technique include real-time rendering on dynamic scenes with high resolutions, while maintaining quality. It showcases potential applications for future developments in film and other media, although there are current limitations regarding the length of motion captured.[7]

See also

References

  1. ^ a b Dou, Mingsong, et al. "Fusion4d: Real-time performance capture of challenging scenes." ACM Transactions on Graphics (TOG) 35.4 (2016): 1-13.
  2. ^ Mustafa, Armin, et al. "Temporally coherent 4d reconstruction of complex dynamic scenes." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016.
  3. ^ Oswald, Martin Ralf, Jan Stühmer, and Daniel Cremers. "Generalized connectivity constraints for spatio-temporal 3d reconstruction." European Conference on Computer Vision. Springer, Cham, 2014.
  4. ^ a b Dong, Jing, et al. "4D crop monitoring: Spatio-temporal reconstruction for agriculture." 2017 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2017.
  5. ^ Kyriakaki, Georgia, et al. "4D reconstruction of tangible cultural heritage objects from web-retrieved images." International Journal of Heritage in the Digital Era 3.2 (2014): 431-451.
  6. ^ Guanjun Wu; Taoran Yi; Jiemin Fang; Lingxi Xie; Xiaopeng Zhang; Wei Wei; Wenyu Liu; Qi Tian; Xinggang Wang (12 Oct 2023). "4D Gaussian Splatting for Real-Time Dynamic Scene Rendering". arXiv:2310.08528 [cs.CV].
  7. ^ a b Franzen, Carl (16 October 2023). "Actors' worst fears come true? New 3D Temporal Gaussian Splatting method captures human motion". venturebeat.com. VentureBeat. Retrieved October 18, 2023.
Wikimedia Commons has media related to 4D reconstruction.


Stub icon

This computer graphics–related article is a stub. You can help Wikipedia by expanding it.