DEVELOPMENT OF FRAMEWORK FOR 2D TO 3D RECONSTRUCTION AND VOLUMETRIC SEGMENTATION

Abstract

There The ever-growing requirements for swift content generation in gaming, e commerce, and VR applications prompt the necessity of fast reconstruction from single view. In this thesis, we present an enhanced pipeline for generating textured 3D models from one image through the integration of deep image segmentation with feed-forward volumetric inference. For background removal, we propose a U²‑Net-based segmentation module that outputs a normalized foreground mask which then goes through TripoSR - a vision transformer encoder paired with a lightweight triplane architecture that predicts an occupancy map. We obtain a watertight mesh using marching cubes. The quality of our reconstruction, measured on a synthetic spherical object, is confirmed through the following metrics: CD of 1.857 × 10⁻³, volumetric IoU of 0.7786, and HD of 0.0849, providing excellent shape recovery performance. The presented method significantly outperforms TripoSR without image segmentation (CD = 10.2 × 10⁻³) and the OccNet [12] model (CD = 12.4 × 10⁻³), achieving a geometric error reduction by almost five times. The whole procedure takes less than two seconds when run on a consumer-grade GPU. Our ablation studies show that the introduction of a separate 2D segmentation stage is crucial for obtaining high-fidelity results.