LIGHTPOSE-LEARNABLEPOINTAND PERSPECTIVE PROBLEM: AN END-TO-END DIFFERENTIABLE 6D POSE ESTIMATION FRAMEWORKUSING EFFICIENT FUSION ANDSELF-CONSISTENCY LEARNABLE PnP LAYERS

Abstract

Estimating the six-degree-of-freedom (6DoF) pose of rigid objects is a foundational challenge in computer vision and robotics, critical for applications like autonomous manipulation and augmented reality. While deep keypoint-based methods have ad vanced robustness against occlusion, standard pipelines are limited by a non-differenti able geometric solver bottleneck at train time, which severs the gradient path between pose loss and feature learning. Conversely, frameworks that bridge this gap typically rely on heavyweight backbones that far exceed the tight memory and compute en velopes of edge hardware. This thesis presents LightPose-LPP, an end-to-end differentiable 6DoF pose es timation framework designed from the ground up for resource-constrained deploy ment. The architecture introduces LightweightPoseNet, which leverages a truncated MobileNetV3-Small RGB backbone and a custom depthwise-separable depth encoder to construct an expressively rich dual-branch feature extractor utilizing just 1.3 million parameters. To align cross-modal traits efficiently, a novel multi-scale Additive At tention Fusion module is proposed. By substituting traditional cross-attention with a global query contraction, the module drops spatial complexity from quadratic O(N2·d) to linear O(N ·d), allowing dense multi-scale RGB-D alignment on a strict memory budget. Features are aggregated via a Feature Pyramid Network into a dense vector field that votes for canonical object keypoints via a fully differentiable, closed-form Cramer’s rule ray intersection. Poses are ultimately recovered using a differentiable Levenberg-Marquardt-style solver built on a continuous 6D rotation parameterization. The entire pipeline is optimized end-to-end via an energy-based EPro-PnP Kullback Leibler (KL) divergence loss, stabilized by a detached log-partition gradient path and a v ten-epoch warmupschedule. Evaluated ontheLineMODbenchmarkusinganNVIDIA GTX 1650 GPU, the framework fits within a 4 GB VRAM budget. Ablation studies demonstrate that end-to-end pose supervision yields a +16.34 percentage-point im provement in ADD-S accuracy over keypoint-only baselines, achieving a headline accuracy of 82.7% on the symmetric eggbox and 90.1% on the iron. These results validate that high-accuracy, multi-modal 6DoF pose estimation can be successfully reconciled with edge-scale computational efficiency.