GADAGeometry-Aware Deformable Aggregation for Image-Based Gaussian Splatting

Gaussian Splatting has achieved significant improvements by incorporating warping-based techniques. These approaches enhance synthesis quality by warping images from source views into the target viewpoint to compensate for missing or residual pixels. However, such methods suffer from pixel-level inaccuracies due to uncertain geometry. This uncertainty leads to spatial misalignments in the warped images, which disrupt residual learning used in warping-based methods and fundamentally limit the gains of correction, particularly on thin structures and high-frequency details. Driven by our insight that useful visual cues are not lost but locally preserved under slight displacement, we propose Geometry-Aware Deformable Aggregation (GADA). This method introduces an iterative refinement module with deformable offsets to actively correct spatial misalignments and recover these displaced cues. Furthermore, to address the limitations of standard pipelines where visibility checks (i.e., thresholding) often discard valid pixels and multi-view warped image fusion relies on naive mean aggregation, our module is coupled with an implicit confidence weighting mechanism that selectively suppresses unreliable evidence. Consequently, our approach outperforms prior warping-based Gaussian Splatting, preserving high-frequency quality while achieving 2.13× faster FPS.

We reinterpret the pixels that fail visibility checks not as discardable noise, but as displaced cues — valid visual data mapped to an incorrect location due to uncertain geometry. Building on this premise, GADA reformulates warped-image processing from a passive thresholding step into a task of local search and active correction. The pipeline proceeds in three stages:

• Geometric Context Embedding. A shared encoder MLP fuses the photometric residual between the warped and base image with the relative camera pose, producing an initial latent state that guides the subsequent search.
• Geometry-Aware Deformable Offsets. A lightweight offset predictor iteratively regresses bounded displacements that actively resample the source image, recovering displaced evidence from the local neighborhood across five refinement stages.
• Geometry-Verified View Aggregation. A confidence network predicts per-view weights from the post-correction features and fuses multi-view evidence through a weighted sum, down-weighting views that remain unreliable after correction.
• Residual decoding. A CNN decoder maps the aggregated feature into the final pixel residual, which is added to the base 3DGS color to produce the rendered image.

By replacing heuristic visibility thresholds with learnable offsets and confidence weighting, GADA boosts valid-pixel density from 33.01% to 79.33% in the warped images while delivering 2.13× faster rendering than the IBGS baseline (47 vs. 22 FPS on Mip-NeRF 360).