Alias-free 4D Gaussian Splatting

Zilong Chen¹ Huan-ang Gao² Delin Qu⁴
Haohan Chi² Hao Tang⁵ Kai Zhang^†1 Hao Zhao^†2,3

¹Tsinghua Shenzhen International Graduate School, Tsinghua University
²Institute for AI Industry Research (AIR), Tsinghua University
³Beijing Academy of Artificial Intelligence, BAAI ⁴Fudan University
⁵Peking University ^†Corresponding author

Paper arXiv Code

Abstract

Existing dynamic scene reconstruction methods based on Gaussian Splatting enable real-time rendering and generate realistic images. However, adjusting the camera's focal length or the distance between Gaussian primitives and the camera to modify rendering resolution often introduces strong artifacts, stemming from the frequency constraints of 4D Gaussians and Gaussian scale mismatch induced by the 2D dilated filter. To address this, we derive a maximum sampling frequency formulation for 4D Gaussian Splatting and introduce a 4D scale-adaptive filter and scale loss, which flexibly regulates the sampling frequency of 4D Gaussian Splatting. Our approach eliminates high-frequency artifacts under increased rendering frequencies while effectively reducing redundant Gaussians in multi-view video reconstruction. We validate the proposed method through monocular and multi-view video reconstruction experiments.

Motivation & Methods

Similar to 3DGS, 4DGS undergoes substantial dilation when the sampling rate is lowered and experiences erosion along with high-frequency artifacts when the sampling rate is increased.This occurs because changes in resolution alter the effective pixel size while the Gaussian scale remains fixed, causing a mismatch between the Gaussian scale and the filter dilation scale, compounded by the absence of a maximum sampling frequency constraint in 4DGS.

The fixed dilation scale used by the 3D smoothing filter proposed in Mip-Splatting can inadvertently render imperceptible Gaussians with extremely small scales visible in 4DGS, and may alter the scale ratios among Gaussian dimensions, affecting Gaussian anisotropy. We present Alias-free-4DGS, which leverages a 4D Scale-adaptive Filter and Scale Regularization to jointly constrain the maximum sampling frequency of 4DGS. This reduces the filter’s minimum dilation scale and avoids significant filtering errors when Gaussian scales are much smaller than the filter’s dilation scale. Moreover, our 4D Scale-adaptive Filter masks out imperceptible Gaussians and adapts the dilation scale using the ratio of Gaussian scales before and after changes in the current time frame, thus mitigating the filter’s impact on Gaussian anisotropy. Alias-free-4DGS effectively eliminates artifacts that arise from varying sampling frequencies without compromising reconstruction quality.

Results

Comparison on D-NeRF Dataset

We first compare the full-resolution reconstruction quality of D3DGS, Mip-Splatting, and our method on the monocular video reconstruction dataset D-NeRF. It can be observed that the ability of D3DGS to model local deformations decreases after incorporating Mip-Splatting. Next, we compare our method with D3DGS and Mip-Splatting under both zoom-in and zoom-out conditions to validate the anti-aliasing capability of our approach.

Full Resolution

Zoom-out

Training was performed at full resolution (800×800), and we subsequently present visualizations rendered at 1/4 resolution.

Zoom-in

Training was performed at 1/4 resolution, and we subsequently present visualizations rendered at full resolution (800×800)

Comparison on N3DV Dataset

We compare our method with Mip-Splatting and 4DGaussian in the zoom-out and zoom-in cases, respectively.

Zoom-out

Training was performed at full resolution (1352×1014), and we subsequently present visualizations rendered at 1/4 resolution.

Zoom-in

Training was performed at 1/4 resolution, and we subsequently present visualizations rendered at full resolution (1352×1014).