Structurally Disentangled Feature Fields Distillation for 3D Understanding and Editing

Recent work demonstrated the ability to leverage or distill pre-trained 2D features obtained using large pretrained 2D foundation models into 3D features, enabling impressive 3D editing and understanding capabilities with only 2D supervision. While powerful, such features contain significant view-dependent components, especially in scenes with complex materials and reflections. When distilled into a single 3D feature field, these inconsistencies are averaged, degrading feature quality and harming downstream tasks like segmentation. We hypothesize that explicitly modeling the physical causes of view-dependence is key to 'cleaning' these features during distillation. To this end, we propose to decompose the 3D feature field into view-independent and view-dependent components, guided by a physically-based reflection model. Our core contribution is demonstrating that this structural disentanglement improves the quality and view-invariance of the distilled semantic features. This leads to improved 3D seg-mentation, particularly in challenging reflective regions,and enables higher-fidelity physically-grounded editing applications. Our project page is available at https://structurallydisentangled.github.io/.