MonoPhysics: Estimating Geometry, Appearance, and Physical Parameters from Monocular Videos
MonoPhysics: 从单目视频估计几何、外观和物理参数
Daniel Rho, Jun Myeong Choi, Matthew Thornton, Biswadip Dey, Roni Sengupta
AI总结 提出MonoPhysics框架,通过可微分MPM模拟和3D高斯泼溅,从单目视频联合优化可变形物体的几何、外观和物理参数,解决尺度模糊和几何不准确问题。
详情
现有的逆物理方法从多视角视频中恢复物理参数,其中跨视角的几何约束解决了尺度和3D结构问题。然而,在单目设置中,这种约束缺失,导致严重的尺度模糊、不准确的几何以及外观优化与物理模拟之间的弱耦合。我们提出MonoPhysics,一个用于可变形物体的单目逆物理估计框架,使用可微分MPM模拟和3D高斯泼溅,从单个相机视角联合优化几何、外观和物理参数。我们通过三个视觉-物理桥梁解决这些挑战:全局尺度对齐、物理感知的几何细化以及可微分位置图,这些共同使得仅从单目观测就能进行准确优化。我们在Vid2Sim和我们新的弹性和塑性物体数据集上评估,结果表明MonoPhysics在单目设置中优于现有基线,并且仅使用单个相机就能达到与多视角基线相当的性能。我们的项目页面可在https://daniel03c1.github.io/MonoPhysics/获取。
Existing inverse physics methods recover physical parameters from multi-view videos, where geometric constraints across views resolve scale and 3D structure. In monocular settings, however, such constraints are absent, leading to severe scale ambiguity, inaccurate geometry, and weak coupling between appearance optimization and physical simulation. We propose MonoPhysics, a framework for monocular inverse physics estimation of deformable objects using differentiable MPM simulation and 3D Gaussian Splatting, which jointly optimizes geometry, appearance, and physical parameters from a single camera view. We address these challenges through three visual-physical bridges: global scale alignment, physics-aware geometry refinement, and a differentiable position map, which together enable accurate optimization from monocular observations alone. We evaluate on Vid2Sim and our new dataset of elastic and plastic objects, showing that MonoPhysics outperforms existing baselines in monocular settings and achieves performance comparable to multi-view baselines using only a single camera. Our project page is available at https://daniel03c1.github.io/MonoPhysics/