Integration and segmentation of non-rigid motion: A computational model

Purpose. Analyzing motion requires integrating some constraints while segmenting others; the visual system must determine which to segment and which to combine. At last year's ARVO (Weiss and Adelson, 1995) we showed a set of phenomena illuminating how the visual system solves this problem. We have now developed a computational model consistent with these results. Method We focus on three basic phenomena: (1) a rigidly rotating ellipse appears rigid if narrow but nonrigid if fat; (2) isolated features near the ellipse contour can strongly influence whether the ellipse appears rigid or non-rigid, depending on how these features move; (3) extrinsic (spurious) features do not influence the perceived motion. We calculated the prediction of three existing motion integration models: (a) Lucas and Kanade, (1981), which does least squares matching within patches; (b) Hildreth (1982), which does integration with smoothing along a contour; (c) Grzywacz and Yuille (1992), which integrates and smooths over space. We also calculated the prediction of a new algorithm (d) which integrates across space but favors measurements along the contour. The algorithm combines motion and form information in a robust estimation framework. Results Each of the three existing models can predict some of the qualitative phenomena observed in human vision; however each one fails badly on at least one of the phenomena. The new model correctly predicts all of the phenomena. Significance The new model incorporates an essential interaction between form and motion analysis and captures some properties of human phenomenology. It is also possible to implement such a model in a biologically plausible way through the use of form modulated interactions between populations of velocity tuned units.