Nonlinear spatio-temporal model based on the geometry of the visual input

E. Barth and A.B. Watson

NASA Ames Research Center, Moffet Field, CA.
Purpose: We propose a nonlinear model of spatio-temporal processing which leads to an hierarchical visual representation and can account for some basic psychophysical and neurophysiological results. Methods: The model is inspired by the differential geometry of the spatio-temporal hypersurface (STHS) corresponding to image intensity . The Riemann tensor of the STHS involves computations which can be expressed as non-linear combinations of spatio-temporal filters. Results: The visual information load is sequentially reduced by extracting intrinsic 2D- and 3D features derived from the Riemannian and Gaussian curvatures respectively. The Riemann tensor of the STHS is shown to incorporate the computation of the optical flow under the constant-gradient assumption, as well as the spatial properties of orientation and end-stopping. Our model further involves the computation of two-dimensional space-time curvatures which are implemented as "and"-type combinations of sustained and transient units. Finally, we suggest top-down strategies for selecting the spatio-temporal locations which contribute to the global motion percept such as to avoid erroneous local motion estimates like those due to the aperture problem. Conclusions: We argue that important properties of the visual system may be understood as resulting from basic geometric processing of the spatio-temporal visual input, as opposed to through terms such as "orientation selectivity", "end-stopping", and "motion".
Supported by DFG grant Ba 1176/4-1 to EB
and NASA grant 199-06-12-39 to ABW.


See also the Riemann-tensor demos.