Estimation of motion vectors and their application to scene retrieval

This paper describes a method for retrieving scenes from a motion picture database by using motion information as a key. The method has three steps: the automatic estimation of motion vectors in frame sequences, the description of motions in spatio-temporal space, and the retrieval of sequences of images. The motion vectors are estimated by block matching. Estimated motion vectors are mapped in spatio-temporal space (x, y, t). Motion vectors in a scene are aggregated into several representative vectors by statistical analysis. The retrieval of scenes is divided into two parts: the specification of query conditions, and matching between the query conditions and the motion database. Similarity is defined in spatio-temporal space as the distance between the query conditions and the stored motion index. Candidate scenes are ranked in order of distance. Experimental results indicate that the proposed method is effective. © 1994 Springer-Verlag.