On Parsing Visual Sequences with the Hidden Markov Model

Abstract

Hidden Markov Models have been employed in many vision applications to model and identify events of interest. Their useis common in applications where HMMs are used to classify previously divided segments of video as one of a set of eventsbeing modelled. HMMs can also simultaneously segment and classify events within a continuous video, without the need fora separate first step to identify the start and end of the events. This is significantly less common. This paper is an exploration of thedevelopment of HMM frameworks for such complete event recognition. A review of how HMMs have been applied to both eventclassification and recognition is presented. The discussion evolves in parallel with an example of a real application in psychology forillustration. The complete videos depict sessions where candidates perform a number of different exercises under the instructionof a psychologist. The goal is to isolate portions of video containing just one of these exercises. The exercise involves rotating thehead of a kneeling subject to the left, back to centre, to the right, to the centre, and repeating a number of times. By designing aHMM system to automatically isolate portions of video containing this exercise, issues such as the strategy of choice of event tobe modelled, feature design and selection, as well as training and testing are reviewed. Thus this paper shows how HMMs can bemore extensively applied in the domain of event recognition in video.