Event Chronography in Multi-modal Data: The BME Method for Quantitative Analyses

Abstract

Methods for investigating multi-modality in human interactions remain open to refinement. Although the annotation process has been facilitated by tools like Elan, synchronising exported cross-tier data for further quantitative analyses remains challenging. We present the BME method: a new approach to data alignment. The idea is straightforward: instead of comparing exact times of onsets, durations, etc., the BME method focuses on their organisation. First, the method describes every annotation by at least two events: its beginning (B) and end (E). Then, it aligns them in chronological order. Middles (M) are precipitated to track events from other tiers which might occur between Bs and Es. We explore three cases in which such an arrangement of multi-modal data can benefit the scientific community: first, in getting insights about the dynamics and dependencies between tiers, second, in contemplating event-based duration rather than time-based ones, and, third, in contributing cross-annotator agreement assessment methods.