journal articles

Reading is one of the most well studied visual activities. Vision research traditionally focuses on understanding the perceptual and cognitive processes involved in reading. In this work we recognise reading activity by jointly analysing eye and head movements of people in an everyday environment. Eye movements are recorded using an electrooculography (EOG) system; body movements using body-worn inertial measurement units. We compare two approaches for continuous recognition of reading: String matching (STR) that explicitly models the characteristic horizontal saccades during reading, and a support vector machine (SVM) that relies on 90 eye movement features extracted from the eye movement data. We evaluate both methods in a study performed with eight participants reading while sitting at a desk, standing, walking indoors and outdoors, and riding a tram. We introduce a method to segment reading activity by exploiting the sensorimotor coordination of eye and head movements during reading. Using person-independent training, we obtain an average precision for recognising reading of 88.9% (recall 72.3%) using STR and of 87.7% (recall 87.9%) using SVM over all participants. We show that the proposed segmentation scheme improves the performance of recognising reading events by more than 24%. Our work demonstrates that the joint analysis of multiple modalities is beneficial for reading recognition and opens up discussion on the wider applicability of this recognition approach to other visual and physical activities.

...

Eye movements are a rich source of information about a person's context. Analyzing the link between eye movements and cognition might even allow us to develop cognition-aware pervasive computing systems that assess a person's cognitive context.

In this work we investigate eye movement analysis as a new sensing modality for activity recognition. Eye movement data was recorded using an electrooculography (EOG) system. We first describe and evaluate algorithms for detecting three eye movement characteristics from EOG signals - saccades, fixations, and blinks - and propose a method for assessing repetitive patterns of eye movements. We then devise 90 different features based on these characteristics and select a subset of them using minimum redundancy maximum relevance feature selection (mRMR). We validate the method using an eight participant study in an office environment using an example set of five activity classes: copying a text, reading a printed paper, taking hand-written notes, watching a video, and browsing the web. We also include periods with no specific activity (the NULL class). Using a support vector machine (SVM) classifier and a person-independent (leave-one-out) training scheme, we obtain an average precision of 76.1% and recall of 70.5% over all classes and participants. The work demonstrates the promise of eye-based activity recognition (EAR) and opens up discussion on the wider applicability of EAR to other activities that are difficult, or even impossible, to detect using common sensing modalities.

Research in eye-based human-computer interaction (HCI) has matured over the past 20 years with current HCI research mostly focusing on stationary eye trackers in laboratory settings. This survey of latest advances in eye tracking equipment and automated eye movement analysis suggests a new generation of mobile eye-based interfaces that will become pervasive and seamlessly integrated into people's everyday lives.

In this article we introduce the analysis of eye motion as a new input modality for activity recognition, context-awareness and mobile HCI applications. We describe a novel embedded eye tracker that, in contrast to common systems using video cameras, relies on Electrooculography (EOG). This self-contained wearable device consists of goggles with dry electrodes integrated into the frame and a small pocket-worn component with a DSP for real-time EOG signal processing. It can store data locally for long-term recordings or stream processed EOG signals to a remote device over Bluetooth. We show how challenges associated with wearability, eye motion analysis and signal artefacts caused by physical activity can be addressed with a combination of a special mechanical design, optimised algorithms for eye movement detection and adaptive signal processing. In two case studies, we demonstrate that EOG is a suitable measurement technique for the recognition of reading activity and eye-based human-computer interaction. Eventually, wearable EOG goggles may pave the way for seamless eye movement analysis and new forms of context-awareness not possible today.