journal articles

:: 2010

• What's in the eyes for context-awareness?
  Andreas Bulling, Daniel Roggen and Gerhard Tröster
  IEEE Pervasive Computing, in press.
  [Abstract BibTeX RIS DOI]
  
  

  Research in pervasive computing has investigated a variety of modalities for developing context-aware systems. A rich source of information on context that has not yet been used is the movements of the eyes. Because we use our eyes in almost everything that we do - everywhere - eye movements are a promising modality that ought to be exploited for context-awareness. Moreover, the link between eye movements and cognition may allow us to develop pervasive computing systems that are able to derive the so-called cognitive context of a person. In this work, we first report on the state-of-the-art in eye-based activity recognition (EAR) and outline the potential but also the research challenges of inferring activity and the cognitive context from eye movements. We present results of two EAR studies and first results towards our vision of cognitive-awareness.
• Eye Movement Analysis for Activity Recognition Using Electrooculography
  Andreas Bulling, Jamie A. Ward, Hans Gellersen and Gerhard Tröster
  IEEE Transactions on Pattern Analysis and Machine Intelligence, in press.
  [Abstract BibTeX RIS DOI PDF]
  
  

  In this work we investigate eye movement analysis as a new sensing modality for activity recognition. Eye movement data was recorded using an electrooculography (EOG) system. We first describe and evaluate algorithms for detecting three eye movement characteristics from EOG signals - saccades, fixations, and blinks - and propose a method for assessing repetitive patterns of eye movements. We then devise 90 different features based on these characteristics and select a subset of them using minimum redundancy maximum relevance feature selection (mRMR). We validate the method using an eight participant study in an office environment using an example set of five activity classes: copying a text, reading a printed paper, taking hand-written notes, watching a video, and browsing the web. We also include periods with no specific activity (the NULL class). Using a support vector machine (SVM) classifier and a person-independent (leave-one-out) training scheme, we obtain an average precision of 76.1% and recall of 70.5% over all classes and participants. The work demonstrates the promise of eye-based activity recognition (EAR) and opens up discussion on the wider applicability of EAR to other activities that are difficult, or even impossible, to detect using common sensing modalities.

:: 2009

• Wearable EOG goggles: Seamless sensing and context-awareness in everyday environments
  Andreas Bulling, Daniel Roggen and Gerhard Tröster
  Journal of Ambient Intelligence and Smart Environments, 1(2):157-171, 2009.
  [Abstract BibTeX RIS DOI PDF]
  
  

  In this article we introduce the analysis of eye motion as a new input modality for activity recognition, context-awareness and mobile HCI applications. We describe a novel embedded eye tracker that, in contrast to common systems using video cameras, relies on Electrooculography (EOG). This self-contained wearable device consists of goggles with dry electrodes integrated into the frame and a small pocket-worn component with a DSP for real-time EOG signal processing. It can store data locally for long-term recordings or stream processed EOG signals to a remote device over Bluetooth. We show how challenges associated with wearability, eye motion analysis and signal artefacts caused by physical activity can be addressed with a combination of a special mechanical design, optimised algorithms for eye movement detection and adaptive signal processing. In two case studies, we demonstrate that EOG is a suitable measurement technique for the recognition of reading activity and eye-based human-computer interaction. Eventually, wearable EOG goggles may pave the way for seamless eye movement analysis and new forms of context-awareness not possible today.

www.andreas-bulling.de | © 2006-2010 Andreas Bulling
last update: 04 May 2010