MARIE

Multimodal Activity Recognition for Interactive Environments

Go <-research home Go <-home


The aim of the MARIE project is to develop new knowledge and technology for recognising human activity.
The main scientific contributions of the project are:
1) the introduction and development of eye movement patterns as a new modality for activity recognition; and
2) the development of a new methodology for evaluating activity recognition performance.


The project has produced a number of significant results over the past two years:
  1. We showed for the first time that a variety of human activities (we used five: writing, reading, copying text on a screen, watching a film, browsing the web) could be spotted and classified using features based solely on eye movement patterns [BWGT10, BWGT09].
  2. We showed that reading activities can be detected during a variety of real-world situations (while sitting, standing, walking - both outdoors and indoors - as well as while riding a tram), using wearable eye movement recording equipment [BWGT08].
  3. We developed and explored a range of different features that can be used for classifying eye movement data [BWGT10,BWGT09,BWGT08].
  4. In an ongoing work (due to be published in autumn 2010), we showed how multi-sensor fusion can be used to improve the robustness of eye-based activity recognition. This work combines sensor data from both eye and head movements.
  5. We helped to establish a consensus among the world's leading researchers in activity recognition about how best to conduct further research in the field [LIW10].
  6. We demonstrated, using a wide selection of papers in activity recognition, the need for a better evaluation methodology. We introduced a new system of event-based evaluation metrics and showed how these offer an improvement over commonly used metrics [WLG10].

Relevant publications


  • Performance metrics for activity recognition
    J.A. Ward, P. Lukowicz and H. Gellersen
    ACM Transactions on Information Systems and Technology (TIST) (available from Sept. 2010)


  • Workshop on Performance analysis in activity recognition research: Experimental methodology, performance evaluation and reproducibility
    P. Lukowicz, S. Intille, J.A. Ward
    in conjunction with Pervasive 2010 in Helsinki, Finland http://eis.comp.lancs.ac.uk/workshops/activity2010



    • The original project abstract can be found here.