Audio- and Video-based Human Activity Recognition Systems in Healthcare

Stefania Cristina, Vladimir Despotovic, Rodrigo Perez-Rodriguez, Slavisa Aleksic*

*Corresponding author for this work

Research output: Contribution to journalArticleResearchpeer-review


In recent years, human activity recognition (HAR) has gained importance in several domains such as surveillance, recognizing indoor and outdoor activities, and providing active and assisted living environments in smart homes and healthcare services. In all these scenarios, audio-, video- and image-based processing algorithms have been applied as well as systems using wearable sensors. This scoping review focuses on audio- and video-based activity recognition systems for healthcare applications. We provide a comprehensive overview of these systems and technologies and discuss their complexity, performance, robustness, stage of development, scalability as well as achievable privacy and security levels. Additionally, we present and discuss datasets that are designed for the evaluation of these activity recognition systems. Although a number of robust approaches have already been proposed, they still pose challenges when it comes to integrating them in larger systems or into clinical practice. We identify challenges for application of audio- and video-based HAR systems in real-word healthcare scenarios, draw recommendations and conclusions based on comparisons of existing approaches, and analyze future trends.

Original languageEnglish
Pages (from-to)8230-8245
JournalIEEE Access
Publication statusPublished - 11 Jan 2024


  • Ambient and Assisted Living (AAL)
  • Assisted living
  • Feature extraction
  • Healthcare
  • Hidden Markov models
  • Human activity recognition
  • Human Activity Recognition (HAR)
  • Medical services
  • Older adults
  • Privacy


Dive into the research topics of 'Audio- and Video-based Human Activity Recognition Systems in Healthcare'. Together they form a unique fingerprint.

Cite this