The Augmented Human Assistant is an ambitious scientific and technological project aimed at alleviating the negative effects of a sedentary lifestyle, particularly for the elderly population.
It mixes expertise in such diverse areas such as Human Functioning and Performance, Augmented Reality (AR) technologies, serious games for health, physiological signal acquisition systems, computer vision systems, robot navigation, and intelligent scene assessment.
The results of my work at Vislab on the project are illustrated in:
- My master thesis, titled: Monitoring Human State in a Robotic Assistive Platform: Data Acquisition and Person Detection Systems.
- A research paper, titled: A Dataset for the Automatic Assessment of Functional Senior Fitness Tests using Kinect and Physiological Sensors, presented at the International Conference on Technology and Innovation in Sports, Health and Wellbeing in Vila Real, Portugal.
System Architecture Schema
I conceptually divided the system in three groups:
First, the sensors, including biosensors connected via bluetooth, a Kinect camera and two omnidirectional cameras using the RTSP protocol.
Then the acquisition and storing system, which reads signals from the sensors using the relative SDKs and then transmits the information through YARP ports. The choice of using YARP was made to follow a standard way of transmitting signals (at that time YARP was quite popular in academical robotics) and allow to add further modules to the system with minimal efforts. The second stage of the system is composed of the data readers, which read the signal form the YARP ports, parse it and store it in a MongoDB database. The programming language used to develop the acquisition and storing system was C++.
Finally, the visualisation and processing Matlab scripts, which access the MongoDB dabase via TCP protocol and allow to perform a series of operations with the stored data, like unwrapping omnidirectional images and running a pedestrian detector.
The project also got good press coverage due to its social impact and innovative aspects. The following is a report made by SIC Noticias, a Portuguese tv channel.