To design a bi-directional user interface with Augmented Reality serious games, wearable sensors and instrumented objects for maximizing the impact of physical interaction in learning contexts (WP5). Description: This objective focuses on the design of novel bi-directional user interface to convey information to the user, through AR-based serious games, and from the user, collecting data from wearable sensors and instrumented objects. The application-driven serious games are specifically designed to exploit the capability of the CONBOTS platform to establish a physical communication between two users. Information from the users are captured by an easy-to-use, non-invasive wearable garment that embeds different kinds of physiological and motion sensors to: i) collect physiological parameters and estimate user’s mental and emotional state for adapting the engagement and the difficulty of the performed task; ii) track movements of the upper limbs for measuring user’s performance and control physical communication during human-human interaction. Moreover, a set of instrumented objects are designed to record how users interact with the environment during the manipulation of the tools necessary for the execution of tasks in the different application scenarios (e.g. instrumented pen or bow).
Source: CONBOTS Project Information