Humantech Institute


Mind Augmented Gesture Interaction

MAGI is a research project, part of a collaboration between the MISG group at EIA-FR and the DIVA group at the UNIFR. Gestures are a natural way of interaction. They can be used as a substitute or a complement to other forms of communication. Two key components in gesture recognition are « detection » and « gesture segmentation ». « Detection » refers to the identification of the actor who performs the gestures and the elements involved (hands, head, etc.). « Gesture segmentation » refers to the task of distinguishing a gesture from another and also from the ones not addressed to the machine (involuntary movements or part of the normal gesticulation). In this context, the main idea of the project MAGI is to analyze the physiological sensors’ capabilities – in particular related to the field of Brain Computer Interface (BCI) – in order to enhance the gesture recognition. Specifically, the project has two objectives: on one hand, to enhance the gesture detection by combining the results from conventional sensors (cameras, accelerometers, etc.) with those from physiological sensors (electroencephalogram, electromyogram, etc.); on the other hand, to provide a new approach to solve the problem of gesture segmentation (gestures will be distinguished from each other based on data produced by the user’s brain).
Moreover, a multimodal approach that aims at navigating in a Virtual Word by the use of the only Head (movements, expressions and thoughts) is under analysis and a prototype is in development.

Phantom Limb Unrelenting Pain: Augmented Reality Treatment

The PLUPART project aims to improve the classical « Mirrorbox » treatment using new technologies and in particular: augmented reality, real-time 3D tracking and 3D modeling. The Mirror box technique is used as treatment for the Phantom Limb Pain. The idea is to « trick the mind » using a box with a mirror in one side and a hole to hide the missing limb: in result, the patient will fill as she/he still has the limb. In this project the mirror box will be reproduced in augmented reality aiming to improving the immersion and, therefore, the therapeutic effect. Finally, exercises based on motor imagery will complete the therapy. The system will record patients’ progress for a series of evaluations conducted by qualified personnel. In order to make possible the use of the system in a domestic environment, the technological choice will be focused on low-cost devices.

Click here to visit the Website !

Combining Semantic Technologies and Interactive Visualization Techniques for Personal Information Management

Click here to visit the Website !

Au jour le jour

This project is part of the Swiss implementation of the “AEMMA program”. This program aims at optimizing the management of stress for close-relatives that take care of people living at home with dementia. The feasibility study involves two groups of close-relatives in Geneva and Fribourg, note that most of them are elders. From a technical point of view, our task was to develop a mobile application to simplify the daily data collection. The application allows the user to accomplish daily data logging tasks exploiting multi-touch and voice interfaces that were explicitly designed for elderly people. Furthermore, the application allows data-sharing and remote monitoring of the participants’ engagement within the system. Partners: HES-SO, HEdS

Click here to visit the Website !

Immersive Mirror for Pain Control and Treatment

The main goal of the IMPACT project is to study, design and implement a system aimed at reducing pain of patients suffering from phantom limb pain after an amputation. The proposed approach is based on the « mirror box » therapy but improved using augmented reality. The system uses 3D modelling and 3D tracking, exploiting new and low cost technologies aiming to provide a tool suitable for everyday domestic therapy. The system is completed by a therapy environment based on motor imagery. IMPACT is the continuation of the PLUPART project. Partners: HES-SO, CHUV – Centre hospitalier universitaire Vaudois.

Click here to visit the Website !


This project aims at testing the feasibility and efficacy of two different intensities of an intervention to enhance resilience and to reduce supportive care needs in cancer patients. Patient engagement in healthcare via self-management is recognized as crucial to improved outcomes for cancer patients. In psychology, resilience refers to a dynamic process encompassing positive adaptation within the context of significant adversity. Individual resilience can be increased by training of professionals and patients living with chronic diseases and thereby positively impact clinically meaningful outcomes. In this context, new technologies can help to simplify the task of nurses and doctors of following the patients’ evolution during the treatment. Within the project, we built a platform to help nurses to collect the patient data about the evolution of resilience and needs during chemotherapy sessions using tablets. Partners: HES-SO, HEdS, Fribourg Hospital

Click here to visit the Website !

Symptom Navi
Mobile application for the support of cancer patients

In this project, we developed a mobile application for the self-management of symptoms (disease and treatment) of cancer patients. The growth of the life expectancy in developed countries is causing an increasing need of medical attention for growing part of the population. For this reason, in the next years, self-management and home care will play a primary role in terms of psychological support for the patient. The developed mobile application is capable of suggesting, according to the state of the patient, the best strategy to follow in order to self-manage 15 delicate psycho-physiological conditions (such as anxiety, fatigue, appetite, etc.). Partners: HES-SO, HEdS

Click here to visit the Website !

a tablet App to keep engaged elderly with cognitive games and social activities

The KeepInTouch is an ongoing project that aims at developing a tablet application that allow elderly people to stay in touch with relatives and friends and to train their cognitive abilities through various games. The system is developed as a contact agenda, which can be easily navigated by flipping the pages of the agenda. Each contact page allows to start a video communication whenever the other user is connected through a similar app. In each contact page, it is also possible to find information about the person, such as her anniversary, the chronology of calls and a photo album, which contains photo uploaded by this person or in which there is this person. Therefore, the application encourages the older adult to stay in touch with her relative and to remember people stories and events through photos. The cognitive games, i.e., memory games and puzzles, exploit photo from albums to keep the user engaged in the training and to foster at the same time long-term memories related to the photos. A web application allows relatives to manage the contacts and the photos of the older adults.

Internet of Things for Urban Innovation

In 2050, over 86 % of the world population will live in cities. This growth poses major challenges to the individual and public mobility, physical safety of people and infrastructure, resource and waste management, the urban development and quality of life of its citizens. The objective of this research program is to address these challenges, using the latest Information and communications technologies. In the initial phase, the research program will consider the major events (e.g., festivals, political summits, sporting events, etc.) as a « concentrated » urbanity. In particular, optimizing mobility, the physical security, the resources and waste management and the urban development are issues that arise during major events on a smaller scale and, therefore, more easily observable and measurable. Subsequent phases of the program will transpose the achieved results at larger scales. The main purpose of this 4 years research program is to establish the « backbone » of this new technological urban space, based on the emerging paradigm of Internet of Things (IoT). It also aims to advance the state of the art of these challenging technologies. A main challenge of this project is to consider the issues of security, trust and protection of privacy. It requires finding the trade-off between the need to provide useful information to the policy makers and the citizens themselves while minimizing the intrusion into the private lives of individuals. A second requirement is also the prevention of the data abuse or misuse by deliberate attacks or excessive zeal of the involved actors.

Click here to visit the Website !


The SmartCrowd projects aims to show the possibility provided for the crowd management using an approach going in the direction of IoT. In particular, the project focuses on the detection of dangerous situations using sensors on the smartphone, for the real-time monitoring, and a crowd simulator to detect obstacles or architectural barriers that may be present in the event location.

Automatic Augmented Reality Occlusion Detection System

This project goal is to study the feasibility of an augmented reality approach to deal with the problem of occlusion in the domain of dental prosthesis. The occlusion problem happens when a real object passes in front of a virtual object. Currently most of the application just neglect the problem or try to avoid it. In this project, we try to deal with the occlusion problem tracking the lips and the jaw of the patient with the goal of allowing the patient to see the effects of a dental prosthesis before embarking on a costly realization.