Human, Interaction, Decision
Digital Systems
Human, Interaction, DEcision (HIDE) research theme
The Human, Interaction, DEcision (HIDE) theme includes 17 permanent and some twenty non-permanent researchers (including PhD students, postdoctoral students, engineers recruited for projects and partnerships).
The HIDE theme focuses on the entire processing chain, from capturing information about humans or a complex system used by humans, to decision-making involving various areas of scientific research (signals, AI algorithms, information theory) and applies all of this to various applications, as shown in the figure below.
This figure has been designed to illustrate the synergy of the different areas of research and expertise within the HIDE thematic group. It represents the infinite cycle between the augmented environment and the augmented human, in which we have broken our areas of expertise down into three parts: data control, complex data representation and decision-making.
The researchers aim to address the need for a better knowledge of humans by combining research expertise from different fields (data mining, decision analysis, signal/image processing). The common point uniting these areas of research is the work on human-related data, with advanced techniques such as the following:
- Information theory, in order to control the capture of data transmitted by the environment (e.g. sensors, cameras, manual input), to optimise data exchanges between humans and the system, and to secure the format of the data processed.
- Advanced data processing, with predictive and/or evolutionary models, to better control input data and their imperfections.
- Work on statistical learning and deep learning to model human behaviour with the aim of predicting it or verifying its “normality” in order to facilitate interactions between humans and augmented systems.
This is all then applied to different themes, whether it be the analysis of human behaviour for medical or well-being purposes, for data security (e.g. authenticity), the digital twin, or virtual or learning environments.
Research themes
To meet these challenges, HIDE focuses on the analysis of human behaviour, on the one hand, and on the quality and reliability of data passing through these systems, on the other.
A. Analysis of human behaviour
HIDE focuses on two levels in the analysis of human behaviour, The individual level, which focuses on humans themselves (e.g. their gestures and body) and their interactions with the systems and their environment — digital or otherwise.
A.1. Human-centred behaviour
Exchanges in interpersonal communication occur both through words and physical movements. Physical behaviour is often considered to be a language with a directly interpretable code. Gaze orientation, body posture and facial expressions are important data that allow systems to obtain accurate information about the interest and intent of a human interacting with them. HIDE processes various types of information in the form of 2D and 3D images, graphs and time series to produce systems capable of interacting with humans quickly, reliably and appropriately. This leads to the creation of non-intrusive assistance and decision support tools.
A.2. Behaviour of digital systems in interactions with humans
The analysis of human behaviour in centralised or/and decentralised systems (e.g. multimedia, telecommunications, computer environments) is required to assist decision-making and more generally facilitate interactions between humans and these systems.
HIDE focuses primarily on the detection of patterns or anomalies (e.g. operating anomalies, rare events, falsifications) as well as modelling and optimising systems with respect to various metrics such as distributions, flow rate and energy efficiency, to improve the way they operate (sharing of resources, coordination between systems, adaptability to humans interacting with these systems).
The primary tools used are advanced machine learning methods – deep learning, reinforcement learning – and Bayesian filtering methods such as particle filtering.
There are a variety of potential applications: forensic analysis and steganalysis, allocation of resources for systems operating on very different types of emerging technologies, generation of digital twins.
HIDE provides solutions which offer new collaborative systems combining both in-person and remote interactions, which also take advantage of new advanced remote work solutions based on virtual and augmented reality technology.
B. Quality and reliability of human data
HIDE is concerned with the quality and reliability of human-centred data in terms of the format of the data acquired and processed by systems with the aim of facilitating interaction between humans and systems. It also concentrates on the relevance of these systems to user profile observed (e.g. behaviour, expertise).
B.1. Strengthening and adapting data used by systems
When systems are centred on humans, they raise several scientific issues related to data. Such systems require the processing of private data. This poses enormous ethical and confidentiality challenges which must be resolved in order for the analysis system to be accepted and applied successfully. HIDE aims to develop mathematical tools which would ensure the compliance and confidentiality of private data while also anonymising sensitive data.
In addition to the sensitive nature of the data, HIDE is concerned with the quality of the data, both in terms of data integrity and representativeness. The quality of the data analysed by the systems has a strong impact on the decisions taken and the accuracy of the interactions between humans and the system.
B.2. System reliability indicators focused on user profiles
When humans interact with a system, it is important to consider the knowledge and human skills required to ensure suitable service. HIDE is addressing this issue by focusing on the design of reliability indicators. There are different ways of approaching the issue, both in terms of the methods for calculating the reliability metric and with regard to its relevance in relation to the proposed task and the user profile interacting with the system. HIDE studies several aspects including the following: how to assess the reliability and acceptance of systems by different humans. How to associate semantics with observed and calculated behaviour. Which display of the indicator should be used?
Application – Projects
1 – Health applications
Part of the research focuses on the analysis of human behaviour by using various types of data such as 2D and 3D images, characteristic points and time series. The researchers are working on gesture analysis, both for the purpose of improving the level of interaction between humans and the system, but also to help the system adapt its decisions to human behaviour. HIDE proposes innovative systems for analysing human behaviour which uses information about body movement (style, gestures, facial expressions, gaze orientation). New avenues for collaboration (U. La Rochelle) on innovative methods based on impulse neural networks and bio-inspired sensors are currently being developed.
In addition, HIDE aims to develop and validate the relevance of new systems for remote collaborative work and telework, based on an unprecedented use of digital twins and mixed reality (AR, VR). Several works on the analysis of human behaviour have been carried out to facilitate interactions between augmented humans and the augmented system.
Examples of work related to health include:
- a thesis on detailed analysis of gestures in non-Euclidean frameworks (supervised by Hazem Wannous)
- A thesis (collaboration with Hippy Medtech Systems) on pre-fall detection for the elderly (supervised by Anthony Fleury)
- Work on the analysis of air quality, the detection of certain substances (e.g. NH3), the detection of local and global anomalies in outdoor air quality. All this work is carried out as part of the IAM Lab joint laboratory project (with TERA Environnement and CERI EE of IMT Nord Europe) (Christelle Garnier, Vincent Itier, Anne Savard, Anthony Fleury).
- Projects involving partnerships with South America will study the analysis of data to detect changes in the behaviour of individuals related to the progression of chronic diseases or ageing. (Anthony Fleury)
2 – Transport applications
Several transport-related projects are under way on autonomous transport in particular. The ECOTRAIN project (led internally by Benjamin Allaert) seeks to develop a fully autonomous light shuttle and lead theses aimed at producing components of this project such as the analysis system (based on deep learning) of the train’s environment to ensure its safety.
The Autonomous Train Passenger Service (Train Autonome Service Voyageur – TASV) project with IRT Railenium aims to automate conventional trains (TER) in their environment. A thesis will therefore focus, for example, on the surroundings of the train doors and their safety using deep learning techniques. (Sébastien Ambellouis)
3 – Data security, confidentiality and reliability applications
The data which passes through the system cannot be used in its raw form. To ensure the effectiveness of the system’s decision-making, it is crucial that the data analysed by the system be confidential, reliable, and of good quality.
Confidentiality – As part of the DATALEASH project (Stockholm City, Karolinska Institutet, KTH – Royal Institute of Technology), focused on data confidentiality, HIDE is working to develop tools for assessing the privacy of learning algorithms (i.e. Membership privacy). This work proposes confidentiality measures (variations in maximum leakage) and seeks to study the compromise between privacy and utility (led by Giulia Cervia).
Reliability – HIDE is pursuing the forensic analysis of digital images based on artefact analysis to ensure data reliability. These methods are based on the approach for solving the problem of generalisation between the training base and test base with the aim of detecting tampering and steganalysis in images. Work is also being conducted on forensic analysis and processing in selectively encrypted images. The ANR CI2(IA) project on this subject, led by Vincent Itier, started in 2023.
Security – the DEPOSIA project focuses on learning methods for detecting intrusions by signal analysis (in particular for Wi-Fi). This applies to both indoor and outdoor environments, with greater complexity in outdoor environments, and aims to detect “radio” attacks (jamming), application attacks (fake access points), and object intrusions, such as drones. (Anne Savard, Anthony Fleury)
4 – Computer environments for human learning applications
HIDE works to represent and model data from learning activities (learning traces), analyse this human-centred data with humans (learning analytics) and establish indicators. This work focuses on semantic and logical reasoning to create decision support tools and recommendation systems for humans interacting with the system.
One of HIDE’s objectives is to apply this work to the modelling resolution of complex education-related problems (e.g. individualised school curriculum with skill erosion). To accomplish this, methods are used to model serious games and learning games (e.g. tangible, digital, VR), including the meta-design of intelligent tutoring system (ITS), multi-modal data collection, human-centred data analysis (learning analytics), and participatory design (for learning dashboards).
The SUCCESS project (I-Site) therefore focuses on modelling learners’ curricula and the relationship with skill acquisition.
The ANR Jeune Chercheur TALE4GDA project (led by Alexis Lebis) aims to model serious games for design assistance.
PROJECTS
Hippy project
Hippy project
The Hippy project is a collaborative project which is part of a CIFRE thesis aimed at adding more advanced methods of data analysis and artificial intelligence to improve the performance of a belt designed to detect falls in advance and…
Want to learn more about
"Human, Interaction, Decision" ?
Our teams at the IMT Nord Europe research centres are here to help
CONTACT US