HumanIS (Human-centric Intelligent Systems)
Digital Systems
Themes and challenges
The HumanIS (Human-centric Intelligent Systems) instrumented platform offers mobile demonstrators and a wide range of services based on CERI SN’s work on human-centered systems in the following areas:
- Understanding human activities and detecting abnormal behavior
- Human-machine collaboration with embedded intelligent vision
- Strengthening human interaction in immersive environments
- Home support and automated monitoring of patient records
This interdisciplinary platform (including physics, data processing and perception) relies on immersive experimental protocols and offers mobile equipment for teaching, research and scientific popularization purposes at trade shows.
Artificial intelligence and embedded vision
Energy consumption and processing time
IMT’s strategic focus is on tapping into the potential of AI to benefit industry and society with guarantees of trust and efficiency. In light of this focus, trusted AI and embedded AI are particularly strategic issues. In addition to optimizing the functional performance of these systems, our research also aims to remove the technological barriers of explicability, frugality and robustness to enable their transition to an industrial scale.
Vision-based human-machine collaboration
Representation, Visualization and Interaction
Analyze human behavior to help the system to identify the optimal action to be taken to ensure the collaborative task will be successful.
Facilitate interactions by optimizing system movements and gestures.
Optimize detection speed and quality when handling complex data by relying on multi-sensor vision systems.
Improve performance of a task by adapting the system according to the person’s level of expertise.
Driving system autonomy
Scene understanding and decision making
Develop systems capable of interpreting the world around the vehicle (signs, behavior) to anticipate collision risks
Study capture systems that ensure reliable and fast decision-making in low stress environments
Propose location tools (visual odometry) that allow the vehicle to locate itself in space and optimize its movements
Optimize energy consumption (neuromorphic camera, spike neural network)
Instrumentation and decision support tools
Make day-to-day activities easier for industry experts
Offer systems that adapt to the experts’ needs without modifying their work environment and experimental protocols
Ensure data retention by providing centralized analysis, fully embedded in the instrumentation tool
Give experts qualitative and quantitative feedback while allowing them to interact with the system directly to make changes
Develop intelligent systems capable of inferring results in real time, while ensuring data integrity and reliability
Enhanced Human Interaction
Collaborative systems and immersive environmentsf
The emergence of mixed reality technologies will revolutionize the way people work, collaborate and interact remotely in the future. Although systems are evolving to improve the quality of immersive environments, the research has focused primarily on the ergonomics and visual quality of these technologies at the expense of the quality of collaborative interactions and data security.
Improve social interactions in mixed reality
Synchronization between the real and virtual world
Localization in multiple static/dynamic environments (BIM)
Categorize real-world elements to define their interest and prioritize updates (processing time optimization)
Multi-sensor data correlation and fusion
Assess the advantages and disadvantages of different representation paradigms for user animation
Mixed environment (real/virtual) design
Representation, Visualization and Interaction
Recognition and localization (orientation, position) of known elements in the real environment
Automatic classification in the 2D and 3D image domain
Inter-element relationship detection (physical or functional proximity)
Understanding human activities (actions, intentions, emotional state)
Improving the security and quality of services
Data quality, security and reliability
Gathering evidence from digital media to understand behavior, remedy an incident and help make informed decisions
Guarantee the confidentiality of documents exchanged via the collaborative tool (steganography, steganalysis) and prevent data leaks.
Observe the consistency of multimodal data (image, sound, text, video) across multiple recordings, programs and platforms.
Personal care assistance and monitoring in the home
Intelligent living and unobtrusive system
Home care requires adapting the living environment and organizing day-to-day assistance for the individual concerned. Deteriorating health, whether physical or mental, requires the implementation of medical measures at home. The prevention and anticipation of a dependent situation is essential in delaying the admission of elderly people to specialized institutions. The use of artificial intelligence largely simplifies the lives of users. When associated with everyday objects, it can simplify and improve the quality of life of older people and facilitate medical monitoring for healthcare staff.
Physiological and cognitive markers
Early detection of risk behaviors
Identify abnormal behavior by analyzing the deviation of an event (fall, frieze, …) from usual human behavioral signatures.
Analyze daily activities (location, frequency of action, rest, postures)
Use of multimodal data to identify complex behaviors (disorders, attention deficit, hyperactivity)
Analyze complex biological signals
Behavioral map of complex activity
Understand the etiology of human movement (abnormal and involuntary movement)
Define a person’s motor signature and behavioral signatures, with the aim of detecting possible neurodegenerative diseases.
Offer innovative, non-intrusive tools to monitor without modifying the patient’s environment (belts, insoles, smartphones).
Visualization and decision support tools
Centralized, reliable multi-person data
Propose assessment metrics based on perceptual-motor improvements (talent, expertise, learning, skills).
Develop intelligent systems capable of inferring results in real time, while ensuring data integrity and reliability
Ensure data retention by providing centralized analysis, based on federated learning networks
Propose systems that adapt to the needs of the expert and the patient without changing their habits
Equipment
The HumanIS platform has extensive equipment, from more general to highly specialized tools to ensure the smooth progress of our research and teaching activities, which cover several subject areas.
Motion
- Qualisys motion capture system, 8 Miqus M3 cameras
- Motion capture Faceware headset, 2 GoPro cameras and facial analysis software
- Qualisys Track Manager (QTM) software
- Inertial Measurement Units (IMU)
- Realsens Depth camera D455
Embedded vision
- Rosmaster X3 Plus robot with Jetson NX 8GB card
- Jetracer Pro cars with Jetson Nano card
- DVXplorer Mini neuromorphic camera (640×480) & Loihi 2 neuromorphic chip
- Yahboom Dofbot robotic arm
- Car simulation racetrack
Connected Health
- Bascom 8-channel video surveillance system, 6 RGB & IR dome cameras
- OpenGo connected insoles incorporating an IMU and 16 pressure sensors
- Smartphone and connected watches
- Connected airbag belt (in partnership with Hippy medtec systems)
Immersive environments
- Station with high-performance GPU computer
- Occulus Quest 2 augmented reality headsets
- Samsung Android tablets
Demonstrators
The HumanIS platform has several demonstrators to which research professors and students (students, engineers, PhD and postdoctoral students) have contributed. The demonstrators are used for teaching, research and scientific popularization purposes.
ECOTRAIN – Autonomous railway shuttle
Obstacle detection and data simulation
The goal is to develop an ultra-light and autonomous rail shuttle suitable for both passenger and micro-freight transport to revive dormant rural lines. The project, called Ecotrain, was selected among the first five winners of the national call for expressions of interest on “Digitalization and decarbonization of rail transport”, as part of the France 2030 plan.
Real-time obstacle analysis (integrated in the vehicle)
Intelligent multi-sensor vision system
Identify elements within the vehicle’s field of vision and their relationships
Analyze the behavior of living obstacles (people, animals)
Simulate scenarios for automatic risk annotation
Analyze the situation around infrastructure
Low power monitoring system
Detect road congestion using vision sensors
Anticipate user behavior when the vehicle passes through
Optimize energy consumption (low energy sensor)
E-health training: augmented reality and test dummy
Fun tool for teaching human anatomy
By combining teaching using test dummies with augmented 3D reality, medical trainers have discovered the possibilities offered by immersive technology and AI in developing their nursing assistant students’ clinical reasoning. This case study allows students to learn human anatomy by working on a test dummy in a real environment, while seeing superimposed virtual elements, in keeping with the physical constraints applied to the manikin (e.g. muscle contraction, blood flow).
Motion capture interaction system
Localization and interaction system in complex environments
Localization of elements in the scene by using a motion capture system and superimposing 3D information in augmented reality
Improve the monitoring of human activities in a complex multi-user environment
Control the latency of flows and the quality of interaction
Interaction and visualization tools
Intelligent multi-sensor vision system
Propose learning scenarios to improve the immersive experience and understand complex concepts.
Study new collaborative interaction solutions in both physical and virtual environments
Want to learn more about
"HumanIS (Human-centric Intelligent Systems)" ?
Our teams at the IMT Nord Europe research centres are here to help
CONTACT US