Thursday 18th December at 2p.m. Paris time, Camille Dupré, Ph.D. Candidate has defended her thesis named “Pad-based Interaction in Mixed Reality environments”. Her thesis defense took place at the LISN, in Gif-sur-Yvette (660 Av. des Sciences Bâtiment, 91190), France. Take a look at the summary below.
Summary
Mixed Reality (MR) environments integrate virtual elements into the user’s physical space, creating immersive experiences where real objects and digital content coexist. This spatial distribution of virtual content is valuable, as it allows information to be displayed where it is most meaningful, but it also complicates interaction with virtual elements scattered throughout the environment.
This thesis investigates the use of a pad, a small touch-sensitive surface, as a means of indirect interaction in MR. It explores the design of opportunistic pads, created from existing surfaces and relying solely on sensors embedded in MR devices, particularly hand-tracking. The goal is to facilitate and enrich interaction in MR, with a particular focus on industrial maintenance scenarios.
The first contribution studies Spatial Window Switching (SWS) techniques, which allow users to locate a specific window among those distributed in the environment. We designed and evaluated techniques based on an overview of all available windows represented as miniatures, visually linked to their actual positions. Results show that these techniques significantly reduce search time and physical movement, helping users locate virtual content of interest more efficiently.
The second contribution explores the use of the non-dominant hand as a support surface for an always-available pad, relying solely on the hand-tracking capabilities of MR devices. Three anchoring strategies were evaluated: a pad floating near the hand, a pad aligned with a flat object held in the hand, and a pad directly placed on the palm. Results show similar performance across all approaches, indicating that hand tracking has reached a level of maturity sufficient to allow the palm to serve as a reliable touch surface. Although these pads perform worse than capacitive screens, they let users keep both hands free and provide proprioceptive and haptic cues.
The third contribution, TriPad, introduces a technique that enables users to transform any passive physical surface into an interactive pad using only hand tracking. The user places their hand flat on the surface, then performs a “tap” by briefly lifting and lowering it. The system recognizes this characteristic movement and infers the plane by analyzing the positions of three fingertips when the hand was stationary, anchoring the virtual pad accordingly. We validated TriPad’s robustness across various surface types (wood, glass, drywall) and compared it to existing direct interaction techniques available in MR devices. Results show that TriPad provides a better speed–accuracy trade-off, particularly on horizontal surfaces where it reduces fatigue and improves comfort.
Taken together, Spatial Window Switching techniques, hand-bound pads, and surface-anchored pads open new possibilities for indirect interaction in Mixed Reality. They address accessibility, comfort, and precision issues inherent to direct interaction.
The thesis concludes with a discussion of future perspectives for improving this paradigm of indirect interaction and its applicability to industrial contexts, especially MR assisted maintenance, where technicians can benefit from more efficient, and safe interaction in complex environments.
The jury was composed of:
Examiners:
- Marchal Maud – Examiner and Advisor, INSA University of Rennes
- Pietrzak Thomas – Examiner and Advisor, University of Lille
- Argelaguet Ferran – Examiner, INRIA University of Rennes
- Coutrix Céline – Examiner, CNRS University of Grenobles Alpes
- Fekete Jean-Daniel – Examiner, INRIA University of Paris-Saclay
Supervisors:
- Pietriga Emmanuel – Director, INRIA Center of Saclay
- Caroline Appert – Co-Director, CNRS | University of Paris-Saclay
- Saidi Houssem – Co-Director in the socio-economic field, Berger-Levrault
Industrial Perspectives:
Carl Software – Berger-Levrault provided computer maintenance management system (CMMS) solutions. Such applications are designed to optimize the management and maintenance of physical assets in the built environments of its clients. With industry 4.0 and 5.0, new technologies like Mixed Reality are now used to enhance the user experience and improve the efficiency of maintenance operations. The integration of MR technologies allows technicians to visualize information in their physical environment, providing them with real-time data and guidance during their tasks.
The work of my thesis brings better understanding of the opportunities and challenges of MR in industrial maintenance. In particular on how to help users locate different content in mixed reality or on the use new interaction techniques based using pad to interact with the content.
Some of her Scientific Publications:
- (2025) Investigating Hand-Bound Pads for AR Input Using Hand-Tracking Only: https://cnrs.hal.science/hal-05140350v1
- (2024) TriPad: Touch Input in AR on Ordinary Surfaces with Hand Tracking Only: https://cnrs.hal.science/hal-04497640v1
