What is Augmented Reality?
Augmented Reality (AR) is the superimposition of digital information within the user’s environment in real time. In AR, users experience their real-world environment with digital elements like images, sounds, or other data overlaid on top of it.
Augmented Reality finds applications across diverse fields including education, healthcare, maintenance, entertainment, etc…
To access this augmentation, users can either wear AR glasses or use a smartphone or tablet as a viewing lens, depending on the application and the context.
How to interact in AR and what are the common limitations?
To engage with Mid-air Augmented Reality content while wearing AR glasses, hand input is typically preferred over motion controllers. This choice allows users to have their hands freely for better usability (i.e. in maintenance, it allows technicians to use their hands to manipulate and repair the real machine).
Interaction commonly involves touching virtual objects by reaching out and manipulating them as if they were physically in the user’s hand, or interacting from a distance using a laser pointer (raycast) originated from the user’s hand.
Despite its advantages, this method has limitations. It can lead to fatigue, lacks tactile feedback for confirming users actions (i.e. the sensation of touching an object), and offers limited motor precision.
These limitations can present significant challenges for new users, making interaction more difficult and potentially frustrating. For experienced users, these constraints can also become a barrier, as they may prevent them from engaging effectively over extended periods due to the fatigue it causes.
Our approche : TriPad
In our approach named TriPad, we give users the possibility to uses physical object as support for their interaction. Users declare a new surface by placing their hand in contact with the target surface and making a quick tap gesture to confirm their intent. This low-friction gesture creates the surface plane, which gets positioned and oriented according to the 3D coordinates of fingertips.

once the plane is created, users are then free to use it at will to interact with UI panels and other 2D interface components. Tripad can be used for direct interaction or indirect interaction like a virtual trackpad.

What does it look like?
Conclusion
We conducted user experiments to evaluate the performance and usability of TriPad. Throughout the experiment, participants demonstrated an ability to quickly and efficiently create TriPads and interact with them. The results also indicate that interacting with a TriPad – especially on a horizontal surface – is often preferred over the mid-air alternative as it improves comfort and minimizes fatigue.
As TriPad creation and interaction relies only on hand tracking, this approach works on a variety of surfaces and can feature macro “irregularities”, such as a perforated panel. This flexibility enhances the practicality and adaptability of TriPad, making it suitable for a variety of real-world environments and use cases.
More information and details are available in the full paper.