How do Gestures Matter for Mutual Awareness in Cockpits?


Thesis work of Valentin Becquet (defended on 2019)

The recent integration of touch screens in the cockpits of commercial airliners has created a problem of sharing information transmitted by gestures between pilots. The objective of Gesture View is to fill this gap by proposing a way to represent, within the digital touch interface, the good properties of physical gestures, in order to preserve the pilots’ ability to coordinate. To do so, I propose an iconic and abstract representation that breaks with representations based on the morphology of the hand.

After conducting an activity analysis, dozens of representations were designed and evaluated by professional pilots during participatory workshops. My work first shows a new way of representing gestures. It then allows to identify the information sought by the pilots so that they are able to ensure their mutual awareness. Finally, the results of the evaluations show that the choices made in terms of graphic design allow the pilots to quickly access the information without reducing the performance of a double task (target tracking task).

On the left, the first pilot operates on a control interface for avionics functions. In accordance with the literature, we designed interactive gestures compatible with the constraints of turbulence. On the right, the second pilot operates on a joystick to control the aircraft. He has a screen representing a target with a cursor symbolizing the trajectory of the plane. He must keep the cursor in the center of the target. A random displacement algorithm has been implemented to require the pilot to maintain a continuous concentration. Gesture View is placed between the target and the external view to optimize the pilot’s visual control scan.

The control interface of the avionic system as well as Gesture View were developed in Smala with Djnn libraries and SVGs from a work done on Adobe Illustrator. The target tracking application (called Tracking) is developed in C#. The connection between the systems is done via the Ivy bus (communication protocol).

Below you will find several examples of proposals evaluated with the pilots that led to the final design of Gesture View.

Related publications

Valentin Becquet, Catherine Letondal, Jean-Luc Vinot, Sylvain Pauchet. How do Gestures Matter for Mutual Awareness in Cockpits? Disclosing Interactions through Graphical Representations. DIS Designing Interactive Systems 2019, Jun 2019, San Diego – California, United States. pp.593-605, ⟨10.1145/3322276.3322306⟩⟨hal-02134640⟩

Valentin Becquet. Conception d’une représentation graphique des gestes numériques pour le cockpit tactile fondée sur les dimensions participant à la conscience mutuelle entre les pilotes. Interface homme-machine [cs.HC]. Institut Supérieur de l’Aéronautique et de l’Espace, 2019. Français. ⟨tel-02492163⟩

Equipe Informatique Interactive.
7 avenue Edouard Belin.
31055 Toulouse Cedex 4
Tél. : +33 5 62 17 40 19
Better understand the interaction phenomenon and master the design of more efficient interactive systems.