Abstract
Mid-air interaction has been investigated for many years, and with the launch of affordable sensors, this type of interaction has become more popular. However, the cursor-based approach, where the hand replicates the movement of the cursor, has influenced the design of mid-air graphical interfaces (GUI) in an attempt to copy the WIMP-based interaction model used in desktop computers. Another paradigm used in mid-air interaction refers to the development of gesture-libraries, where different gestures are assigned to the functionalities of a system (e.g. thumbs up to confirm an action). Nevertheless, in this approach the user must memorise the gestures to be used, potentially generating cognitive overload due to the need for recall over recognition.Since both approaches present several shortcomings and usability issues – through the adoption of a Research through Design framework, this thesis proposes an alternative approach for midair interaction entitled gestures-only, based on a combination of different strategies: (i) the use of mainly swipe gestures performed in the vertical and horizontal axis and reducing the need for memorisation; (ii) application of distal interaction and relative position to the gestures performed, not requiring precision to be performed; (iii) a navigational structure that allowshierarchical navigation in “n” levels, all mediated by a (iv) graphical interface (GUI) based on manipulation, cursorless, using visual affordances to reveal the gestures to be performed, along
with 11 interaction principles defined to support the approach comprehension.
In order to demonstrate and validate the gestures-only approach, six Design Patterns were developed as a tool to generate and transfer knowledge in design, evaluated through two empirical studies. The first one identifies if gestures suggested by participants matched with the ones originally designed, along with the validation of strategies defined to communicate mid-airgestures through the GUI. The second study presents a comparative analysis of performance and satisfaction between cursor-based and gestures-only, consolidating the proposed approach as the preferred one, identified in the overall user experience. Both studies revealed design recommendations for mid-air gesture-driven interfaces.
Resultant contributions to the field of user experience design include: (i) a novel approach for mid-air interaction based on manipulation and cursorless named gestures-only; (ii) a set of strategies to design mid-air graphical interfaces for the proposed approach (iii) six Design Patterns used to mediate informational content; (iv) a detailed analysis comparing the performance and satisfaction of cursor-based interfaces against the proposed approach and (v) empirically validated design recommendations that inform designers, developers and future research concerning the design of mid-air applications.
| Date of Award | Jul 2019 |
|---|---|
| Original language | English |
| Awarding Institution |
|
| Supervisor | Derek Covill (Supervisor) & Lyn Pemberton (Supervisor) |