Gesture Me: A Machine Learning Tool for Designers to Train Gesture Classifiers

Research output: Chapter in Book/Conference proceeding with ISSN or ISBNChapterpeer-review


This paper contributes to the body of work examining how designers can be sup-ported in integrating machine learning (ML) capabilities into their designs for novel applications and services. It presents an online tool enabling designers and other non-specialist audiences to define body gestures, interactively and iterative-ly train and test a classifier to recognise these gestures, and integrate the trained classifier into a template web application. An empirical evaluation with MSc User Experience Design students and practitioners, all of whom had previous experi-ence in web development but not in ML, found that the tool enables them to de-fine, train and test a gesture recognition classifier with little or no help, and that engagement with the tool advances their understanding of the capabilities, limita-tions and operational aspects of ML. The evaluation confirmed the value of visu-alising the ML perspective and encouraging designers to experiment with ML to support their experiential learning. The study led to design recommendations that can inform the development of tools supporting designers to ideate and prototype ML-enhanced applications.
Original languageEnglish
Title of host publicationComputer-Human Interaction Research and Applications. CHIRA 2023.
EditorsH.P. da Silva, P. Cipresso
Place of PublicationCham
PublisherSpringer Nature
ISBN (Electronic)9783031494253
ISBN (Print)9783031494246
Publication statusPublished - 23 Dec 2023

Publication series

Name Communications in Computer and Information Science
PublisherSpringer Nature
ISSN (Print)1865-0929
ISSN (Electronic)1865-0937


  • UX Design
  • Machine Learning
  • Gesture Recognition
  • Usability
  • Learning


Dive into the research topics of 'Gesture Me: A Machine Learning Tool for Designers to Train Gesture Classifiers'. Together they form a unique fingerprint.

Cite this