Low-power machine learning for visitor engagement in museums

Description

Much of the discussion of Machine Learning (ML) over the past decades has focused on extracting value from vast amounts of data. Applications in this tradition are typically developed by ML experts, involve bespoke ML models and run on high-powered workstations or cloud infrastructure. Emerging low-power ML technologies, by contrast, are optimised to process data locally on consumer-level hardware and typically involve pre-trained, off-the-shelf ML models for generic problem classes. They are well suited for interactive applications that require access to devise sensors and have privacy or network constraints. This talk gives a brief introduction to low-power ML and presents a case study of using it for audience engagement in museums. It presents an application using lightweight ML models for human pose estimation and gesture classification to enable visitor engagement with interactive projections of interior designs. An empirical evaluation found the application is highly engaging and motivates visitors to learn more about the designs. Most participants noticed the effects of uncertainty in ML predictions, as inaccuracies, jitter, or gesture recognition problems, however, these had little impact on their positive user experience. While the findings cannot be extrapolated to other contexts and use cases, they indicate the great potential of these technologies for non-critical interactive applications.

Period	28 Sept 2022
Held at	Norwegian University of Science and Technology, Norway
Degree of Recognition	International