Expanding Perspectives to Improve Access to Visual Archives through Multimodal Image Enrichment

The research tackles key challenges for improving the discovery of large-scale visual collections within the Cultural Heritage (CH) domain, particularly in museums and archives. Its contribution is a multimodal content understanding approach designed for image collections that lack relevant metadata, hindering effective discovery. The proposed method utilises AI-assisted image classification and unified vision-language understanding, combining visual features with semantic context to generate rich and meaningful metadata. The proposed approach enables experts to enrich and visualise large-scale datasets of image collections by assigning both expert and non-expert labels, aligning with FAIR principles (Findable, Accessible, Interoperable, Reusable). Thus, the novel workflow broadens access to this visual material for diverse audiences through search and browse interfaces in a web browser. The proposed approach is demonstrated using the previously unclassified Design Archives’ glass plate negatives dataset, which consists of approximately 10,000 digitised images depicting 20th-century historical designs. Through an AI workflow, the dataset is enriched with expert and non-expert information. Users can search and browse results using 2D and 3D visualisations, as well as text-based search. The research also explores the advantages and current limitations of the proposed visualisation approach in creating more meaningful search and browsing functionalities for large-scale CH collections. The results demonstrate that while 3D visualisations offer more affordances than their 2D counterpart, users require further support to interact with the large-scale datasets meaningfully. Hence, there is a need for discovery interfaces that support interactivity, visual cues, and text-based search to enhance the users’ discovery journey.