Using large language models for narrative analysis: a novel application of generative AI

This study, a collaboration between the University of Southampton and Ipsos UK, aimed to develop and test a novel method for analysing qualitative data using generative artificial intelligence (AI). It compared large language model (LLM)-conducted analysis with human analysis of the same qualitative data, explored optimisation of LLMs for narrative analysis and evaluated their benefits and drawbacks. Using existing data, 138 short stories written by young people (aged 13–25 years) about social media, identity formation and food choices were analysed separately three times: by human researchers, and by two different LLMs (Claude and GPT-o1). The method was developed iteratively, combining Ipsos' artificial intelligence (AI) expertise and tools with researchers’ qualitative analysis expertise. Claude and GPT-o1 each conducted a narrative analysis of all 138 stories using the same analytic steps as the human researchers. Findings between the humans and both LLMs were then compared. Both LLMs quickly and successfully conducted a narrative analysis of the stories. Their findings were comparable to those of the human researchers and were judged by the researchers to be credible and thorough. Beyond replication, the LLMs provided additional insights into the data that enhanced the human analysis. This study highlights the significant potential benefits of LLMs to the field of qualitative research and proposes that LLMs could one day be seen as valuable tools for strengthening research quality and increasing efficiency. Additionally, this study discusses ethical concerns surrounding responsible AI use in research and proposes a framework for using LLMs in qualitative analysis.