Seminar:
Measuring and Mitigating Bias in Parliamentary Discourse Summarisation - Eoghan Cunningham (University College Dublin)
14:00-15:00 (GMT) Wednesday, October 1.
Please join our mailing list (opens in a new window)here to receive bi-weekly invitations to all events. All events take place at UCD and are also live-streamed on Zoom.
Abstract: Public understanding of parliamentary activities is worryingly low, despite their fundamental role in representative democracy. The automated summarisation of parliamentary debates using large language models (LLMs) offers a promising way to make complex legislative discourse more accessible to the public. However, such summaries must not only be accurate and concise but also equitably represent the views and contributions of all speakers. We explore the use of LLMs to summarise plenary debates from the European Parliament and investigate the algorithmic and representational biases that emerge in this context. We propose a structured, multi-stage summarisation framework that improves textual coherence and content fidelity, while enabling the systematic analysis of how speaker attributes – such as speaking order, language or political affiliation – influence the visibility and accuracy of their contributions in the final summaries. Through our experiments using both proprietary and open-weight LLMs, we find evidence of consistent positional and partisan biases, with certain speakers systematically under-represented or misattributed. We show that these biases vary by model and summarisation strategy, with hierarchical approaches offering the greatest potential to reduce disparity. Our findings indicate that LLMs present a promising means of making complex legislative information more accessible to the public, while also underscoring the need for domain-sensitive evaluation metrics and ethical oversight in the deployment of LLMs for democratic applications.
About the speaker: Eoghan is a postdoc on the ParliView project, a transparency initiative led by James Cross and Derek Greene in UCD, and Zachary Greene at the University of Strathclyde in Glasgow. Eoghan completed his PhD at the Insight Centre for Data Analytics in the School of Computer Science, UCD. His thesis studied network analysis and natural language processing methods to measure and facilitate the transfer of knowledge in research networks. Eoghan's current research is focused on AI for democracy and transparency, but he is equally interested in democracy and transparency for AI.