Seminar: When Republicans See Red but Democrats Feel Blue: Why Labeler-Characteristic Bias Matters for Image Analysis - Nora Webb Williams (University of Illinois)
14:00-15:00 (IST) Wednesday, September 22.
Please register for this event (opens in a new window)here.
Abstract: Image analysis studies, big or small, rely on human annotators. Human-generated labels are treated as ground truth for analysis and for the training of machine-learning algorithms. Annotators may label for image content (e.g. whether an image includes a protest) or for reactions to images (e.g. whether it evokes enthusiasm or sadness). In this research note we explore whether partisan and gender identities impact how annotators see or react to the same images. We then examine how labeler-characteristic bias could affect results from common analyses. Using nearly 7,500 images from left-leaning social movements on Twitter, we find that partisans can disagree about what an image depicts and that they report very different emotional reactions to the same images. We also find significant differences in content labels and emotional reactions between male and female annotators. Finally, we demonstrate why these systematic labeling differences matter by estimating the effects of content and reactions on retweets. We find that the results from these models vary based on whose labels are included in the analysis.
About the speaker: Nora Webb Williams is an Assistant Professor in the Department of Political Science at the University of Illinois at Urbana-Champaign. She holds a Political Science PhD from the University of Washington and a dual Master’s degree in Public Affairs and Central Eurasian Studies from Indiana University. Her research addresses economic resilience and the long-term impacts of colonialism on social trust, with a regional focus on the former Soviet Union. She also writes about the impact of social media and images on protest mobilization, examining diverse cases such as the 2010 revolution in Kyrgyzstan and the Black Lives Matter movement in the United States. Her primary methodological interest is in images as data for social science research, with related interests in machine learning, text as data, and causal inference. Notable experiences outside of the university setting include serving as a Peace Corps volunteer in Kazakhstan and Liberia. Her work has been published or is forthcoming in the Cambridge Elements series in Quantitative and Computational Methods for the Social Sciences, Policy Studies Journal, Political Research Quarterly, and Europe-Asia Studies, among other outlets.