Abstract:  The media’s capacity to stimulate public concern and create a common ground for issues can counteract the fragmentation of society. Assessing the intactness of the media’s agenda-setting function can be an important diagnostic tool for scholars. However, the manifold design choices in agenda-setting research raise the question of how design choice impacts analysis results and potentially leads to methodological artefacts. I compare how the choice between 20 plausible analysis configurations impacts tests of the agenda-setting hypothesis, coefficients, and explanatory power. I also explore changes in agenda-setting effect size over time. I develop a typology of analysis configurations from five basic study design types by four ways of linking content analysis to survey data (5 × 4 = 20). The following design types are compared: three single-survey/between designs (aggregate-cross-sectional, aggregate-longitudinal, and individual-level) and two panel-survey/within designs (aggregate-change and individual-change). I draw on the German Longitudinal Election Study data (2009, 2013, and 2017). All 20 tests of the agenda-setting hypothesis support the hypothesis, independent of the analytical configuration used. The choice of analysis configuration substantially impacts the coefficients and explanatory power attributed to media salience. The individual-level analyses indicate that agenda-setting effects became significantly weaker at later elections, though not linearly. This study provides strong empirical support for the agenda-setting hypothesis independent of design choice.

Keywords:  agenda-setting; aggregation; design choice; data analysis; data linkage; methodological artefacts

Published:   31 August 2022

Issue:   Vol 10, No 3 (2022): Enlightening Confusion: How Contradictory Findings Help Mitigate Problematic Trends in Digital Democracies