The Internet, Policy & Politics Conferences

Oxford Internet Institute, University of Oxford

Katharina Esau, Dennis Friess, Christiane Eilders: Design Matters! An empirical analysis of online deliberation on different news platforms

Katharina Esau, Institute of Social Sciences at the University of Düsseldorf, Germany
Dennis Friess, Institute of Social Sciences at the University of Düsseldorf, Germany
Christiane Eilders, Institute of Social Sciences at the University of Düsseldorf, Germany

The rise of the Internet offers various opportunities for political communication. One elaborately discussed form of political communication within the Internet is online deliberation. From a theoretical point of view the Internet provides the exact infrastructure for the public sphere that deliberation advocates have dreamed of. Not surprisingly, deliberative democracy is one of the most influential theoretical concepts in the ongoing debate on the relationship of democracy and Internet technology.

Since deliberation research has experienced a sharp increase the proliferation of online deliberation has been conceptualised in very different ways. The growing body of empirical literature has stretched the concept including almost every type of communication (Bфchtiger et al., 2010). In contrast, this paper rests on a narrow definition: Drawing on normative theories of deliberative democracy (e.g., Habermas, 1996) we define deliberation as a demanding type of communication following certain rules. It is therefore characterised by particular standards (e.g. argumentation, interactivity, respect and constructiveness) which are empirically assessable.

So far scholars have mainly focused on parliaments (e.g., Steiner et al., 2004), traditional media (e.g., Gastil, 2008) or deliberative polls (e.g., Fishkin, 2009). Online deliberation was analysed particular in newsgroups (e.g., Wilhelm, 1999), government forums (e.g., Coleman et al., 2002), online deliberative polls (e.g. Iyengar et al., 2005) or on newspaper websites (e.g., Zhou et al., 2008). However, only few studies have compared deliberativeness across different online platforms (e.g., Rowe, 2015; Jensen, 2003).

This paper builds on the strand of literature that empirically analyses various design features which are likely to have an effect on the quality of online deliberation (e.g., Towne & Herbsleb, 2012; Janssen & Kies, 2005). Accordingly, we do not perceive the Internet as one coherent public sphere but rather as a network where different public spheres provide different opportunities for public deliberation. Against this background we argue that online deliberation research should focus on the particular platforms where deliberation is supposed to take place.

In this contribution we focus on three different types of platforms which differ in design and scope: official Facebook pages of newspapers, newspaper websites, and specific news media forums for deliberation. The comparison between these platforms is particularly meaningful as online media pursue different strategies to enable public debate (some newspapers have closed their comment functions completely and outsourced discussions to social networks, others set up special discussion forums). With regard to the corresponding platform differences we expect that the quality of deliberation varies across platforms. Hence, the main research question is:

""How does the quality of deliberation vary across different online platforms?""

In order to answer this question we analysed more than 1800 user comments scraped from the different online platforms. The degree of deliberativeness was assessed via a quantitative content analysis. The coding scheme draws on Habermas’discourse ethics. Furthermore, we integrated suggestions from empirical research on the measurement of deliberation in the coding scheme (e.g. Stromer-Galley, 2007).

The analysis compared online user comments on newspaper articles shared on the social network Facebook (N= 979), user comments on Newspaper Websites on the same news articles (N = 591) and user comments in a specific Discussion Forum of a newspaper (N = 231). In order to avoid bias through different topics all discussions regarded the Refugees Crisis and Military Engagement in Syria, two topics controversially discussed in December 2015.

Findings indicate that deliberative quality differs significantly among the analysed platforms. Based on a deliberation index consisting of six measures (topic relevance, argumentation, interactivity, argumentative engagement, respect, constructiveness) the results show that specific design deliberation forums are most likely to produce deliberative debate. While users’comments on newspaper websites are only slightly less deliberative, Facebook comments are significantly less deliberative compared to the other platforms.

A closer look at the individual quality criteria shows relevant details. There is more interactivity on newspaper websites and on Facebook than in the forum which is specifically designed for deliberation. Moreover, deliberative quality is less likely in humorous and negative emotional comments. Since negative emotions and humour are more present on Facebook, this may explain the low degree of deliberativeness compared to the other platforms. Positive emotions and narratives also affect deliberativeness, but they increase the degree of deliberativeness. This casts light on the relationship between type I and type II deliberation (Bфchtiger et al., 2010).


Bфchtiger, A., Niemeyer, S., Neblo, M., Steenbergen, M. R. & Steiner, J. (2010). Symposium: Toward More Realistic Models of Deliberative Democracy. Disentangling Diversity in Deliberative De-mocracy: Competing Theories, Their Blind Spots and Complementarities. The Journal of Political Philosophy, 18(1), 32-63.

Coleman, S., Hall, N. and Howell, M. (2002) Hearing Voices. The Experience of Online Public Consultation and Discussion in UK Governance, UK: Hansard Society

Fishkin, J. S. (2009). When the People Speak. Deliberative Democracy and Public Consultation. Oxford: Oxford University Press. Gastil, J. (2008). Political communication and deliberation. Thousand Oaks: Sage Publications

Gutmann, A. & Thompson, D. (2004). Why Deliberative Democracy. Princeton: Princeton University Press. Habermas, J. (1996). Between Facts and Norms. Contributions to a Discourse Theory of Law and Democracy. Cambridge: MIT Press.

Janssen, D. & Kies, R. (2005). Online forums and deliberative democracy. Acta Politica, 40(2005), 317-335. Jensen, J.L. (2003) ‘Public spheres on the internet: anarchic or government-sponsored ‘a comparison’, Scandinavian Political Studies 26(4): 349’374.

Iyengar, S., Luskin, R. C. & Fishkin, J. S. (2005). Deliberative Prefrences in the Presidential Nomina-tion Campaign: Evidence from an Online Deliberative Poll. Research Paper, Center for Deliberative Democracy, Stanford: Stanford University.

Steiner, J., Bфchtiger, A., SpЎrndli, M. & Steenbergen, M. (2004). Deliberative Politics in Action: Analysing Parliamentary Discourse. Cambridge: Cambridge University Press

Stromer-Galley, J. (2007). Measuring Deliberation's Content: A Coding Scheme. Journal of Public Deliberation, 3(1), 1-33.

Towne, W. B. & Herbsleb, J. D. (2012). Design Consideration of Online Deliberations Systems. Journal of Information Technology & Politics, 9(1), 97-115.

Wilhelm, A. G. (1999). Virtual sounding boards: how deliberative is online political discussion? In B.N. Hague and B.D. Loader (eds.) Digital Democracy. Discourse and Decision Making in the Information Age. London: Routledge.

Katharina Esau, Dennis Friess, Christiane Eilders