Critiquing Evidence Criticisms: The Condition and Challenge of Evidence Criticisms for Democratically Constituted Knowledge Societies

digital (Villa Vigoni, Menaggio)
DFG-Forschungsgruppe 2448 „Practicing Evidence – Evidencing Practice“; Mariacarla Gadebusch Bondio, University of Bonn; Karin Zachmann, Technical University of Munich
23.03.2021 - 24.03.2021
Claudia Göbel, Institut für Hochschulforschung, Universität Halle-Wittenberg

In its current second phase of operation[1], the interdisciplinary DFG research group 2448 “Practicing Evidence – Evidencing Practice” is concerned with dynamics of de- and re-stabilisation of evidence. Evidence can be understood as socially consented knowledge emerging from negotiation processes. Evidence criticisms served as conceptual lens to explore the inherent contingencies of evidence practices. The symposium departed from the observation that – as key resource in knowledge societies – evidence is increasingly contested. Guiding questions were “What are conditions and challenges of evidence criticisms along with their contingencies? How are evidence criticisms related to democratic knowledge societies? How can forms of evidence criticism beyond a generalised deconstruction of science be understood?” This report summarises cornerstones of the debate, which gathered perspectives from history, philosophy and the social sciences.

A historical perspective allows tracing how the decreasing lifespan of evidence and ferocious public mistrust in science today, are influenced by past uprisings of evidence criticism and their re-stabilisation. In this way, KARIN ZACHMANN (Munich) linked the current so-called post-factual situation to what happened in the 1970s in Western Europe and the United States. This period saw strong social movements accompanied by clusters of what can be called „counter“ science pushing for alternative evidence practices, especially regarding knowledge on industrial hazards and pollution. This de-stabilisation of evidence regimes was followed by a re-stabilisation that included the creation of risk research and policy advice bodies as well as the institutionalisation of „counter“ research.

STEFAN ESSELBORN (Munich) and Karin Zachmann illustrated this for civil-society-based „Gegenforschung“ (counter or alternative research) in the history of probabilistic risk assessment for nuclear power plants in Germany. They showed how a de-stabilisation of institutional evidence-regimes between the 1960s and 1980s was characterised by the emergence of alternative expertise and increasing establishment and networking of organisations, like the Öko-Institut in Freiburg or IFEU Heidelberg. From the 1990s onwards, the integration of the Gegenforschung into the mainstream research and expertise ecosystem followed. This professionalisation was accompanied by a split from the social movement base and could be interpreted as a re-stabilisation of evidence.

ANDREAS WENNINGER, KEVIN ALTMANN (Munich), SASCHA DICKEL and MICHAEL KITZING (Mainz) scrutinised conditions for the possibility of evidence criticism by external actors further. For the case of Citizen Science, they argued that the transformative potentials of this form of participatory research for classical science are more likely to be accepted by the latter, as long as Citizen Science projects exhibited a certain connectivity. Expressed in a positioning close to or inside academia and in the acceptance of evidential standards of mainstream research, such intelligibility also entailed a certain degree of professional control.

Evidence criticisms occur in manifold contexts of knowledge generation and application. ELIF ÖZMEN and DANIEL FÜGER (Giessen) offered a point of entry into this variety and ways to study it. They provided foundational philosophical remarks on contingencies, flexibilities and ambiguity of evidence within science. Following Helen Longino‘s social epistemology, they argued that science needs to be understood as a social process. In such a perspective, evidence appeared as a pluralistic, ambiguous and contested concept with inherent contingencies. The latter fulfilled an important normative function for scientific inquiry: Contingencies of evidence, and some forms of evidence criticism, needed to be seen as interdependent with criticism as key element of the scientific ethos.

SAANA JUKOLA and MARIACARLA GADEBUSCH BONDIO (Bonn) explored such inner-scientific negotiations of knowledge standards in evidence-based medicine. For a controversy on surgical clothing, they examined disagreement on what constitutes sufficient evidence. The opposing positions they found shed light on broader ethical implications: The gold standard of evidence-based medicine – epidemiological evidence, i.e. statistical evidence and randomised control trials – was often not available for fundamental measures such as surgical wear, surgical hand disinfection or face masks.

Mariacarla Gadebusch Bondio also highlighted the key role of media for communicating scientific evidence claims in the public sphere. The dense debates of the past months about the pandemic illustrated the importance but also the dilemmas, uncertainties and costs associated with seeking solid evidence in complex situations. In this context, the scandalisation of dissent and (self)criticism in the scientific community by the media had paved the road for a diffuse scepticism of evidence. This prompted the question: How much evidence criticism can we actually afford?

SUSANNE KINNEBROCK, HELENA BILANDZIC and THERESA STAHLHUT (Augsburg) investigated forms of evidence criticism in German print media coverage of genomic research. Their focus was on narrative as form of communicating inner-scientific evidence criticism to non-expert audiences. While storytelling did not result to be a frequent mode of evidence criticism in journalistic coverage of genomic research, several common rhetoric strategies could be identified for when it did occur.

The role of mass media was also a big issue in the nutritional sciences, where it came together with a weakly established corpus of evidence, plenty contradictory recommendations and a strong presence of commercial interests. EDOARDO PELLI and JUTTA ROOSEN (Munich) examined how consumers interpret and treat such controversial nutritional evidence, thus approaching another key group of actors concerned with contingencies of evidence in knowledge societies.

Further evidence critiques emerge in the relations between science and other spheres of society. EVA BARLÖSIUS (Hanover) gave an overview of three forms of contestation of scientific expertise linked to political processes: (1) contestation of the scientific knowledge, on which the evidence is based, regarding scientific validity; (2) contestation of the appropriateness of political decisions, in which scientific expertise had a formative part, regarding policy-related criteria; and (3) an escalated contestation of the relevance of scientific expertise for political decisions by opposing the view that only scientific knowledge is valued as objectively reasonable. At the same time, public policy bodies had developed and institutionalised measures to handle contestation to some extend by various scientific procedures, such as consensus building, and political ones, such as the precautionary principle.

Contestations of forms (1) and (2) can be observed, for instance, in the field of knowledge on global environmental challenges. CHRISTOPH KÜFFER (Zurich) shared a first perspective in this regard from the field of ecology. He put forward critique on the dominant evidence regime at the science-policy interface, which was shaped by standards from climate change research. The more diverse sources and pluralistic understanding of evidence in ecology required adaptations of science-policy interfaces in order to accurately represent and protect biodiversity. SARAH EHLERS and HELMUTH TRISCHLER (Munich) addressed knowledge on global environmental challenges for the case of hazardous pesticides. They studied evidence strategies by the environmental (justice) movement and other voices from the Global South in the context of international development policy during the 1980s. The Pesticide Action Network, an international coalition of NGOs, developed a critique and alternatives to the dominant evidence regime on industrial hazards. The latter had led to increasing bans of toxic pesticides in the Global North, while exports to the Global South flourished and related health problems were not well captured.

To explain how pseudoscientific information fuels science denialism and populist positions on scientific topics, i.e. contestations of type (3), CARLO MARTINI (Milan) proposed the concept of „vicious evidence“. Based on philosophical concepts of evidence and disinformation he carved out the problem that evidence denialism could be masked as evidence questioning, the latter being generally legitimate in science. Cases of scientific disinformation could then be better understood as not rejecting evidence per se, but rather as adhering to the wrong kind of evidence. Vicious evidence appeared as „wolf in sheep’s clothing“: Fake news and scientific misinformation that mimicked scientific evidence by bearing most of its trappings or by falsely appearing to be relevant to the issue at hand. However, vicious evidence deviated from standards of evidence in important ways that revisiting debates on the demarcation of science from pseudoscience should address.

Drawing boundaries around science or facilitating connectivity is also a matter of judgement. This puts practices of assessing evidence to the fore, along with their respective contingencies and critiques. RUTH MÜLLER and MALLORY JAMES (Munich) presented a critical examination of how time matters in evidence practices in research funding. For presenting and judging evidence on the excellence of candidates for grants by the European Research Council (ERC), they highlighted two dimensions: (1) the temporal organisation of review processes and how time pressure affects individual and collective review practices; (2) the use of temporal categorisation as judgement devices to facilitates comparative evaluation and thus co-shape who is seen as excellent researcher.

OLGA SPARSCHUH (Munich) was concerned with evidence practices regarding competence for migrant qualifications. Since they are tied to national settings, academic and professional certificates, experiences from other countries were turned into contested resources. In the history of foreign qualifications in Germany over the 20th century the “Zentralstelle für ausländisches Bildungswesen” was a central actor performing and regulating the assessment of foreign qualifications.

Another vital set of questions was present merely in the background of the discussion: On what grounds can one criticise evidence critiques? What are potential roles of reflexive research in the humanities and social sciences for addressing the escalating evidence critiques today? The answer could simply be to continue working carefully, since philosophical, historical and sociological studies of science already constitute forms of analysing and criticising evidence practices in their own right. Beyond that, adaptations might harbour opportunities. Eva Barlösius emphasised that the rising economic stratification of society is a new driving factor for science denialism and advocated to change our view from science studies to (include) social structural analysis. Other contributions explored strategies on the boundary between science, arts and engagement.

Drawing on feminist science studies, MARTHA KENNEY (San Francisco/Munich) proposed speculative fiction as a tool to interrogate relations between narration and evidence in evolutional biology. She called for acknowledging storytelling as an important component of scientific practice and using it to generate alternative hypotheses. ELENA ROCCA and RANI LILL ANJUM (Oslo) work for more transparency through interdisciplinary scientific networks, in interaction with students as well as in cooperation with practitioners and patients. They use the concept of „philosophical bias“ for the explication and joint reflection of ontological claims on causality in evidence-based medicine. Carlo Martini experiments with debunking vicious evidence[2], in addition to developing tools and programs for critical thinking and spotting pseudoscience.

Altogether, the concept of evidence critique contributed to a more nuanced understanding of evidence cultures in at least three dimensions as summarised by Sascha Dickel: (1) Criticism of evidence drew on evidence itself, albeit evidence of alternative forms. This allowed for a sharpening of analyses by asking what kind of evidence is used in evidence criticisms. (2) Different levels of evidence critique could be distinguished. (3) The symposium shed light on ways in which various social systems deal with evidence and its criticism, for instance science and journalism. Future investigations might explore potential links between such levels and modes of evidence critique in different social systems.

Another area for further research concerns dynamics of evidence criticism inside and outside of scientific communities as well as their interdependencies. How are changes of evidence criticism within science related to possibilities of the institutionalisation of external critiques? How is evidence accepted or rejected in public discourses? Opportunities also exist in linking results to other debates in the history and philosophy of science, such as on epistemic justice, objectivity, legitimacy and trust. The sociology of controversies with its alternative conception of pluralism, future studies regarding the (prospective) governance of evidence regimes and recent accounts in Science and Technology Studies on everyday-workings of democracy might hold potentials for a fruitful exchange. A final area for further collective exploration is how evidence criticisms can be addressed as part of teaching, inter- and transdisciplinary cooperation, science communication and beyond.

*Conference overview:_

Karin Zachmann (Technical University of Munich), Mariacarla Gadebusch Bondio (University of Bonn): Welcome and Setting the Scene

Panel 1

Christoph Küffer (OST Eastern Switzerland University of Applied Sciences, Zurich): Alternative Practices of Evidence in Ecology amidst a Perfect Storm of Environmental Crises

Sarah Ehlers / Helmuth Trischler (German Museum and Rachel Carson Center, Munich): Complicating the Equation – Environmentalism and the Debate on Hazardous Pesticides in the Global South during the 1980s

Panel 2

Eva Barlösius (Leibniz University Hannover): Three Different Forms of Contestation – Questioning the Validity of Scientific Expertise

Stefan Esselborn, Karin Zachmann (Technical University of Munich): Evidence against the “Atomic State” – The Nuclear Energy Controversy and the Rise of “Gegenforschung” in West Germany (1970s to 1990s)

Panel 3

Carlo Martini (Vita-Salute San Raffaele University, Milan): Vicious Evidence – How Pseudoscience Fuels Evidence Criticism and Destabilizes Science

Andreas Wenninger / Kevin Altmann (Bavarian Institute for Digital Transformation, Munich) / Sascha Dickel / Michael Kitzing (University of Mainz): Alternativity or Incorporation – Institutionalisation of Citizen Science as Evidence Critique?

Panel 4

Edoardo Maria Pelli / Jutta Roosen (Technical University of Munich): The Relevance of the Moral Foundation Theory to Explore Consumer Interpretation of Contested Nutritional Evidence

Susanne Kinnebrock / Helena Bilandzic / Theresa Stahlhut (University of Augsburg): Stories about Villains, Mad Scientists and Failure? Patterns of Evidence Criticism in Media Coverage of Genomic Research

Panel 5

Saana Jukola / Mariacarla Gadebusch Bondio (University of Bonn): Caps, Scrubs and Trouble with Evidence – What the Debate about Surgical Wear Tells us about Guideline Development

Elena Rocca / Rani Lill Anjum (Norwegian University of Life Sciences, Oslo): Expanding the Meaning of Causal Evidence in Medicine – Challenges, Achievements and Future Perspectives of the CauseHealth Approach

Panel 6

Elif Özmen / Daniel Füger (Justus Liebig University, Giessen): Evidence between Contingency and Necessity – Some Philosophical Remarks

Martha Kenney (San Francisco State University / Technical University of Munich): How Many Plots Can the Data Hold? Reconciling Stories and Evidence in Evolutionary Biology

Panel 7

Ruth Müller / Mallory James (Technical University of Munich): Time as a Judgment Device – How Time Matters When Reviewers Generate Evidence for Assessing Applicants for ERC Starting and Consolidator Grants

Olga Sparschuh (Technical University of Munich): Evidence of Competence – Professional Qualifications of Migrants as a Contested Resource

Final Discussion

Evidence Criticism Reconsidered – What have we learnt?

[1] In its first three years of operation, the research group investigated concrete evidence practices, cf. Daniel Füger, Tagungsbericht: Practicing Evidence – Evidencing Practice. How is (Scientific) Knowledge Validated, Valued and Contested?, 19.02.2020 – 21.02.2020 München, in: H-Soz-Kult, 27.04.2020, (23.04.2021).
[2] Cf. Folco Panizza, et al., Can professional fact-checkers' techniques advance users' understanding of scientific content on social media?, (23.04.2021).

Tagungsbericht: Critiquing Evidence Criticisms: The Condition and Challenge of Evidence Criticisms for Democratically Constituted Knowledge Societies, 23.03.2021 – 24.03.2021 digital (Villa Vigoni, Menaggio), in: H-Soz-Kult, 26.05.2021, <>.