Comparisons represent a core operation of the production of meaning and social order. Although of general social relevance, comparisons are of particular importance in academia, where comparisons represent both a meta-method and a social practice. The ascent of rankings, altmetrics and other comparative devices suggests that comparisons have lost their innocence as a scientific meta-method and are becoming increasingly important as a ubiquitous social practice that has turned onto academics themselves. Reflecting on this development, the participants of our international and interdisciplinary symposium discussed transformations and continuities of academic comparisons from an empirical and methodological perspective in four panels: 1. What can be learned from historical perspectives on comparisons? 2. What is the interrelation between the comparator and the compared? 3. How do devices and instruments of comparison operate and how can they be compared to each other? 4. What are the consequences of these comparisons?

In our introduction, we made four provocative propositions to instigate discussions and provide reference points for the following contributions. Regarding the history and trajectories of comparisons, we argued that we are in an “Age of Comparison 2.0.” In this new age, comparisons go beyond a mere methodical operation and strike back onto academia. In contrast to comparisons that are applied by researchers, comparisons that are applied to academia often follow a meritocratic ideology. Concerning the interrelation of the comparator and the compared, we suggested that the comparators within and of academia are changing. Within academia, comparative research is increasingly drawing on digital databases, and conducted in research networks that collaborate globally. Comparisons of academia are increasingly conducted by international media and (educational) companies which establish data hubs to compare academic practices and actors. With view on the devices of comparison, we proposed that digitalization leads to an extension, acceleration and deepening of comparative horizons due to new technical possibilities of data collection. Regarding the consequences of comparison, we argued that an increase of evaluative and hierarchical comparisons would lead to assimilation, standardization, and convergence of the compared entities.

The introduction was followed by a panel discussion by TOBIAS WERRON (Bielefeld University), ANGELIKA EPPLE (Bielefeld University), MERLE HUMMRICH (Frankfurt University) and BERNHARD EBBINGHAUS (Oxford University). The panelists lined out their respective notions of comparison and took up the proposition of a “New Age of Comparison.” Epple and Werron warned from a historical perspective against simplifying assumptions of uniform epochs and one-directional development lines. Yet, the discussion emphasized some genuinely new aspects of comparisons. For example, digitalization has revolutionized the means of conducting large scale comparisons and the availability of data. Another new development is that data access has shifted from scientists to companies, who gained access to large datasets and can compare on an ongoing basis. The aforementioned shift leads to a third aspect which may not be entirely new, but appears in a new light: Authority on the significance of datasets, the appropriateness of comparative methods or the suitability of measures is not exclusively in the hands of academics anymore. In particular, the increase in commissioned research by political actors leads to the establishment of key metrics in comparisons.

The introductory session was concluded with a keynote by DAVID FRANK (University of California, Irvine). Firmly rooted in sociological neo-Institutionalism, Frank argued that universities act as anchors of a modern cultural framework. Universities’ worldwide diffusion would foster universalization and thus level differences. First, differences between universities at the nation-state level would disappear due to their global spread. Second, increasing participation rates in academic education would make differences between individuals disappear. Third, differences in cultural domains would make way for homologated university knowledge. The university has colonized society, Frank concluded, “but we don’t mind, because we like universities – we don’t think of them as invasive species!”

In the first contribution on historical perspectives on comparisons, TOBIAS WERRON and JELENA BRANKOVIC (Bielefeld University) called for a study of comparisons as an establishment of relationships, which they illustrated with their research on rankings. They claimed that to understand the present of rankings, one would need to reconstruct their history. However, sociology analyses the consequences of rankings as comparative devices, but mostly ignores their historical origins and trajectories. Werron and Brankovic used the examples of rankings in arts and sports to show that the emergence of rankings and their establishment would be facilitated by existing discursive and media infrastructure as well as the agreement of the illusion of the field with performance-based comparisons. ANGELIKA EPPLE and WALTER ERHART (Bielefeld University) also followed a historicizing perspective. They aimed to merge methodological and empirical perspectives on comparisons by establishing a notion of comparison as a practice. In doing so, they introduced fundamental analytical components: two comparata, a tertium comparationis, and a situated actor. Epple and Erhart defined comparing as a productive activity which includes negotiations between differences and similarities and ultimately changes the object of comparison. Instead of assuming an increase in comparisons, they proposed to use the concepts of regimes of comparison for analyzing comparability within specific historical contexts. ANNETTE SIMONIS (University of Gießen) concluded the first panel with a view from comparative literature. She was concerned with a methodological trajectory, namely the ascent of the field of comparative media studies. In contrast to traditional notions of texts and literature, a media comparative perspective is sensitive to the dynamics and co-evolution of media phenomena, for example, regarding the spatial presence and dimensionality.

The second panel was concerned with the relations between the comparator and the compared. RUTH MÜLLER (Technical University of Munich) approached this topic by introducing a new comparator on the academic landscape: the European Research Council (ERC). Describing how curricula vitae (CV) are compared and evaluated at the ERC’s peer review processes, Müller discussed two aspects of temporality. First, the temporal dimension of the life course, whereby supposed excellence is subject to normative assessments. Second, the time pressure on the decision-makers, which leads to pragmatic decisions. In CVs, there is a linear and uniform model of academic careers embedded that perpetuates bias. MICHAEL GUGGENHEIM (Goldsmiths, University of London) raised methodological questions about good comparisons and called for overcoming the methodological obsession with comparability. Researchers do not only create comparison by choosing comparative units, they also construct those units in the first place. Good comparators would not necessarily make good comparisons, because they would acknowledge asymmetries between cases and reflect their imperfect operations. By reflexively using constraints, much could be gained through unusual comparisons like hypothetical or asymmetrical comparisons. In the third contribution, JUSTIN POWELL and MARCELO MARQUES (University of Luxembourg) focused on the Research Excellence Framework (REF) in the UK and the implications that REF-based media rankings have on the organizational behavior of departments of educational research. They argued that ratings and rankings trigger strategic behavior among departments, which use ratings and rankings as marks of distinction to secure or strive for reputation within their environment.

The first day ended with a poster session. Seven early career researchers presented posters and pitched their research. The session facilitated international perspectives on new comparative devices, comparative research, and different aspects of comparisons in academia.

BERNHARD EBBINGHAUS (Oxford University) opened the second day with a keynote on the relation between typologies and comparisons. He argued that we have to question the generalizability of typologies if they are used as “inductive instruments” for the classification of observations or developments. When they are used as “deductive instruments” in the form of constructed models for causal explanations, we have to reflect on how they refer to reality and whether they can explain it. Ebbinghaus explained these objections based on the welfare state typology, which are often criticized for being too general, too holistic and too static. Additionally, they are based on the assumption of path dependency and stability of the regimes and some aspects are underdeveloped (gender, health care, family policy). All of this raises questions about the relationship between typologies and empirical cases.

The third panel focused on devices and instruments of comparison. MICHAEL SAUDER (University of Iowa) stressed that numbers are typically taken at face value while underlying subjective decisions are ignored. This is remarkable because quantitative comparisons, in particular, exert different forms of pressure. To understand those forms of pressure, we need to differentiate between forms of comparison and ask questions like: Are they rankings or ratings? How are they presented, which types of information do they use and which power and legitimacy do the rankers have? WOLFGANG KALTENBRUNNER (CWTS, Leiden) discussed academic CVs as devices of comparison in peer review processes. He argued that referees use CVs as judgment devices and interpret them against the backdrop of their own academic work experience. Because referees have diverging experiences, different evaluative registers exist. Kaltenbrunner emphasized that CVs are a malleable infrastructure that dynamically changes with the development of academic research. MARTINA FRANZEN (WZB Berlin Social Science Center) introduced altmetrics as digital devices of comparison. Altmetrics widen the scope of quality measurement as they include citations from non-academic sources like Wikipedia or Twitter. She argued that the way research quality is measured impacts the motivational structure of research. Thus, the use and diffusion of altmetrics raises the question of whether they contribute to changes in the academic social order.

The final contributions were devoted to the consequences of comparison. LARS ALBERTH (Leibniz University Hannover) presented his empirical research on performance-based allocations in universities. Universities define performance criteria as they rate the importance of, e.g., teaching or research and evaluate and compare their organizational members through such criteria. He showed that they establish new inequalities and justify these inequalities by demonstrating their appropriateness and the correctness of the comparisons. In a conceptual contribution, FRANK MEIER (Helmut Schmidt University, Hamburg) challenged the proposition that comparative judgments are a device of control. He emphasized that people react differently to their measurements. Reactivity should not be seen as a deterministic force, as the world is full of judgments people do not care about. Thus, he called for a differentiated analysis that also considers the social structure of a given field to understand people's reactions. Meier suggested that we should study (non-)reactivity and decision-making problems by examining evaluative constellations.

In a closing panel ANDREA MENNICKEN (London School of Economics and Political Science, London) and MARTA NATALIA WRÓBLEWSKA (National Centre for Research and Development, Warsaw) commented on the past two days. Mennicken pointed to some omissions from an economics perspective. In particular, she reminded of the practice of counting as a form of comparison and of the role of money and prices. Money implies comparability as it negates qualitative differences among objects and relates everything to one price scale. Mennicken was surprised not to hear more about power relations, particularly because the symposium was based on the assumption that comparisons are not neutral. Wróblewska reflected on the double role of comparison in academia based on her own double role: She works in a R&D research council and thus much of the criticism articulated against non-academic comparators also concerns her work. From this position, Wróblewska raised the question if it is enough to criticize non-academic comparators or whether academic actors should not try to inform and influence policymakers when the impact of evaluations is seen as problematic.

We would like to highlight two questions that have been raised by the conference and that deserve further attention: Do we indeed witness the dawn of a “New Age of Comparison?” However the answer might look like, the discussions underlined that we need to study the paths of the (social) history of comparisons to understand their present use and (social) impact. Although there is a lot of continuity, it seems like a new quality of comparisons is indeed introduced by new non-academic comparators and their devices of comparison. The proliferation of such devices requires us to reflect on their use in our (comparative) research. Are comparisons in academia really a special case, or are they just social practices as in many other fields? The discussions conveyed that academic comparisons are not necessarily more rational and controlled compared to comparisons in other social fields. Such a perspective would call for an investigation of comparative methods as a social practice.

Conference overview:

Introduction: Comparing and being Compared. Academia and the Double Role of Comparison
Anna Kosmützky (LCSS, Hanover) / Julian Hamann (LCSS, Hanover) / Frerk Blome (LCSS, Hanover)

Opening panel: From Assessment to Quantification – On the Analytical Leverage of “Comparison” as a Term and Concept:
Chair: Anna Kosmützky (LCSS, Hanover)

Bernhard Ebbinghaus (Oxford University) / Angelika Epple (Bielefeld University) / Merle Hummrich (Goethe University Frankfurt) / Tobias Werron (Bielefeld University)

Keynote I
Chair: Julian Hamann (LCSS, Hanover)

David J. Frank (University of California, Irvine): The University in Comparative Perspective: Sameness and Difference

Panel I: Histories and Trajectories of Comparison
Chair: Jennifer Dusdal (University of Luxembourg)

Tobias Werron (Bielefeld University) / Jelena Brankovic (Bielefeld University): Where Do Rankings Come from? The Emergence and Institutionalization of Rankings in Different Societal Fields

Angelika Epple (Bielefeld University) / Walter Erhart (Bielefeld University): Is There an Age of Comparison? Comparative Practices from Antiquity to Present Times

Annette Simonis (University of Gießen): From Comparison of Arts to Comparative Media Studies

Panel II: The Comparator and the Compared
Chair: Eva Maria Vögtle (DZHW, Hanover)
Commentator: Anne K. Krüger (HU, Berlin)

Ruth Müller (Technical University München): Bigger, Better, Faster, More? Narratives of Excellence in Contemporary Academia

Joe Deville (Lancaster University) / Michael Guggenheim (Goldsmiths, University of London): How to Make Good Comparators

Justin Powell (University of Luxembourg) / Marcelo Marques (University of Luxembourg): From Ratings to Rankings: Research Evaluation and Strategic Organizational Actors

Poster and Pitche(r)s Session: PhDs and Post-Docs Presentations
Chair: Frerk Blome (LCSS, Hanover)

Anoud Abusalim (Lancaster University): Disciplinary Writing Practices in Social Sciences and STEM

Aliakbar Akbaritabar (DZHW, Berlin): A Spatial Scientometric and Network Analysis of Higher Education Research and Science Studies (HERSS)

Tamara Dagen (University of Zagreb): The Impact of Globalization on the Internationalization of European Public Universities

Marita McGrory (Europa-Universität Flensburg): How do the International and National Students‘ Success Rates Compare in Germany’s Higher Educational Institutions?

Hanne Kvilhaugsvik (University of Bergen): Re-connecting to the Labor Market? Labor Market Panels for Higher Education in Denmark and Norway

Marianne Noel (Université Paris-Est): Three Tales of Marketization. The Role of Publication Programs in the Entry into an Era of Post-Comparison in Chemistry

Pedro Pineda (University of Kassel): Studying Worldwide Diffusion of Teaching Evaluations through Case-Centered and Variable-Centered Comparisons

Keynote II
Chair: Anna Kosmützky (LCSS, Hanover)

Bernhard Ebbinghaus (Oxford University): Are Welfare Regime Typologies an Ideal or Realistic Comparison of and for Public Policy?

Panel III: Devices and Instruments of Comparison
Chair: Tim Seidenschnur (INCHER, Kassel)
Commentator: Torger Möller (DZHW, Berlin)

Michael Sauder (University of Iowa): Rankings as Devices of Comparison

Wolfgang Kaltenbrunner (CWTS, Leiden): Filling in the Gaps: The Interpretation of CVs in Peer Review

Martina Franzen (WZB Berlin Social Science Center): Measuring Scholarly Impact. Altmetrics as Devices of Comparison in the Digital Age

Panel IV: Consequences of Comparison
Chair: Stephanie Beyer (LCSS, Hanover)
Commentator: Terhi Nokkala (University of Jyväskylä)

Lars Alberth (University of Hannover): Universities between Comparability and Incomparability

Frank Meier (Helmut Schmidt University, Hamburg): Competition between Higher Education Institutions

Final discussion and closing remarks:
Andrea Mennicken (LSE, London)
Marta Natalia Wróblewska (Nation Centre for Research and Development, Warsaw)