The focus of the international symposium was on application of mixed methods and triangulation in research designs in the discipline of History Education Research. It was conducted within the framework of the Austrian project “Competence and Academic Orientation in History Textbooks (CAOHT)”, in which a sequential qualitative-quantitative triangulation design is being applied to study the use of history textbooks in the context of history education in school on a large scale empirical project.1
In the keynote “Triangulation and Mixed Methods – two methodological concepts and debates and their interrelations” the internationally recognised specialist in the use of empirical methods UDO KELLE (Hamburg, Germany) introduced the participants into the sociological fundamentals of triangulation and mixed methods research. Kelle showed that the methodological concept of triangulation dates back to the 1950s and outlined the scientific debates since then. Different authors have proposed elaborations and modifications of the concept of triangulation so that an empirical researcher who wants to employ it may find very different understandings of the term. Kelle showed that a further debate about methodological triangulation between qualitative and quantitative methods started in the late 1980s leading to what is called “mixed methods research” today. Under this umbrella term a whole community of scholars discusses important aspects of mixed methods on international conferences related to that topic. In addition, a “Mixed Methods International Research Association“2 and an “International Journal of Mixed Methods Research” has been founded and established. Kelle clarified the terms ‘triangulation’ and ‘mixed methods’ and their different usages and showed the interrelations between both debates.
CARLA VON BOXTEL´s (Amsterdam, Netherlands) presentation about analyzing students' reasoning in the history classroom by using domain-specific and general educational theories showed how different theories (Theory Triangulation) can be used to study or analyze a certain phenomenon. Van Boxtel’s research combines domain specific theories on historical thinking and reasoning3, theories on domain specific expertise development, theories on argumentation in the classroom and theories on reasoning in speech in small groups or whole-classroom and in historical writing. She showed that theory triangulation helped to create more powerful coding schemes to capture the complexity of students’ historical reasoning. It allowed to gain a more diverse and comprehensive picture of what is needed to develop students’ historical reasoning ability with the help of triangulation of different theories.
The next contributor, ROLAND BERNHARD (Salzburg, Austria), talked about “Researching history education and historical thinking” focusing on the role of qualitative data in methodological triangulation design. In his contribution, Bernhard presented the research design of the aforementioned Austrian CAOHT-Project. This involved the participation of 1.000 students and hundreds of history teachers; the fieldwork was done in 27 different schools in Austria in 2016 and 2017. Bernhard pointed out the benefits of using mixed research to gain a deeper and broader understanding and corroborating evidence from combining qualitative and quantitative data. He also presented some preliminary results from participant observation in 50 history lessons in schools in Vienna that indicate that the textbook is used very extensively and in many different ways in history education in Austria. He also presented interview-data that showed that some of the approaches required in competence-oriented history education (historical thinking approaches) are seen as important for history teachers. Nevertheless, teachers often do not associate these approaches with “competence orientation of historical thinking”.
MARIO CARRETERO (Madrid, Spain) asked whether students’ historical ideas and history textbooks coincide. He presented a study in which both quantitative and qualitative methodologies were applied. The study was determined as an cross-national research about how one and the same historical topic is being presented in textbooks in different countries and how the same topic is represented by students.4 Therefore pupils from Chile, Argentina, Spain and Mexico were asked about their knowledge and interpretations of the famous engraving of De Bry in which Columbus is to be seen landing in Guanahaní. The results were compared to those of the textbook analysis concerning this picture. Carretero found a strong correlation between students’ ideas and those presented in textbooks for Mexico, Chile and Argentina, but not for Spain, and concluded that factors outside of schools – such as the public history about the national past – strongly contribute to students’ ideas.
The last presentation of the first day was held by CHRISTOPH KÜHBERGER (Salzburg, Austria), who presented a case study with a triangulated approach to measure the complexity of learning tasks in history textbooks. Therefore he developed a categorical framework, which includes different models of complexity, such as General Task Complexity (component and coordinative complexity), Linguistic Complexity and Domain-specific Task Complexity (i.a. performance level, lifeworld relations, domain specific concepts and competencies and multimodality, which describes the modal structure that creates historical narratives in textbooks5). Kühberger showed that the different models to represent the complexity of tasks lead to very different results, so that the chosen triangulated approach did not lead to a practical conclusion in this case study. Therefore, further differentiations of the analysis framework are needed. The contribution was followed by a controversial and productive discussion about what the term complexity in the context of historical learning tasks in general means, or could mean.
The second day of the symposium started with a contribution by MONIKA WALDIS (Aarau, Switzerland) about “The assessment of pre-service history teacher’s pedagogical content knowledge (PCK): Cross-validation of different assessment methods”. Waldis described an elaborated mixed methods approach to determine pedagogical content knowledge and presented some preliminary results of an ongoing project with 484 student teachers in six universities of teacher education in German-speaking Switzerland.6 They used open- and closed-ended questions and group discussions about video clips of history teaching and surveys and asked for principles and concepts that student teachers learned during initial teacher education. Waldis et al. found small changes in noticing and knowledge-based reasoning between the beginning and the end of the history didactic courses of student teachers. In this context Waldis sees mixed methods as a way to increase the validity of the research
CHRISTIANE BERTRAM (Konstanz, Germany) talked about how history didactics can meet computational linguistics and described five steps in a computer based evaluation of students’ text using methods of computational linguistics to analyze linguistic complexity and to assess the content in students’ writings. Bertram refers to the competence model of the Group FUER ‘historical consciousness’ (‘Geschichtsbewusstsein’), which structured historical thinking processes in a circular or spiral framework7, and held that the process of deconstructing a historical narrative and creating one’s own narrative about the past are considered to be central historical thinking competencies. Such competencies of historical thinking should generally be assessed by open-ended tasks. According to Bertram, these tests are often time-consuming to write and expensive to assess. Taking this as a starting point, she described how her research group tried to find a way to assess the linguistic complexity and content of students’ responses in an efficient and effective way.8
TERRY HAYDN (Norwich, England) showed in his presentation “Triangulation in history education research – and its limitations: a view from the UK”, how different forms of triangulation have been used in recent research. He discussed the strengths and weaknesses of different approaches to triangulation. Haydn especially focused on the limitations of triangulation as a means of making claims about the validity of research outcomes. In spite of the pronouncements of policymakers in the UK that education reforms will be ‘evidence based’, there are many examples of distortion and misrepresentation in the field of history education research. Haydn argued that without an underpinning commitment to veracity and respect for evidence, neither sample size, nor research approach, nor range of triangulation methods can ensure that reasonable claims are made for the outcomes of research.
DANIJELA TRŠKAN (Ljubljana, Slovenia) showed how methodological triangulation was used at her university to study the quality of teaching practice in history and to provide suggestions for improvements in planning, implementation and evaluation in the future. Trškan et al. used interviews, questionnaires, self-evaluation reports and documents and found that the teaching practice in history requires, on the one hand, a precise plan and a description of the tasks and requirements of all participants involved in the teaching practice, and, on the other hand, an examination of the level of competences acquired by students and the satisfaction of all participants in the preparation, organization, implementation and evaluation of the teaching practice.
The last contributor was BODO VON BORRIES (Hamburg, Germany) who talked about a big research project about “Ideal, reality, use and understanding of history textbooks” that he led in 2002.9 The study was mainly done by asking students (6th, 9th, 12th grade as well as teacher trainees) and teachers about many different aspects of history textbooks. In a quantitative approach, surveys for teachers and students were used and in a qualitative strand short essays and interviews were undertaken with some students after they had completed the questionnaires. The same was done with other students after video-documented history lessons with the use of textbooks. Thus, a type of “two-stage study” developed; individual persons (cases) can be combined with their position in a bigger group (micro and macro analysis). Von Borries reflected on the achievements and advantages of triangulation designs with different approaches that control each other. However, he also pointed towards the limitations of such (often expensive) research designs.
Altogether, the symposium provided an interesting insight into present methodological approaches in history education research. The contributions and discussions showed that, while still in an early stage, the acceptance and implementation of triangulated and mixed methods based research designs in history education have increased within the last years. Therefore, the small number of participants at the symposium provided much space– not only for a productive and pleasant atmosphere, but for insightful discussions of forward-looking research designs in the international field of history education research.
Conference Overview:
Moderation (11th of May): Christoph Bramann (Salzburg University of Education)
Christoph Kühberger (Salzburg University of Education): Opening and introduction
Udo Kelle (Helmut-Schmidt-University Hamburg): Triangulation and Mixed Methods – two methodological concepts and debates and their interrelations
Carla von Boxtel (University of Amsterdam): Analysing students' reasoning in the history classroom: using domain-specific and general educational theories
Roland Bernhard (University of Education Salzburg): Researching history education and historical thinking. The role of qualitative data in a Methodological Triangulation design
Mario Carretero (University Autónoma Madrid): Do students' historical ideas and history textbooks coincide?
Christoph Kühberger (Salzburg University of Education): Triangulation in History Education – a Case Study
Moderation (12th of May): Roland Bernhard (Salzburg University of Education)
Monika Waldis (University of Applied Sciences of Northwestern Switzerland / University of Teacher Education): Assessment of history teacher’s pedagogical content knowledge. Development of a videosurvey using cross-validation of different assessment methods
Christiane Bertram (University of Konstanz): History didactic meets computational linguists: The computer-based evaluation of student texts in the field of history
Terry Haydn (University of East Anglia): Triangulation in history education research and its limitations: a view from the UK
Danijela Trskan (University of Ljubljana): Methodological Triangulation for quality indicators of teaching practice in history. A new approach for evaluation of history pedagogical master study programme
Bodo von Borries (University of Hamburg): Experience with and reflection about Triangulation, considering a study on "Ideal, Reality, Use and Understanding of History Textbooks" (2002)
Notes:
1 CAOHT website: http://www.phsalzburg.at/index.php?id=1050 (20.07.2017).
2 MMIRA web site: http://mmira.wildapricot.org/ (20.07.2017).
3 For further information about the model of historical reasoning see http://www.uva.nl/en/profile/b/o/c.a.m.vanboxtel/c.a.m.vanboxtel.html#tab_2 (20.07.2017); see also Jannet Van Drie / Carla van Boxtel, Historical Reasoning: Towards a Framework for Analyzing Students’ Reasoning about the Past. In: Educational Psychology Review 20 (2008), pp. 87–110.
4 See also Mario Carretero, History Learning Research
in Spain and Latin America, in: Manuel Köster / Holger Thünemann / Meik Zülsdorf-Kersting (eds.), Researching History Education. International Perspectives and Disciplinary Traditions, Schwalbach/Ts. 2014, pp. 56–80.
5 For the concept of multimodality in history textbooks see also Christoph Kühberger, Intertextual and multi-modal construction of history via textbooks and its reception, in: Katja Lehmann / Michael Werner / Stephanie Zabold (eds.), Historisches Denken jetzt und in Zukunft. Wege zu einem theoretisch fundierten und evidenzbasierten Umgang mit Geschichte, Berlin 2016, pp. 67–81.
6 See also Monika Waldis / Martin Nitsche / Philipp Marti / Jan Hodel / Corinne Wyss, „Der Unterricht wird fachlich korrekt geleitet“ – theoretische Grundlagen, Entwicklung der Instrumente und empirische Erkundungen zur Unterrichtsreflexion angehender Geschichtslehrpersonen, in: Zeitschrift für Geschichtsdidaktik 13 (2014), pp. 32–49.
7 Andreas Körber / Waltraud Schreiber / Alexander Schöner (eds.), Kompetenzen historischen Denkens. Ein Strukturmodell als Beitrag zur Kompetenzorientierung in der Geschichtsdidaktik, Neuried 2007.
8 See also Christiane Bertram / Wolfgang Wagner / Elisabeth Schaser, Historische Kompetenzen mit offenen Antwortformaten messen – Eine Studie auf Basis der „Sechser-Matrix“ des FUER-Modells, in Monika Waldis / Béatrice Ziegler (eds.), Forschungswerkstatt Geschichtsdidaktik 13. Beiträge zur Tagung „geschichtsdidaktik empirisch 13“, Bern 2015, pp. 165–180.
9 See also Bodo von Borries / Claudia Fischer / Sibylla Leutner-Ramme / Johanes Meyer-Hamme, Schulbuchverständnis, Richtlinienbenutzung und Reflexionsprozesse im Geschichtsunterricht. Eine qualitativ-quantitative Schüler- und Lehrerbefragung im Deutschsprachigen Bildungswesen 2002, Neuried 2005.