Finding the needle in the haystack: archival research in European political science

Abstract

This short article offers a practical introduction to archival research for political scientists working on European politics. Archival documents are increasingly recognized as a relevant data source for process tracing analyses in small-N or mixed methods studies. Previously classified archival documents are exceptionally trustworthy due to their original confidentiality. Their rich and detailed content facilitates the understanding of causal mechanisms. Still, the hurdles for working with archival sources are high for political scientists. Lack of experience, no special training in handling historic documents, and a shortage of textbooks meeting their demands are a few of the problems political scientists planning archival research face. In the article, I highlight the opportunities of archival research and demonstrate how challenges can be overcome. I emphasize that the archival field trip should only be planned once researchers have gained substantive context knowledge. In their preparation, researchers should use all the resources archives offer and develop measurable expectations from theories.

Archival research in European political science

Political scientists, particularly those interested in historical cases, increasingly recognize the enormous potential of archival research. The evidence derived from the analysis of archival documents is well suited to inform process tracing analyses. Process tracing is the dominant choice in the discipline for small-N studies and the qualitative part of mixed methods studies, which continue shaping the evolving landscape of political science research. Political science has overcome dogmatic debates between small and large-N research and recognized the inherent benefits of combining both (Humphreys and Jacobs 2023). Researchers particularly value the opportunity to generate internally and externally valid results with one study (Brady and Collier 2010; Dunning 2015). Due to its significance, process tracing is becoming increasingly sophisticated (Beach and Pedersen 2013; Bennett and Checkel 2015; see also Fairfield and Charman 2022 on formal Bayesian process tracing; Humphreys and Jacobs 2023 on integrating qualitative and quantitative inferences).

Nevertheless, scholars still encounter practical issues. One relates to collecting and analysing causal process observations from archival research. Even though archival files are increasingly available online, the digitization of content is still in the early stages and incomplete, so field trips to archives remain essential. Researchers relying exclusively on digitized content risk bias resulting from convenience sampling (King et al. 2021).

Compared to their potential fellow visitors in the archival reading room, political scientists usually have less experience handling archival primary sources. While a political science program’s curriculum typically includes, for example, software programme training or qualitative content analysis, dedicated courses on archival research are missing (Frisch et al. 2012a, b). This leads to several problems once a research puzzle has been identified to which the answer might lie in an archive. Historians can count on their undergraduate training or a range of guiding textbooks when approaching an archive. On the contrary, political scientists lack experience and cannot rely on routines for archival fieldwork (Cook and Schwartz 2002). Existing textbooks from and for historians on archival research do not address the demands of political scientists as they, for instance, do not reflect on the role of theory. While there is a small range of publications specifically devoted to archival research for political scientists, this literature in comparison with other methodological approaches remains underdeveloped. These nevertheless helpful publications are either broad and unspecific or focused on US politics and work in US archives (cf. Frisch and Kelly, 2012b; Kapiszewski et al. 2015). For the field of European politics,¹ therefore, subject-specific guidance is missing. As a result, scholars might abandon their research projects or turn to more familiar data sources. If they still go on the archival field trip, it might become an inefficient adventure with poor results.

This short article is directed to political scientists in the field of European politics for whose projects archival research might be a worthwhile endeavour.² I discuss why and when political scientists should consider a field trip to an archive and offer practical guidance for planning and executing the field trip. In essence, successful preparation comes down to fulfilling three tasks. Even though I focus on European archives, the guidance is to a large degree also helpful for researchers in other subfields.³

The article proceeds as follows: First, I consider different the forms of data that political scientists typically rely on and discuss possible bias. Then, I demonstrate how archival documents once they have been declassified can avoid many of those limitations. After that, I turn to practical matters that scholars should consider when preparing their archival field trips. I emphasize the importance of theory for organizing archival research. Finally, I discuss typical challenges related to the data collection and analysis of archival files and also address the opportunity of using archival material for text-as-data approaches. In the last section, I summarize the article’s implications before I end with an outlook on the future of archival research in European political science. The article is inspired by field trips I made to different archives such as the Historical Archives of the European Union (HAEU) in Florence, Italy, the National Archives of the UK (NAUK) in London, and the German Federal Archives (GFA) in Koblenz, Germany. For the sake of illustration and to emphasize the article’s practical element, I frequently refer to examples from my own research and the field of European politics more broadly.

Field trip to an archive: why and when?

Many political scientists share a strong interest in history. There is a long tradition of studying historical events’ impact on contemporary political phenomena (Putnam et al. 1993; Simpser et al. 2018). Recent work demonstrates that visiting the reading room of an archive is not a necessary condition for publishing on historical topics. Quantitive-minded scholars provide new historical insights through innovative statistical methods (e.g. Bueno de Mesquita and Bueno de Mesquita 2023; Gingerich and Vogler 2021). Despite their advantages, these approaches cannot inform us about the “cogs and wheels” (Elster 1989, p. 3) generating an outcome. If a researcher aims to understand a causal mechanism completely, she must use a process tracing approach (Beach and Pedersen 2013; Bennett and Checkel 2015; Fairfield and Charman 2022).

Archival documents are, of course, not the only type of data typically used for process tracing. Authors consider interviews, memoirs, speeches, newspapers, and secondary literature, which can be analysed with considerably less time and resources. Nevertheless, the evidence from archival documents will most of the time prevail and significantly improve the quality of political science research (Darnton 2018; Frisch et al. 2012a, b; Larson 2001, 2017).

No matter which type of data one draws on in the analysis, reflecting on potential bias is an inevitable part of the research process. This is particularly crucial for interviews, the most popular form of political science field research (Kapiszewski et al. 2015, p. 190). Bias can be introduced by both the interviewer and the interviewee and  impact the validity and reliability of a study’s results (Berry 2002). The selection of interview partners already involves difficulties. When analysing a bargaining process, for instance, researchers should interview both the “winners” and the “losers” of the negotiations. An unbalanced selection of interviewees from both camps would significantly bias the results and should, therefore, be avoided (Beach and Pedersen 2013, pp. 134–136). In many cases, the “losers” might be unavailable for an interview about their defeat so the results will suffer from “non-response bias” (Kapiszewski et al. 2015, p. 219). The personal characteristics of the researcher herself also matter and inhibit interviewees from expressing themselves freely. This affects the validity but also reliability of the results. Another researcher conducting the interviews might get very different responses, for instance, due to ethical issues (Berry 2002; Kapiszewski et al. 2015, pp. 190–223). In terms of validity, the content created from interviews does not represent facts. Instead, the statements of interviewees often contain untruths, whether intentional or not. If intentional, interviewees might want to present themselves in a good light. On the contrary, unintentional error can be caused by the interviewee who has never known the truth and can, therefore, not speak the truth. Moreover, vanishing memory can cause incorrect statements (Mosley, 2013, pp. 14–28). Imagine you plan to conduct interviews to study the role of the Delors-Commission in the creation of the European Monetary Union (EMU). In fact, the results of interviews with involved decision makers will have a lower validity nowadays than if the interviews had already been conducted in the 1990s. Moreover, important decision makers, such as the late Delors, might already have passed away, making interviews problematic as the more distant the events date back.

Other types of data frequently used as evidence for process tracing analyses also tend to be biased. Relying exclusively on publications by historians is problematic for several reasons. Historians have little interest in theoretical explanation or generalization and, therefore, omit relevant documents (Larson 2001, 2017). Apart from being selective on the primary sources, the secondary literature includes bias as history is explained from the author’s point of view (Lustick 1996; Thies 2002). Memoirs tend to be subjective and full of self-praise. Like speeches,⁴ they are directed to the public and may, therefore, leave out essential details. Newspapers are informative but, after all, commercial products in which journalists potentially exclude information lacking attractiveness to the reader (Beach and Pedersen 2013, pp. 132–143).

Even though archival research avoids many of these problems, it is no “panacea” (Darnton 2018, p. 125). The type of documents stored in archives is diverse and can broadly be grouped into public, private and classified.⁵ This categorization is well illustrated by the example of the Historical Archives of the European Commission (HAEC).⁶ The HAEC holds items such as interviews by Commissioners, their newspaper appearances, or speeches that are meant to be publicly available. Whereas this data can be informative, it can include similar bias as discussed previously. This is different for private documents such as private letter exchanges between Commissioners or transcriptions of phone calls. Private documents have no legal status or classification but are usually not proactively made public unless their authors or recipients choose to do so, for instance, in their memoirs. Documents such as internal Commissioner briefings, memoranda, or non-papers are usually classified by the EU legislation as restricted, confidential, secret, or top-secret. As a common rule, classified EU documents remain under lock and key for 30 years.⁷ This period might be extended if the documents relate to strictly personal information. The exact classification period is subjected to a country’s legislation and, therefore, varies across archives.

Checking the status of declassification is obligatory for researchers to ensure that files of interest are actually accessible. The archives will not be open for many issues that political scientists are interested in such as, for example, the negotiations preceding the EU’s Treaty of Lisbon 2009. In this case, all classified and the majority of the private documents will remain locked up in the archives for at least the next 15 years. Thus, researchers must rely on other data sources or be patient and wait another 15 years.

Assuming the common 30-year rule, a record from 1 December 1989 will still not become automatically accessible on 1 December 2019. Declassification is no automatized process and requires archivists to check the documents before they are made available. Since the number of documents in archives is large and the size of staff is relatively small, this might take another couple of years. From my experience, this process can be accelerated by requesting access to specific documents, but policies differ from archive to archive. There can also be dedicated committees for making declassification decisions on an ad-hoc or regular basis such as the US Advisory Committee on Historical Diplomatic Documentation.⁸ Declassification is not always a routine process, but it may also become politicized (Trachtenberg 2020). A striking example is the Stasi Records Agency that was installed by the German parliament to manage the release of millions of documents the former East German secret service had collected.⁹

Once the documents of interest have been declassified, scholars can make use of that data and significantly improve the quality of their research. Previously classified documents, and to a large extent also private documents, are trustworthy and constitute “hard primary sources” (Moravcsik 1998, p. 80). Due to their original confidentiality, they are less likely to be biased. ”Documents after all, are generated from a government’s own internal purposes, and what would be the point of keeping records if those records were not even meant to be accurate?” (Trachtenberg 2006, p. 147). Hence, archival documents should be consulted whenever possible as they permit an unbiased look into the functioning of state bureaucracies, ministries, or international organizations (Frisch et al. 2012a, b; Larson 2001; Lieshout et al. 2004; Shifrinson 2022).

Depending on the research topic, different types of archives can be of interest. Typical archive holders are states, cities, international organizations, and foundations, but also former state officials often create archival collections with private documents after leaving office.¹⁰ In fact, archives are similar to gold mines in which you cannot simply walk in and pick up the treasure. In the following, I will provide practical advice on how the mining of evidence in an archive will be successful.

Preparing for the field trip: how to find the needle in the haystack

Archival research is often compared to searching for a needle in the haystack. This relates to the overwhelming number of documents that are stored in archives. The NAUK’s collection alone consists of more than 11 million files. For sound reasons, archives prefer to measure their records in kilometres. Eventually, it can be one or a handful out of 11 million records speaking as evidence to a research question. In the following, I introduce three tasks that every political scientist should complete in the preparation for their archival field trip so that the needle in the haystack will be found.

The first task is to acquire extensive background knowledge. The reading room of an archive should only be entered if the researcher has become an expert on the case. A good strategy for this is studying the secondary literature.¹¹ Most likely, historians or sometimes social scientists have already written about your topic or at least touched on related issues. For my research on the history of the European integration, a large corpus of literature is available in which historians deal with the different enlargements of the EU. Even though this literature is rather descriptive and does not directly address my research questions, this existing scholarship provided me with essential background knowledge. Furthermore, studying the secondary literature facilitates narrowing down the general research interest into a concrete research question. Of course, there are other ways of familiarizing oneself with a research topic, such as interviews at a preliminary stage of research (Lynch 2013, pp. 34–35). Helpful, are also collected volumes that are occasionally published and contain a selection of private or classified documents. For my research, several of those were available, such as a special issue published by the UK’s Foreign and Commonwealth Office (FCO) (Salmon et al. 2010). Special issues like these can be very beneficial while preparing for an archival stay. Moreover, they facilitate the identification of relevant holdings. Nevertheless, it is not advisable to entirely build on these collected volumes since the selection of documents for these volumes should raise concerns. Indeed, publishers have intentions. Why should the FCO, for instance, publish files that show it in a bad light?

After a rich understanding of the research topic has been gained and the research question has been narrowed down sufficiently, researchers can move on to the second task of the preparation stage and select the archives to visit. Since field trips to archives are time and resource-intensive, most researchers must focus on the most relevant ones for their research question. The best strategy to identify archives, whose reading rooms are worth travelling to, is to approach the archives directly by sending them a research inquiry. This is a formal request for some archives, and for others, an informal e-mail. The message should specify where the research interest lies and how the research would benefit from visiting the archive. After some time, the request will be answered by an archivist. Typically, requests showing that the author has informed herself about the archive’s holdings receive quicker responses. In my experience, you should keep the connection to the archivist answering your inquiry. The archivists usually check the documents before declassification and sort them into holdings. Thus, they have an excellent understanding of what to find where. In some archives like the HAEU, archivists are also researchers themselves and might recall particularly insightful holdings. Furthermore, researchers should engage with archives’ online catalogues. Even though archives’ catalogues may not have the quality of big search engines, they allow for browsing the holdings to determine whether the archive stores relevant files. Searching through online catalogues is also a great tool to shrink the metaphorical haystack of archival documents, as an example from the GFA demonstrates. The files in the GFA’s online catalogue sum up to more than 500 km. For my research, I was interested in the implications of the German unification on the European integration process. After searching for “German Unification”, the database indicates more than 1000 files. This is still a massive haystack as a file consists, on average, of several hundred documents. By specifying the search with “German Unification European Integration” the number of files reduces to 180. With added specifications, the number of files will further shrink and make the haystack manageable.

The final task completing the preparation phase of the archival trip relates to the role of theory. Contrary to historians, political scientists aim to generalize, meaning that theorizing is an essential part of their research (Larson 2001). Beach and Pedersen (2013, p. 137) advocate a “theory-driven search for data”. Before going to an archive, a political scientist should already have engaged with theory and derived expectations that can be tested with the documents held in the archive.¹² Prior to each of my field trips, I had studied the theoretical literature and had a list of process expectations ready. Central to my research on European integration was Moravcsik’s (1998) theory of liberal intergovernmentalism. From this theory, I derived the process expectation that states form their preferences on economic considerations. In more general terms, I assumed that if the theory is true, I should expect to see X, Y, and Z. Having these process expectations or observable implications at hand, facilitates the identification of relevant documents since you know what type of evidence to search for.

To prevent the creation of bias already in the data collection process, process expectations from rival theories must be considered. Including alternative explanations and rival hypotheses already in the data collection stage follows the Bayesian approach that also encourages researchers to think about the probability of a piece of evidence and currently update these probabilities during the research process. In order words, researchers should reflect on the “prior” and “posterior probability” of observations and engage in “Bayesian updating” (Fairfield and Charman 2022, pp. 3–4). While theory and observable implications are essential for political scientists, they should remain open to inductive insights. Archival research often includes serendipities and leads to rather unexpected discoveries (Frisch et al. 2012a, b; Kapiszewski et al. 2015, pp. 18–19). In this case, scholars should not see their previously specified research strategies as “straitjackets” (Trachtenberg 2020, p. 140) but instead engage in “soaking and poking” (Bennett and Checkel 2015, p. 18), meaning that they should be open to inductive insights and update general research strategies including theoretical assumptions and observable implications.

BOX 1: Tasks to complete in preparation of the archival field trip
• Task 1: Acquire case-specific background knowledge
• Study secondary literature, interviews, and special collections
• Task 2: Select archive(s) with relevant holdings
• Send research inquiries to archives
• Contact archivists
• Browse online catalogue
• Task 3: Engage with theory
• Make a list of process expectations
• Incorporate that observable implications have different likelihoods
• Under rival expectations
• Leave space for rival expectations and inductive insights

Typical challenges in the reading room

Before entering the reading room, visitors are asked to pass a security check and place all their working materials in a transparent plastic bag. Then, visitors are free to enter, and after some initial disorientation, political scientists will find out that there is no reason to feel alien among their peers in the reading room. At the latest, after having a document on the desk that carries a President’s signature, every political scientist will be under the spell of history. Even though the reading room is video monitored and full of regulations, most archives no longer oblige visitors to wear white gloves and permit photographing or scanning the files. This is a great help since it allows scanning many of documents and analysing them back in the office. On the other hand, it includes the danger of tireless scanning and filling up the online cloud with documents that are actually not relevant. I recommend skimming documents and scanning them only if they appear relevant. This way, it is possible to go through 20 to 25 files daily. This rule of thumb should, however, be treated with caution. First, the volume of files varies greatly. I had files on my desk with less than ten pages but also ones with more than 1000 pages. Second, a file might be so interesting that it requires hours to skim and scan, whereas on other occasions, a file is understood as “irrelevant” after 5 min. Researchers should pre-order documents before their trip to avoid time in the archive if possible.

Depending on the length of the field trip, the number of scanned documents rapidly grows to a large size. To avoid confusion, researchers should, therefore, develop an efficient system to keep track of the files that they have examined. There are special programmes available such as Obsidian or Devonthink, but a simple Excel file will also suffice. Every scanned document should be included in the database with its title, keywords, and a short summary. This facilitates the analysis of the documents back in the office.

The mere scanning of documents takes on a different meaning if researchers employ a text-as-data approach for which archival documents are suitable (Grimmer and Stewart 2013; Wilkerson and Casas 2017). Advances in optical character recognition (OCR) allow importing scanned documents directly as computer-readable text and facilitate the quantification of archival content in datasets. There are different software and apps available such as Adobe, Abby Finereader, or Readiris. Some of them, such as Transkribus, can also read historic handwritten documents. OCR is already well established among scholars working with archival documents as it tremendously reduces the time and resources otherwise required (Bukhari et al. 2017; Martínek et al. 2020). Moving to data analysis, scholars can then use the gained large corpus of text for rather simple exercises such as word counts but also more complex endeavours (Egami et al. 2022). With constantly evolving AI-based large language models, text-as-data approaches are becoming ever more sophisticated (e.g. Gilardi et al. 2023). This also opens new avenues for archival research. No matter which of these avenues is taken, the analysed documents have to be selected carefully and comprehensively to avoid selection bias.

No matter to which end, working in an archive is also learning by doing so that a routine will be quickly developed. Still, some challenges may occur in the reading room. Firstly, it is not possible to examine every file you wish. This might be due to time constraints, or the limit of files that can be ordered in a day might be reached. I have also experienced that documents were unavailable because they were being restored or borrowed by a ministry. In addition, the archivist checking the record after declassification can consider it “unworthy of archival preservation”.¹³ Furthermore, records can be incomplete, lost, stolen, or hidden in a former President’s bathroom. If we assume that the documents were destroyed intentionally, the findings of an analysis built on archival research should be interpreted with caution. Even though archival documents (private and classified) are on the whole less biased than other types, this does, of course, not mean that they are unbiased. Take for instance a European Council meeting that a researcher might be interested in. Usually, the minutes are the only available record of the meeting and constitute a reliable source. However, when analysing the minutes, one must not forget that the document’s author might have had intentions such as, for example, leaving out the most controversial parts or downplaying heated exchanges. Moreover, the minutes will most likely not capture the emotions participants expressed which might significantly alter the interpretation of the meeting. When analysing the documents, it should also be kept in mind that participants sometimes do not express their actual opinions due to instrumental motivations or are hesitant because they know that the minutes will eventually be declassified.

A common phenomenon of archival research is called “e silentio evidence” (Beach and Pedersen 2013, p. 126; Bennett and Checkel 2015, pp. 17–20). It relates to missing crucial information in a document or the absence of a document concerning salient issues. Even if a document is absent or does not mention something, it might still have inferential value. An example from my own research relates to the “10-point-programme” presented by Germany’s Chancellor Kohl as a major advance towards Germany’s reunification. I have found evidence that the programme created major discussions among the governments of other European states but there is not much about it in the German archive. Reading the memoirs of the involved German decision makers, I learned that the programme was developed under utmost secrecy to guarantee a surprise effect. Only the Chancellor’s inner circle was involved, so even other government members were not informed. The programme was not prepared via official channels and is largely absent in official documents by Germany’s federal chancellery. Thus, it represents e silentio evidence. The best way to cope with these challenges and avoid bias as much as possible is to triangulate within and across sources (cf. Beach and Pedersen 2013, pp. 126–129). Returning to the example of the European Council meeting, a good approach for within-sources triangulation is to analyse the meeting records of every involved state. If there are no major discrepancies in them, this is a good sign. The confidence in the documents can be increased when triangulating across sources, for instance, by analysing interviews with participants or checking if they have written about the meeting in their memoirs.

Outlook

In this article, I have introduced and discussed archival research for political scientists working in the field of European politics. Even though the value of archival documents is generally recognized, political scientists usually lack subfield-specific practical guidance on why and how to use archival documents as causal process observations.

In a nutshell, I have emphasized the advantages of archival research for political scientists. The observations collected in an archive’s reading room generally outperform other types of data, such as interviews, memoirs, and speeches, in terms of bias and validity. I establish practical guidance in the form of three tasks to complete in the preparation phase of the archival stay. I emphasized that scholars should enter an archival reading room only once they have a rich understanding of their case. Only with a narrow research question and clear theoretical expectations will the number of relevant documents stored in an archive reduce to a manageable size. Even after a solid preparation for their field trip, researchers might still face challenges in the reading room, which have been addressed in the final section. The article has several implications for political scientists also beyond the field of European politics. Despite the challenges rooted in archival research, political scientists should feel encouraged to approach an archive’s reading room if this could be beneficial for their research. After all, collecting data in an archival room is not a Herculean task if basic rules are followed. Moreover, political scientists will quickly overcome the disadvantage of having received no training on archival research during their studies and develop a routine (Cook and Schwartz 2002). Indeed, some archival research education would also enrich the typical political science curriculum. As a result, political scientists will become familiar with archival research early on in their careers and use it more often. The advantages of archival documents over other data will increase the validity of results from small-N, large-N, and mixed methods research that builds on process tracing analyses but also on evolving text-as-data approaches.

The good news is that archival documents are getting more accessible as archives start digitizing their content (Kim 2022). In the future, researchers can possibly save resource-intense trips to archives and analyse documents from their desks. So far, however, only a fraction of archival records has been digitized, making a visit to the reading room indispensable. Establishing the norm of making collected documents available online would be a great achievement for the scholarly community. Until that norm is established, research relying exclusively on partially digitized archival holdings faces the downsides of convenience sampling. Hence, digitized material from archives should be treated with caution. In addition to these shortcomings of digitized archival documents, the visit to archives brings a particular scholarly flavour beyond the touristic value added.