Introduction

Measuring individual learning outcomes and comparing them among individuals and nation states by means of worldwide league tables have become predominant strategies for monitoring and assessing the efficiency of education and training systems. Thus, numbers and testing are being used as policy tools (Lingard 2011; Sellar 2015; Smith 2017; Lingard et al. 2017). This rise of test-based “infrastructures of accountability” (Anagnostopoulos et al. 2013), which extend pure data collection and management to “complex assemblages of technology, people and policies” (ibid., p. 13), also applies to academia and the policy sector in terms of their approach to dealing with the issues of adult literacy and numeracy (Hamilton et al. 2015; Evans 2019). Over the past decades, large-scale skills assessments, such as the Programme for International Student Assessment (PISA) or the Programme for the International Assessment of Adult Competencies (PIAAC), have substantially contributed to increased knowledge and data production on a global level, administered by transnational organisations such as the Organisation for Economic Co-operation and Development (OECD 2019).Footnote 1

Framing adult basic education as a global education policy concern has also been supported by the establishment of worldwide educational goals by intergovernmental organisations. One of the most prominent endeavours is the United Nations 2030 Agenda with its seventeen Sustainable Development Goals (SDGs) (UN 2015). The fourth of these goals, SDG 4, aims to

ensure inclusive and equitable quality education and promote lifelong learning opportunities for all (ibid., p. 14).

Among the ten targets of this education-focused goal, SDG target 4.6 calls on United Nations (UN) Member States to ensure that “all youth and a substantial proportion of adults, both men and women, achieve literacy and numeracy” by 2030 (ibid., p. 17).

Over time, all these measurement and monitoring endeavours of academia and the policy sector have helped to substantiate the relevance of adult numeracy, derived from its

recognized role in contributing to the empowerment, effective functioning, economic status, and well-being of citizens and their communities (Gal 2000, p. ix).

However, some studies are also drawing attention to the impact and the consequences of a development towards extensive testing regimes. For example, Leon Tikly has argued that indicators and quantitative monitoring exercises have evolved in academia and the policy sector into a sort of “gold standard” (Tikly 2015, p. 239), taking the nature of testing and its standards for granted.

In this article, we pursue Tikly’s (2015) line of argument, and use William Smith’s (2017) analysis of a global testing culture as our conceptual framework. Our aim is to explain why we consider certain adult abilities to be overlooked in international (literacy and) numeracy surveys because they remain off the radar of testing regimes. Two aspects guide our elaboration. First, we discuss the concepts of abilities and skills used in testing regimes such as PIAAC for measuring individual skills and classifying them based on standardised scores. In our view, these concepts follow a rather deficit-oriented logic,Footnote 2 and we suggest using a resource-oriented perspective,Footnote 3 which is particularly relevant in the context of disability.

The second aspect concerns the blind spots of large-scale international performance surveys which occur due to the scope of these surveys’ methodological approach and are thus inherent in their findings on adult numeracy. We agree with some scholars’ concerns (Jackson et al. 2018, p. 248; Evans 2019, p. 68) that large-scale standardised definitions and tests in numeracy may not satisfactorily draw a comprehensive picture of adults’ diversity and the broad range of their everyday numerate social practices. Having identified the blind spots of large-scale surveys, we then complement the detection of their presence with the insights and added value which can be drawn from the use of small-scale qualitative approaches and principles of participatory research.Footnote 4

The structure of this article is as follows: We begin with an overview of the key paradigms of a global testing culture. Next, we outline our two areas of interest, disability and numeracy, identifying the blind spots of large-scale international surveys, and we consider ways of extending the range of methodological approaches to investigating (literacy and) numeracy. We present the theoretical and conceptual frameworks which serve as our basis for discussing testing regimes in the field of adult numeracy and disability. Next, we debate the benefits and the limitations of using principles of participatory research in small-scale qualitative studies. After identifying five guiding principles, we showcase and reflect on them by presenting selected data from our current researchFootnote 5 on numeracy practices of adults with learning difficultiesFootnote 6 (also referred to by some as intellectual disabilities). We conclude our article by summing up the benefits and the limitations of using such principles in the context of investigating numeracy among all members of a country’s adult population.

Key paradigms of a global testing culture

The discourse in the field of education and lifelong learning about the epistemological foundations, methodological approaches and the scope of findings of large-scale surveys has been enriched by Smith (2017). According to his analysis of the emergence of a global testing culture in the light of world culture theory (Ramirez 2003),Footnote 7 the global testing culture is driven by two key paradigms – positivism and individualismFootnote 8 – thus favouring an environment where high-stakes standardised “testing becomes synonymous with accountability, which becomes synonymous with education quality” (Smith 2017, p. 7).

Pursuant to these paradigms, Smith distinguished among five core values of such a culture (ibid., pp. 12–13). These are: (1) education as a human right, leading to mandatory worldwide policies to ensure education equity; (2) prioritisation of academic intelligence, privileging cognitive and scientific dimensions; (3) faith in science and its findings as the “objective arbiter of truth”; (4) decentralisation of education systems; and (5) neoliberalism as a political-economic concept that fuels most of society and uses accountability measures for assessing education systems’ efficiency. Smith further argued that these values set the legitimate behavioural standards for all parties involved and their respective social roles, which stipulate that individual “behaviour that fails to align with the cultural model provided is considered deviant and formally or informally sanctioned” (ibid., p. 14).

In our view, these key paradigms and core values point to factors which are most relevant factors for the analysis we present in this article, which aligns with other scholarly work in social sciences. In his work on scientific history, Nikolas Rose elaborated on how societal systems became dependent on evolving their citizens’ human capital and thus “sought to codify, calculate, supervise, and maximize the level of functioning of individuals” (Rose 1988, p. 197). This helped constitute a “system of individualization in terms of measurement and diagnosis rather than status and worth” (ibid., p. 197). More recently, Camilla Addey has brought forward the same line of argument, explaining why low adult skills levels of literacy (also applicable to numeracy) are considered threatening to society’s “threshold of functionality” (Addey 2018, p. 326). In this sense, testing regimes have served as the backbone of a “global literacy”, having become the predominant and “single story of literacy” (ibid., p. 324).

The necessity of debating these conceptual frameworks and drivers for policy and research becomes particularly evident when taking into account adults with learning difficulties and their numeracy-related abilities, which we have chosen as our particular areas of interest in this article.

Disability and numeracy-related abilities

Disability

Taking a closer look at disability (our first area of interest in this article) in the context of large-scale international surveys reveals that the respective terminologies and conceptualisations of impairment and disability differ substantially worldwide and are far from clear-cut (WHO 2007, pp. 17–24; Neuhäuser and Steinhausen 2013, pp. 16–21; Schreiber-Barsch et al. 2020, p. 583). To distinguish among the respective approaches, three conceptual models are commonly used: (1) medical/individualistic; (2) social; and (3) cultural/relational models of disability (Barnes 2014).

Medical models interpret disability as a primarily individual and biological factor of deviation to be “fixed” by medical or other treatments. The standard definition of the World Health Organization in its International Classification of Functioning, Disability and Health (WHO 2001) seeks to broaden the strict medical and social understanding by recognising the category’s complexity, and captures disability as arising from a dynamic interaction between a person’s health condition (individual impairment) and the contextual factors of activities and participation.

Likewise taking social participation into account, the UN Convention on the Rights of Persons with Disabilities states:

Persons with disabilities include those who have long-term physical, mental, intellectual or sensory impairments which in interaction with various barriers may hinder their full and effective participation in society on an equal basis with others (UN 2006, p. 3).

In this article, we follow the cultural model, which again broadens the analytical view and strongly scrutinises the legitimacy of widely unquestioned normative standards and classifications. It emphasises the intersections across biological, social, cultural and psychological aspects, as well as attitudes, norms and personal perceptions; it also pays attention to power structures. This model is advocated in the field of disability studies (e.g. Waldschmidt 2017; Goodley 2017),Footnote 9 where disability is considered

neither as only an individual fate, as in the individualistic-reductionist model of disability, nor as merely an effect of discrimination and exclusion, as in the social model (Waldschmidt 2017, p. 24).

Instead, the cultural model recognises “impairment, disability and normality as effects generated by academic knowledge, mass media, and everyday discourses” (ibid., p. 24).

This understanding is also informed by the theoretical approach of ableism (Campbell 2009). Like Smith (2017) and Addey (2018), Fiona Kumari Campbell criticised the one-sided and often invisible social architecture of an ability regime, where skills and competencies are believed to determine a person’s degree of functionality. She defined ableism as

a network of beliefs, processes and practices that produces a particular kind of self and body (the corporeal standard) that is projected as the perfect, species-typical and therefore essential and fully human (Campbell 2009, p. 5).

Disability in this deficit-oriented view is thus “cast as a diminished state of being human” (ibid., p. 5). In contrast to this deficit-oriented view and in accordance with the cultural model, our own ambition was to choose and use a term which would reflect a resource-oriented perspective on disability. The outcome of our quest for such a term is that we decided to adopt the term learning difficulties instead of intellectual disabilities, as suggested by self-advocacy groups such as People First (Stalker 2014; Walmsley and Johnson 2016; Di Lorito et al. 2018, p. 669). However, we also recognise that this term does not simply erase social stigma or the need for reflecting on its inherent norms and attributions.

Numeracy

Over the past decade, numeracy, our second area of interest in this article, has received increased scholarly attention. It has also been incorporated in large-scale assessments, establishing the benefit of providing a standard definition and comparable data. PIAAC defines numeracy as

the ability to access, use, interpret, and communicate mathematical information and ideas, in order to engage in and manage the mathematical demands of a range of situations in adult life (Gal et al. 2009, p. 21).

This definition was extended by Vince Geiger, Merrilyn Goose and Helen Forgasz to a rich interpretation of numeracy, which they circumscribed as

the knowledge and capabilities required to accommodate the mathematical demands of private and public life, and to participate in society as informed, reflective, and contributing citizens (Geiger et al. 2015, p. 531).

This takes into account both the mastery of school mathematics and

the capability to: make sense of non-mathematical contexts through a mathematical lens; exercise critical judgement; and explore and bring to resolution real world problems (ibid., p. 531).

It is this rich interpretation of numeracy which guides the rationale of our article and the research methodology we used in our study.

Despite this growing attention, the extant body of scholarship on adult numeracy and disability, particularly on the subcomponent of learning difficulties, is still very limited (see for example Prendergast et al. 2017; Lane et al. 2019). This dearth echoes data collection omissions in large-scale assessment surveys. While the PIAAC background questionnaire (BQ) used in the United States (US) collected data on disability status, it did not gather any information on intellectual disability (Patterson and Paulson 2015, p. 11). The 2012 German PIAAC survey (GESIS 2012) left out persons living in institutional settings (e.g., prisons and homes for the elderly or for adults with disabilities), which massively reduced the probability that adults with learning difficulties would be included in the sample (see Schreiber-Barsch et al. 2020, p. 582). Furthermore, stocktaking papers on the respective epistemological foundations of survey designs have documented the predominance of cognitive testing and behaviourism rationales that favour medical models of disability, and thereby deficit-oriented perspectives (Magne 2003; Tan et al. 2019). Taking into account Smith’s key paradigms of a global testing culture outlined above thus strengthens our appraisal that individualism and positivism also seem to have a strong impact on adult numeracy and disability research.

Yet, across these theoretical and conceptual approaches, what becomes evident is the key relevance accorded to the concept of human abilities when measuring skills and debating numeracy and literacy. Researchers largely agree that in contrast to understandings of skills and competencies (Rychen and Salganik 2003, p. 43), abilities constitute a basic blend of a human individual’s mostly innate reservoir of physical and mental aptitudes (Straka and Macke 2009, p. 15). This reservoir provides a substantial yet extendable foundation – extendable because abilities mirror an individual combination of knowledge and of cognitive, corporal and motivational-emotional dispositions to do what one intends to do (or not do). The gradually more future-oriented meaning of still unrealised potential is often called capability. The concept of skills differs from that of abilities and capabilities by also drawing on learned or acquired routinised abilities and proficiencies with reference to performing a particular task (ibid., p. 15).

The dilemma that arises from the disability-specific component of a deficit-oriented perspective of human abilities ties in with commonly used standards imposed on disability, particularly learning difficulties. As declared by Robert Edgerton as early as the 1960s, from the standpoint of sociological stigma theory,Footnote 10

no other stigma is as basic as mental retardation in the sense that a person so labeled is thought to be so completely lacking in basic competence (Edgerton 1967, p. 207).

Assuming that these labelling mechanisms are still at work today reinforces our view that the development towards global testing regimes with their paradigms of positivism and individualism fuels this deficit-orientation. Methodologies which involve classifications of skills that assess individual deviations from society’s “threshold of functionality” (Addey 2018, p. 326) tend to exclude parts of the population and/or prioritise survey designs which preclude data collection on abilities beyond the given standards, as outlined above.

Arguing from the perspective of disability studies, Jan Weisser therefore called disability a phenomenon that triggers an “ability conflict” (Weisser 2018, p. 99). The conflict arises out of a suspected or proven “clash” between a person’s (perceived) abilities and an environment’s context-contingent expectations of certain degrees of functioning that are considered basic in that environment. The conflict culminates in the case of disability within the context of an ability regime (see also Tan et al. 2019). Such conflicts cannot be ultimately erased, they are a regular part of learning processes and might also (unintentionally) occur within the context of participatory research, as exemplified below in the section where we discuss selected data from our small-scale qualitative study on numeracy practices of adults with learning difficulties. However, it is imperative to identify and question normative frames, power-driven mechanisms and deficient impacts on learning.

Factoring in the key paradigms and core values of a global testing culture (Smith 2017) as components which are also fuelling the scholarly debate on adult numeracy and disability stresses the need for extending methodological approaches in order to diminish blind spots of testing regimes and promote a resource-oriented perspective on human abilities. In this sense, we understand all adults as having an innate ability to engage (more or less) in numeracy-related learning activities.

Extending the range of methodological approaches

To extend the range of methodological approaches which seem suitable for assessing adult numeracy, we use Jeff Evans, Keiko Yasukawa, David Mallows and Brian Creese’s (2017) distinction between top-down and bottom-up analyses. Top-down (generalising) analyses follow the positivist and individualist key paradigms and “seek to explain numeracy at a global level” (Evans et al. 2017, p. 18; emphasis in original). By contrast, bottom-up approaches operate in a “grounded” way. They aim to explore “the sets of practices that a particular group or community of adults may engage in” (ibid., p. 19), thus taking such practices as starting points for their analyses, rather than the items and scores used to measure predefined and standardised skills.

In our view, this understanding of practices has the potential to diminish blind spots of testing regimes and should be distinguished from a quantitative, generalising concept of practices as a real-context “numerate behaviour”, which is measured, for example, in terms of its frequency (OECD 2019, p. 24). By contrast, we align our theoretical approach with the rich tradition of scholarly works on literacy and numeracy based on social practice theoryFootnote 11 (Lave 1988; Street et al. 2008; Yasukawa, Rogers et al. 2018). Andreas Reckwitz (2003), for example, emphasised that a practice is not just a behaviour, but rather incorporates a more or less visible pattern of how particular things “get done”, which involves a person’s body (enactment of the practice), tacit and explicit forms of embodied knowledge, capabilities and meanings and material resources (artefacts). In this way, social practice theories “seek to overcome a subject/object dualism” (Yasukawa, Jackson et al. 2018, p. 6).

Social practice theorist Reckwitz (2003) and also disability studies scholar Anne Waldschmidt (2017) both base their conceptual approaches on cultural studies research and refer to the philosophy of pragmatismFootnote 12 (Joas 1992) in order to respond to that dualism and to strengthen the subject’s viewpoint in its social, historical, cultural and power-related embeddedness.

Research on adult numeracy through this lens thus

focuses on what people do with numeracy through social interactions in particular contexts, rather than on people’s performance of mathematical skills in isolation of context (Yasukawa, Jackson et al. 2018, p. 13; emphasis in original).

In this sense, knowledge on numeracy practices allows researchers

to explore the conceptualizations, the discourse, the values and beliefs and the social relations that surround numeracy events as well as the context in which they are located (Street et al. 2008, p. 20).

This context-contingent and subject-oriented perspective on human action and interaction within societal structures (of power) and socially shared meaning is coherent with the theoretical assumptions of research in disability studies, as they seek “to investigate the relations between symbolic (knowledge) systems, categorization and institutionalisation processes, material artefacts, practices and ‘ways of doing things’” (Waldschmidt 2017, p. 24).

Based on this line of argument and with regard to research methodologies, the suitability of approaches such as a grounded theory methodology (GTM),Footnote 13 particularly in its interpretation by Anselm Strauss and Juliet Corbin (1990), is widely supported (Strübing 2014). We highlight the fruitful encounter of social practice theories, disability studies and GTM in the following sections by discussing the application of participatory research in exploring the numeracy-related abilities of adults with learning difficulties.

Participatory research: identifying its guiding principles

Participatory research is associated with a range of theoretical strands, such as critical theory or pragmatism (Wright et al. 2010, p. 222).Footnote 14 However, these overlap to form a common ground in participatory research, based on its rejection of a strict positivist view on social science research and on its emphasis on the subject and the subjective in researchFootnote 15 – embedded in societal structures (of power) (see also Schreiber-Barsch and Rule, forthcoming). In participatory research, participation is perceived as not only the aim but also the key process of undertaking research (Bergold and Thomas 2012). In our view, participatory research methodologically enables research designs that correspond with the multi-perspective analysis demanded by the cultural model of disability (see above, Waldschmidt 2017).

This approach fundamentally alters the understanding of the epistemological relationship among science, research and human action (Wright et al. 2010, p. 227; Strübing 2014, p. 38; Waldschmidt 2017). According to Hella von Unger’s seminal work on the topic, “participatory research” is a generic term for approaches that aim to explore, reconstruct and ultimately influence social reality by conducting research in a participatory way (von Unger 2014, p. 1). Implementing this kind of research in the context of disability has noticeably increased over the last few decades (see for example Chappell 2000; Rule and Modipa 2012; Townsend 2013; Stalker 2014).

As for research on learning difficulties, this is not yet widespread (Stalker 2014; Di Lorito et al. 2018; Hollomotz 2018, p. 154) and moreover still tends to prioritise research on, not with, people (Buchner and Koenig 2008, p. 32). Its conceptual strands vary according to the role assigned to persons with disabilities (Chappell 2000; Walmsley and Johnson 2016). In this article, and in the research approach we adopted in our study, we focus on those strands that conduct participatory research in the form of a partnership between researchers and persons with disabilities, a so-called alliance (Chappell 2000).

Across scholarly works which emerged from the fields of social and educational sciences, social work and psychology, we have identified five guiding principles of participatory research approaches (see Table 1). This list provides the heuristic framework for debating our research on numeracy and for presenting selected data from our small-scale qualitative study on numeracy practices of adults with learning difficulties in the following sections. Moreover, these five principles once again demonstrate the distinction of participatory research from the key paradigms and core values of a global testing culture (in particular its values (2) and (3)).

Table 1 Five guiding principles of participatory research approaches
Full size table

We will revisit these guiding principles as we explore the selected data from a small-scale qualitative study we conducted on numeracy practices of adults with learning difficulties. Below, we present a methodological outline of our study. It is followed by our reflections on our own experiences in applying participatory research with adults with learning difficulties, for which the five guiding principles form a useful presentational structure.

Conducting participatory research: exploring numeracy practices of adults with learning difficulties

To demonstrate the benefits and the limitations of participatory research approaches, we use our current research on numeracy practices of adults with learning difficulties as an empirical example. In this article, we focus on methodological considerations (for a detailed literature review and discussion focused on adult numeracy, see Schreiber-Barsch et al. 2020). We begin with a brief introductory overview of our research design, which frames our discussion of selected data from our small-scale qualitative study.

The limited extant knowledge on numeracy practices of adults with learning difficulties made us opt for an exploratory study (Creswell 2003, pp. 198–199).Footnote 16 Our study was guided by three interrelated questions:

  1. (1)

    In which ways do adults with learning difficulties engage in numeracy practices in their everyday contexts?

  2. (2)

    To what extent are numeracy-related abilities mirrored in these practices?

  3. (3)

    In particular, what are the meanings they ascribe to their practices?

Methodology

To answer the above set of questions, we designed an in-situ study (Yasukawa, Jackson et al. 2018, p. 14), following a bottom-up approach in adult basic education research (Evans et al. 2017, p. 18). Such small-scale qualitative studies do not intend to make generalisations, but seek an in-depth understanding of subjective sense-making processes as they unfold in everyday contexts. The setting of our research was the city of Hamburg in Northern Germany, and the study was conducted in German. Our unit of analysisFootnote 17 was social practices (not behavioural or cognitive facets of individuals), which shifted the attention from assessing the (deficient) skills of individuals “towards a consideration of how things get done and how social practices are shaped” (Williams et al. 2018, p. 170).

To carry out this research endeavour in the context of disability with a particular interest in learning difficulties, we decided to intersect the methodological principles of participatory research and qualitative research (Buchner and Koenig 2008, p. 16; O’Brien et al. 2014; von Unger 2014, p. 5) – the latter in the form of grounded theory methodology (GTM) (Strauss and Corbin 1990; Strübing 2014). It recognises adults with learning difficulties as experts in their ways of acting out numeracy.

For the iterative GTM process of data sampling, analysis and theory development (called theoretical sampling), we followed three sensitising concepts (Kelle and Kluge 2010, pp. 28–30):Footnote 18 First, we took on board the rich interpretation of numeracy compiled by Geiger et al. (2015), outlined above in the subsection on numeracy. Second, we reviewed the extant knowledge on what to identify as facets of numeracy (for details, see Schreiber-Barsch et al. 2020, pp. 585–586). We synthesised numeracy facets constituting the analytical reference framework for our selected context of adults with learning difficulties: space and shape; numbers and calculating; size and measurement; patterns and structures; probabilities, frequencies and chances; and artifacts (ibid., p. 586). Third, in exploring ways of acting out numeracy in everyday life, we applied the following widely used classification of the four arenas of adult everyday life (ibid., p. 584): (1) work, (2) education and training, (3) living conditions (including financial and material resources; type of living [residential care, assisted, with parents, on their own, with partners]; taking care of basic needs [e.g., shopping for food], etc.) and (4) free time (e.g., pursuing a personal life and interests; health and well-being; taking part in society and community).

Against this backdrop, the GTM process led us to develop a multilevel participatory research design consisting of three steps based on different methods of data collection, including a between-method triangulation. The steps were (1) shadowing adults with learning difficulties in their ways of acting out numeracy, as in-situ observations (n=11); (2) interviewing adults with learning difficulties on their numeracy practices (n=13); and (3) conducting focus group discussions with professionals working with adults with learning difficulties in the listed arenas of everyday life (n=3). In accordance with the principles of participatory research, we set up a parallel group of co-researchers (adults with learning difficulties) as a main pillar of the alliance (Chappell 2000) between researchers and adults with learning difficulties (the co-researchers were not participants in the shadowing or the interviews).

The sampling for the three levels of the research design, as well as for the group of co-researchers, was in accordance with the GTM process of theoretical sampling that did not pursue a statistical representation (generalisation), but a conceptual or analytical representation (Strübing 2014, p. 31; Merriam and Tisdell 2016, p. 33). Footnote 19 In this case, it was a representation of a set of social practices as the unit of analysis, not of individuals according to (among others) their cognitive facets, such as scores on a scale measuring their intelligence quotient (IQ). Accordingly, we ensured a balanced distribution of participants across contexts and genders, and we aimed for maximal contrast in living conditions and ages (Schreiber-Barsch et al. 2020, p. 586).

To conduct shadowing and interviews, it was necessary for the participants to have abilities at their command that allowed them to communicate verbally with us, move without personal assistance and be able and willing to give us insights into their ways of acting out numeracy. This profile corresponded to persons who would be described as experiencing mild-to-moderate difficulties in the WHO “activities and participation” component in terms of capacity and performance qualifiers (WHO 2001, pp. 237–239). This led us to use snowball sampling and to seek participants via channels (such as particular journals) or places that were frequently and commonly used by members of the target group. Finally, we addressed potential ethical issues by placing emphasis on obtaining informed consent for our procedures of data collection, analysis and dissemination (Porter and Lacey 2005, pp. 92–95; Buchner 2008).

By using the principles of participatory research, we were thus able to gather data from persons who are underrepresented in the extant scholarly literature and difficult to reach using traditional test methods. This was reflected in the common worry amongst our participants that this would “be a maths test”. Throughout the data collection processes, the participants were able to ask questions about the methods and were also aware that, if they requested this, they could stop participating in the study (n=0).

Co-researchers

We organised the so-called participatory research group (PRG) with 9 co-researchers (adults with learning difficulties). Our first concern was to provide the PRG with information and learning opportunities to become familiar with the university and the nature of research. Moreover, we provided information about our research, preliminary findings and challenges, and put our way of conducting research up for debate. This meant involving the co-researchers in the steps of planning and implementing the shadowing and the interviews. In this process, we discussed, for example, the intelligibility of the interview questions and asked for advice. This led one co-researcher to suggest a question that we then successfully placed as the opening question in the interviews in order to collect rich descriptions of acting out numeracy in daily life: “Please tell us about your daily life: what does your normal day look like?”

This PRG subsequently led to a more formalised setting as part of the study programme at the Faculty of Education, University of Hamburg (the participatory research workshop [PRW 1]). In this setting, students and adults with learning difficulties collaboratively work together on small self-organised research projects. While we thought of continuing with the focus on the numeracy topic, the co-researchers expressed their interest in a broader range of issues. Consequently, we continued to provide information on the ongoing research and put methodological questions and preliminary findings up for debate, complemented by organised multiplier events to which we also invited the participants of the shadowing and interview sessions. However, the participants of the PRG/PRW chose their own research areas of interest, also in the subsequent series (PRW II and III) (for details, see Gundlach et al. 2019).

Shadowing, interviews and focus groups

We used the shadowing method (Bøe et al. 2017) for our in-situ explorations of numeracy practices (lasting between 2 and 7.5 hours). The 11 participants’ ages ranged from 18 to 80 years (6 males, 4 females and 1 transgender), 6 were living with their parents, 3 were living in supportive housing or assisted living facilities, and 2 were living alone. Regarding their personal circumstances, 2 received an (invalidity) pension, 4 were employed in a sheltered job or workshop, 4 attended vocational schools, and 1 was completing an internship. We asked the participants to allow us to shadow them while they performed their daily activities as they usually did; thus, they were in full control of the circumstances of the shadowing (McDonald and Simpson 2014). We agreed on a meeting point, and from that point, a researcher followed the participants wherever they went. The researcher used a recording templateFootnote 20 (observational record sheet) throughout the shadowing.

However, as Marit Bøe et al. (2017, p. 608) aptly remarked, researching social practices requires a complementary method to capture their complexity. After completing the shadowing, we therefore conducted semi-structured interviews (n=13) (lasting from 30 minutes to 2.5 hours each). Due to our use of snowball sampling and to individual preferences, there was only a small overlap (n=3) between the shadowing participants and the interview sample. The adults who were interviewed were aged 18–42 (7 male, 6 female). Four of them lived with parents, 5 alone (one together with a spouse, one with a daughter) and 4 in supportive housing or assisted living facilities. The majority (7 people) were employed at a sheltered job or workplace, 3 at a school and 3 were completing an internship. For our interview method, we combined principles from problem-oriented interviews (Witzel 2000)Footnote 21 and from guidelines for conducting qualitative interviews with adults with learning difficulties (Buchner 2008).Footnote 22 We offered flexibility in terms of where and when the interview would be conducted, used plain language (Buchner 2008, p. 522; Hollomotz 2018, p. 153), a clear structure and questions that were assumed to be relevant to the participant’s experiences (tested by the co-researchers). To this end, we also used seven picture cards with motives in the broadest sense of numeracy which we presented during one interview sequence to initiate further reflection on everyday numeracy.

In the iterative process of data analysis, the data collected in the shadowing and the interviews demonstrated that professionals working in the listed arenas of daily life play a pivotal role for acquiring, using and developing numeracy-related abilities. This insight led to the third step of our research design. We conducted three focus group interviews (Mangold 1960; Barbour 2007), each lasting approximately 45–60 minutes and involving a total of 15 professionals (working in a vocational school for the group under scrutiny, in supportive housing or assisted living facilities, and in support structures for sheltered workplaces). The aim was to gather collective experiences and opinions on numeracy practices of adults with learning difficulties, particularly on ways of professionally supporting numeracy learning and development in everyday life contexts.

In this article, due to our focus on participatory research with adults with learning difficulties, we limit the following glance at selected data from the study described in the previous section to those parts of the data collection that were conducted with these adults.

A glance at selected data from our small-scale qualitative study

To illustrate and reflect on the benefits and the limitations of participatory research approaches, in this section we provide insights into selected data collected during the shadowing and the interviews (the participants’ names have been anonymised; a CAPITALISED word signifies the participant’s emphasis).Footnote 23 We present these selected data using the five guiding principles of participatory research (see Table 1).

(1) Dual objective: participation in research and participation in society

We endeavoured to realise this dual objective in several methodological ways. These included the research design with the part of the co-researchers who were recognised in their expertise in giving advice for planning, implementing and analysing the different stages of data collection. This role also included participating physically as a member of a research group in the usually highly exclusive venue of a university. Likewise, the multiplier events with debates on preliminary findings and feedback about the methods used in the data collection opened a space for a dialogical exchange on numeracy, on the relationship between researchers and participants and on fostering abilities through research as a way of empowerment. Several participants reported that by being part of the research, they had started to reflect on their ways of acting out numeracy and realised abilities that they had never identified as having something to do with “mathematics”.

This dialogical process of capacity building and empowerment also became apparent in the process of the data collection itself due to the participatory methodology we had opted for. During the shadowing, several times, the participants asked about what the researcher wrote in the observational protocol and what phrases such as “social interactions” meant (see footnote 20 describing our recording template), which started a dialogue with the researcher. In an interview sequence, a participant had explained how he made sure to arrive punctually at a specific place, and the interviewer had asked for an example of how he had learned that (e.g., using a street map). The participant was puzzled, then laughed:

Participant: “That is difficult to answer. One just learns it. Or not? [laughing] I do not know.”

Researcher: “Yes.”

Participant: “How have YOU learnt it, [name of the researcher]?”

(I_1_#176–178#)

(2) Practising participation in research: addressing equivalence

The participatory research approach clearly made voices visible and audible which remain largely unheard and underrepresented in the extant scholarly body of knowledge. Considering these numeracy-related abilities in their own right thus contributed not only to striking a better balance by complementing/extending the existing spectrum of research findings, but also to identifying controversial interests and societal conflicts. This became apparent when, on the one hand, our participants showed evidence of their successful numeracy proficiencies and strategies and then, on the other hand, devalued their abilities by closing their narratives with remarks such as “but I am only a learner of a special needs school” (I_10_#360#), or “Well, I was always only told by other people … but I would say, from the level, wait, how do you say that? 5th grade or so at the most” (I_8_#524#).

The limitations of realising this principle in our case became evident due to the funding structure of the research, which meant a given time schedule and an a priori set research question. In these parts of the research process, that is, choosing the research topic and planning the overall design as a collaborative outcome of the alliance, participation had to be kept very limited. Nonetheless, throughout our research, we paid particular attention to several ways and forms of realising participation, considering adapted methods, information, consultation, involvement and co-determination (Schnurr 2005, p. 1336; von Unger 2014, pp. 39–52). We therefore designed the data collection process from the perspective and subjective viewpoints of adults with learning difficulties, recognising their expertise and prioritising it as the key reference.

(3) Dealing with knowledge: construction and ownership

For the third principle, the development of a multilevel participatory research design played a pivotal role. To produce a rich database, it was necessary to obtain observation data (shadowing) and descriptions (interviews) of numeracy practices – through methods that would recognise the participants as experts and be suitable to their life situations. In general, this principle demands the researcher’s awareness of subtle mechanisms of knowledge construction processes. The reason for this is that if these mechanisms are not reflected and based only on the researcher’s knowledge and expectations, they might, as demonstrated in the next scene, unintentionally give rise to an ability conflict (Weisser 2018) between a person’s (perceived) abilities and an environment’s context-contingent expectations of certain degrees of functioning that are considered basic in that environment.

As explained above, we used picture cards in the interviews, with the intention of initiating further reflection on issues of numeracy in matters of daily life. However, our analysis showed that in several cases, this disrupted the participants’ flow of conversation. For example, one participant was clearly puzzled when shown a picture card of a woman holding a note with numbers written on it:

Participant: “Yes. I have no clue about this, about this note.”

Researcher: “About this note, you mean this one here, right?”

Participant: “Yes.” [pause]

Researcher: “But maybe … So that is, what could that be?”

Participant: “Calculating?”

Researcher: “Calculating?”

Participant: “Mathematics? [pause] Learning difficulty? I do not know.” […]

[Looking at another picture card that visualises a street traffic situation with road signs.]

Participant: “I have… well, about this road sign, I have no idea because I do not drive a car.”

Researcher: “Okay.”

Participant: “And I come …, or I just do not know what this means.”

(I_10_#326–332#; #392–394#)

Thus, the abilities that this participant articulated or perceived with regard to specific tasks and situations (“understanding” a pattern of numbers on a note; driving a car) clashed with the researcher’s expectations of what could be matters of daily life and relevance. Thus, in this case, this particular method had its limitations, because it failed to link to relevant matters of the participant’s daily life and unintentionally produced the feeling of being in a test situation, which obviously influenced the production of knowledge. The principles of participatory research, however, helped to discover and reflect on this limitation.

(4) Reflecting on issues of power and control

Whereas in the interviews, the control of the procedures (choreography of questions and answers) relied more strongly on the researcher conducting the interview, the participants controlled the consent for the interview as such – the place and the time and the extent and the content of their narratives. In the shadowing, the possibility of exercising control became even more obvious. One researcher met with a participant near a train station after she had finished her vocational training course; the only information given was that we would follow her in spending free time and doing her shopping. The following event occurred when changing trains (noted in the recording template):

Participant: “Do we want to go to the [shopping centre]?”

Researcher: “You decide. I’m just following you today.”

Participant: “Then we’ll go to the [shopping centre].”

She goes purposively to the platform. We stand at the station and wait.

Participant: “We can also go to the airport. I was there the other day with my friend.”

Researcher: “Yes, we can do that, too.”

Participant: “Or to the [shopping centre].”

Two minutes before the train arrives, the participant changes her mind.

Participant: “We’ll take my route to school. Going home, that’s not that far. That’s better.”

She turns and goes back downstairs to change platforms.

Participant: “The train is already there!”

She quickly runs up the stairs and jumps into the train.

I [the researcher] run after her.

(Sh_3_para 7_1–7)

On the one hand, this sequence illuminates that shadowing allows gaining a first-hand view of numeracy practices (routines) and numeracy-related abilities incorporated in everyday contexts. The participant demonstrated her familiarity with the train station (spatial orientation); she knew the destinations of certain trains and could adapt to the variety of routes and spontaneously change her mind and desired destination (patterns and routines; understanding numbers). Doing this, she planned ahead and decided on the basis of comparing how long the journey would take on different routes (handling the unit of time). On the other hand, the researcher was required to take a back seat, follow the participant and interpret the data being offered. By contrast, the participant gained control of the research situation, made decisions and changed her mind.

(5) Rethinking one’s role as a researcher

Continuously reflecting on the research process is inherent in the bodies of knowledge consulted, the methods’ benefits and limitations and/or the different interests of the parties involved (including academia, the policy sector, researchers, participants and the funding agency). The aim is to expose and to debate these contested terrains, exemplified by, for example, unintentionally giving rise to an ability conflict due to the chosen method (picture cards). Likewise, using shadowing as a method requires well-balanced and careful data collection. Sharing the participant’s daily life for hours and engaging (cautiously) in conversations (Bøe et al. 2017, p. 609) could give them the impression of having made a new friend, which would then need to be corrected by the researchers as being only part of their professional role and task. In general, intersecting principles of participatory research and GTM require the researcher to maintain a higher degree of openness with regard to prior planning, designing and conducting the research than would be necessary in the use of standardised and quantitative methodologies. This is illuminated, for example, by the decision-making processes of the co-researcher group or by sharing control while shadowing and conducting interviews.

Conclusion

Against the backdrop of key paradigms and core values of a global testing culture (Smith 2017), we argued in this article that using a rich interpretation of numeracy and a cultural model of disability in intersection with methodological principles of participatory research, GTM and bottom-up approaches in literacy and numeracy research has the potential of diminishing blind spots of testing regimes. It not only enables the inclusion of persons and their social practices in the literacy and numeracy spectrum that are not yet part of the extant scholarly knowledge, but also strengthens a resource-oriented perspective on human abilities. In this way, the theoretical encounter among social practices theories, disability studies and GTM is further nuanced and substantiated for scholarly debate. Moreover, we predict that the heuristic framework we have presented of the guiding principles of participatory research, with its illustrated benefits and limitations, will also be usable for research beyond the context of disability, thereby fostering research not on, but with hard-to-reach and marginalised learners and social groups.

Within this field of enquiry, our evidence has shown that the shadowing method is suitable for research in the context of social practices and learning difficulties, thus broadening the methodological debate in education sciences. Applying this method proved to be particularly pertinent in the case of very limited extant knowledge (on numeracy practices of this group) and of participants who might show cognitive limitations in verbally explaining their points of view (though this subsequently proved to be the case only to a very small extent in some of the interviews). However, deploying methods such as shadowing and interviews still requires certain abilities and might, therefore, not be suitable in cases of e.g. more complex impairments.

To sum up (and as emphasised elsewhere; Schreiber-Barsch and Rule, forthcoming), our aim is to contribute evidence towards the epistemological, methodological, ethical and political importance of a participatory ethos in research, not only – but in particular – with regard to persons with disabilities such as adults with learning difficulties. This ethos strives for learners’ capacity building and empowerment through research as opposed to the use of learners merely for the benefit of academic research or the policy sector. The ultimate goal of research in this participatory approach is, thus, the collective transformation of social reality.