Background

Universal, repeated, weekly testing followed by strict household isolation after a positive test and continued normal life after a negative test, was promoted in the UK as an important exit strategy from the COVID-19 (SARS-CoV-2) pandemic [1,2,3,4,5]. A novel pilot programme to assess the feasibility of weekly community RT-LAMP (Reverse transcription loop-mediated isothermal amplification) testing [6, 7] for the SARS-CoV-2 virus using saliva samples collected at home was developed and piloted in a unique partnership (the Southampton COVID-19 Saliva Testing Programme) between Southampton City Council and the University of Southampton. Immediately on waking in the morning, the saliva samples were self-collected at home into specimen pots, which participants then dropped off at collection points for transfer to the testing laboratory. Direct RT-LAMP testing was undertaken using the OptiGene SARS-CoV-2 assay [8]; an evaluation of the assay found an overall sensitivity of 79% and specificity of 100%, with a sensitivity of 94% in samples with a higher viral load(RT-qPCR ORF1ab CT values of < 25) [6]. Between June and October 2020, participants from two general practices in Southampton, staff and students at the University of Southampton and staff and pupils from one infant, one junior, one primary and one secondary school in the city were invited to participate in two phases of a pilot of the Southampton COVID-19 Saliva Testing Programme. The four schools had catchment areas in more deprived parts of the city, and the pupils were from diverse ethnic backgrounds. More than a quarter of students at the University of Southampton come from outside the UK. Interactive engagement activities took place alongside the testing programme to maximise uptake of saliva testing, with extensive development and deployment of educational materials targeting different age groups of students and pupils in the schools, alongside work to engage university students.

Guidance from the UK Scientific Advisory Group for Emergencies (SAGE) draws on experiences of previous public health emergencies and identifies the need for high levels of engagement with communities and individuals in the creation of successful mass testing programmes [9]. They suggested that high levels of engagement with the community builds trust, shared goals, and a sense of fairness [10, 11]; engagement also bridges cultural and language gaps [12]. Concern was expressed about the lack of community engagement in the government’s response to COVID-19 [13]. In previous epidemics, highly engaged communities and community organisations were found to play important and active roles in prevention and control of infection [13]. Data are lacking on the factors that improve engagement with efforts to prevent and control infection, including community-testing, in the context of the COVID-19 pandemic.

Rapid qualitative evaluation can be used when the public sector requires immediate feedback on programme process and impacts in order to shape policy [14]. Hence, this type of approach lends itself to providing insights into the response to public health recommendations and programmes during the COVID-19 pandemic. Data generated describing people’s lived experiences of the infection, their behaviour and responses to the government efforts to contain the virus have complemented and explained the epidemiological data [15]. Other studies involving rapid qualitative methods during the COVID-19 pandemic have provided ‘real-time’ feedback on current public health practices [16] and have informed the design of the UK’s COVID-19 response [17]. This approach was established to inform iterative development and testing of programmes and interventions and has been shown to produce findings comparable to those of more traditional qualitative research and thematic analysis [18, 19].

To generate insights and inform the design and modification of the Southampton COVID-19 Saliva Testing Programme and the next phase of future mass-testing, we conducted a rapid qualitative evaluation exploring the experiences of individuals and organisations who took part, of those individuals who declined to take part, and of those in the educational and healthcare organisations involved in the pilot testing who were responsible for roll-out.

Methods

Between 4th June and 7th November 2020, we conducted 77 interviews and 20 focus groups with 223 staff, pupils, students and household members, from four schools, one university, and two GP practices belonging to one Clinical Commissioning Group in the city of Southampton in the South-East of England. This evaluation of the Southampton COVID-19 Testing Pilot Programme was guided by the NIHR framework for community engagement and involvement in health research and, as such, was underpinned by a participatory approach, modifications recommended to the programme were driven by the participants’ priorities and we endeavoured to be inclusive of all ages, ethnicities and genders [20].

Three groups of participants, selected purposively, took part in interviews and focus groups: i) individuals who had been approached about taking part in the pilot Saliva Testing Programme including some who tested positive; ii) those who were approached but declined to take part; and iii) senior university, primary care and school representatives responsible for delivering the Saliva Testing Programme in their organisations. The majority of participants in this qualitative study were recruited via the testing programme’s online registration process, a feedback form on the Southampton City Council website, during a phone call with the testing programme team to those who tested positive or inconclusive, or through the schools’ weekly newsletters about the programme. Senior university and school representatives and a minority of other participants were recruited through the testing team’s and participants’ professional networks. Participants gave permission for their contact details to be shared with the qualitative evaluation team and were offered a small financial incentive to take part. Participants aged under 18 years and their parents signed consent forms. Participants from all groups except those who declined to take part continued to be recruited and interviewed until the point of data saturation was believed to had been reached. Those who declined to take part were more difficult to recruit and so the number of these participants interviewed was determined by those who were available within the timeframe. Participants’ characteristics are outlined in Table 1.

Twelve experienced qualitative researchers were involved in this evaluation. Each researcher observed at least one focus group or interview to ensure our approach was standardised. Focus groups, averaging 1 h long, were conducted using Zoom Pro and audio-recorded using Open Broadcast Software; telephone interviews, averaging 30 min long, were recorded using Olympus recorders and a microphone earpiece. Participants gave verbal consent for discussions to be recorded at the start of each interview and focus group. Focus groups and interviews were conducted using semi-structured discussion guides (see Additional files) developed by experienced qualitative researchers informed by the needs of the programme team and revised iteratively in the light of insights gained. Observation guides were used to capture key findings and contextual factors.

Findings from the focus groups and interviews were rapidly synthesised and added to a report, which was edited collectively by the evaluation team and continuously updated as new data and insights were generated [16, 21, 22]. This report was used to provide immediate feedback to the programme team enabling them to make modifications to the running of the programme, and to offer insights to other groups in the UK developing similar testing programmes. At the end of the pilot phases of the programme, rapid analysis of the key messages gathered in the shared report was conducted to produce a framework of themes which addressed three questions [23]: (i) what made people engage with the testing programme?; (ii) how could engagement with the testing programme be improved?; and (iii) what were the impacts on participants of engaging with the testing programme? A descriptive approach was taken to the analysis, meaning that notes taken during the focus groups and interviews were coded into the framework of themes. Findings under each theme were summarised and presented in the findings section as they answer the three questions above [24]. Quotes from participants were transcribed directly from the audio-recordings [22] and illustrate the range of responses as they relate to the three questions are provided in Table 2. The themes and key messages were synthesised and discussed widely with authors to form recommendations (see Table 4).

In a final phase of analysis, a consultation activity was undertaken with public contributors hosted by the Wessex Public Involvement Network, during which our interpretation of the data was validated and promoted transparency and reflectivity [25]. Theory was not directly applied to the conceptualisation of the evaluation or the data analysis in the first instance. An inductive approach to analysis was taken to ensure that the codes and themes were grounded in the views of participants themselves. Psychological theory was, however, applied retrospectively to assist with interpretation of the findings described in the discussion [26]. Theory on promotion- and -prevention-focused motivation was used to explain why people did and did not take part. Promotion- and -prevention theory suggests that people balance the motivation to fulfil basic needs with the desire to maintain safety and security [27]. Promotion activities generate gains for people in terms of social connectedness and reductions in anxiety, for example, where prevention activities are those that minimise risk and loss of those gains. Bandura’s concept of collective efficacy was applied to descriptions of differences in engagement between the organisations and communities involved [28]. We used COREQ guidance to structure our reporting [29].

Results

Participants in the rapid qualitative evaluation were staff or students at the University, pupils, parents and staff from four schools and employees at General Practices in the city.

Table 1 Characteristics of participants (n = 223)
Full size table

What made people engage with the testing programme?

Communication

Participants emphasised the need for open and transparent communication from programme implementers of the reasons they should register for the programme, how to go about registering and why they should stay registered. They felt these communications should be motivating in content. To do this, they needed to address participants’ sense of community and make clear how easy and convenient the testing regime was for both those who took part and for those who were managing the programmes within their organisations. Parents, school pupils and staff described finding open communication with the programme team reassuring and therefore motivating. This open communication was led by senior representatives of the organisations taking part, who also emphasised preparation, leadership buy-in and using data management systems to scale up as important to the success of the programme within their organisations. Transparent information sharing between the testing team, participants, local and national stakeholders was seen as a strength by programme implementers. Programme communications were also translated into seven languages in Phase II to increase uptake by minority ethnic groups, seen by participants as another strength.

Community

Participants’ decisions to take part and engage in the programme were, they felt, influenced by a pull on their sense of community. Participants from schools and GP surgeries felt this pull more strongly than participants from the University. The schools and GP surgeries were smaller, more cohesive organisations, where staff and pupils saw one another every day, spoke about the saliva testing frequently and encouraged one another to take part. This was less evident in the University. Automatic registration of all school students may also have normalised participation in the testing programme, where university students had to make the decision to register as individuals. University students suggested that taking part in the testing programme could be normalised by employing testing champions in their halls and courses to promote the programme and by using an ‘opt-out’ process for participation. School pupils and university students were offered interactive educational activities being run to increase engagement with the testing programme. Those who took part in these activities reported understanding more about the science involved in managing the pandemic, which they felt increased their motivation to protect their communities. Senior school and university representatives appreciated the role that the programme played in connecting senior community stakeholders across Southampton in a way that had not been evident before. Finding common cause through using the Saliva Testing Programme was seen as an unexpected benefit from their involvement.

Convenience

Most participants found the programme procedures easy and convenient; registration processes were simple, drop off points were accessible, and testing instructions were clear. It was perceived to be easier to carry out than nasopharyngeal PCR swab tests and test results were received quickly. They felt that making participation as convenient and easy as possible was key to increasing uptake. Parents reported that the test was simple enough for children to take responsibility for carrying out tests independently. From the perspective of the organisations, initial engagement was motivated by the cost-effectiveness of the programme and the value of having data on infection to manage outbreak hotspots by integrating programme data with data from symptomatic PCR testing, to keep the schools, the University and GP practices running.

How could engagement with the testing programme be improved?

Building trust

A major reason some chose not to take part in the programme was that they did not trust the government with their data. Many of those who declined to take part in the Saliva Testing Programme were anxious about the possibility of losing control of their data when the programme passed them to NHS Test and Trace in the event of a positive test. The local NHS Foundation Trust and its partnership with the University and Southampton City Council, however, was trusted; it was seen to have scientific integrity, and as a local organisation, was felt to be answerable to the Southampton community in a way NHS Test and Trace was not. Some suggested that they would have been more likely to take part if the programme was run solely by local organisations. Participants and decliners were clear that building trust was necessary to improve engagement in the programme. They suggested that this would be helped by receiving more directed information from credible sources such as the University and Southampton City Council about the rationale for and design of the programme, about data protection and the accuracy of the tests, and about the progress of the programme. This information would increase transparency and help dispel myths, particularly about the accuracy of the saliva test. One reason some people declined to take part in the programme was a concern about the personal consequences of a false-positive result.

Extra support for testing positive

Those who had experienced a positive test result asked for more efficient data management by the testing programme, NHS Test and Trace and their general practice, and more coordinated messaging. Participants requested more personalised support for those testing positive and having therefore to self-isolate. This included financial aid if unable to work, receiving food and medication supplies and mental health support. Some felt, however, that there were too many support calls from NHS Test and Trace for those testing positive. Participants were particularly worried about the possibility of spreading the virus to others whilst they waited for a test result, and feared the stigma of testing positive, which suggests that they would need reassurance and social support in dealing with a positive test result. Fear of a positive test result was enough to make some decline to take part; they were concerned that if they had to isolate they would lose income, their employer would be unsympathetic and that a history of infection with the virus might affect their ability to get a mortgage and life-insurance. These people preferred not to know their viral status.

Increasing accessibility

Both decliners and participants recommended making the programme more inclusive and accessible to a wider group of those eligible to take part. Drop-off points for saliva samples that catered for people living outside Southampton and more frequent, convenient and better sign-posted drop-off points within Southampton would have reduced the distances people had to travel. Some suggested postal deliveries and returns. Comprehensive translation of all programme communications would have increased uptake and engagement of those from minority ethnic communities.

Practical improvements

Many improvements suggested by participants were incorporated into the programme as it evolved (see Table 3). A small minority of participants, especially parents of younger children, had issues producing enough saliva, which may have produced an inconclusive result. They suggested the tubes be marked with a clear indicator of the amount of saliva necessary. Some decliners missed the University registration email and recommended putting key information highlighted at the top of emails. Senior representatives of organisations involved were clear that the programme gave their staff added responsibilities and added to their workload. Whilst accepting that this extra work was in a good cause, some suggested a ‘toolkit’ of instructions and tips for those implementing the programme to help manage the expectations of both staff and participants.

What were the broader consequences for participants of engaging with the testing programme?

Participants reflected on the positive and negative impacts of participating in the testing programme. Positive outcomes included:

Reassurance

They generally felt reassured by knowing their viral status and expressed a sense of relief and reduced feelings of anxiety when they tested negative. They appreciated knowing that they were not spreading the virus, felt that this enabled a “near normal” life to continue and were more confident going to school or work and visiting vulnerable family and friends. This was perceived to be an incentive for taking part in the programme.

Pride

Participants expressed pride in knowing that they were contributing to a programme that was part of the national effort to manage the pandemic. Some viewed this as a privilege and others were excited. Southampton University students spoke of being envied by those from other universities and felt that the testing programme made the university attractive to prospective students. Senior university representatives spoke of the reputational benefit to the organisation of having provided a testing programme for the local community.

Participants expressed a number of concerns:

Environmental impact

Some were concerned about the amount of plastic in testing kits and the environmental impact of an expansion of the testing programme.

Increased chance of infection

Concern was expressed about the potential of those who received a negative test result to become less vigilant in applying social-distancing and hygiene measures. They were also worried that the test kit drop-off points were sites of potential infection; prior Health and Safety risk assessments had however ensured that these carried no or minimal risk.

Feelings of anxiety

Participants reported feelings of anxiety whilst waiting for their test results, worrying about the personal consequences of having to self-isolate or of having unknowingly passed the virus on to others whilst awaiting their test result. Others were anxious about the possibility of the testing programme ending.

Cultural barriers

Some participants were concerned that aspects of culture were a barrier to participation. These included language barriers and a cultural mistrust in central government. Those who had lived in countries where governments were believed to misuse personal data were more anxious about taking part in the Saliva Testing Programme because of the perceived threat to their privacy. Those who needed the testing programme most may in this way have been prevented from taking part.

Table 2 Illustrative quotes
Full size table

Discussion

Individual participants and organisational representatives provided valuable insights into the experience of engaging with the Southampton COVID-19 Saliva Testing Programme, leading to changes in programme processes and communications, which undoubtedly increased engagement and adherence to programme requirements. They also made important observations about the benefits they experienced from engaging with the testing programme. This is known as promotion-focused motivation which describes a focus on what can be gained from a health behaviour or intervention [27]. Participants suggested that promotion-focused motivation for engaging in testing should be emphasised to potential participants by those running the programme. One of the benefits that participants in the programme emphasised was increased knowledge of the science behind management of the pandemic that school and university students gained from the educational engagement activities. As well as providing insights, it was clear from the focus groups and interviews that the process of seeking feedback from participants also increased the engagement of local people with the testing programme. Prevention-focused motivation theory would suggest that programme promotors should understand and address the value to potential participants of preventing the loss of what they already have [27]. In the case of the saliva testing programme, this would refer to their ability to go to work or school or to socialise if they tested positive. It was felt by decliners and by those who knew people who had declined to take part, that it might be best to remain ignorant of their viral status if it meant that they could continue earning money, for example. Programme implementors need to work on schemes that provide support such as financial aid to those who cannot work if they test positive and to increase emphasis of the benefits such as being able to socialise on receipt of a negative test and continuing to follow national guidance.

Involving local people and organisations in the development and piloting of programmes in this way may well have been important in ensuring community buy-in [30]. They appreciated the chance to contribute to this programme, which they saw to be of national importance. The process of holding the conversations reported in this paper may also have helped to address two key underlying issues affecting engagement with the testing programme: the need for trust between participants and testing programmes, and the role of ‘collective efficacy’ within organisations. Local organisations, such as schools and the University were seen to be answerable to local people and hence more trustworthy than more national ‘faceless’ and private organisations such as NHS Test and Trace. Collective efficacy is a group’s shared belief in its capability to organise and execute actions required to achieve goals [28], and a sense of collective efficacy appeared to be stronger in the smaller, more cohesive organisations such as the schools and GP practices than in the University, for example. Schools and GP practices had staff and student communities within which members through daily conversation and mutual encouragement made regular testing the ‘social norm’. The educational engagement activities appeared to play an important role in this too. Students’ increased knowledge appeared to make them more engaged with the programme and motivated by the need to protect their communities, as well as making them feel like they had more agency in controlling the spread of the virus and its damaging consequences. Schools and the University were thus building collective efficacy for regular testing.

A rapid qualitative evaluation enabled real-time feedback of insights into perceptions and functioning of the testing programme, improving the participants’ experience and was therefore likely to increase uptake of testing. Whilst some of these insights confirm what has been learnt during previous epidemics, our study was necessary to understand more precisely how to improve specific features of the programme which would facilitate participation. Table 3 details how insights gathered through the interviews and focus group discussions led to modifications to the programme in order to increase engagement. Insights gained were also used to inform university, schools, and city-wide strategies for managing the pandemic and to feed-back to national and local government [31]. This included evaluation reports to the UK Department of Health and Social Care [31] and presentations to the UK Department for Education based on the findings in this rapid qualitative evaluation. Figure 1 illustrates the way in which the rapid qualitative evaluation influenced Saliva Testing Programme development and wider testing strategy. This study has also produced actionable recommendations for improving engagement with COVID-19 testing programmes across the UK (Table 3). See Table 4 for details of these recommendations.

Fig. 1
figure 1

How participants’ experiences informed the Southampton COVID-19 Saliva Testing Programme wider strategy

Full size image
Table 3 Modifications to the Saliva Testing Programme
Full size table
Table 4 Recommendations from the Southampton Saliva Testing Programme to ensure mass engagement in testing for COVID-19
Full size table

Strengths and limitations

The speed with which insights were needed to optimise the testing programme meant that a full and rigorous traditional qualitative analysis of the data was not possible. Steps taken to ensure that our findings were robust included regular team meetings to discuss interpretations, checking these interpretations with participants in focus groups and interviews and consultation with experts from a wide range of backgrounds. These steps are part of an established process for conducting rapid qualitative evaluation [16]. Accessing those who declined to take part in the programme was challenging, and we may not have identified all the reasons people have for not taking part in this type of testing programme. Those who were consulted, however, offered important insights that informed significant programme development. In common with other qualitative data, ours may not represent the views of all participants and decliners of the saliva testing programme. It is the intention of qualitative research to represent instead the range and diversity of participant views, that we believe is what this study has achieved. Future evaluations of testing programmes should seek to engage more people who decline to be tested and explore participants’ experiences of indirect NHS Test and Trace contact tracing and household isolation as these are key components of the effectiveness of population testing.

Conclusions

High levels of communication, trust and convenience were necessary to ensure people’s engagement with the programme. Community leaders and stakeholder organisations should be involved throughout programme development and implementation to optimise these features of the testing. Participants’ and stakeholders’ motivations, challenges and concerns need to be understood and these insights used to modify the programme in a continuous, real-time process to ensure and sustain engagement with testing over the extended period necessary.