Background

Rheumatic fever (RF) is an autoimmune disease which may follow group A Streptococcus pyogenes (GAS) pharyngitis [1, 2]. Repeated RF episodes may cause rheumatic heart disease (RHD). RHD may be prevented through long-term antibiotic prophylaxis, usually with monthly administration of intramuscular benzathine penicillin G (BPG) [3].

RF and RHD produce a significant burden of disease in New Zealand (NZ) [4, 5]. RF is associated with social deprivation and household crowding. It almost exclusively affects Maori and Pacific children [5, 6].

The NZ Government aims to reduce the RF incidence by two-thirds, to 1.4 cases per 100,000 people, by June 2017 [7]. However, it is actually unclear what the true rate of RF is, or how current interventions are altering it, due to major flaws affecting every RF surveillance system operating on a national scale [8–11].

Public health surveillance is vital for supporting infectious disease prevention and control measures [12–20]. Surveillance may be defined as, ‘…the on-going systematic collection, analysis, interpretation and dissemination of data regarding a health-related event for use in public health action to reduce morbidity and to improve health’ [20]. Surveillance activities can be broadly categorised into two types: control-focused and strategy-focused [13]. Control-focused surveillance aims to identify affected individuals to enable treatment. Strategy-focused surveillance aims to provide information to support prevention strategies, such as improving housing conditions [13].

In NZ there are three major systems used to monitor RF: national hospitalisation data [21]. national notification data (based on notifications to Medical Officers of Health) [22] and regional patient registers [9]. Both initial episodes of RF and recurrences are notifiable in NZ. RHD in children and young adults less than 20 years old is also legally notifiable [23]. Extensive under-reporting has been well documented in RF notification data [11, 24] and register data [10, 25]. Miscoding and misdiagnoses affect hospitalisation data, which may over-count cases by 25%-33% [8, 10].

From 2007, the International Health Regulations, released by the World Health Organisation, required member states to assess their core capacity for surveillance [26]. A surveillance sector review framework was devised in 2010 as a means for supporting countries to meet this assessment goal. This methodology seeks to systematically review the surveillance sector as a whole. It has a number of advantages over reviewing individual surveillance systems; chiefly that it can detect gaps which may otherwise be missed, such as identifying important events that are not covered by existing systems. This approach also identifies issues affecting multiple surveillance systems, such as a lack of information integration [13].

We have successfully used this novel framework to perform a surveillance sector review with the aim of identifying potential improvements which would support optimal RF control and prevention activities.

Methods

Surveillance sector review framework

The surveillance sector review framework contains seven stages, described in Figure 1. A key feature of this framework is that it covers all stages of the disease pathway (Figure 2). When reporting on qualitative components, we ensured that the RATS guidelines for reporting on qualitative studies were adhered to.

Figure 1
figure 1

Surveillance sector review framework (From Baker et al. [13]).

Full size image
Figure 2
figure 2

Revised summary of the RF causal pathway and potential surveillance points.

Full size image

Planning the surveillance sector review

This project was commissioned by the NZ Ministry of Health. They helped to refine the aims and identify important surveillance roles and data end-users.

Full ethics approval was granted by the University of Otago Ethics Committee. Interviews with key informants (KIs) considered to have experience and expertise in RF prevention, control or surveillance were a major information source. An initial list of potential KIs and important surveillance sector roles was first devised during meetings with senior MoH staff in the Communicable Diseases team. This list included people working in a diverse range of roles across the sector, from national policy advisors to local level clinical service providers. Additional KIs were identified by reading RF reports and asking other KIs whether they knew of anyone else who should be interviewed.Information gained in interviews was used to identify and describe existing RF surveillance systems, assess their performance, describe information needs of end-users, identify surveillance gaps and prioritise improvements. Consultations with the Ministry of Health and within our research team were necessary when assigning priorities for improvement. Finally recommendations on potential surveillance sector improvements were made to the Ministry of Health.

Collecting data

The surveillance sector review framework (Figure 1) permitted the burden of disease to be divided into categories according to whether it was directly attributable to GAS pharyngitis, RF or RHD.Literature searches identified existing surveillance systems which could provide information on RF disease outcomes, hazards, interventions and determinants. This information was used to construct show cards displaying key points in the perceived RF causal pathway (Figure 2) and identify what sources (if any) collect surveillance information at these points.Fifty people were invited to participate via email. Fourteen either did not respond or suggested somebody else to speak to instead. Thirty-six people indicated they would like to participate and were interviewed. They were assured they would remain anonymous throughout the research process and the information they provided was confidential. Data collection ceased when the interviewer felt a ‘saturation point’ had been reached, whereby little new information was attained during interviews. Interviews were recorded on a Dictaphone. Show cards were presented to KIs, who were asked to suggest how the accuracy of these representations could be improved. Suggestions were noted and some incorporated into the show cards following discussion among the research team. The refined description of the surveillance sector is shown in Figure 2.

KIs were asked what RF surveillance information they and their agencies required. Future requirements were discussed. KIs were asked to assess the performance of the current RF surveillance systems, describe gaps, and discuss the perceived effects of these limitations. Ideas for closing gaps were discussed. Where appropriate, the interviewer would ask the KIs their opinion on ideas which other KIs had suggested for closing the gap in question. The interviewer attempted to ask open questions and respond respectfully and neutrally to the KI at all times. Responses covered steps 3–5 of the surveillance sector review framework.

KIs were presented with a third show card outlining major categories of surveillance gaps, as identified by Baker et al. [13]. KIs were asked to select three gap categories which they regarded as top priorities for RF surveillance improvement.

We also performed a quantitative assessment of the major RF surveillance systems (notifications, hospitalisations, registers) and the distribution of RHD in cases younger than 20 years of age. These analyses are described elsewhere [24].

Analysing data

Interview recordings were transcribed and summarized as a list of bullet points and direct quotes. These lists were emailed to the KI with a request that they altered them until they felt that their viewpoint had been accurately represented. Final versions of the interview summaries were read. Themes were noted under major headings corresponding to questions asked during the interview. Where an issue was addressed by multiple KIs, the proportion who responded in the same way (for example, with agreement) was recorded in a semi-quantitative manner, using a scale with seven levels. These levels ranged through ‘none’, ‘a few’, ‘several’, ‘about half’, ‘many’, ‘most’ and ‘almost all’ KIs. All surveillance gaps noted were identified in each set of interview notes. The researchers noted which major category each gap fell under, according to whether it concerned a surveillance system’s leadership or resourcing, design or structure, or operation and function.The frequency that major gap categories were selected was graphed (Figure 3). More consideration was given to gaps that many KIs had identified as being high-priority problems. Discussion with the KIs, among the research team and with the Ministry of Health led to recommendations being made to the Ministry of Health on how to address surveillance sector gaps.

Figure 3
figure 3

Priorities for surveillance sector improvement.

Full size image

Results

The surveillance sector review framework supported a comprehensive review of the NZ RF surveillance sector. We found the process straightforward and allowed us to identify potential surveillance improvements which would support optimal RF control and prevention activities.

Characteristics of responders

All 36 KIs had experience and expertise in aspects of RF surveillance. They occupied a wide range of positions across the country and held national, regional and local level roles. Primary healthcare providers, surveillance system operators and strategic information users were included.

Description of RF surveillance systems and their aims

The RF surveillance sector has a strong emphasis on control-focussed surveillance. Notification and register data were the most commonly identified control-focussed surveillance systems. These support delivery of antibiotic prophylaxis to identified RF cases. Echocardiography screening similarly supports patient management by identifying undiagnosed cases. GAS pharyngitis screening is emerging as a major national disease control strategy. Identifying household contacts of RF cases in order to treat GAS positive individuals is another, less well established, aim of control-focussed surveillance activities.

Strategy-focussed RF surveillance is limited by the scope of existing monitoring systems. Commonly identified monitoring systems include hospitalisation data, mortality data, systems monitoring GAS virulence and antibiotic resistance, and systems monitoring household crowding, socioeconomic conditions and population factors (e.g. the census). For the full list of identified NZ RF surveillance systems, see Additional file 1: Table S1.

Surveillance information needs of major end users

Not surprisingly, primary healthcare providers and clinicians expressed a need for information on RF and GAS pharyngitis rates in their localities. System operators needed very timely, accurate and complete information on new cases. Strategic information users needed information on national and regional disease rates and intervention strategies so they could rapidly assess and compare progress. All KIs felt that up-to-date, reasonably detailed epidemiological information should be available at a national and regional scale with accompanying information, such as throat swabbing program coverage information, to put it into context.

Performance of existing rheumatic fever surveillance systems

These systems perform poorly in several important areas. Gaps in control-focused surveillance include the inability to effectively track highly mobile cases, lack of consistency in patient management and data collection across District Health Boards (responsible for health care delivery and public health services at local and regional levels). A limited ability to monitor compliance with secondary prophylaxis was also noted. An absence of systems to monitor general practitioner’s delivery of secondary prophylaxis was considered a gap.

The key strategy-focused surveillance gap was the inability to generate consistent national and regional case totals. All KIs thought under-notification of RF cases is a key problem. The major reason for under-notification was attributed to a lack of prevention or patient management related incentives to notify. Many KIs pointed out that checking case notes before notifying may improve the accuracy of notification data. Limited monitoring of RF recurrences was widely criticised. Many thought it would be a great improvement if the notification form could be transmitted electronically.

KI’s views were supported by a parallel project which quantitatively assessed the performance of RF surveillance systems. This capture-recapture analysis found hospitalisation data were 82% sensitive when detecting RF cases, and notification data 61% sensitive. PPVs for the datasets were 71% and 88% respectively [24].

Although RHD in cases younger than 20 years is notifiable, [22, 23] many KIs were unaware of this and described confusion surrounding which conditions were notifiable. Around half of all patients hospitalised for RHD had not been previously hospitalised with RF, so are likely to have missed out on the benefits of antibiotic prophylaxis [24]. Clarifying that RHD in younger cases is notifiable could help improve RF control.

Other strategy-focused surveillance gaps included a lack of information on GAS pharyngitis and carriage rates. Uncertainty about the significance of GAS pharyngitis in areas considered high-risk for RF was a common concern. The very limited ability to monitor the burden exerted by RHD was criticised.

Surveillance sector gaps

The most frequently identified surveillance sectors gaps concerned leadership and coordination, information integration, analysis and communication (Figure 3).

Leadership and resource gaps

Many KIs said only recently had there been visible national advocacy promoting and co-ordinating RF prevention and control initiatives. Most felt that until now, leadership in this field was largely left up to the Heart Foundation. Many pointed out, "There is no person with ownership of all that data and collection systems who is able put it all together, instead everybody is just doing their own little bit." A common theme was that the Ministry of Health needs to decide on one surveillance platform (such as notification data or a national register) and use it to take a consistent approach to RF surveillance. To ascertain the best way to monitor RF, the Ministry of Health should consult key players, then work to implement the agreed strategy nationally. KIs agreed, "Change must be driven at a national level if it is to occur", as, "Leadership and co-ordination are vital elements in disease prevention and control".

Most thought protocols should be clarified and standardised. "There should be more consistency in how people are trained, ensuring staff understand RF as a whole".

Design and structural gaps

Almost all KIs felt limited RHD surveillance was a major gap. Comprehensive RHD surveillance could be achieved through combining data from RF registers, cardiology departments and echocardiography records.

Nearly all KIs strongly felt that prevention interventions should primarily operate at the primordial level (ie. should aim to remove risk factors themselves, thus minimising the likelihood of disease occurring). This approach is likely to not only prevent RF but reduce many other diseases associated with deprivation; however, "We should only monitor determinants if it would help direct action in a cost-effective fashion."

Almost all KIs thought it highly worthwhile to monitor rates of GAS pharyngitis and relate these to RF rates, especially in high incidence areas. Concerns were raised about the coverage of throat swabbing programs. A lack of control sites to compare intervention areas with was commonly identified as a gap limiting the ability to monitor the effect of throat swabbing programs.

Researching GAS emm types, carrier transmission and other host and microbial characteristics was commonly proposed to improve understandings of RF.

As the rate of positive contact swabs is not monitored; we cannot know whether contact tracing RF cases is a worthwhile activity. Most felt this was an important knowledge gap.

Educating high-risk people and their doctors about RF primary prevention and monitoring awareness levels was thought to be highly beneficial. Several KIs thought epidemiological information about disease determinants should be better linked with RF surveillance data. Multiple sources of information should be reported on in order to clearly capture trends.

The effectiveness of secondary prophylaxis programs is not monitored. This was a widespread concern. The information collected about secondary prophylaxis and its delivery should be standardised. The limited ability to ensure continuing treatment of cases transferring to a new region was a deep concern shared by all KIs. In addition, several mentioned, "Some registers do not incorporate non-compliant cases very well, so attempts to track them may be hampered." All agreed there should be a system ensuring cases receive free, easy-to-access BPG.

Operational/functional gaps

Almost all KIs criticised the surveillance systems’ inability to evaluate the effect of interventions or to identify true changes in the RF incidence rate. Major criticisms were that rates are published using flawed data and calculations may be performed in different ways, therefore rate comparisons may be inappropriate. Misinterpretation of data was also a concern. Most KIs with local-level roles felt gaining access to RF surveillance information is, "Complicated and ineffective".

Priorities for surveillance sector improvements

Specific recommended improvements to RF surveillance in NZ are listed in Table 1. Support for a national register was overwhelming, on the grounds that it would improve the ability to treat patients and produce standardised surveillance information as a by-product of its patient management objective. It was often said, "It is silly and pointless to have notifications going to two different data sources".

Table 1 Specific rheumatic fever surveillance improvements identified by the surveillance sector review
Full size table

Discussion

Key findings and implications

Applying the surveillance sector review framework to RF was successful at both a technical and managerial level. At a technical level it successfully identified necessary changes to surveillance systems that constitute the RF surveillance sector (Table 1). As noted, a strength of this approach is that it can identify issues that only emerge when looking at multiple related surveillance systems, rather than at individual systems [13]. The existing surveillance systems fail to fulfill their purposes and thus require modification.

At a managerial level, results of this review have been used to advise Government policy makers in an evidence-based fashion, placing an emphasis on the opinions of a wide range of RF surveillance information end-users. This method therefore provides a foundation from which the Ministry of Health can take a collegial approach for making surveillance improvements.

There is a need for national leadership and co-ordination to support development of an effective, integrated, RF surveillance sector. The Ministry of Health appears best placed to lead this development. A comprehensive RF surveillance strategy should be developed in order to communicate and coordinate these improvements. Consultation with key stakeholders is vital when developing this strategy to ensure changes are useful and acceptable. Any improvements made must be supported with on-going mandating and funding if they are to be maintained.

A tangible way to begin integrating surveillance activities would be to implement a periodic, national RF surveillance report. This report could relate regional and local RF rates to interventions, intermediate outcomes and RHD. Reporting would help increase the profile of RF and inform end users of the current situation and progress.

There is a clear need for a well-run online national RF register. This tool has potential to close many of the surveillance gaps we noted. The RF notification dataset is effectively operating as a national register already. This web-based national data system has most of the critical features required of a national register, and is supported by the legislative requirement for RF notification. To improve sensitivity, it would be useful to supplement this dataset with automatic reporting based on hospitalisation data as a check on notifications. All reported cases would need to meet standard case-definition requirements to improve the validity of the data. The main additional functional requirements are for greater case management utility, revision of data fields and associated documentation, and processes to ensure effective clinical engagement. Furthermore, if cardiac status could be included on the register and kept up-to-date, then changes in the prevalence of RHD and in the severity of cardiac damage could be monitored. If cases or their families were asked simple questions at the time of notification (e.g. about exposure to risk factors, recent pharyngitis, and participation in intervention programs) then the register could also record this information and so provide a means of evaluating the effects of risk factors and interventions.

Limitations

The surveillance sector review was comprehensive and included interviews with a large number of KIs (n = 36), but inevitably we could not interview all individuals with valuable perspectives on RF surveillance in NZ. Subjects held a diverse range of roles throughout the RF surveillance sector. Despite this, many common viewpoints were identified. Most agreed with findings of previous literature concerning the surveillance sector, indicating KIs were well-informed. Many also ventured their own views and ideas, indicating that they had spent time considering matters individually as well.

Further research

This surveillance sector review identified a ‘grey’ boundary between surveillance and research. Some end-users with policy roles had a great need for strategy-focussed information that might be better met by specific research studies rather than on-going surveillance. There is a need to develop a better understanding of the contribution of ‘upstream’ determinants and risk factors to the burden of disease produced by RF. Some of these questions could be monitored through expanded surveillance, but others are better suited to specific research studies, such as a case–control study.

The surveillance sector review approach could be equally effective in resource poor countries where disease such as RF have a particularly high health burden [27, 28]. The surveillance sector review framework works well when reviewing the capacity to effectively monitor a single condition. It would be interesting to apply it to other health-related conditions, or even review several conditions simultaneously.

Conclusions

The surveillance sector review approach facilitated a thorough review of the RF surveillance sector. It supported wide sector engagement in the review process, which should facilitate adoption of the proposed changes. The surveillance sector review approach could be added to the small set of tools currently available for developing and evaluating surveillance systems [20]. This new approach is likely to prove useful as we confront the challenges of combating new emerging infectious, responding to global environmental changes, and reducing health inequalities.

Consent statement

Written informed consent to be interviewed was obtained from all participants.