Background

Antenatal Care (ANC) comprises a number of activities and procedures aimed at preserving the health of pregnant women while ensuring delivery of preventive measures, early detection of complications and the adequate management of pre-existing maternal disease states [1]. The reduction of maternal mortality remains a priority under “Goal 3: Ensure healthy lives and promote well-being for all at all ages” in the new Sustainable Development Goals (SDGs) agenda through 2030 [2]. In this context, information on maternal and child health is fundamental to ensure antenatal care quality and reduction of maternal and perinatal mortality. In Brazil, a pregnancy card records health information and details of antenatal and postpartum care visits. The card is a tool included in the Prenatal and Birth Humanization Program, part of the Ministry of Health medical record systems, and its use is mandatory to ensure that healthcare for pregnant women meets the national minimum quality standards. The card includes information on ANC visits, diagnostic findings and patient behaviors and links the different stages of the health care process. The systematic collection of data during each consultation can therefore be used to provide optimal health services at any point of care.

An alternative method for collection of data on health care received during the pregnancy-postpartum cycle is the use of individual questionnaires. The in-person interview or questionnaire is frequently used in epidemiology research because it provides a comprehensive and standardized assessment of the topic of interest. It is a relatively inexpensive tool that enables collection of data from a large population sample.

Medical records are considered the gold standard and are generally the preferred data source over interviews/questionnaires. However, evidence indicates that reliability of any individual source of data is imperfect, possibly inaccurate, and thus the combination of sources may improve reliability and completeness of information [1, 3,4,5,6]. The objective of this study was to evaluate for the first time the agreement between self-reported questionnaire information and data recorded in the ANC cards from women in the 2015 city of Pelotas Birth Cohort study, Rio Grande do Sul, Brazil.

Materials and methods

Study settings

The data is representative of births in the municipality of Pelotas in 2015. However, this birth cohort is not representative of the total number of births in Brazil for that period. In 2015, 5598 newborn babies were identified, of whom 4329 were children of mothers who were residents in the urban area of Pelotas (cohort study targeted population) and were invited to participate. The refusal rate was 1.3% in the perinatal period.

Sampling

This was a cross-sectional study based on the mothers of children eligible for the 2015 Pelotas Birth Cohort who were interviewed during pregnancy and/or at birth. The total number of eligible pregnant mothers was 4329 (in the case of multiple pregnancies only one record was kept for each mother). Fifty-nine records corresponded to multiple births, totaling 4270 mothers in the birth cohort. Of these mothers 4172 had attended ANC but 249 did not have an antenatal card available at the time of interview, resulting in a final sample of 3923 mothers included in this study (Additional file 1: Figure S1).

The questionnaires used during prenatal and perinatal visits of the 2015 Pelotas Birth Cohort are available at http://epidemio-ufpel.org.br/site/content/coorte_2015/questionarios.php. More detailed information on the cohort methods and follow-up visits is provided elsewhere [7].

Variables

The information was collected through interviews during pregnancy and/or the perinatal period. During antenatal care, two antenatal visits were performed, one before gestational week 16 and the second between gestational weeks 16 and 24. The third follow-up was at delivery. One questionnaire was used at weeks 16, 24 and at delivery to collect sociodemographic characteristics including maternal age, education, marital status, skin color, family income, pre-pregnancy health, and if the mother was seen by the same health professional during ANC and type of ANC provider. The perinatal questionnaire asked for reproductive health history variables – delivery characteristics, number of pregnancies, and if the mother was seen by the same health professional during ANC and type of ANC provider. ANC data were also collected at delivery and documented service utilization, details of clinical examination, occurrence of disease during pregnancy, prophylactic interventions (vaccination and supplements), and educational interventions (counseling). Additional file 2: Table S1 shows the questionnaire contents and data collection tools.

At the end of the interview the ANC card was photographed and information on ANC routines extracted and double entered into a database (Epi Info 6.04, Centers for Disease Control and Prevention, Atlanta, USA) by two trained research assistants. A consistency analysis with a frequency verification was performed using Stata 12® (Statacorp, College Station, TX, USA).

If there was no record of a given procedure on the card, it was considered as not performed. The agreement analysis was carried out only when the card was available with at least one record present. The questionnaire items and antenatal card variables used in this study are described in Additional file 3: Table S2.

Statistical analysis

The agreement between the card and the questionnaire for each variable of interest was estimated using Kappa coefficient (κ) with corresponding 95% confidence intervals (95% CI). Kappa and Prevalence-Adjusted-and-Bias-Adjusted Kappa (PABAK) [8] values were interpreted using Landis and Koch categorization [9] into almost perfect (>0.80), substantial (0.61–0.80), moderate (0.41–0.60), fair (0.21–0.40), slight (0.00–0.20) and poor (<0.00). Sensitivity and specificity are reported with 95% CI and were calculated assuming the ANC card to be the gold standard or assuming the questionnaire to be the gold standard.

The association of maternal characteristics with an agreement between the card and the questionnaire was evaluated by building a score ranging from 1 to 10 using those ANC indicators with adjusted Kappa coefficients in the categories of almost perfect to moderate (ten indicators as shown in Table 3). A value of 1 was assigned to each indicator agreement (1 = yes/yes or no/no). The score was dichotomized into low (≤7 points as zero) and high agreement (≥ 8 points as one) to conduct bivariate analysis using the score as the dependent variable. Heterogeneity chi-squared tests were used to measure the difference between low and high agreement categories. The analyses were performed in Stata 15® (Statacorp, College Station, TX, USA).

Results

Sample characteristics

Within the cohort of 3923 women 82.6% were between 20 and 39 years of age, 14.4% were under 20 years of age, 71.9% were white, 33.7% had up to 8 years of schooling and 86.4% were living with a partner at the time of interview. With regard to socioeconomic data, 59.7% were between the first and third quintiles of family income. In relation to obstetrical data, 50.4% of the women were primiparous, 64.8% had undergone c-section delivery, 52.8% were seen by the same health professional during ANC, and 45.0% received ANC in the public health sector. When comparing our sample with mothers who did not attend or receive ANC and lacked an antenatal card, a statistically significant difference for maternal education, marital status, family income, and parity was found (Table 1). Table 2 shows the distribution of data from ANC cards and from the self-reported questionnaire.

Table 1 Maternal characteristics in the 2015 Pelotas Birth Cohort, Brazil
Full size table
Table 2 Distribution of ANC indicator data by information source
Full size table

Agreement between ANC card indicators and questionnaire information

From the 21 indicators in the ANC card we observed that agreement strength with self-reported questionnaire information was poor in 1 indicator (K < 0.00), slight for 16 (K between 0.00–0.20), fair for 2 (K between 0.21–0.40), and moderate for 2 (K between 0.41–0.60) (Table 3). The agreement strength category obtained with the PABAK was higher than that obtained with the Kappa coefficient in 9 (43%) indicators – antenatal care number of visits ≥6, reported date of last menstrual period, weight, symphysis-fundal height, blood pressure, syphilis test, hypertension during pregnancy, diabetes during pregnancy, and sexually transmitted diseases (Table 3). Agreement strength categories for urinary tract infection during pregnancy, breast exam, hepatitis B vaccination, and counseling on physical activities such as walking were moderate, slight, fair, and poor respectively, and were equally categorized by PABAK and Kappa values. Agreement strength categories were lower with PABAK relative to those obtained with Kappa for gynecological exam, cervical cancer screening test, counseling on risks of alcohol use during pregnancy and counseling on risks of smoking during pregnancy (Table 3).

Table 3 Concordance measures of ANC indicators from the ANC card and self-reported questionnaire information
Full size table

Sensitivity and specificity

Self-reported questionnaire with the ANC card as the gold standard

The sensitivity of the self-reported questionnaire was high (>90%) for 28.6% of the indicators evaluated, all of them in the components of service use and clinical examination. Good sensitivity (≥80 and < 90%) was present in 23.8% and low sensitivity (<80%) in 47.6% of the indicators. Specificity was low (<80%) for all indicators except for diabetes and sexually transmitted diseases that had >90% specificity (Table 4.).

Table 4 Validity of ANC indicators from self-reported questionnaire information or the antenatal card
Full size table

Antenatal card with the self-reported questionnaire as the gold standard

The sensitivity of antenatal card was high (>90%) for 19.0% of the indicators evaluated, all of them in the clinical examination component. The number of visits showed good sensitivity (81.1%) and the rest presented low sensitivity. Specificity was high for 42.8% of the indicators, mainly those related to diseases during pregnancy and counselling. Good specificity (≥80 and < 90%) was present in 19.0% of the indicators (Table 4).

Maternal factors associated with an agreement between data sources

Table 5 presents the maternal characteristics associated with the concordance between the ANC card and self-report questionnaire. There was a significantly higher agreement among pregnant women with younger age, more years of maternal schooling, primiparous status and those using a public sector health care provider.

Table 5 Maternal characteristics associated with agreement between the ANC card and self-reported questionnaire
Full size table

Discussion

Main findings

To our knowledge, this is one of the few studies evaluating the agreement between self-reported questionnaire from pregnant women and the antenatal card that recorded procedures and interventions during ANC [10, 11]. The assessment of agreement between sources of information on ANC assistance is an issue that has aroused the interest of the global scientific community. Our key findings indicate firstly, that assessment of ANC from self-reported questionnaire has moderate to high agreement to ANC card indicators of service utilization, clinical examination and diseases during pregnancy whereas indicators of counseling performed poorly. Secondly, self-reported information presented higher data frequency and higher sensitivity but slightly lower specificity when compared to the ANC card. Thirdly, factors associated with higher agreement between both data sources included young maternal age, more maternal schooling years, being a primiparous mother, and health care received in the public sector.

Moderate to high agreement on indicators of service utilization, clinical examination and diseases during pregnancy. Poor agreement on indicators of counseling

The moderate to high agreement in these indicators suggests that assessment of ANC through self-reported questionnaire or the antenatal card could be equivalent. However, caution is advised when interpreting these results, especially for indicators related to diseases during pregnancy. As has been shown in other studies [12], the performance of antenatal clinical exams such as measurements of weight, symphysis-fundal height, and blood pressure reached almost perfect agreement [10, 12]. These findings may be related to the ability of the patient to identify the reason for the ANC procedure/action recorded by the health professional during the ANC visit. For example, self-reporting weight may be more accurate than self-reporting a cervical cancer screening test. Patients are more likely to understand that weight is being measured when asked to step on a scale but less likely to discern whether a cervical cancer screening test is the reason for a gynecological examination [13,14,15]. Indeed we found a very low agreement on reporting of gynecological exam, cervical cancer screening test and breast exam (Table 3). Given that the frequency of these procedures was lower in the ANC cards relative to self-reports, the low agreements may also stem from poor record-keeping by health professionals or that these tests, that are mandatory at the national level, were not carried out, as reported in other studies [10,11,12,13,14, 16,17,18,19,20,21,22,23]. This would raise serious concerns regarding the quality of healthcare that women and their offspring receive.

Efforts to improve the quality of care for pregnant women in Brazil include a National Oral Health Policy [24] which mandates a dental consultation when initiating ANC. Our study found a fair agreement. However, the results should be analyzed with caution as 70.9% (1002/1412) of the pregnant women who reported having consulted with a dentist during pregnancy also participated in the oral health sub-study of this cohort [7], therefore in the general population agreement could be lower than reported.

For gestational diabetes, agreement was classified as almost perfect, a finding confirmed by other studies [25, 26] where agreement was often higher than those reported for other chronic diseases [27, 28]. Studies evaluating self-reported medical information have shown that patients can provide reasonably good reports on their illnesses [29, 30]. Recent publications on urinary tract infection have shown moderate agreement between self-reporting and medical records with a low sensitivity [31, 32], similar to the findings obtained by our study.

Some conditions such as diabetes, hypertension, anemia, and urinary tract infection may not have been recorded on the ANC cards, but women still refer to having experienced them. Similar findings have been reported in other studies [10, 33, 34]. One possible reason for over reporting of gestational diabetes is that the diagnosis requires a positive result on two different glucose tests. Thus, women with a positive first test but negative on the second may have reported gestational diabetes. Sexually transmitted diseases may have been under-reported possibly due to stigmatization.

Counselling indicators were more frequently self-reported than reported in the antenatal card. Reports of counselling pertaining to alcohol use and smoking during pregnancy had the lowest Kappa values and the highest difference between sources, perhaps due to a low probability of reporting of these habits to healthcare providers during antenatal care. Previous studies have shown that counseling and referrals (e.g., cervical cancer screening tests) are underreported in medical records compared to self-reported records, indicating that health professionals do not consistently register these interventions [35, 36].

. Self-reported information presented higher data frequency and higher sensitivity but slightly lower specificity when compared to the ANC card

The use of questionnaires in epidemiologic studies is considered a valid tool with many advantages for research. However, the quality of the information obtained by self-reporting is dependent on the type of disease [37, 38], the characteristics of the participants [30, 39], the design of the questionnaire, and the method used to administer it [40, 41]. In addition, information obtained by self-reporting or by review of medical records may not be consistent. Several studies have shown that the agreement between the two sources of information is dependent on the type of variable collected [42,43,44].

When self-report was compared to the card as gold standard, we found that specificity and sensitivity were > 80% for 9.5% (2/21) and 52% (11/51) of ANC indicators respectively. The high sensitivity in this comparison may be related to the use of self-administered questionnaires as sources of information on exposures and outcomes, which may be more complete and with fewer omissions in responses by participants.

When we compared the antenatal card with self-report as the gold standard, we found that the specificity and sensitivity were > 80% for 60% (13/21) and 25% (5/21) of ANC indicators respectively. This high specificity suggests that data recorded on the cards corresponded to the most relevant aspects identified during medical consultation. The low sensitivity suggests that registration of other ANC procedures/activities was incomplete. One reason may be that data recorded on antenatal cards do not accurately reflect all events occuring during a medical visit [45, 46]. Furthermore the content of antenatal cards is not standardized for data collection in a private versus public setting or designed for data collection by health professionals with responsibility for specific aspects of ANC.

Despite these limitations, data collected in the ANC card are preferable for making decisions on potentially significant clinical interventions during antenatal and perinatal stages [47].

. Maternal characteristics associated with the agreement between the ANC card and the self-reported questionnaire

Sociodemographic characteristics may contribute towards the strength of agreement between data sources and determine ANC quality [48]. Our findings show that higher maternal age, lower educational status, pregnant women with ≥2 children and type of healthcare provider associate with a lower probability of high agreement between the antenatal card and the self-reported questionnaire. We were unable to find other studies on factors associated with ANC indicator agreements for the data sources used in our study. However, there is evidence that the demand for health services and ANC quality are typically related to the sociodemographic characteristics of pregnant women [49].

Limitations and implications

Due to the intrinsic limitations of the data sources analyzed herewith, judgements about their respective validity should be made with caution.

The quality of medical record documentation may be affected by omissions in reporting, for example tests may have been performed elsewhere and not transcribed onto the antenatal card or incomplete documentation due to time constraints for record-keeping by health professionals. In addition to problems with recording-keeping, accuracy of medical record extraction may be compromised by bias, fatigue or distraction of the systematic reviewer. These findings suggest that a purely quality-based assessment on medical records data could fail to find information on performed procedures and interventions that the pregnant woman might have supplied. This can result in the loss of important information on some ANC indicators.

The quality of an evaluation that depends solely on self-reported information can be influenced by recall bias, by subjectivity or misunderstanding of the questions pertaining to diagnoses, procedures and interventions received during ANC. In addition, the accuracy of a self-report may be affected by culturally influenced factors such as the importance of events, awareness and knowledge of health conditions [50].

Conclusions

Our study shows that data from questionnaires or the ANC cards provide substantially different information on indicator performance. From a public health perspective this raises questions with regard to reliance on indicators derived from a single data source. It may be more prudent to assess the quality of ANC programs using multiple data sources to determine quality and effectiveness of health promotion and disease prevention programs in pregnant women and their offspring. Researchers should explore alternative methods and data sources to obtain consistent estimates of ANC quality, various different indicators, be aware of factors that may influence the accuracy of data sources and conduct sub-studies to collect such data when not available. The implications of choosing a questionnaire or medical records should be carefully considered when evaluating health services for clinical practice, research, and public health. Deciding which data source to use will also depend on the outcome of interest and if the data is used for clinical decision-making, performance tracking, or public health.