Background

Provision of quality of healthcare services is the tenet of success in patient outcomes. The importance of quality of healthcare services in the nursing profession and by the nursing professional has been addressed since the time of Florence Nightingale [1]. In more recent times, the nursing scholars and the nursing administrators and leaders have brought the concept of Quality Nursing Care (QNC) to the frontline of the nursing profession. Several milepost achievements such as establishment of the National Database of Nursing Quality Indictors by the American Nursing Association (ANA) in 1998 or the formation of the Task and Finish Group for a rapid appraisal of the evidence on nursing measures in April of 2012, at the behest of the Chief Nursing Officer, by the UK National Health Services underline the prominence of QNC in the nursing profession [2, 3]. The indicators for QNC developed by the ANA or by the UK National Health Services, can be used as guidelines for the nursing professionals in other countries, but their applications might not totally be relevant and/or applicable to countries such as Mongolia. For example, some of the proposed QNC indicators are abstruse and difficult to either utilize and/or measure. This argument is supported by the extent of publications, by the nursing professionals across the globe, addressing QNC in various healthcare settings [4,5,6].

Delivery healthcare services by the nursing professionals is complex. This complexity arises from the wide range of healthcare services, physical, psychological, emotional, social and spiritual care, delivered by the nursing professionals [7]. Additionally, the diversity in patients’ needs adds another layer of complexity to the delivery of healthcare services and to the concept of QNC [1, 7, 8]. Moreover, variations in cultural and social norms and political ambiances across the globe adds another tier of complexity to the delivery of healthcare services by the nursing professionals and to the concept of QNC. Finally, QNC is a subjective perception, at least in some of the domains of nursing care. This subjectivity can be influenced by personal experiences and social and cultural norms and values of patients and/or the nursing management [9,10,11]. Therefore, application of a universal set of quality indicators for assessment of QNC in different countries can not be justified; because this application potentially can lend itself to spurious and deleterious results. We report here the development and field testing of an instrument to assess the psychometric properties of the Quality Nursing Care Scale (QNCS) in Mongolia, hereafter, referred to as QNCS-M.

Methods

Research design and approach

We implemented a two-phase exploratory sequential mixed-method design to develop and validate QNCS-M. During the phase I of this project, we dedicated time and effort to develop an instrument merited on the 5-step proposed by DeVellis [12] while, phase II was dedicated to ascertaining psychometric performance of QNCS-M.

Phase I: instrument design and development

This phase began by drafting the blue-print of QNCS-M which was based on the overall concept of QNC, first proposed by Kunaviktikul [7]. However, the components of QNCS-M and its 5 dimensions were established based on the results of a previously implemented qualitative study in Mongolia and conducting a comprehensive literature review [13]. We then proceeded with establishing the operational definition of each of the five dimensions of QNCS-M. These operational definitions were unanimously agreed by the members of our research team. We defined the Physical Care as the interventions provided by the nursing professionals with the objective of symptom management and assistance with activities of daily life. The operational definition of Psychological Care was focused on activities provided by nursing professionals that encourage patients in assuming self-directed activities to relieve mental distress. We defined the operational operation of Emotional Care as supports and activities provided by the nursing professionals to understand and to support the frame of minds and emotional states of patients because of their illness. We explicated the operational definition of Social Care as the process of building positive social relationships with the patients and creating a positive and supportive ambiance to prevent depression among patients due to their sense of isolation. Finally, the operational of definition of Spiritual Care was established as recognition of and respect for patients’ religious beliefs and proclivity. In addition, we included the practice of acceptance and respect for patients’ cultural diversity by the nursing professionals in the operational definition of Spiritual Care. This step was followed by drafting and scaling of a total of 66 items in English language. Each item was scaled based on a five-point Likert scale, ranging 0–4. The sum of scores reflected the level of quality of nursing care with value of 0 indicating the lowest quality of nursing healthcare services, while the value of 264 indicated the highest possible level.

The English translated version of QNCS-M was reviewed and evaluated by a panel of seven academicians from US, Thailand and Mongolia with expertise in instrument development and content validity assessment. Per the recommendations of the panel, seven questions were deleted because the item-content validity index (I-CVI) < .78 [14]. Item-content validity indices (I-CVIs) of the remaining items ranged from 0.85 to 1.0. The revised version of QNCS-M was translated back to Khalkha dialect and was reviewed by a panel of 10 nursing academicians and directors in Mongolia. The panel assessed our instrument for its clarity, readability, accuracy and its length. The panel of reviewers flagged 12 items as ambiguous. These 12 items were revised and restructured to ascertain clarity in Khalkha dialect. The latest version of this 59-item instrument was pretested among 20 nursing professionals working in the State Central Third Hospital in Mongolia. The Cronbach’s alpha coefficients of the instrument was .94.

Phase II: instrument psychometric performance

Assessment of the psychometric performance of QNCS-M consisted of two steps. During step one, we focused on field testing of the 59-item instrument for its construct validity and internal consistency reliability. Field Testing of Instrument was conducted in nine public hospitals in city of Ulaanbaatar. These hospitals with 984 strong nursing professionals are secondary and tertiary level hospitals and serve as the referral system for the entire country [15]. A total of 485 nursing professionals using multi-stage sampling technique were randomly identified and recruited to take part in our study [16]. First, we applied a proportional random sampling approach to determine the number of participants; our sampling approach was adjusted for the size of the nursing staff in each hospital. In the second step, proportionate stratified sampling method was used to select the number of nurses from different departments in each hospital. Finally, prospective study participants randomly were selected from the name list of each nursing department. Only those who met the eligibility criteria were recruited to participate in our study. We have outlined location of the participating hospitals in the city of Ulaanbaatar, Mongolia (Fig. 1, Additional file 1: Appendix A). Additionally, the list of hospitals and percentage of sample size from each hospital are provided in Additional file 1: Appendix A.

Fig. 1
figure 1

Location of the Participating Hospitals in the City of Ulaanbaatar, Mongolia

Full size image

In calculating the total sample size, we assumed participant-to-item ratio of 7:1 [17], we further inflated our sample size by 18% to account for the sample attrition due to no-response. This approach was justified based on previous reports suggesting a sample size of 300–500 as adequate for factor analysis procedure [12, 18]; additionally, the relatively small number of the nursing professionals (n = 985) working in the nine public hospitals was another justification for our sample size calculation.

The nursing professionals were eligible to participate in our study if: A) They were actively employed by the public hospital system in the city of Ulaanbaatar; B) They were performing their professional duties in one of the nine accessible secondary or tertiary care hospitals in Mongolia; C) They had at least 1 year of work experience in direct patient-care health services and D) They were willing to participate in our study. Nurses who had participated in the pre-testing stage were not eligible to participate. This restriction was imposed to reduce the likelihood of information bias.

A trained research assistant introduced the objective of the study to the prospective study participants and obtained their signed informed consent forms. The research assistant then distributed copies of our 59-item QNCS-M and Marlowe-Crowne Social Desirability Scale (MCSDS) and Demographic questionnaires to the study participants and instructed them to return the completed instruments within 2-weeks in sealed envelopes in the drop-boxes which were placed in the Nursing Departments of the participating hospitals. Returned surveys were anonymous. This section of our study started in February of 2019 and was completed in December 2019.

Statistical analysis

We used descriptive statistics to summarize the demographic characteristics of the study participants and distributions of responses to each item listed in our 59-item instrument. Inter-item correlation was examined by using the Spearman rank-order correlation. Item-total correlation was calculated by using reliability analysis. The criterion for selecting qualified items to constitute a consistent scale included item-total correlation and a corrected item-subscale correlation of .30 or higher [18]. Inter-item correlations ranged from .30 to .70 [19]. The internal consistency reliability was analyzed using Cronbach’s alpha coefficient. A reliability coefficient above .70 is considered acceptable for a new scale [12]. Finally, we applied Spearman rank-order correlation used to assess the relationship between MCSDS score and each item of QNCS-M.

We then proceeded with calculating the Kaiser-Meyer-Olkin (KMO) value to ascertain the adequacy of our sample size. Additionally, we performed Bartlett’s test of sphericity to evaluate for item redundancy of QNCS-M [20]. The criteria for determining the factor solution of factor extraction included: 1) a factor with an eigenvalue of 1.00 or above, 2) item with a factor loading of .40 or above, 3) no factor with fewer than three items [21]. Finally, we performed exploratory factor analysis (EFA) to assess construct validity of QNCS-M. This approach permitted us to investigate if the internal structure of the scale of QNCS-M fit with the original theoretical definition and conceptual framework [7].

Step 2. Performance comparison against QNCQ-RN, the criterion tool

In this step, we assessed the empirical relationship between QNCS-M and the criterion tool, the Quality of Nursing Care Questionnaire-Registered Nurse (QNCQ-RN) [22]. The QNCQ-RN has been widely to assess quality nursing care instruments in hospital settings [23, 24]. Permission was obtained from the authors; QNCQ-RN was translated into Khalkha dialect and then was translated back to English to ascertain the validity of the translated version.

We recruited 54 nursing professionals from State Central First Hospital, Bayangol District Health Center and Bayanzurkh District General Hospital. We imposed three inclusion criteria:1) Providing healthcare services in the role of a nursing professional and 2) Having at least 1 year of work experience in direct patient-care health services and 3) Willingness to participate in our study. The sample size of 54 was based on the expected correlation coefficient value of 0.4, power of 80% and, alpha = .05; our calculation yielded a minimum sample size of 46 [25]; however, we inflated our sample size by 18% to account for non-responders.

The lead investigator introduced the objective of this segment of our study and copies of QNCS-M and the QNCQ-RN were distributed among the study participants. The study participants were instructed to return the completed instruments within 2-weeks in sealed envelopes in the drop-boxes which were placed in the Nursing Departments of the two participating hospitals. All the 54 participants returned their anonymous completed questionnaires; the returned questionnaires were evaluated by the lead investigator for completeness; data were entered into the databases for statistical analysis. To assess the validity scale of QNCS-M, we calculate the correlation between QNCS-M and the QNCQ-RN using the Spearman rank-order statistical technique [26]. Our calculation yielded a statistically significant correlation between the two instruments (r = .28, P < .03).

Ethical considerations

This study was approved by the Research Ethics Committee of the Faculty of Nursing, Chiang Mai University (Approval-020/2019), the Bio-Medical Research Ethics Committee of Mongolian National University of Medical Sciences, and the participating hospitals in Mongolia. Informed written consents were obtained from all participants. Confidentiality of study participants were respected by the research team. All personal identifiers were deleted after the final data quality and assurance. Data were stored in a password protected database behind the IT firewall of the Mongolian National University of Medical Sciences.

Results

Instrument psychometric performance

Of the 485 study participants, 456 (94%) returned their questionnaires; we excluded 16 (3.2%) of the returned questionnaires because of missing information. Questionnaires from a total of 440 (90.7%) participants were complete and deemed as acceptable for further analysis. The mean age of study participants was 36 years (SD = 10; range 22–59). The majority of the nurses were female (n = 431, 98.0%), and more than half of the subjects were married (n = 313, 71.1%). More than half of the nursing professionals (n = 262, 59.5%) who contributed to our study, had earned a bachelor’s degree in nursing science (Table 1).

Table 1 Demographic Characteristics of the Nurses (n = 440)
Full size table

Item analysis, item-total correlation and inter-item correlation of QNCS-M

Results from the item analysis, yielded item mean values ranging from 2.57 to 4.64 (SD = .57–2.01). Findings from the total-item correlation analysis yielded coefficient values ranging between .25 to .64. For three items, #5 (r = .26), # 18 (r = .29) and # 42 (r = .25), the total-item correlation coefficient values were less than .3 [18]; therefore, we decided against including these three items in the revised version of QNCS-M. A total of 56 items comprised the latest version of QNCS-M. Findings from inter-item correlation were assessed and coefficient values between a minimum value of .30 and a maximum value of .7 were considered as acceptable [19]. Our calculation yielded correlation coefficient values for 12 items below .3. These 12 items were removed which reduced the number of items in QNSC-M from 56 to 44 items. (Additional file 1: Appendix B).

Results from the Spearman rank-order correlation analysis of the remaining 44 items, indicated relationship between MCSDS and 6 items, #Q11 (r = .11, p = .02), #Q20 (r = −.10, p = .03), #Q31 (r = .09, p = .05), #Q35 (r = .10, p = .02), #Q56 (r = .10, p = .03) and #Q57 (r = −.09, p = .03). We decided to exclude these items to reduce the likelihood of potential information bias. A total of 38 items remained for exploratory factor analysis. (Additional file 1: Appendix B).

Exploratory factor analysis

Results from Kaiser-Meyer-Olkin measure of sampling adequacy yielded a value of .90, which indicates a strong support for the adequacy of items (n = 38) for factor analysis [21]; furthermore, results from Bartlett’s test of sphericity (Xi2 = 7021.82, P = .0001) supported that the items were in a linear relationship [19]. Therefore, we proceeded with performing exploratory factor analysis (EFA). Our EFA analysis of the 38 items, yielded eigenvalues greater than 1.0 for a total of 9 domains. However, upon further assessment of the results of EFA, three domains (factors, 7–9) contained items fewer than the minimum requirements of 3 items; thereby, we eliminated these domains [21].

Results from scree plot factor analysis showed six domains above the elbow of the curve. Therefore, we decided to retain the six domains within the overall structure of the QNCS-M. These six domains were labeled as: “Independent Nursing Role”, “Inter-dependent Nursing Role”, “Psychological Element”, “Social Milieu”, “Personal Milieu” and “Spiritual Force”. Two items, # 27 and 36, in the 38 QNCS-M showed no loading (coefficient < .4) to any of the six domains. These two items (#27 and 36) were excluded, yielding to a 36-item QNCS-M. A second EFA was conducted to ascertain loading adequacy of these 36 items. Results indicated that these 36 items maintained the loading values greater than .40. These six domains explained 55.41% of the total variance with eigenvalues ranging between 9.86 to 1.17 (Table 2).

Table 2 Item descriptions, item-total correlations, Cronbach’s alpha and factor loadings (n = 440)
Full size table

Reliability assessment and concurrent validity

Results from internal consistency reliability of QNCS-M yielded an overall Cronbach’s alpha coefficient value of .92; while, the coefficient values for the six domains ranged between .72 and .85 (Table 2). Test-retest reliability of the field-tested items was evaluated among 40 volunteers 2 weeks after the completion of field testing. Results from Pearson correlation for test-retest reliability yielded a coefficient value of .82 (r = .82, P < .001). Finally, we detected a positive and statistically significant correlation (r = .28, P = .03) between QNCS-M and the QNCQ-RN, the reference criterion. This correlation further lends supporting evidence of the concurrent validity of our instrument.

Discussion

We implemented a sequential exploratory mixed-method study with the primary objective of developing a valid instrument to assess Quality Nursing Care in Mongolia. The necessity of developing a Quality Nursing Care Scale that targets the Mongolian healthcare system has been a priority for the Mongolian nursing and the other healthcare professionals for some time. Implementation of internationally developed instruments in Mongolian hospitals is not practical primarily because of three reasons. First, health practices in Mongolia, similar to any other nation, are influenced by its history, its culture and its social norms and values. Second, perception of Mongolian patients about healthcare services and quality of these services are influenced by their religious health practices and cultural values about health and well-being. Third, delivery and quality of healthcare services in Mongolia, similar to other nations, are influenced by its political structure and its economy. Therefore, we rest our justification for developing a Quality Nursing Care Scale in Mongolian (QNCS-M) based on the incompatibility of the already developed scales with the Mongolian society [8, 22, 27]. The development of QNCS-M is a major stride in addressing the concerns of the Mongolian healthcare professionals and administration.

The premise of QNCS-M was an extensive based on a previously implemented qualitative research and literature review [13]. However, the final version of QNCS-M, which contains 36 items and loaded into 6 domains, was morphed to the specifics of Mongolian healthcare systems and cultural values about the quality of healthcare services. Initially we attempted to combine the first (interdependent nursing role) and second (independent nursing role) domains because both domains were addressing the concept of physical care. However, findings from our factor analysis suggested strong correlation within each domain, most likely due to the number of items within each factor; therefore, we opted for listing these two domains separately. The QNCS-M is quick to complete, the average completion time is about 15 min, and easy to score.

The QNCS-M was designed to demonstrate conformities with previously developed quality nursing care instruments and yet to address the specifics of the Mongolian healthcare system. For instance, the importance of application of quality of standards and attention to the physical status of patients have been reported as important in creating positive perceptions about the quality of healthcare services [28]. Several previously developed Quality of Nursing Care Scales have allocated one domain, labeled as Physical Care, to assess the quality of healthcare services delivered by the nursing professionals [8, 22, 29]. However, in designing and developing QNCS-M, we decided to allocate two domains, Independent Nursing Role and Interdependent Nursing Role, to assess the quality of healthcare services delivered by Mongolian nursing professionals. We justified our decision based on the adequacy of numbers of the items and the strong correlation among the items within each domain. But equally important, our decision was based on the independent role of the Mongolian nursing professionals in the delivery of healthcare services that are classified under the domain of Independent Nursing Role. The Mongolian nursing health professionals are the sole responsible entities in decision making and delivery of nursing care such as feeding, toileting, bathing, transferring and dressing. While, the Inter-dependent Nursing Role emphasizes the collective approach and inter-dependency of healthcare professionals in assessing patients’ health status, delivering healthcare services and managing their health conditions.

Limited or no concern from the nursing professionals about patients’ psychological wellbeing, i.e., limited patient-communication, can negatively affect perception of quality of healthcare services delivered by the nursing professionals [28]. The importance of patients’ psychological states in the delivery of healthcare services, provision of information and/or assessing patients’ physical well-being have been well documented, emphasized and applied in scales such as GNCS [8, 22, 27, 29]. In consequence, we designed and developed QNCS-M to capture the most significant aspect of psychological needs and psychological care that are congruent with Mongolian cultural health values and healthcare practices. For example, QNCS-M contains items that address the nursing professionals’ responsibilities for psychological state of patients or their roles in inspiring patients to recovery and boosting their confidence in assuming and maintaining self-care.

Supportive environment, human interaction and religiosity/spirituality are important components of quality of healthcare services; furthermore, these attributes of a healthcare system which reflect the quality of services can accelerate the recovery rate and improve overall health of patients [30, 31]. Under the domain of Social Milieu, the focus of quality of nursing care is on the efforts and attention of the nursing professionals in creating a social environment for patients to promote patients’ safety and to ascertain their security and privacy. In the domain of Personal Milieu, the focus of quality of nursing care is on the efforts and attention of the nursing professionals on establishing constructive communication and “human touch” with their patients. The domain of Personal Milieu of QNCS-M is comparable to either psychological or emotional domains of previously developed scales [8, 22, 27, 29]. Finally, in the domain of Spiritual Force of the QNCS-M the focus of quality of nursing care pivots on the awareness of the nursing professionals about their patients’ religious affiliation. Additionally, attention has been given to the nursing professionals’ respect and support for their patients’ desires to perform their religious rituals and religious cultural inclination, i.e., use of religious healers or observance of dietary restrictions. The QNSC-M has dedicated one full domain to the concept of religiosity/spirituality and the quality of nursing care. This is in contrast to previously published scales, in which the concept of religiosity/spirituality is embedded within the domains of emotional care or psychological domain [22, 27, 29].

Conclusion

QNCS-M is a 6-domain, 36-item instrument designed and developed to assess the perception of quality of healthcare services delivered by the nursing professionals in publicly owned and operated hospital systems in Mongolia. QNCS-M demonstrates a high level of content and construct validity with acceptable reliability. The QNCS-M is quick to complete, the average completion time is about 15 min, and easy to score.

The development of QNCS-M is a major stride in addressing the concerns of the Mongolian healthcare professionals and administration.

Strengths and limitations

The main strength of our study is its design approach which involved both the nursing professionals and patients in generation of items. Our study has two limitations. First, the instrument was validated only by the nursing professionals. Second the study was implemented in the hospital systems, owned and operated by the Mongolian government. Therefore, its applicability might be limited to other settings such as private for profit or not-for-private hospitals.

Implications

QNCS-M is the segue to improvement of quality of healthcare services delivered by the nursing professionals in Mongolia. Although QNCS-M was designed and developed for application in Mongolian society, this newly developed instrument can be utilized as a guideline in other countries with healthcare systems similar to Mongolia. Finally, QNCS-M can be used in academic nursing focusing as an educational tool in teaching the concept of quality nursing healthcare services.