Abstract
There are numerous scientific articles of studies on the prevalence of disorders with non-standardised examination and diagnostic protocols. Because their quality is heterogeneous, a new instrument has been developed for the assessment of such studies. The new instrument is based mainly on statistical criteria. The points assigned for each of the main criteria according to the information gained from each paper are summed up to form a Total Quality Score (TQS). The interrater reliability of the instrument was tested by employing Kappa and Interrater Correlation Coefficient (ICC) statistics. The latter was assessed on the results of three independent investigators. The new quality instrument appeared to be easy to use, and the instructions were comprehensible. The ICC(2,1) for the TQS ranged between 0.94 and 1.00 indicating almost perfect agreement between the investigators. The reliability of the new instrument enables its use for scientific review purposes. In this way, its validity will also be tested. The instrument could be adopted for assessment of scientific articles of studies on the prevalence of disorders in many, similar, scientific areas.
Similar content being viewed by others
References
Gross AJ, Rivera-Morales WC, Gale EN (1988) A prevalence study of symptoms associated with TM disorders. J Craniomandib Disord 2:191–195
Cherkin DC, Deyo RA, Wheeler K, Ciol MA (1994) Physician variation in diagnostic testing for low back pain. Who you see is what you get. Arthritis Rheum 37:15–22
Koes BW, van Tulder MW, Ostelo R, Kim Burton A, Waddell G (2001) Clinical guidelines for the management of low back pain in primary care: an international comparison. Spine (Phila Pa 1976) 26:2504–2513, Discussion, 2513–2504
Bamji AN, Erhardt CC, Price TR, Williams PL (1996) The painful shoulder: can consultants agree? Br J Rheumatol 35:1172–1174
Luime JJ, Koes BW, Hendriksen IJ, Burdorf A, Verhagen AP, Miedema HS, Verhaar JA (2004) Prevalence and incidence of shoulder pain in the general population; a systematic review. Scand J Rheumatol 33:73–81
Egger M, Smith G, Altman D (2009) Systematic reviews in health care: meta-analysis in context, 2nd edn. BMJ Books, London
von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC, Vandenbroucke JP (2007) The strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet 370:1453–1457
Shamliyan T, Kane RL, Dickinson S (2010) A systematic review of tools used to assess the quality of observational studies that examine incidence or prevalence and risk factors for diseases. J Clin Epidemiol 63:1061–1070
Boyle MH (1998) Guidelines for evaluating prevalence studies. Evid Based Mental Health 1:37–39
Beller EM, Gebski V, Keech AC (2002) Randomisation in clinical trials. Med J Aust 177:565–567
Krogh-Poulsen WG (1969) Management of the occlusion of the teeth. Examination, diagnosis, treatment. In: Chayes LSCM (ed) Facial pain and mandibular dysfunction. Saunders, Philadelphia, pp 251–258
Helkimo M (1974) Studies on function and dysfunction of the masticatory system. Ii. Index for anamnestic and clinical dysfunction and occlusal state. Sven Tandläk Tidskr 67:101–121
Levitt SR, Lundeen TF, McKinney MW (1988) Initial studies of a new assessment method for temporomandibular joint disorders. J Prosthet Dent 59:490–495
Levitt SR, McKinney MW, Lundeen TF (1988) The TMJ scale: cross-validation and reliability studies. Cranio 6:17–25
Lundeen TF, Levitt SR, McKinney MW (1986) Discriminative ability of the TMJ scale: age and gender differences. J Prosthet Dent 56:84–92
Fricton JR, Schiffman EL (1986) Reliability of a craniomandibular index. J Dent Res 65:1359–1364
Fricton JR, Schiffman EL (1987) The craniomandibular index: validity. J Prosthet Dent 58:222–228
McNeill C (1993) Temporomandibular disorders-guidelines for classification, assessment and management, 2nd edn. Quintessence Publishing, Chicago
Okeson J (1996) Orofacial pain: guidelines for assessment, diagnosis, and management. Quintessence Publishing, Chicago
Dworkin SF, LeResche L (1992) Research diagnostic criteria for temporomandibular disorders: review, criteria, examinations and specifications, critique. J Craniomandib Disord 6:301–355
Wahlund K, List T, Dworkin SF (1998) Temporomandibular disorders in children and adolescents: reliability of a questionnaire, clinical examination, and diagnosis. J Orofac Pain 12:42–51
Schmitter M, Kress B, Leckel M, Henschel V, Ohlmann B, Rammelsberg P (2008) Validity of temporomandibular disorder examination procedures for assessment of temporomandibular joint status. Am J Orthod Dentofacial Orthop 133:796–803
Dworkin SF, Sherman J, Mancl L, Ohrbach R, LeResche L, Truelove E (2002) Reliability, validity, and clinical utility of the research diagnostic criteria for Temporomandibular Disorders Axis II Scales: depression, non-specific physical symptoms, and graded chronic pain. J Orofac Pain 16:207–220
John MT, Dworkin SF, Mancl LA (2005) Reliability of clinical temporomandibular disorder diagnoses. Pain 118:61–69
Schmitter M, Ohlmann B, John MT, Hirsch C, Rammelsberg P (2005) Research diagnostic criteria for temporomandibular disorders: a calibration and reliability study. Cranio 23:212–218
List T, John MT, Dworkin SF, Svensson P (2006) Recalibration improves inter-examiner reliability of tmd examination. Acta Odontol Scand 64:146–152
Cohen J (1960) A coeffiecient of agreement for nominal scales. Educ Psychol Meas 20:37–46
Shrout PE, Fleiss JL (1979) Intraclass correlations: uses in assessing rater reliability. Psychol Bull 86:420–428
Aggarwal VR, Macfarlane TV, Macfarlane GJ (2003) Why is pain more common amongst people living in areas of low socio-economic status? A population-based cross-sectional study. Br Dent J 194:383–387, Discussion, 380
Dworkin SF, Huggins KH, LeResche L, Von Korff M, Howard J, Truelove E, Sommers E (1990) Epidemiology of signs and symptoms in temporomandibular disorders: clinical signs in cases and controls. J Am Dent Assoc 120:273–281
Heikinheimo K, Salmi K, Myllarniemi S, Kirveskari P (1989) Symptoms of craniomandibular disorder in a sample of Finnish adolescents at the ages of 12 and 15 years. Eur J Orthod 11:325–331
Locker D, Grushka M (1987) The impact of dental and facial pain. J Dent Res 66:1414–1417
Nilsson IM, List T, Drangsholt M (2005) Prevalence of temporomandibular pain and subsequent dental treatment in Swedish adolescents. J Orofac Pain 19:144–150
Pedroni CR, De Oliveira AS, Guaratini MI (2003) Prevalence study of signs and symptoms of temporomandibular disorders in university students. J Oral Rehabil 30:283–289
Rantala MA, Ahlberg J, Suvinen TI, Savolainen A, Kononen M (2004) Chronic myofascial pain, disk displacement with reduction and psychosocial factors in finnish non-patients. Acta Odontol Scand 62:293–297
Raustia AM, Peltola M, Salonen MA (1997) Influence of complete denture renewal on craniomandibular disorders: a 1-year follow-up study. J Oral Rehabil 24:30–36
Sadowsky C, Muhl ZF, Sakols EI, Sommerville JM (1985) Temporomandibular joint sounds related to orthodontic therapy. J Dent Res 64:1392–1395
Tallents RH, Hatala M, Katzberg RW, Westesson PL (1993) Temporomandibular joint sounds in asymptomatic volunteers. J Prosthet Dent 69:298–304
Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33:159–174
Schouten H (1986) Nominal scale agreement among observers. Psychometrika 51:453–466
DuRant R (1994) Checklist for the evaluation of research articles. J Adolesc Health 15:4–8
Downs SH, Black N (1998) The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J Epidemiol Community Health 52:377–384
Macfarlane TV, Glenny AM, Worthington HV (2001) Systematic review of population-based epidemiological studies of oro-facial pain. J Dent 29:451–467
Genaidy A, Lemasters GK, Lockey J, Succop P, Deddens J, Sobeih T, Dunning K (2007) An epidemiological appraisal instrument - a tool for evaluation of epidemiological studies. Ergonomics 50:920–960
Juni P, Altman D, Egger M (2001) Assessing the quality of controlled trials. BMJ 323:42–46
Berlin J, Rennie D (1999) Measuring the quality of trials: the quality of quality scales. JAMA 282:1083–1085
Greenland S (1994) Quality scores are useless and potentially misleading. Am J Epidemiol 140:300–302
Moher D, Pham B, Jones A, Cook DJ, Jadad AR, Moher M, Tugwell P, Klassen TP (1998) Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet 352:609–613
Whiting P, Harbord R, Kleijnen J (2005) No role for quality scores in systematic reviews of diagnostic accuracy studies. BMC Med Res Methodol 5:19–27
Tooth L, Ware R, Bain C, Purdie DM, Dobson A (2005) Quality of reporting of observational longitudinal research. Am J Epidemiol 161:280–288
Soares HP, Daniels S, Kumar A, Clarke M, Scott C, Swann S, Djulbegovic B (2004) Bad reporting does not mean bad methods for randomised trials: observational study of randomised controlled trials performed by the radiation therapy oncology group. BMJ 328:22–24
Nguyen Q, Bezemer P, Habets L, Prahl-Andersen B (1999) A systematic review of the relationship between overjet size and traumatic dental injuries. Eur J Orthod 21:503–515
Ariëns G, van Mechelen W, Bongers P, Bouter L, van der Wal G (2000) Physical risk factors for neck pain. Scand J Work Environ Health 26:7–19
Loney P, Chambers LW, Benett KJ, Roberts JG, Stratford PW (1998) Critical appraisal of the health research literature: prevalence or incidence of a health problem. Chronic Dis Can 19:170–176
Woodbury M, Houghton PE (2004) Prevalence of pressure ulcers in Canadian healthcare settings. Ostomy/Wound Manage 50:22–38
Silva LC, Ordunez P, Paz Rodriguez M, Robles S (2001) A tool for assessing the usefulness of prevalence studies done for surveillance purposes: the example of hypertension. Rev Panam Salud Pública 10:152–160
Barclay P, Hollender LG, Maravilla KR, Truelove EL (1999) Comparison of clinical and magnetic resonance imaging diagnosis in patients with disk displacement in the temporomandibular joint. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 88:37–43
Schmitter M, Kress B, Rammelsberg P (2004) Temporomandibular joint pathosis in patients with myofascial pain: a comparative analysis of magnetic resonance imaging and a clinical examination based on a specific set of criteria. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 97:318–324
Emshoff R, Rudisch A (2001) Validity of clinical diagnostic criteria for temporomandibular disorders: clinical versus magnetic resonance imaging diagnosis of temporomandibular joint internal derangement and osteoarthrosis. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 91:50–55
Acknowledgements
The authors of this article would like to acknowledge the dentists J. Mahabadi, A. Hassel, R. Shahin, and W. Bömicke for their help with assessment of the new instrument's reliability.
Conflict of interest
The authors declare that they have no conflict of interest.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Giannakopoulos, N.N., Rammelsberg, P., Eberhard, L. et al. A new instrument for assessing the quality of studies on prevalence. Clin Oral Invest 16, 781–788 (2012). https://doi.org/10.1007/s00784-011-0557-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00784-011-0557-4