Skip to main content

Advertisement

Log in

A new instrument for assessing the quality of studies on prevalence

  • Original Article
  • Published:
Clinical Oral Investigations Aims and scope Submit manuscript

Abstract

There are numerous scientific articles of studies on the prevalence of disorders with non-standardised examination and diagnostic protocols. Because their quality is heterogeneous, a new instrument has been developed for the assessment of such studies. The new instrument is based mainly on statistical criteria. The points assigned for each of the main criteria according to the information gained from each paper are summed up to form a Total Quality Score (TQS). The interrater reliability of the instrument was tested by employing Kappa and Interrater Correlation Coefficient (ICC) statistics. The latter was assessed on the results of three independent investigators. The new quality instrument appeared to be easy to use, and the instructions were comprehensible. The ICC(2,1) for the TQS ranged between 0.94 and 1.00 indicating almost perfect agreement between the investigators. The reliability of the new instrument enables its use for scientific review purposes. In this way, its validity will also be tested. The instrument could be adopted for assessment of scientific articles of studies on the prevalence of disorders in many, similar, scientific areas.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Gross AJ, Rivera-Morales WC, Gale EN (1988) A prevalence study of symptoms associated with TM disorders. J Craniomandib Disord 2:191–195

    PubMed  Google Scholar 

  2. Cherkin DC, Deyo RA, Wheeler K, Ciol MA (1994) Physician variation in diagnostic testing for low back pain. Who you see is what you get. Arthritis Rheum 37:15–22

    Article  PubMed  Google Scholar 

  3. Koes BW, van Tulder MW, Ostelo R, Kim Burton A, Waddell G (2001) Clinical guidelines for the management of low back pain in primary care: an international comparison. Spine (Phila Pa 1976) 26:2504–2513, Discussion, 2513–2504

    Article  Google Scholar 

  4. Bamji AN, Erhardt CC, Price TR, Williams PL (1996) The painful shoulder: can consultants agree? Br J Rheumatol 35:1172–1174

    Article  PubMed  Google Scholar 

  5. Luime JJ, Koes BW, Hendriksen IJ, Burdorf A, Verhagen AP, Miedema HS, Verhaar JA (2004) Prevalence and incidence of shoulder pain in the general population; a systematic review. Scand J Rheumatol 33:73–81

    Article  PubMed  Google Scholar 

  6. Egger M, Smith G, Altman D (2009) Systematic reviews in health care: meta-analysis in context, 2nd edn. BMJ Books, London

    Google Scholar 

  7. von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC, Vandenbroucke JP (2007) The strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet 370:1453–1457

    Article  Google Scholar 

  8. Shamliyan T, Kane RL, Dickinson S (2010) A systematic review of tools used to assess the quality of observational studies that examine incidence or prevalence and risk factors for diseases. J Clin Epidemiol 63:1061–1070

    Article  PubMed  Google Scholar 

  9. Boyle MH (1998) Guidelines for evaluating prevalence studies. Evid Based Mental Health 1:37–39

    Article  Google Scholar 

  10. Beller EM, Gebski V, Keech AC (2002) Randomisation in clinical trials. Med J Aust 177:565–567

    PubMed  Google Scholar 

  11. Krogh-Poulsen WG (1969) Management of the occlusion of the teeth. Examination, diagnosis, treatment. In: Chayes LSCM (ed) Facial pain and mandibular dysfunction. Saunders, Philadelphia, pp 251–258

    Google Scholar 

  12. Helkimo M (1974) Studies on function and dysfunction of the masticatory system. Ii. Index for anamnestic and clinical dysfunction and occlusal state. Sven Tandläk Tidskr 67:101–121

    PubMed  Google Scholar 

  13. Levitt SR, Lundeen TF, McKinney MW (1988) Initial studies of a new assessment method for temporomandibular joint disorders. J Prosthet Dent 59:490–495

    Article  PubMed  Google Scholar 

  14. Levitt SR, McKinney MW, Lundeen TF (1988) The TMJ scale: cross-validation and reliability studies. Cranio 6:17–25

    PubMed  Google Scholar 

  15. Lundeen TF, Levitt SR, McKinney MW (1986) Discriminative ability of the TMJ scale: age and gender differences. J Prosthet Dent 56:84–92

    Article  PubMed  Google Scholar 

  16. Fricton JR, Schiffman EL (1986) Reliability of a craniomandibular index. J Dent Res 65:1359–1364

    Article  PubMed  Google Scholar 

  17. Fricton JR, Schiffman EL (1987) The craniomandibular index: validity. J Prosthet Dent 58:222–228

    Article  PubMed  Google Scholar 

  18. McNeill C (1993) Temporomandibular disorders-guidelines for classification, assessment and management, 2nd edn. Quintessence Publishing, Chicago

    Google Scholar 

  19. Okeson J (1996) Orofacial pain: guidelines for assessment, diagnosis, and management. Quintessence Publishing, Chicago

    Google Scholar 

  20. Dworkin SF, LeResche L (1992) Research diagnostic criteria for temporomandibular disorders: review, criteria, examinations and specifications, critique. J Craniomandib Disord 6:301–355

    PubMed  Google Scholar 

  21. Wahlund K, List T, Dworkin SF (1998) Temporomandibular disorders in children and adolescents: reliability of a questionnaire, clinical examination, and diagnosis. J Orofac Pain 12:42–51

    PubMed  Google Scholar 

  22. Schmitter M, Kress B, Leckel M, Henschel V, Ohlmann B, Rammelsberg P (2008) Validity of temporomandibular disorder examination procedures for assessment of temporomandibular joint status. Am J Orthod Dentofacial Orthop 133:796–803

    Article  PubMed  Google Scholar 

  23. Dworkin SF, Sherman J, Mancl L, Ohrbach R, LeResche L, Truelove E (2002) Reliability, validity, and clinical utility of the research diagnostic criteria for Temporomandibular Disorders Axis II Scales: depression, non-specific physical symptoms, and graded chronic pain. J Orofac Pain 16:207–220

    PubMed  Google Scholar 

  24. John MT, Dworkin SF, Mancl LA (2005) Reliability of clinical temporomandibular disorder diagnoses. Pain 118:61–69

    Article  PubMed  Google Scholar 

  25. Schmitter M, Ohlmann B, John MT, Hirsch C, Rammelsberg P (2005) Research diagnostic criteria for temporomandibular disorders: a calibration and reliability study. Cranio 23:212–218

    PubMed  Google Scholar 

  26. List T, John MT, Dworkin SF, Svensson P (2006) Recalibration improves inter-examiner reliability of tmd examination. Acta Odontol Scand 64:146–152

    Article  PubMed  Google Scholar 

  27. Cohen J (1960) A coeffiecient of agreement for nominal scales. Educ Psychol Meas 20:37–46

    Article  Google Scholar 

  28. Shrout PE, Fleiss JL (1979) Intraclass correlations: uses in assessing rater reliability. Psychol Bull 86:420–428

    Article  PubMed  Google Scholar 

  29. Aggarwal VR, Macfarlane TV, Macfarlane GJ (2003) Why is pain more common amongst people living in areas of low socio-economic status? A population-based cross-sectional study. Br Dent J 194:383–387, Discussion, 380

    Article  PubMed  Google Scholar 

  30. Dworkin SF, Huggins KH, LeResche L, Von Korff M, Howard J, Truelove E, Sommers E (1990) Epidemiology of signs and symptoms in temporomandibular disorders: clinical signs in cases and controls. J Am Dent Assoc 120:273–281

    PubMed  Google Scholar 

  31. Heikinheimo K, Salmi K, Myllarniemi S, Kirveskari P (1989) Symptoms of craniomandibular disorder in a sample of Finnish adolescents at the ages of 12 and 15 years. Eur J Orthod 11:325–331

    PubMed  Google Scholar 

  32. Locker D, Grushka M (1987) The impact of dental and facial pain. J Dent Res 66:1414–1417

    Article  PubMed  Google Scholar 

  33. Nilsson IM, List T, Drangsholt M (2005) Prevalence of temporomandibular pain and subsequent dental treatment in Swedish adolescents. J Orofac Pain 19:144–150

    PubMed  Google Scholar 

  34. Pedroni CR, De Oliveira AS, Guaratini MI (2003) Prevalence study of signs and symptoms of temporomandibular disorders in university students. J Oral Rehabil 30:283–289

    Article  PubMed  Google Scholar 

  35. Rantala MA, Ahlberg J, Suvinen TI, Savolainen A, Kononen M (2004) Chronic myofascial pain, disk displacement with reduction and psychosocial factors in finnish non-patients. Acta Odontol Scand 62:293–297

    Article  PubMed  Google Scholar 

  36. Raustia AM, Peltola M, Salonen MA (1997) Influence of complete denture renewal on craniomandibular disorders: a 1-year follow-up study. J Oral Rehabil 24:30–36

    Article  PubMed  Google Scholar 

  37. Sadowsky C, Muhl ZF, Sakols EI, Sommerville JM (1985) Temporomandibular joint sounds related to orthodontic therapy. J Dent Res 64:1392–1395

    Article  PubMed  Google Scholar 

  38. Tallents RH, Hatala M, Katzberg RW, Westesson PL (1993) Temporomandibular joint sounds in asymptomatic volunteers. J Prosthet Dent 69:298–304

    Article  PubMed  Google Scholar 

  39. Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33:159–174

    Article  PubMed  Google Scholar 

  40. Schouten H (1986) Nominal scale agreement among observers. Psychometrika 51:453–466

    Article  Google Scholar 

  41. DuRant R (1994) Checklist for the evaluation of research articles. J Adolesc Health 15:4–8

    Article  PubMed  Google Scholar 

  42. Downs SH, Black N (1998) The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J Epidemiol Community Health 52:377–384

    Article  PubMed  Google Scholar 

  43. Macfarlane TV, Glenny AM, Worthington HV (2001) Systematic review of population-based epidemiological studies of oro-facial pain. J Dent 29:451–467

    Article  PubMed  Google Scholar 

  44. Genaidy A, Lemasters GK, Lockey J, Succop P, Deddens J, Sobeih T, Dunning K (2007) An epidemiological appraisal instrument - a tool for evaluation of epidemiological studies. Ergonomics 50:920–960

    Article  PubMed  Google Scholar 

  45. Juni P, Altman D, Egger M (2001) Assessing the quality of controlled trials. BMJ 323:42–46

    Article  PubMed  Google Scholar 

  46. Berlin J, Rennie D (1999) Measuring the quality of trials: the quality of quality scales. JAMA 282:1083–1085

    Article  PubMed  Google Scholar 

  47. Greenland S (1994) Quality scores are useless and potentially misleading. Am J Epidemiol 140:300–302

    Google Scholar 

  48. Moher D, Pham B, Jones A, Cook DJ, Jadad AR, Moher M, Tugwell P, Klassen TP (1998) Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet 352:609–613

    Article  PubMed  Google Scholar 

  49. Whiting P, Harbord R, Kleijnen J (2005) No role for quality scores in systematic reviews of diagnostic accuracy studies. BMC Med Res Methodol 5:19–27

    Article  PubMed  Google Scholar 

  50. Tooth L, Ware R, Bain C, Purdie DM, Dobson A (2005) Quality of reporting of observational longitudinal research. Am J Epidemiol 161:280–288

    Article  PubMed  Google Scholar 

  51. Soares HP, Daniels S, Kumar A, Clarke M, Scott C, Swann S, Djulbegovic B (2004) Bad reporting does not mean bad methods for randomised trials: observational study of randomised controlled trials performed by the radiation therapy oncology group. BMJ 328:22–24

    Article  PubMed  Google Scholar 

  52. Nguyen Q, Bezemer P, Habets L, Prahl-Andersen B (1999) A systematic review of the relationship between overjet size and traumatic dental injuries. Eur J Orthod 21:503–515

    Article  PubMed  Google Scholar 

  53. Ariëns G, van Mechelen W, Bongers P, Bouter L, van der Wal G (2000) Physical risk factors for neck pain. Scand J Work Environ Health 26:7–19

    Article  PubMed  Google Scholar 

  54. Loney P, Chambers LW, Benett KJ, Roberts JG, Stratford PW (1998) Critical appraisal of the health research literature: prevalence or incidence of a health problem. Chronic Dis Can 19:170–176

    PubMed  Google Scholar 

  55. Woodbury M, Houghton PE (2004) Prevalence of pressure ulcers in Canadian healthcare settings. Ostomy/Wound Manage 50:22–38

    Google Scholar 

  56. Silva LC, Ordunez P, Paz Rodriguez M, Robles S (2001) A tool for assessing the usefulness of prevalence studies done for surveillance purposes: the example of hypertension. Rev Panam Salud Pública 10:152–160

    Article  PubMed  Google Scholar 

  57. Barclay P, Hollender LG, Maravilla KR, Truelove EL (1999) Comparison of clinical and magnetic resonance imaging diagnosis in patients with disk displacement in the temporomandibular joint. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 88:37–43

    Article  PubMed  Google Scholar 

  58. Schmitter M, Kress B, Rammelsberg P (2004) Temporomandibular joint pathosis in patients with myofascial pain: a comparative analysis of magnetic resonance imaging and a clinical examination based on a specific set of criteria. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 97:318–324

    Article  PubMed  Google Scholar 

  59. Emshoff R, Rudisch A (2001) Validity of clinical diagnostic criteria for temporomandibular disorders: clinical versus magnetic resonance imaging diagnosis of temporomandibular joint internal derangement and osteoarthrosis. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 91:50–55

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

The authors of this article would like to acknowledge the dentists J. Mahabadi, A. Hassel, R. Shahin, and W. Bömicke for their help with assessment of the new instrument's reliability.

Conflict of interest

The authors declare that they have no conflict of interest.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Nikolaos Nikitas Giannakopoulos.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Giannakopoulos, N.N., Rammelsberg, P., Eberhard, L. et al. A new instrument for assessing the quality of studies on prevalence. Clin Oral Invest 16, 781–788 (2012). https://doi.org/10.1007/s00784-011-0557-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00784-011-0557-4

Keywords

Navigation