Abstract
Motion sensor devices such as actigraphs are increasingly used in studies that seek to obtain an objective assessment of activity level. They have many advantages, and are useful additions to research in fields such as sleep assessment, drug efficacy, behavior genetics, and obesity. However, questions still remain over the reliability of data collected using actigraphic assessment. We aimed to apply generalizability theory to actigraph data collected on a large, general-population sample in middle childhood, during 8 cognitive tasks across two body loci, and to examine reliability coefficients on actigraph data aggregated across different numbers of tasks and different numbers of attachment loci. Our analyses show that aggregation greatly increases actigraph data reliability, with reliability coefficients on data collected at one body locus during 1 task (.29) being much lower than that aggregated across data collected on two body loci and during 8 tasks (.66). Further increases in reliability coefficients by aggregating across four loci and 12 tasks were estimated to be modest in prospective analyses, indicating an optimum trade-off between data collection and reliability estimates. We also examined possible instrumental effects on actigraph data and found these to be nonsignificant, further supporting the reliability and validity of actigraph data as a method of activity level assessment.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Abikoff, H., Courtney, M., Pelham, W. E., Jr., & Koplewicz, H. S. (1993). Teachers’ ratings of disruptive behaviors: The influence of halo effects. Journal of Abnormal Child Psychology, 21, 519–533.
Armitage, P., Berry, G., & Matthews, J. N. S. (2001). Statistical methods in medical research (4th ed.). Oxford: Blackwell.
Börger, N., & van der Meere, J. (2000). Motor control and state regulation in children with ADHD: A cardiac response study. Biological Psychology, 51, 247–267.
Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6, 284–290.
Cole, R. J., Kripke, D. F., Gruen, W., Mullaney, D. J., & Gillin, J. C. (1992). Automatic sleep/wake identification from wrist activity. Sleep, 15, 461–469.
Cronbach, L. J., Gleser, G. C., Nanda, H., & Rajaratnam, N. (1972). The dependability of behavioral measurements: Theory of generalizability for scores and profiles. New York: Wiley.
Dane, A. V., Schachar, R. J., & Tannock, R. (2000). Does actigraphy differentiate ADHD subtypes in a clinical research setting? Journal of the American Academy of Child & Adolescent Psychiatry, 39, 752–760.
Eaton, W. O. (1983). Measuring activity level with actometers: Reliability, validity, and arm length. Child Development, 54, 720–726.
Eaton, W. O., McKeen, N. A., & Saudino, K. J. (1996). Measuring human individual differences in general motor activity with actometers. In K. Ossenkopp, M. Kavaliers, & P. R. Sanberg (Eds.), Measuring movement and locomotion: From invertebrates to humans (pp. 79–92). Austin, TX: Landes.
Eaves, L., Rutter, M., Silberg, J. L., Shillady, L., Maes, H., & Pickles, A. (2000). Genetic and environmental causes of covariation in interview assessments of disruptive behavior in child and adolescent twins. Behavior Genetics, 30, 321–334.
Epstein, S. (1983). Aggregation and beyond: Some basic issues on the prediction of behavior. Journal of Personality, 51, 360–392.
Inoue, K., Nadaoka, T., Oiji, A., Morioka, Y., Totsuka, S., Kanbayashi, Y., & Hukui, T. (1998). Clinical evaluation of attentiondeficit hyperactivity disorder by objective quantitative measures. Child Psychiatry & Human Development, 28, 179–188.
Jean-Louis, G., von Gizycki, H., Zizi, F., Spielman, A., Hauri, P., & Taub, H. (1997). The actigraph data analysis software: II. A novel approach to scoring and interpreting sleep-wake activity. Perceptual & Motor Skills, 85, 219–226.
Kuntsi, J., Andreou, P., Ma, J., Börger, N. A., & van der Meere, J. (2005). Testing assumptions for endophenotype studies in ADHD: Reliability and validity of tasks in a general population sample. BMC Psychiatry, 5, 40.
Kuntsi, J., Rogers, H., Swinard, G., Börger, N., van der Meere, J., Rijsdijk, F., & Asherson, P. (2006). Reaction time, inhibition, working memory and “delay aversion” performance: Genetic influences and their interpretation. Psychological Medicine, 36, 1613–1624.
Kuntsi, J., Stevenson, J., Oosterlaan, J., & Sonuga-Barke, E. J. S. (2001). Test-retest reliability of a new delay aversion task and executive function measures. British Journal of Developmental Psychology, 19, 339–348.
Morgenthaler, T., Alessi, C., Friedman, L., Owens, J., Kapur, V., Boehlecke, B., et al. (2007). Practice parameters for the use of actigraphy in the assessment of sleep and sleep disorders: An update for 2007. Sleep, 30, 519–529.
Mushquash, C., & O’Connor, B. P. (2006). SPSS and SAS programs for generalizability theory analyses. Behavior Research Methods, 38, 542–547.
O’Brien, R. M. (1995). Generalizability coefficients are reliability coefficients. Quality & Quantity, 29, 421–428.
Puyau, M. R., Adolph, A. L., Vohra, F. A., & Butte, N. F. (2002). Validation and calibration of physical activity monitors in children. Obesity Research, 10, 150–157.
Rousson, V., Gasser, T., & Seifert, B. (2002). Assessing intrarater, interrater and test-retest reliability of continuous measurements. Statistical Medicine, 21, 3431–3446.
Sadeh, A., Sharkey, K. M., & Carskadon, M. A. (1994). Activitybased sleep-wake identification: An empirical test of methodological issues. Sleep, 17, 201–207.
Saudino, K. J., Cherny, S. S., & Plomin, R. (2000). Parent ratings of temperament in twins: Explaining the “too low” DZ correlations. Twin Research, 3, 224–233.
Saudino, K. J., Ronald, A., & Plomin, R. (2005). The etiology of behavior problems in 7-year-old twins: Substantial genetic influence and negligible shared environmental influence for parent ratings and ratings by same and different teachers. Journal of Abnormal Child Psychology, 33, 113–130.
Saudino, K. J., Wertz, A. E., Gagne, J. R., & Chawla, S. (2004). Night and day: Are siblings as different in temperament as parents say they are? Journal of Personality & Social Psychology, 87, 698–706.
Schachar, R., Sandberg, S., & Rutter, M. (1986). Agreement between teachers’ ratings and observations of hyperactivity, inattentiveness, and defiance. Journal of Abnormal Child Psychology, 14, 331–345.
Simonoff, E., Pickles, A., Hervas, A., Silberg, J. L., Rutter, M., & Eaves, L. (1998). Genetic influences on childhood hyperactivity: Contrast effects imply parental rating bias, not sibling interaction. Psychological Medicine, 28, 825–837.
Spinath, F. M., & O’Connor, T. G. (2003). A behavioral genetic study of the overlap between personality and parenting. Journal of Personality, 71, 785–808.
Stevens, J., Quittner, A. L., & Abikoff, H. (1998). Factors influencing elementary school teachers’ ratings of ADHD and ODD behaviors. Journal of Clinical Child Psychology, 27, 406–414.
Teicher, M. H., Ito, Y., Glod, C. A., & Barber, N. I. (1996). Objective measurement of hyperactivity and attentional problems in ADHD. Journal of the American Academy of Child & Adolescent Psychiatry, 35, 334–342.
Thapar, A., Harrington, R., Ross, K., & McGuffin, P. (2000). Does the definition of ADHD affect heritability? Journal of the American Academy of Child & Adolescent Psychiatry, 39, 1528–1536.
Treuth, M. S., Schmitz, K., Catellier, D. J., McMurray, R. G., Murray, D. M., Almeida, M. J., et al. (2004). Defining accelerometer thresholds for activity intensities in adolescent girls. Medicine & Science in Sports & Exercise, 36, 1259–1266.
Trouton, A., Spinath, F. M., & Plomin, R. (2002). Twins early development study (TEDS): A multivariate, longitudinal genetic investigation of language, cognition and behavior problems in childhood. Twin Research, 5, 444–448.
Tryon, W. W. (2005). The reliability and validity of two ambulatory monitoring actigraphs. Behavior Research Methods, 37, 492–497.
Van Coevering, P., Harnack, L., Schmitz, K., Fulton, J. E., Galuska, D. A., & Gao, S. (2005). Feasibility of using accelerometers to measure physical activity in young adolescents. Medicine & Science in Sports & Exercise, 37, 867–871.
van der Meere, J., Stemerdink, N., & Gunning, B. (1995). Effects of presentation rate of stimuli on response inhibition in ADHD children with and without tics. Perceptual & Motor Skills, 81, 259–262.
Verhulst, F. C., Achenbach, T. M., Althaus, M., & Akkerhuis, G. W. (1988). A comparison of syndromes derived from the child behavior checklist for American and Dutch girls aged 6–11 and 12–16. Journal of Child Psychology & Psychiatry, 29, 879–895.
Wechsler, D. (1991). Wechsler Intelligence Scale for Children (3rd ed.). London: Psychological Corporation.
Welk, G. J., Blair, S. N., Wood, K., Jones, S., & Thompson, R. W. (2000). A comparative evaluation of three accelerometry-based physical activity monitors. Medicine & Science in Sports & Exercise, 32, S489-S497.
Welk, G. J., Schaben, J. A., & Morrow, J. R., Jr. (2004). Reliability of accelerometry-based activity monitors: A generalizability study. Medicine & Science in Sports & Exercise, 36, 1637–1645.
Wood, A. C., Saudino, K. J., Rogers, H., Asherson, P., & Kuntsi, J. (2007). Genetic influences on mechanically-assessed activity level in children. Journal of Child Psychology & Psychiatry, 48, 695–702.
Author information
Authors and Affiliations
Corresponding author
Additional information
The Study of Activity and Impulsivity Levels in Children (SAIL) is funded by a project grant from the Wellcome Trust (GR070345MF). K.J.S. is supported by Grant MH062375 from the National Institute of Mental Health. A.C.W. is supported by the Economic and Social Research search Council.
Rights and permissions
About this article
Cite this article
Wood, A.C., Kuntsi, J., Asherson, P. et al. Actigraph data are reliable, with functional reliability increasing with aggregation. Behavior Research Methods 40, 873–878 (2008). https://doi.org/10.3758/BRM.40.3.873
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/BRM.40.3.873