Skip to main content
SpringerLink
Log in

The Latent Change Score Model: A More Flexible Approach to Modeling Time in Self-Regulated Learning

  • Original Paper
  • Published:
Journal of Business and Psychology Aims and scope Submit manuscript

Abstract

Purpose

This research advances understanding of empirical time modeling techniques in self-regulated learning research. We intuitively explain several such methods by situating their use in the extant literature. Further, we note key statistical and inferential assumptions of each method while making clear the inferential consequences of inattention to such assumptions.

Design/Methodology/Approach

Using a population model derived from a recent large-scale review of the training and work learning literature, we employ a Monte Carlo simulation fitting six variations of linear mixed models, seven variations of latent common factor models, and a single latent change score model to 1500 simulated datasets.

Findings

The latent change score model outperformed all six of the linear mixed models and all seven of the latent common factor models with respect to (1) estimation precision of the average learner improvement, (2) correctly rejecting a false null hypothesis about such average improvement, and (3) correctly failing to reject true null hypothesis about between-learner differences (i.e., random slopes) in average improvement.

Implications

The latent change score model is a more flexible method of modeling time in self-regulated learning research, particularly for learner processes consistent with twenty-first-century workplaces. Consequently, defaulting to linear mixed or latent common factor modeling methods may have adverse inferential consequences for better understanding self-regulated learning in twenty-first-century work.

Originality/Value

Ours is the first study to critically, rigorously, and empirically evaluate self-regulated learning modeling methods and to provide a more flexible alternative consistent with modern self-regulated learning knowledge.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  • Antonakis, J., Bendahan, S., Jacquart, P., & Lalive, R. (2010). On making causal claims: A review and recommendations. The Leadership Quarterly, 21(6), 1086–1120.

    Article  Google Scholar 

  • Austin, J. T., & Vancouver, J. B. (1996). Goal constructs in psychology: Structure, process, and content. Psychological Bulletin, 120(3), 338–375.

    Article  Google Scholar 

  • Ballard, T., Yeo, G., Loft, S., Vancouver, J. B., & Neal, A. (2016). An integrated formal model of motivation and decision making: The MPMM*. Journal of Applied Psychology, 101(9), 1240–1265.

    Article  PubMed  Google Scholar 

  • Bandura, A. (2012). On the functional properties of perceived self-efficacy revisited. Journal of Management, 38(1), 9–44.

    Article  Google Scholar 

  • Bandura, A., & Locke, E. A. (2003). Negative self-efficacy and goal effects revisited. Journal of Applied Psychology, 88(1), 87.

    Article  PubMed  Google Scholar 

  • Beier, M. E., & Kanfer, R. (2010). Motivation in training and development: A phase perspective. In S. J. Kozlowski, E. Salas, S. J. Kozlowski, & E. Salas (Eds.), Learning, training, and development in organizations (pp. 65–97). New York, NY: Routledge/Taylor & Francis Group.

    Google Scholar 

  • Bell, B. S., & Kozlowski, S. W. J. (2008). Active learning: Effects of core training design elements on self-regulatory processes, learning, and adaptability. Journal of Applied Psychology, 93(2), 296–316. doi:10.1037/0021-9010.93.2.296.

    Article  PubMed  Google Scholar 

  • Bjork, R. A., Dunlosky, J., & Kornell, N. (2013). Self-regulated learning: Beliefs, techniques, and illusions. Annual Review of Psychology, 64, 417–444.

    Article  PubMed  Google Scholar 

  • Bledow, R. (2013). Demand-perception and self-motivation as opponent processes: A response to Bandura and Vancouver. Journal of Management, 39(1), 14–26.

    Article  Google Scholar 

  • Braun, M. T., Kuljanin, G., & DeShon, R. P. (2013). Spurious results in the analysis of longitudinal data in organizational research. Organizational Research Methods, 16(2), 302–330.

    Article  Google Scholar 

  • Brown, K. G., Howardson, G. N., & Fisher, S. W. (2016). Learner control: Taking stock and moving forward. Annual Review of Organizational Psychology and Organizational Behavior, 3, 267–291.

    Article  Google Scholar 

  • Carver, C. S., & Scheier, M. F. (1998). On the self-regulation of behavior. Cambridge, UK: Cambridge University Press.

    Book  Google Scholar 

  • Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155–159.

    Article  PubMed  Google Scholar 

  • Curran, P. J. (2003). Have multilevel models been structural equation models all along? Multivariate Behavioral Research, 38(4), 529–569.

    Article  PubMed  Google Scholar 

  • Debowski, S., Wood, R. E., & Bandura, A. (2001). Impact of guided exploration and enactive exploration on self-regulatory mechanisms and information acquisition through electronic search. Journal of Applied Psychology, 86(6), 1129–1141.

    Article  PubMed  Google Scholar 

  • Enders, C. K., & Tofighi, D. (2007). Centering predictor variables in cross-sectional multilevel models: A new look at an old issue. Psychological Methods, 12(2), 121.

    Article  PubMed  Google Scholar 

  • Ford, J. K. (2008). Transforming our models of learning and development: How far do we go? Industrial and Organizational Psychology, 1, 468–471.

    Article  Google Scholar 

  • Ford, J. K., & Oswald, F. L. (2003). Understanding the dynamic learner: Linking personality traits, learning situations, and individual behavior. In M. R. Barrick & A. M. Ryan (Eds.), Personality and work: Reconsidering the role of personality in organizations (pp. 229–260). San Francisco, CA: Jossey-Bass/John Wiley & Sons, Inc.

    Google Scholar 

  • Gully, S., & Chen, G. (2010). Individual differences, attribute-treatment interactions, and training outcomes. In S. W. J. Kozlowski & E. Salas (Eds.), Learning, training, and development in organizations (pp. 3–64). New York: Routledge.

    Google Scholar 

  • Heggestad, E. D., & Kanfer, R. (2005). The predictive validity of self-efficacy in training performance: Little more than past performance. Journal of Experimental Psychology: Applied, 11(2), 84.

    PubMed  Google Scholar 

  • Hox, J. J. (2010). Multilevel analysis: Techniques and applications. New York, NY: Routledge.

    Google Scholar 

  • Hughes, M. G., Day, E. A., Wang, X., Schuelke, M. J., Arsenault, M. L., Harkrider, L. N., et al. (2013). Learner-controlled practice difficulty in the training of a complex task: Cognitive and motivational mechanisms. Journal of Applied Psychology, 98(1), 80.

    Article  PubMed  Google Scholar 

  • Johnson, R. E., Chang, C. H., & Lord, R. G. (2006). Moving from cognition to behavior: What the research says. Psychological Bulletin, 132(3), 381.

    Article  PubMed  Google Scholar 

  • Kanar, A. M., & Bell, B. S. (2013). Guiding learners through technology-based instruction: The effects of adaptive guidance design and individual differences on learning over time. Journal of Educational Psychology, 105(4), 1067–1081.

    Article  Google Scholar 

  • Kanfer, R., & Ackerman, P. L. (1989). Motivation and cognitive abilities: An integrative/aptitude-treatment interaction approach to skill acquisition. Journal of Applied Psychology, 74(4), 657–690. doi:10.1037//0021-9010.74.4.657.

    Article  Google Scholar 

  • Kanfer, R., & Ackerman, P. L. (1996). A self-regulatory skills perspective to reducing cognitive interference. In I. G. Sarason, G. R. Pierce, & B. R. Sarason (Eds.), Cognitive interference: Theories, methods, and findings. The LEA series in personality and clinical psychology (pp. 153–171). Hillsdale, NJ: Lawrence Earlbaum.

    Google Scholar 

  • Kanfer, R., Ackerman, P. L., & Heggestad, E. D. (1996). Motivational skills and self-regulation for learning: A trait perspective. Learning and Individual Differences, 8(3), 185–209.

    Article  Google Scholar 

  • Keith, N., & Frese, M. (2005). Self-regulation in error management training: Emotion control and metacognition as mediators of performance effects. Journal of Applied Psychology, 90(4), 677–691.

    Article  PubMed  Google Scholar 

  • Kozlowski, S. W., & Bell, B. S. (2006). Disentangling achievement orientation and goal setting: Effects on self-regulatory processes. Journal of Applied Psychology, 91(4), 900.

    Article  PubMed  Google Scholar 

  • Kuljanin, G., Braun, M. T., & DeShon, R. P. (2011). A cautionary note on modeling growth trends in longitudinal data. Psychological Methods, 16(3), 249.

    Article  PubMed  Google Scholar 

  • Liu, S., Rovine, M. J., & Molenaar, P. C. M. (2012). Selecting a linear mixed model for longitudinal data: Repeated measures analysis of variance, covariance patter model, and growth curve approaches. Psychological Methods, 17(1), 15–30.

    Article  PubMed  Google Scholar 

  • Locke, E. A., & Latham, G. P. (2002). Building a practically useful theory of goal setting and task motivation: A 35-year odyssey. American Psychologist, 57(9), 705.

    Article  PubMed  Google Scholar 

  • Lord, R. G., Diefendorff, J. M., Schmidt, A. M., & Hall, R. J. (2010). Self-regulation at work. Annual Review of Psychology, 61, 543–568.

    Article  PubMed  Google Scholar 

  • McArdle, J. J. (2009). Latent variable modeling of differences and changes with longitudinal data. Annual Review of Psychology, 60, 577–605.

    Article  PubMed  Google Scholar 

  • Mitchell, T. R., & James, L. R. (2001). Building better theory: Time and the specification of when things happen. Academy of Management Review, 26(4), 530–547.

    Google Scholar 

  • National Research Council. (2012). Education for life and work: Developing transferable knowledge and skills in the 21st century. In J. W. Pellegrino & M. L. Hilton (Eds.), Committee on defining deeper learning and 21st century skills, board on testing and assessment and board on science education, division of behavioral and social sciences and education. Washington, DC: The National Academies Press.

    Google Scholar 

  • Noe, R. A., Clarke, A. D. M., & Klein, H. J. (2014). Learning in the twenty-first-century workplace. Annual Review of Organizational Psychology and Organizational Behavior, 1, 245–275.

    Article  Google Scholar 

  • Pinheiro, J. C., & Bates, D. M. (2000). Mixed-effects models in S and S-PLUS. New York, NY: Springer.

    Book  Google Scholar 

  • Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods. Thousand Oaks, CA: Sage Publications Inc.

    Google Scholar 

  • Schmidt, A. M., & DeShon, R. P. (2010). The moderating effects of performance ambiguity on the relationship between self-efficacy and performance. Journal of Applied Psychology, 95(3), 572.

    Article  PubMed  Google Scholar 

  • Sitzmann, T., Bell, B. S., Kraiger, K., & Kanar, A. M. (2009). A multilevel analysis of the effect of prompting self-regulation in technology-delivered instruction. Personnel Psychology, 62, 697–734.

    Article  Google Scholar 

  • Sitzmann, T., & Ely, K. (2010). Sometimes you need a reminder: the effects of prompting self-regulation on regulatory processes, learning, and attrition. Journal of Applied Psychology, 95(1), 132.

    Article  PubMed  Google Scholar 

  • Sitzmann, T., & Ely, K. (2011). A meta-analysis of self-regulated learning in work-related training and educational attainment: What we know and where we need to go. Psychological Bulletin, 137(3), 421.

    Article  PubMed  Google Scholar 

  • Sitzmann, T., Ely, K., Bell, B. S., & Bauer, K. N. (2010). The effects of technical difficulties on learning and attrition during online training. Journal of Experimental Psychology: Applied, 16(3), 281–292.

    PubMed  Google Scholar 

  • Sitzmann, T., & Johnson, S. K. (2012a). The best laid plans: Examining the conditions under which a planning intervention improves learning and reduces attrition. Journal of Applied Psychology, 97(5), 967.

    Article  PubMed  Google Scholar 

  • Sitzmann, T., & Johnson, S. K. (2012b). When is ignorance bliss? The effects of inaccurate self-assessments of knowledge on learning and attrition. Organizational Behavior and Human Decision Processes, 117(1), 192–207.

    Article  Google Scholar 

  • Sitzmann, T., & Wang, M. (2015). The survey effect: Does administering surveys affect trainees’ behavior? Learning and Individual Differences, 37, 1–12.

    Article  Google Scholar 

  • Sitzmann, T., & Weinhardt, J. M. (2015). Training engagement theory a multilevel perspective on the effectiveness of work-related training. Journal of Management. doi:10.1177/0149206315574596.

    Google Scholar 

  • Skrondal, A., & Rabe-Hesketh, S. (2004). Generalized latent variable modeling: Multilevel, longitudinal, and structural equation models. Boca Raton: CRC Press.

    Book  Google Scholar 

  • Soderstrom, N. C., & Bjork, R. A. (2015). Learning versus performance: An integrated review. Perspectives on Psychological Science, 10(2), 176–199.

    Article  PubMed  Google Scholar 

  • Steel, P., & König, C. J. (2006). Integrating theories of motivation. Academy of Management Review, 31(4), 889–913.

    Article  Google Scholar 

  • Tett, R. P., & Burnett, D. D. (2003). A personality trait-based interactionist model of job performance. Journal of Applied Psychology, 88(3), 500.

    Article  PubMed  Google Scholar 

  • Vancouver, J. B. (2005). The depth of history and explanation as benefit and bane for psychological control theories. Journal of Applied Psychology, 90(1), 38.

    Article  PubMed  Google Scholar 

  • Vancouver, J. B. (2008). Integrating self-regulation theories of work motivation into a dynamic process theory. Human Resource Management Review, 18(1), 1–18.

    Article  Google Scholar 

  • Vancouver, J. B., & Carlson, B. W. (2015). All things in moderation, including tests of mediation (at least some of the time). Organizational Research Methods, 18(1), 70–91.

    Article  Google Scholar 

  • Vancouver, J. B., & Kendall, L. N. (2006). When self-efficacy negatively relates to motivation and performance in a learning context. Journal of Applied Psychology, 91(5), 1146–1153.

    Article  PubMed  Google Scholar 

  • Vancouver, J. B., Weinhardt, J. M., & Schmidt, A. M. (2010). A formal, computational theory of multiple-goal pursuit: integrating goal-choice and goal-striving processes. Journal of Applied Psychology, 95(6), 985.

    Article  PubMed  Google Scholar 

  • Vancouver, J. B., Weinhardt, J. M., & Vigo, R. (2014). Change one can believe in: Adding learning to computational models of self-regulation. Organizational Behavior and Human Decision Processes, 124(1), 56–74.

    Article  Google Scholar 

  • Wood, R. E., Kakebeeke, B. M., Debowski, S., & Frese, M. (2000). The impact of enactive exploration on intrinsic motivation, strategy, and performance in electronic search. Applied Psychology: An International Review, 49(2), 263–283.

    Article  Google Scholar 

  • Yeo, G., Loft, S., Xiao, T., & Kiewitz, C. (2009). Goal orientations and performance: Differential relationships across levels of analysis and as a function of task demands. Journal of Applied Psychology, 94(3), 710–726.

    Article  PubMed  Google Scholar 

  • Yeo, G. B., & Neal, A. (2004). A multilevel analysis of effort, practice, and performance: Effects; of ability, conscientiousness, and goal orientation. Journal of Applied Psychology, 89(2), 231–247.

    Article  PubMed  Google Scholar 

  • Yeo, G. B., & Neal, A. (2006). An examination of the dynamic relationship between self efficacy and performance across levels of analysis and levels of specificity. Journal of Applied Psychology, 91(5), 1088.

    Article  PubMed  Google Scholar 

  • Yeo, G., & Neal, A. (2008). Subjective cognitive effort: A model of states, traits, and time. Journal of Applied Psychology, 93(3), 617.

    Article  PubMed  Google Scholar 

  • Zyphur, M. J. (2009). When mindsets collide: Switching analytical mindsets to advance organization science. Academy of Management Review, 34(4), 677–688.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Psychology, Hofstra University, 135 Hofstra University, Hempstead, NY, 11549, USA

    Garett N. Howardson

  2. Tuple Work Science, Limited, Arlington, VA, 22202, USA

    Garett N. Howardson

  3. Fors Marsh Group LLC, 1010 North Glebe Road, Number 510, Arlington, VA, 22201, USA

    Michael N. Karim

  4. The George Washington University, 600 21st St NW, Washington, DC, 20052, USA

    Ryan G. Horn

Authors
  1. Garett N. Howardson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael N. Karim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ryan G. Horn
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Garett N. Howardson.

Additional information

The authors would like to thank Kevin Murphy and two anonymous reviewers for the helpful direction in revising this manuscript.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Howardson, G.N., Karim, M.N. & Horn, R.G. The Latent Change Score Model: A More Flexible Approach to Modeling Time in Self-Regulated Learning. J Bus Psychol 32, 317–334 (2017). https://doi.org/10.1007/s10869-016-9475-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10869-016-9475-4

Keywords

Search

Navigation