Skip to main content
SpringerLink
Log in

Use of Latent Profile Analysis to Model the Translation of University Research into Health Practice and Policy: Exploration of Proposed Metrics

Research in Higher Education volume 64, pages 1058–1070 (2023)Cite this article

Abstract

The aim of this study is to profile academic institutions (n = 127) based on publications, citations in the top 10% of journals, patent citations in Food and Drug Administration (FDA) approvals, clinical trials with uploaded results, contributions to clinical practice guidelines, awarded patents, start-ups, and licenses generating income in response to the Association of University Technology Managers (AUTM) Licensing Activity Survey: Fiscal Years 2011–2015. Latent variable modeling (LVM) was conducted in Mplus v.8.1, specifically latent profile analysis (LPA) was utilized to predict institutional profiles of research, which were compared with the 2015 Carnegie Classification System ranks. Multivariate regression of profile assignment on research expenditure and income generated by licensure was used to show concurrent validity. The LPA resulted in three profiles as the most parsimonious model. Mantel-Haenszel test of trend to the Carnegie Classification found a positive and significant association among institution rankings (r = 0.492, χ2(1) = 26.69, p < 0.001). Profile assignment significantly predicted differences in research expenditure and income generated by licensure. By classifying academic institutions into improving, mobilizing and thriving translational research profiles allows for a universal metric of translation of science from basic or bench to practice or policy.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3

Data Availability

Non-proprietary data available upon request from the corresponding author.

References

  • Abbott, A., Cyranoski, D., Jones, N., Maher, B., Schiermeier, Q., & Van Noorden, R. (2010). Do metrics matter? Many researchers believe that quantitative metrics determine who gets hired and who gets promoted at their institutions. With an exclusive poll and interviews, Nature probes to what extent metrics are really used that way. Nature, 465(7300), 860–863.

    Article  Google Scholar 

  • Aguillo, I., Bar-Ilan, J., Levene, M., & Ortega, J. (2010). Comparing university rankings. Scientometrics, 85(1), 243–256.

    Article  Google Scholar 

  • Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19(6), 716–723. https://doi.org/10.1109/TAC.1974.1100705.

    Article  Google Scholar 

  • Ashworth, M., Schofield, P., Doran, T., Cookson, R., Sutton, M., Seed, P. T., Howe, A., & Fleetcroft, R. (2013). The Public Health Impact score: a new measure of public health effectiveness for general practices in England. British Journal Of General Practice, 63(609), e291–299. https://doi.org/10.3399/bjgp13X665260.

    Article  Google Scholar 

  • Association of University Technology Managers (2015). 2014 AUTM Licensing Survey (

  • Bercovitz, J., & Feldman, M. (2006). Entpreprenerial universities and technology transfer: A conceptual framework for understanding knowledge-based economic development. The Journal of Technology Transfer, 31(1), 175–188.

    Article  Google Scholar 

  • Bonnell, A. G. (2016). Tide or tsunami?: The impact of metrics on scholarly research. Australian Universities’ Review The, 58(1), 54.

    Google Scholar 

  • Bornmann, L. (2013). What is societal impact of research and how can it be assessed? A literature survey. Journal of the American Society for Information Science and Technology, 64(2), 217–233.

    Article  Google Scholar 

  • Bornmann, L. (2017). Measuring impact in research evaluations: a thorough discussion of methods for, effects of and problems with impact measurements. Higher Education, 73, 775–787.

    Article  Google Scholar 

  • Bornmann, L., Mutz, R., & Daniel, H. D. (2008). Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine. Journal of the American Society for Information Science and Technology, 59(5), 830–837.

    Article  Google Scholar 

  • Collins, F. S., & Tabak, L. A. (2014). NIH plans to enhance reproducibility. Nature, 505(7485), 612–613. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4058759/.

    Article  Google Scholar 

  • Collins, L., & Lanza, S. (2010). Latent class and latent transition analysis. Hoboken. NJ: Wiley.

    Google Scholar 

  • Collins, L., & Lanza, S. (2010). Latent class and latent transition analysis: With applications in the social, behavioral, and health sciences. John Wiley & Sons Inc. https://doi.org/10.1002/9780470567333.

  • Dembe, A. E., Lynch, M. S., Gugiu, P. C., & Jackson, R. D. (2014). The translational research impact scale: development, construct validity, and reliability testing. Evaluation & the health professions, 37(1), 50–70.

    Article  Google Scholar 

  • Dill, D. D., & Soo, M. (2005). Academic quality, league tables, and public policy: A cross-national analysis of university ranking systems [journal article]. Higher Education, 49(4), 495–533. https://doi.org/10.1007/s10734-004-1746-8.

    Article  Google Scholar 

  • Edwards, M. A., & Roy, S. (2016). Academic Research in the 21st Century: Maintaining Scientific Integrity in a Climate of Perverse Incentives and Hypercompetition. Environmental Engineering Science. https://doi.org/10.1089/ees.2016.0223.

    Article  Google Scholar 

  • Florida, R., Knudsen, B., Stolarick, K., & Lee, S. Y. (2006). Talent and Creativity in the Software Industry. vol. Talent and Creativity in the Software Industry.

  • Heinrich, C. J., & Marschke, G. (2010). Incentives and their dynamics in public sector performance management systems. Journal of Policy Analysis and Management, 29(1), 183–208.

    Article  Google Scholar 

  • Hicks, D. (2012). Performance-based university research funding systems. Research Policy, 41(2), 251–261. https://doi.org/10.1016/j.respol.2011.09.007.

    Article  Google Scholar 

  • Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015). The Leiden Manifesto for research metrics. Nature, 520, 429–431.

    Article  Google Scholar 

  • Horton, R. (2015). Offline: What is medicine’s 5 sigma. The Lancet, 385(9976), 1380.

    Article  Google Scholar 

  • Jiang, Y., Gervais, F., Gauthier, A., Baptiste, C., Martinon, P., & Bresse, X. (2015). A comparative public health and budget impact analysis of pneumococcal vaccines: The French case. Human Vaccines & Immunotherapeutics, 11(9), 2188–2197. https://doi.org/10.1080/21645515.2015.1011957.

    Article  Google Scholar 

  • Jones, M., & Huh, J. (2014). Toward a multidimensional understanding of residential neighborhood: A latent profile analysis of Los Angeles neighborhoods and longitudinal adult excess weight. Health & place, 27, 134–141.

    Article  Google Scholar 

  • Kongsted, A., & Nielsen, A. M. (2017). Latent class analysis in health research. Journal of physiotherapy, 63(1), 55–58.

    Article  Google Scholar 

  • Lanjouw, J. O., & Schankerman, M. (2004). Patent Quality and Research Productivity: Measuring Innovation with Multiple Indicators*. The Economic Journal, 114(495), 441–465. https://doi.org/10.1111/j.1468-0297.2004.00216.x.

    Article  Google Scholar 

  • Lanza, S. T., Flaherty, B. P., & Collins, L. M. (2003). Latent class and latent transition analysis (Handbook of psychology: Research methods in psychology, Vol. 2 (pp. 663–685). John Wiley & Sons Inc. https://doi.org/10.1002/0471264385.wei0226.

  • Larkin, M. A., & Marks, J. S. (2012). Public Health Services and Systems Research An Agenda Where Research and Practice Matter to Each Other and to Our Nation’s Health [Editorial Material]. American Journal of Preventive Medicine, 42(5), S79–S81. https://doi.org/10.1016/j.amepre.2012.01.025.

    Article  Google Scholar 

  • Leshner, A. I., Terry, S. F., Schultz, A. M., & Liverman, C. T. (2013). The CTSA program at NIH: Opportunities for advancing clinical and translational research. National Academies Press.

  • Lubke, G. H., & Muthén, B. (2005). Investigating population heterogeneity with factor mixture models. Psychological methods, 10(1), 21.

    Article  Google Scholar 

  • Luke, D. A., Sarli, C. C., Suiter, A. M., Carothers, B. J., Combs, T. B., Allen, J. L., Beers, C. E., & Evanoff, B. A. (2018). The translational science benefits model: a new framework for assessing the health and societal benefits of clinical and translational sciences. Clinical and translational science, 11(1), 77–84.

    Article  Google Scholar 

  • McCutcheon, A. L. (2002). Basic concepts and procedures in single-and multiple-group latent class analysis.Applied latent class analysis,56–88.

  • Miller, F. A., Mentzakis, E., Axler, R., Lehoux, P., French, M., Tarride, J. E., Wodchis, W. P., Wilson, B. J., Longo, C., Bytautas, J. P., & Slater, B. (2013). Do Canadian researchers and the lay public prioritize biomedical research outcomes equally? A choice experiment. Academic Medicine, 88(4), 519–526. https://doi.org/10.1097/ACM.0b013e31828577fe.

    Article  Google Scholar 

  • Mongeon, P., Brodeur, C., Beaudry, C., & Larivière, V. (2016). Concentration of research funding leads to decreasing marginal returns.Research Evaluation,rvw007.

  • Muthén, B. (2004). Latent variable analysis. The Sage handbook of quantitative methodology for the social sciences, 345, 368.

    Google Scholar 

  • Muthén, B. O. (2001). Latent variable mixture modeling (New developments and techniques in structural equation modeling (pp. 21–54). Psychology Press.

  • Muthén, L., & Muthén, B. (1998–2015). Mplus User’s Guide. Muthén & Muthén..

  • National Guideline Clearinghouse (2016). National Guideline Clearninghouse. Agency for Healthcare Quality and Research (AHRQ). Retrieved February 22 from http://www.guideline.gov.

  • Nylund, K. L., Asparouhov, T., & Muthén, B. O. (2007). Deciding on the number of classes in latent class analysis and growth mixture modeling: A Monte Carlo simulation study. Structural equation modeling, 14(4), 535–569.

    Article  Google Scholar 

  • Pollitt, A., Potoglou, D., Patil, S., Burge, P., Guthrie, S., King, S., Wooding, S., & Grant, J. (2016). Understanding the relative valuation of research impact: a best–worst scaling experiment of the general public and biomedical and health researchers. British Medical Journal Open, 6(8), https://doi.org/10.1136/bmjopen-2015-010916.

  • Rasmussen, E., & Borch, O. J. (2010). University capabilities in facilitating entrepreneurship: A longitudinal study of spin-off ventures at mid-range universities. Research Policy, 39(5), 602–612. https://doi.org/10.1016/j.respol.2010.02.002.

    Article  Google Scholar 

  • Ruiz-Castillo, J., & Waltman, L. (2015). Field-normalized citation impact indicators using algorithmically constructed classification systems of science. Journal of Informetrics, 9(1), 102–117.

    Article  Google Scholar 

  • Schwarz, G. (1978). Estimating the Dimension of a Model. The Annals of Statistics, 6(2), 461–464. http://www.jstor.org.proxy.kumc.edu:2048/stable/2958889.

    Article  Google Scholar 

  • Sclove, S. L. (1987). Application of model-selection criteria to some problems in multivariate analysis. Psychometrika, 52(3), 333–343.

    Article  Google Scholar 

  • Smaldino, P. E., & McElreath, R. (2016). The natural selection of bad science. Royal Society Open Science, 3(9), 160384.

    Article  Google Scholar 

  • Stanley, L., Kellermanns, F. W., & Zellweger, T. M. (2017). Latent profile analysis: Understanding family firm profiles. Family Business Review, 30(1), 84–102.

    Article  Google Scholar 

  • Todd, M., Adams, M. A., Kurka, J., Conway, T. L., Cain, K. L., Buman, M. P., Frank, L. D., Sallis, J. F., & King, A. C. (2016). GIS-measured walkability, transit, and recreation environments in relation to older Adults’ physical activity: a latent profile analysis. Preventive medicine, 93, 57–63.

    Article  Google Scholar 

  • US Department of Education, I. f. E. S (2015). National Center for Education Statistics,. Integrated Postsecondary Education Data System. National Center for Education Statistics,. Retrieved May 15 from http://nces.ed.gov

  • Vernon, M. M., Balas, E. A., & Momani, S. (2018). Are university rankings useful to improve research? A systematic review.PloS one, 13(3), e0193762.

  • Vernon, M. M., Danley, C. M., & Yang, F. M. (2021). Developing a measure of innovation from research in higher education data.Scientometrics,1–10.

  • Waltman, L., & Schreiber, M. (2013). On the calculation of percentile-based bibliometric indicators. Journal of the Association for Information Science and Technology, 64(2), 372–379.

    Google Scholar 

  • Waltman, L., & Schreiber, M. (2013). On the calculation of percentile-based bibliometric indicators. Journal of the Association for Information Science and Technology, 64(2), 372–379.

Download references

Acknowledgements:

The authors would like to acknowledge Dr. Andrew Balas for his contributions to the early development of evaluation metrics. We received no funding for this work.

Funding

The authors did not receive support from any organization for the submitted work.

Author information

Authors and Affiliations

  1. Department of Medicine, Medical College of Georgia, Georgia Cancer Center, Augusta University, 1410 Laney Walker Blvd. CN-2152, Augusta, GA, 30912, USA

    Marlo M. Vernon PhD, MPH

  2. School of Nursing, University of Kansas Medical Center, Kansas City, KS, USA

    Frances M. Yang

Authors
  1. Marlo M. Vernon PhD, MPH
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Frances M. Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by Marlo Vernon and Frances Yang (supervised data analysis). The first draft of the manuscript was written by Marlo Vernon and Frances Yang and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Marlo M. Vernon PhD, MPH.

Ethics declarations

Conflicts of Interest

The authors have no conflicts of interest to disclose.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Vernon, M.M., Yang, F.M. Use of Latent Profile Analysis to Model the Translation of University Research into Health Practice and Policy: Exploration of Proposed Metrics. Res High Educ 64, 1058–1070 (2023). https://doi.org/10.1007/s11162-023-09735-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11162-023-09735-w

Keywords

search

Navigation