Skip to main content
SpringerLink
Log in

Subjective Outcome Evaluation Based on the Perceptions of the Program Implementers

  • Chapter
  • First Online:
Development and Evaluation of Positive Adolescent Training through Holistic Social Programs (P.A.T.H.S.)

Part of the book series: Quality of Life in Asia ((QLAS,volume 3))

  • 1216 Accesses

Abstract

In this chapter, the views of the program implementers on the Tier 1 Program, program implementers, and program effectiveness were examined in the 10-h and 20-h programs. Based on the Subjective Outcome Evaluation Form for Instructors (Form B), results showed that program implementers generally had positive views of the program, implementer qualities, and program effectiveness, with the 20-h program associated with relatively higher subjective outcome evaluation scores than did the 10-h program in terms of perceptions of the program quality and effectiveness. Multiple regression analyses showed that perceived program quality and implementers predicted perceived program benefits. Reliability analysis showed that Form B was internally consistent. Confirmatory factor analyses also revealed that the scale assesses three hypothesized dimensions, including perceived program quality, perceived implementer quality, and perceived effectiveness of the program.

The authorship of this article is equally shared by the two authors. The preparation for this chapter and the Project P.A.T.H.S. were financially supported by The Hong Kong Jockey Club Charities Trust. Address all correspondence to Cecilia Ma, Department of Applied Social Sciences, The Hong Kong Polytechnic University, Hunghom, Hong Kong (e-mail address: cecilia.ma@polyu.edu.hk).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Aitkin, M., & Longford, N. (1986). Statistical modeling issues in school effectiveness studies. Journal of the Royal Statistical Society: Series A, 149, 1–43.

    Article  Google Scholar 

  • Alker, H. R. (1969). A typology of ecological fallacies. In H. Dogan & S. Rokkan (Eds.), Quantitative ecological analysis in the social sciences (pp. 69–86). London: MIT Press.

    Google Scholar 

  • Baker, A., & Bruner, B. (2006). Evaluation capacity and evaluative thinking in organizations (Vol. 60). Cambridge, MA: Bruner Foundation Inc.

    Google Scholar 

  • Bearinger, L. H., Sieving, R. E., Ferguson, J., & Sharma, V. (2007). Global perspectives on the sexual and reproductive health of adolescents: Patterns, prevention, and potential. The Lancet, 369, 1220–1231.

    Article  Google Scholar 

  • Bowen, S., & Martens, P. J. (2006). A model for collaborative evaluation of university-community partnerships. Journal of Epidemiology and Community Health, 60, 902–907.

    Article  PubMed  Google Scholar 

  • Brandon, R. (1998). Stakeholder participation for the purpose of helping ensure evaluation validity: Bridging the gap between collaborative and non-collaborative evaluations. American Journal of Evaluation, 19, 325–337.

    Google Scholar 

  • Catalano, R. F., Berglund, M. L., Ryan, J. A. M., Lonczak, H. S., & Hawkins, J. D. (2002). Positive youth development in the United States: Research findings on evaluations of positive youth development programs. Prevention & Treatment, 5, Article 15.

    Google Scholar 

  • Catalano, R. F., Gavin, L. E., & Markham, C. M. (2010). Future directions for positive youth development as a strategy adolescent sexual and reproductive health. Journal of Adolescence Health, 46, S92–S96.

    Article  Google Scholar 

  • Chinman, M., Hunter, S. B., Ebener, P., Paddock, S. M., Stillman, L., Imm, P., et al. (2008). The getting to outcomes demonstration and evaluation: An illustration often prevention support system. American Journal of Community Psychology, 41, 206–224.

    Article  PubMed  Google Scholar 

  • Curran, P. J., West, S. G., & Finch, J. F. (1996). The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. Psychological Methods, 1(1), 16–29.

    Article  Google Scholar 

  • Dane, A. V., & Schneider, B. H. (1998). Program integrity in primary and early secondary prevention: Are implementation effects out of control. Clinical Psychology Review, 18, 23–45.

    Article  PubMed  Google Scholar 

  • Durlak, J. A., & DuPre, E. P. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41, 327–350.

    Article  PubMed  Google Scholar 

  • Durlak, J., & Weissberg, R. (2007). The impact of after-school programs that promote personal and social skills. Chicago: Collaborative for Academic, Social, and Emotional Learning.

    Google Scholar 

  • Durlak, J., &Weissberg, R. (2005). A major meta-analysis of positive youth development programs. Paper presented at the annual meeting of the American Psychological Association, Washington, DC.

    Google Scholar 

  • Elliott, D. S., & Mihalic, S. (2004). Issues in disseminating and replicating effective prevention programs. Prevention Science, 5(1), 47–53.

    Article  PubMed  Google Scholar 

  • Finney, S. J., & DiStefano, C. (2006). Non-normal and categorical data in structural equation modeling. In G. R. Hancock & R. O. Mueller (Eds.), Structural equation modeling: A second course (pp. 269–312). Greenwich, CT: Information Age Publishing.

    Google Scholar 

  • Forss, K., Rebien, C. C., & Carlsson, J. (2002). Process use of evaluations: Types of use that precede lessons learned and feedback. Evaluation, 8, 29–45.

    Article  Google Scholar 

  • Freedman, D. A. (2001). Ecological inference and the ecological fallacy. International Encyclopedia of the Social & Behavioral Sciences, 6, 4027–4030.

    Article  Google Scholar 

  • Gallagher, K. M., Stanley, A., Shearer, D., & Mosca, C. (2005). Implementation of youth development programs: Promise and challenges. Journal of Adolescent Health, 37, S61–S68.

    Article  PubMed  Google Scholar 

  • Gottfredson, D. C., & Gottfredson, G. D. (2002). Quality of school-based prevention programs: Result from a national survey. Journal of Research in Crime and Delinquency, 39, 3–36.

    Article  Google Scholar 

  • Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55.

    Article  Google Scholar 

  • Kegler, M., Young, K., Marshall, L., Bui, D., & Rodine, S. (2005). Positive youth development linked with prevention in a Vietnamese-American community: Successes, challenges, and lessons learned. Journal of Adolescence Health, 37, 69–79.

    Article  Google Scholar 

  • Kline, R. B. (2005). Principles and practices of structural equation modeling. New York: Guilford Press.

    Google Scholar 

  • Lee, T. Y. (2008). A case study on the implementation of a positive youth development program (Project P.A.T.H.S.) in a changing education policy environment. The Scientific World Journal, 8, 1010–1026.

    Article  Google Scholar 

  • Lerner, R. M., Fisher, C. B., & Weinberg, R. A. (2000). Towards a science for and of the people: Promoting civil society through the application of developmental science. Child Development, 71, 11–20.

    Article  PubMed  Google Scholar 

  • Malvin, J. H., Moskowitz, J. M., Schaeffer, G. A., & Schaps, E. (1984). Teacher training in affective education for the primary prevention of adolescent drug abuse. The American Journal of Drug and Alcohol Abuse, 10, 223–235.

    Article  PubMed  Google Scholar 

  • McCreary, L. L., Kaponda, C. P. N., Kafulafula, U. K., Ngalande, R. C., Kumbani, L. C., Jere, D. L. N., et al. (2010). Process evaluation of HIV prevention peer groups in Malawi: A look inside the black box. Health Education Research, 25, 965–978.

    Article  PubMed  Google Scholar 

  • Moskowitz, J. M., Schaps, E., Schaeffer, G. A., & Malvin, J. H. (1984). Evaluation of a substance-abuse prevention program for junior high school students. International Journal of the Addictions, 19, 419–430.

    PubMed  Google Scholar 

  • Pind, J., Gunnarsdóttir, E. K., & Jóhannesson, H. S. (2003). Raven’s standard progressive matrices: New school age norms and a study of the test’s validity. Personality and Individual Differences, 34, 375–386.

    Article  Google Scholar 

  • Schumacker, R. E., & Lomax, R. G. (2004). A beginner’s guide to structural equation modeling (2nd ed.). Mahwah, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Shek, D. T. L. (2010). Using students’ weekly diaries to evaluate positive youth development programs: Are findings based on multiple studies consistent? Social Indicators Research, 95(3), 475–487.

    Article  Google Scholar 

  • Shek, D. T. L., & Ma, C. M. S. (2012). Program implementers’ evaluation of the Project P.A.T.H.S.: Findings based on different datasets over time. The Scientific World Journal. Advance online publication. doi:10.1100/2012/918437.

  • Shek, D. T. L., Ma, C. M. S., & Tang, C. Y. P. (2011). Subjective outcome evaluation of the Project P.A.T.H.S.: Findings based on different datasets. International Journal of Adolescent Medicine and Health, 23(3), 237–243.

    PubMed  Google Scholar 

  • Shek, D. T. L., & Ma, H. K. (2008). Evaluation of Project P.A.T.H.S. (Secondary 1 program) by the program implementers: Findings based on the full implementation phase. The Scientific World Journal, 8, 492–501.

    Article  Google Scholar 

  • Shek, D. T. L., & Ng, C. S. M. (2009). Secondary 1 program of Project P.A.T.H.S.: Process evaluation based on the co-walker scheme. The Scientific World Journal: TSW Child Health & Human Development, 9, 704–714.

    Article  Google Scholar 

  • Shek, D. T. L., & Ng, C. S. M. (2011). Evaluation of the tier 1 program (Secondary 2 program) of Project P.A.T.H.S.: Conclusions drawn by the program implementers. International Journal of Adolescent Medicine and Health, 4(1), 41–51.

    Google Scholar 

  • Shek, D. T. L., & Sun, R. C. F. (2009a). Implementation of a positive youth development program in a Chinese context: A case study based on Project P.A.T.H.S. International Journal of Child and Adolescent Health, 2(4), 431–446.

    Google Scholar 

  • Shek, D. T. L., & Sun, R. C. F. (2009b). Qualitative evaluation of the Project P.A.T.H.S. (Secondary 1 program) based on the perceptions of the program implementers. International Public Health Journal, 1(3), 255–265.

    Google Scholar 

  • Shek, D. T. L., & Sun, R. C. F. (2010). Subjective outcome evaluation based on secondary data analyses: The Project P.A.T.H.S. in Hong Kong. The Scientific World Journal: TSW Child Health & Human Development, 10, 224–237.

    Article  Google Scholar 

  • Shek, D. T. L., Sun, R. C. F., & Kan, V. W. M. (2009). Full implementation of the Secondary 1 program of Project P.A.T.H.S.: Observations based on the co-walker scheme. The Scientific World Journal: TSW Child Health & Human Development, 9, 982–991.

    Article  Google Scholar 

  • Shek, D. T. L., Sun, R. C. F., & Lung, D. W. M. (2008). Evaluation of Project P.A.T.H.S. (Secondary 2 program) by the program implementers: Findings based on the experimental implementation phase. The Scientific World Journal, 8, 536–546.

    Article  Google Scholar 

  • Shek, D. T. L., Sun, R. C. F., & Siu, A. M. H. (2008). Interim evaluation of the Secondary 2 program of a youth development program: Insights based on the experimental implementation phase. International Journal of Child and Adolescent Health, 1(2), 185–193.

    Google Scholar 

  • Siu, A. M. H., & Shek, D. T. L. (2010). Secondary data analyses of conclusions drawn by the program implementers of a positive youth development in Hong Kong. The Scientific World Journal: TSW Child Health & Human Development, 10, 238–249.

    Article  Google Scholar 

  • Taut, S. M. (2007). Studying self-evaluation capacity building in a large international development organization. American Journal of Evaluation, 28, 45–59.

    Article  Google Scholar 

  • Wallace, T. L. (2008). Integrating participatory elements into an effectiveness evaluation. Studies in Educational Evaluation, 34, 201–207.

    Article  Google Scholar 

  • Weissberg, R. P., & O’Brien, M. U. (2004). What works in school-based social and emotional learning programs for positive youth development. Annual American Academy of Political & Social Science, 591, 86–97.

    Article  Google Scholar 

  • Wilson, D. K., Griffin, S., Saunders, R. P., Kitzman-Ulrich, H., Meyers, D. C., & Mansard, L. (2009). Using process evaluation for program improvement in dose, fidelity and reach: The ACT trial experience. International Journal of Behavioral Nutrition and Physical Activity, 6, 79–89.

    Article  PubMed  Google Scholar 

  • Yohalem, N., & Wilson-Ahlstrom, A. (2010). Inside the black box: Assessing and improving quality in youth programs. American Journal of Community Psychology, 45, 350–357.

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Applied Social Sciences, The Hong Kong Polytechnic University, Hung Hom, Hong Kong

    Cecilia M. S. Ma Ph.D. & Daniel T. L. Shek Ph.D., FHKPS, BBS, JP

Authors
  1. Cecilia M. S. Ma Ph.D.
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daniel T. L. Shek Ph.D., FHKPS, BBS, JP
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cecilia M. S. Ma Ph.D. .

Editor information

Editors and Affiliations

  1. Department of Applied Social Sciences, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong SAR

    Daniel T.L. SHEK

  2. Faculty of Education, The University of Hong Kong, Pok Fu Lam Rd, Hong Kong, Hong Kong SAR

    Rachel C. F. Sun

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer Science+Business Media Singapore

About this chapter

Cite this chapter

Ma, C.M.S., Shek, D.T.L. (2013). Subjective Outcome Evaluation Based on the Perceptions of the Program Implementers. In: SHEK, D., Sun, R. (eds) Development and Evaluation of Positive Adolescent Training through Holistic Social Programs (P.A.T.H.S.). Quality of Life in Asia, vol 3. Springer, Singapore. https://doi.org/10.1007/978-981-4451-54-3_8

Download citation

Publish with us

Policies and ethics

Search

Navigation