Skip to main content
SpringerLink
Log in

The Dependability of NSSE Scalelets for College- and Department-level Assessment*

  • Published:
Research in Higher Education Aims and scope Submit manuscript

College and university administrators, as well as faculty members, are more likely to take responsibility for student learning if they believe that the assessment data represent their students and suggest specific actions for improvement. This study examined whether it is feasible to develop scalelets (i.e., focused measures, usually consisting of four or five items) that provide dependable metrics for assessing student learning at the college or department level. A generalizability analysis of 12 scalelets from the National Survey of Student Engagement (NSSE) indicated that the scalelets provided dependable measures of educational effectiveness with 25–50 respondents.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • A. W. Astin (1977) Four Critical Years Jossey-Bass San Francisco

    Google Scholar 

  • A. W. Astin (1984) ArticleTitleStudent involvement: A developmental theory for higher education Journal of College Student Personnel 25 297–307

    Google Scholar 

  • A. W. Astin (1993) What Matters in College? Four Critical Years Revisited Jossey-Bass San Francisco

    Google Scholar 

  • L. L. Baird (1976) Using Self-reports to Predict Student Performance The College Board New York

    Google Scholar 

  • T. W. Banta (1993) Summary and conclusion: Are we making a difference? T. Banta (Eds) Making a Difference: Outcomes of a Decade of Assessment in Higher Education Jossey-Bass San Francisco 357–376

    Google Scholar 

  • T. W. Banta (2002) A call for transformation T. Banta (Eds) Building a Scholarship of Assessment Jossey-Bass San Francisco 284–292

    Google Scholar 

  • R. Berdie (1971) ArticleTitleSelf-claimed and tested knowledge Educational and Psychological Measurement 31 629–636

    Google Scholar 

  • R. L. Brennan (1983) Elements of Generalizability Theory ACT Publications Iowa City, IA

    Google Scholar 

  • A. W. Chickering Z. F. Gamson (1987) ArticleTitleSeven principles for good practice in undergraduate education AAHE Bulletin 7 IssueID3 3–7

    Google Scholar 

  • L. J. Cronbach G. C. Gleser H. Nanda N. Rajaratnam (1972) The Dependability of Behavioral Measurements: Theory of Generalizability for Scores and Profiles Wiley New York

    Google Scholar 

  • W. J. Dixon (1992) BMDP Statistical Software Manual University of California Press Berkeley, CA

    Google Scholar 

  • El-Khawas, E. (2003). Using NSSE data for assessment and institutional improvement. Documenting Effective Educational Practices: National Roundtable Series (Issue 5). Indiana University Center for Postsecondary Research, Bloomington, IN

  • P. T. Ewell (1991) To capture the ineffable: New forms of assessment in higher education G. Grant (Eds) Review of Research in Education NumberInSeries17 American Educational Research Association Washington, D.C. 75–126

    Google Scholar 

  • P. T. Ewell (2002) An emerging scholarship: A brief history of assessment T. Banta (Eds) Building a Scholarship of Assessment Jossey-Bass San Francisco 3–25

    Google Scholar 

  • K. A. Feldman T. M. Newcomb (1969) The Impact of College on Students Jossey-Bass San Francisco

    Google Scholar 

  • Higher Education Research Institute. College Student Survey. University of California Los Angeles, Los Angeles, CA

  • R. Hughes C. R. Pace (2003) ArticleTitleUsing NSSE to study student retention and withdrawal Assessment Update: Progress, Trends, and Practices in Higher Education 15 IssueID4 1–2, 15

    Google Scholar 

  • InstitutionalAuthorNameIndiana University Center for Postsecondary Research (2001) Improving the College Experience: National Benchmarks of Effective Educational Practice Author Bloomington, IN

    Google Scholar 

  • InstitutionalAuthorNameIndiana University Center for Postsecondary Research (2002) From Promise to Progress: How Colleges and Universities are using Student Engagement Results to Improve Collegiate Quality Author Bloomington, IN

    Google Scholar 

  • InstitutionalAuthorNameIndiana University Center for Postsecondary Research (2003) Converting Data into Action: Expanding the Boundaries of Institutional Improvement Author Bloomington, IN

    Google Scholar 

  • InstitutionalAuthorNameIndiana University Center for Postsecondary Research (2004) Student Engagement: Pathways to Collegiate Success Author Bloomington, IN

    Google Scholar 

  • M. T. Kane G. M. Gillmore T. J. Crooks (1976) ArticleTitleStudent evaluations of teaching: The generalizability of class means Journal of Educational Measurement 13 171–183 Occurrence Handle10.1111/j.1745-3984.1976.tb00009.x

    Article  Google Scholar 

  • Kezar, A. (2002). Faculty developers using the0 National Survey of Student Engagement (NSSE) to be change agents. Documenting Effective Educational Practices: National Roundtable Series (Issue 1). Indiana University Center for Postsecondary Research, Bloomington, IN

  • Kezar, A. (2003). Student affairs administrators: building collaborations with students and academic affairs for institutional improvement. Documenting Effective Educational Practices: National Roundtable Series (Issue 3). Indiana University Center for Postsecondary Research, Bloomington, IN

  • G. D. Kuh (2001a) ArticleTitleAssessing what really matters to student learning: Inside the National Survey of Student Engagement Change 33 IssueID3 10–17

    Google Scholar 

  • G. D. Kuh (2001b) The National Survey of Student Engagement: Conceptual Framework and Overview of Psychometric Properties Indiana University Center for Postsecondary Research Bloomington, IN

    Google Scholar 

  • G. D. Kuh R. M. Gonyea D. P. Rodriguez (2002) The scholarly assessment of student development T. Banta (Eds) Building a Scholarship of Assessment Jossey-Bass San Francisco 100–128

    Google Scholar 

  • G. D. Kuh J. C. Hayek R. M. Carini J. A. Ouimet R. M. Gonyea J. Kennedy (2001) NSSE Technical and Norms Report Indiana University Center for Postsecondary Research Bloomington, IN

    Google Scholar 

  • G. D. Kuh C. R. Pace N. Vesper (1997) ArticleTitleThe development of process indicators to estimate student gains associated with good practices in undergraduate education Research in Higher Education 38 435–454 Occurrence Handle10.1023/A:1024962526492

    Article  Google Scholar 

  • S. Messick (1989) Validity R. Linn (Eds) Educational Measurement EditionNumber3 Macmillan New York 13–103

    Google Scholar 

  • J. A. Muffo M. A. Bunda (1993) Attitude and opinion data T. Banta (Eds) Making a Difference: Outcomes of a Decade of Assessment in Higher Education Jossey-Bass San Francisco 168–178

    Google Scholar 

  • InstitutionalAuthorNameNational Center for Higher Education Management Systems (1994) A Preliminary Study of the Feasibility and Utility for National Policy of Instructional “Good Practice” Indicators in Undergraduate Education U. S. Department of Education Washington, D.C.

    Google Scholar 

  • InstitutionalAuthorNameNational Survey of Student Engagement. (2004) College Student Report Indiana University Bloomington, IN

    Google Scholar 

  • C. R. Pace (1984) Measuring the Quality of College Student Experiences Center for the Study of Evaluation, University of California Los Angeles Los Angeles

    Google Scholar 

  • C. R. Pace (1985) The Credibility of Student Self-reports Center for the Study of Evaluation, University of California Los Angeles Los Angeles

    Google Scholar 

  • C. R. Pace G. D. Kuh (1998) College Student Experiences Questionnaire EditionNumber4 Indiana University Bloomington, IN

    Google Scholar 

  • E. T. Pascarella P. T. Terenzini (1991) How College Affects Students: Findings and Insights from Twenty Years of Research Jossey-Bass San Francisco

    Google Scholar 

  • E. T. Pascarella P. T. Terenzini (2005) How College Affects Students: A Third Decade of Research NumberInSeriesVol. 2 Jossey-Bass San Francisco

    Google Scholar 

  • M. W. Peterson M. K. Einarson C. H. Augustine D. S. Vaughan (1999) Institutional Support for Student Assessment: Methodology and Results of a National Survey National Center for Postsecondary Improvement Stanford, CA

    Google Scholar 

  • M. W. Peterson D. S. Vaughan (2002) Promoting academic improvement: Organizational and administrative dynamics that support student assessment T. Banta (Eds) Building a Scholarship of Assessment Jossey-Bass San Francisco 26–46

    Google Scholar 

  • G. R. Pike (1994) Applications of generalizability theory in higher education assessment research J. Smart (Eds) Higher Education: Handbook of Theory and Research NumberInSeriesVol. X Agathon New York 45–87

    Google Scholar 

  • G. R. Pike (1995) ArticleTitleThe relationship between self reports of college experiences and achievement test scores Research in Higher Education 36 1–21 Occurrence Handle10.1007/BF02207764

    Article  Google Scholar 

  • G. R. Pike (1999) Assessment measures T. W. Banta (Eds) Assessment Update: The First Ten Years National Center for Higher Education Management Systems Boulder, CO 418–522

    Google Scholar 

  • G. R. Pike (2002) Measurement issues in outcomes assessment T. Banta (Eds) Building a Scholarship of Assessment Jossey-Bass San Francisco 131–147

    Google Scholar 

  • G. R. Pike G. D. Kuh (2005) ArticleTitleA typology of student engagement for American colleges and universities Research in Higher Education 46 185–210 Occurrence Handle10.1007/s11162-004-1599-0

    Article  Google Scholar 

  • G. R. Pike G. D. Kuh R. M. Gonyea (2003) ArticleTitleThe relationship between institutional mission and students’ involvement and educational outcomes Research in Higher Education 44 243–263 Occurrence Handle10.1023/A:1022055829783

    Article  Google Scholar 

  • J. Pohlmann D. Beggs (1974) ArticleTitleA study of the validity of self-reported measures of academic growth Journal of Educational Measurement 11 115–119 Occurrence Handle10.1111/j.1745-3984.1974.tb00980.x

    Article  Google Scholar 

  • L. A. Schreiner S. L. Juillerat (1994) Student Satisfaction Inventory: 4-Year College and University Version USA Group Noel-Levitz Iowa City, IA

    Google Scholar 

  • R. J. Shavelson N. M. Webb (1991) Generalizability Theory: A Primer Sage Newbury Park, CA

    Google Scholar 

  • D. Thissen R. L. Mislevy (1990) Testing algorithms J. Wainer (Eds) Computerized Adaptive Testing: A Primer Lawrence Erlbaum Hillsdale, NJ 103–136

    Google Scholar 

  • H. Wainer G. L. Kiely (1987) ArticleTitleItem clusters and computer adaptive testing: A case for testlets Journal of Educational Measurement 24 185–201 Occurrence Handle10.1111/j.1745-3984.1987.tb00274.x

    Article  Google Scholar 

  • H. Wainer N. J. Dorans B. F. Green R. J. Mislevy L. Steinberg D. Thissen (1990) Future challenges H. Wainer (Eds) Computerized Adaptive Testing: A Primer Lawrence Erlbaum Hillsdale, NJ 233–271

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Mississippi State University, Mississippi State, USA

    Gary R. Pike (Director of Institutional Research)

  2. Mississippi State University, 269A Allen Hall, P.O. Box EY, Mississippi State, MS, 39762-5708, USA

    Gary R. Pike (Director of Institutional Research)

Authors
  1. Gary R. Pike
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gary R. Pike.

Additional information

*SAIR 2004 Best Paper. Presented at the annual meeting of the Association for Institutional Research, Sandiego, CA, March 2005.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Pike, G.R. The Dependability of NSSE Scalelets for College- and Department-level Assessment*. Res High Educ 47, 177–195 (2006). https://doi.org/10.1007/s11162-005-8885-3

Download citation

  • Received:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11162-005-8885-3

Keywords

Search

Navigation