Are the Effects of Student–Faculty Interaction Dependent on Academic Major? An Examination Using Multilevel Modeling

Abstract

Using multilevel models, this study examined whether and why the strength of association between student–faculty interaction and student cognitive skills development varies across academic majors. The study utilized data from the 2008 University of California Undergraduate Experience Survey (UCUES) and a sample of 43,014 students from 119 academic majors across nine campuses. The results indicate that the impact of interaction with faculty on students’ cognitive skills development significantly varies by academic major. Findings also suggest that some aspects of departmental climate can potentially magnify the effects produced by student–faculty interaction.

Background and Literature Review

College impact research has extensively studied the relationship between student–faculty interaction and college student outcomes, revealing wide-ranging benefits in terms of students’ academic and social integration, self-confidence, satisfaction, retention, and longer-term educational aspirations (see Pascarella 1980; Pascarella and Terenzini 2005 for an extensive literature review on student–faculty interaction in relationship with college outcomes). Although historically such research has focused on “general” positive effects of faculty contact on educational outcomes utilizing aggregate student samples, in the past decade studies in higher education have begun paying attention to “conditional” effects of faculty contact using disaggregated student subsamples by race, gender, and other student characteristics (e.g., Colbeck et al. 2001; Cole 2004; Kezar and Moriarty 2000; Kim 2006; Kim and Sax 2009; Lundberg and Schreiner 2004; Sax et al. 2005). For example, Sax et al. (2005) found that, compared to female students, male students experienced greater gains in political engagement, social activism, and liberalism resulting from their interactions with faculty. By contrast, the positive effects of student–faculty interaction on students’ sense of physical, emotional, and academic well-being were more evident among females. With respect to race, Kim (2006) showed that student–faculty interaction had a significantly positive effect on White students’ educational aspirations, but not on African Americans, Asian Americans, and Latinos. Kim also found that student–faculty interaction had no significant effect on racial tolerance for African American and Latino students, as opposed to a significant positive effect for White and Asian American students. Recently, Kim and Sax (2009) found that the effects of student–faculty interaction on college student outcomes also varied depending on students’ social class as well as gender and race. Their study revealed that course-related student–faculty interaction significantly and positively related to students’ overall satisfaction with college for all social class categories, but the association was significantly stronger for upper-class students than middle-class students. Still, these studies tend to investigate individual- (or student-) level differences in the effects of student–faculty interaction, relatively ignoring the examination of group-level differences (e.g., disaggregated by institutions, or academic majors or departments).

Given the increasing size of higher education institutions and diversity of academic disciplines, it is worth examining whether and how the impact of students’ college experience (in this case, student–faculty interaction) varies by academic sub-environment (major). College impact research on students’ academic sub-environments suggest that distinctive academic sub-environments (e.g., majors, departments, or academic disciplines) within institutions play a more critical role than institution-level environments in contributing to student outcomes (Feldman and Newcomb 1969; Feldman et al. 2008; Jessor 1981; Laird et al. 2008; Pascarella and Terenzini 2005; Umbach and Wawrzynski 2005). Informed by previous higher education literature (e.g., Braxton and Hargens 1996; Feldman and Newcomb 1969; Parsons and Platt 1973; Vreeland and Bidwell 1966) and their own empirical studies using Holland’s (1966, 1973, 1985, 1997) theory of careers, Smart et al. (2000) argue that understanding different academic disciplines and their environments is essential to explaining college students’ experiences and their development in college. There have been other numerous studies relying on Holland’s theory to examine disciplinary-based patterns of college students’ experiences and outcomes such as academic and job satisfaction, academic and social involvement, and involvement in diversity-related activities (e.g., Feldman et al. 2001; Feldman et al. 2004; Milem and Umbach 2003; Umbach 2006; Umbach and Milem 2004; Wolniak and Pascarella 2005) and the findings of the studies generally support Smart et al.’s (2000) argument.

Smart et al. (2000) also suggests that the differential effects of academic majors or departments on student development “is assumed to be manifested in large part through student interactions with departmental faculty” (p. 13). That is, faculty members are important socializing agents for college students, and through the interactions with departmental faculty, students experience a unique academic sub-environment that may distinctively influence their learning and development. Indeed, the findings of a series of higher education studies (which also employed Holland’s work) suggest that the professional attitudes and behaviors of faculty are essential to differential effects of academic disciplines on student development (Morstain and Smart 1976; Smart 1982; Smart and Thompson 2001; Smart and Umbach 2007). For example, Smart and Thompson (2001) found that faculty in Investigative environments encourage students to develop their analytical, mathematical, and scientific abilities more so than their colleagues in other academic environments, while faculty in Artistic environments more strongly encourage students to develop their innovation, creativity, and literary abilities. Also, findings from other studies suggest that faculty in different academic environments tend to use different educational strategies to promote students’ learning and development (Milem and Umbach 2003; Milem et al. 2004; Umbach 2006; Umbach and Milem 2004). When it comes to student–faculty interaction, Umbach (2006) found that faculty in Social and Artistic environments are more likely to interact with their students than their peers in other academic environments.

Consequently, it seems reasonable, based on the aforementioned findings, to expect that the influence of student–faculty interaction may be conditioned by academic discipline. The current study attempts to improve our understanding of the disciplinary effects of college by examining differential effects of student–faculty interaction on students’ growth in self-assessed cognitive skills across academic majors. General cognitive or intellectual abilities and skills are among the most desired student educational outcomes (Pascarella and Terenzini 1991, 2005) and have been considered some of the key byproducts of student–faculty interaction (Astin 1993; Endo and Harpel 1982; Kuh and Hu 2001; Pascarella and Terenzini 2005; Sax et al. 2005; Strauss and Terenzini 2007; Terenzini and Pascarella 1980; Volkwein et al. 1986). Through the use of multilevel modeling, this study seeks to answer the following two research questions: (1) Is the relationship between student–faculty interaction and student cognitive skills development different across academic majors? (2) If yes, what aspects of departmental climate contribute to differences in the relationship between faculty interaction and cognitive skills development? Answers to these questions are intended to inform scholarship that typically presumes general effects of student–faculty interaction, and to provide useful insights to faculty aiming to support students in optimal ways across the disciplines.

Research Framework

Holland’s theory of careers (1985), Astin’s involvement theory (1984), and models of college impact suggested by Astin (1991) and Pascarella (1985) are especially relevant to the current study, in both a conceptual and methodological sense. Holland’s (1985) theory argues that the interaction between individuals and their environment shapes their behavior. He describes six different personality types that both contribute to and are reinforced by individual disciplines. These types are: Realistic, Investigative, Artistic, Social, Enterprising, and Conventional. Based on his theory, we assume that structural and contextual environments of different academic majors reflect these different “types” and uniquely shape the pattern and effect of student–faculty interaction within each academic major. Though the theory is primarily interested in accounting for vocational behavior, Holland (1997) argues that the underlying basis of his theory is applicable to an educational context. Smart et al. (2000) provide evidence that faculty and student characteristics do indeed vary in ways generally consistent with Holland’s types.

Astin’s involvement theory stresses “behavioral mechanisms or processes that facilitate student development” (Astin 1984, p. 301). He suggests that students are more likely to learn and develop when they invest more time and energy in meaningful college experiences. As such, when examining the effects of student–faculty interaction, the present study considers the role played by other aspects of students’ academic engagement in addition to considering the role played by disciplinary climate.

Also relevant to the current study are Astin’s (1991) Input–Environment–Outcome (I–E–O) model and Pascarella’s (1985) causal model for assessing the impact of college on cognitive development. Astin’s model, with its emphasis on accounting for relevant student background characteristics (inputs) when assessing the impact of aspects of the college environment on student outcomes, provides an overall framework for estimating the unique predictive power of student–faculty interaction on the outcome measure, controlling for student inputs, college environments, and experiences. Pascarella’s model also acknowledges the biasing effects of students’ pre-college characteristics, but is more explicit in addressing the role of student–faculty interaction and the institutional (or in this case, the disciplinary) environment in the prediction of students’ cognitive development.

Methods

Data Source and Sample

This study used data from the 2008 University of California Undergraduate Experience Survey (UCUES), which is a biennial online survey of all University of California (UC) undergraduate students administered by the UC Berkeley Office of Student Research and managed by the UC Office of the President. The survey instrument includes a Core Module completed by all respondents, as well as five modules, each of which is administered to a randomly selected 20% of students at each institution. This study utilized items solely from the Core Module, which gathers information on student background characteristics, academic and personal development, academic engagement, satisfaction, and evaluation of the major (Brint et al. 2007). The UCUES 2008 population includes all undergraduate students at nine UC campuses who were enrolled in winter quarter 2008 (at Davis, Irvine, Los Angeles, Riverside, San Diego, Santa Barbara, and Santa Cruz) or spring semester 2008 (at Berkeley and Merced) and who were 18 years or older on April 1, 2008. The systemwide response rate for the 2008 survey was 39.2%, reporting 63,528 respondents out of 162,061 eligible participants. Generally speaking, the UCUES respondent sample is a good reflection of the UC undergraduate population (Chatman 2007a).

Given that this study attempted to examine the differential effects of student–faculty interaction as dependent upon students’ academic majors and departmental climate, the sample for this study was limited to those students who met all of the following conditions: (1) completed the 2008 UCUES Core Module, (2) declared their academic majors, and (3) had valid Classification of Instructional Programs (CIP) code. Given that academic major fields may have different departmental names across campuses in the UC system, this study identified student academic majors using each department’s CIP code. Initially developed by the U.S. Department of Education’s National Center for Educational Statistics (NCES) in 1980, the CIP is a taxonomic scheme of students’ academic major field and is used for the purpose of accurate student tracking and assessment (for more information regarding the taxonomy, refer to the website http://nces.ed.gov/pubs2002/cip2000/). Moreover, to improve the reliability of aggregate measures of student-level variables that were used as level-2 variables (i.e., department¹-level variables) in hierarchical linear modeling (HLM) analysis, academic majors that had less than 50 systemwide respondents were excluded from the study sample.

Consequently, the final sample used for this study was composed of 43,014 students from 119 academic majors across nine campuses. The sample included 58.9% female students and 41.1% male students. The racial composition of the final sample was as follows: 39.2% Asian American (including Filipino and Pacific Islander), 34.6% White, 13.6% Latino, 2.4% African American, and 10.2% other races. Of the sample, 46.4% were senior students, 31.2% juniors, 12.9% sophomores, and 9.5% were first-year students. When we grouped students based on Holland’s (1985) classification of academic environments, 32.5% came from Social, 28.2% from Investigative, 12.0% from Artistic, 10.4% from Enterprising, 4% from Realistic, and 13.0% from academic majors that do not fall into any of the categories of Holland Classification. Of the total 119 academic majors included in the final sample, 26.1% were Investigative, 25.2% Social, 20.2% Artistic, 9.2% Enterprising, 1.7% Realistic, and 17.6% were in majors not represented in the Holland Classification. (See Table 1 for distributions of students and departments by Holland’s categories.)

Table 1 University of California academic majors classified by Holland’s academic environments

Variables

Given that the UCUES data were collected from nine undergraduate campuses in a research university system, they share numerous structural and organizational similarities (Chatman 2007b). Therefore, we assumed for the purpose of data analysis that the UCUES data have a two-level hierarchy (i.e., student- and department-level) rather than a three-level (i.e., student-, department-, and institution-level). Indeed, using the 2006 UCUES data, Chatman (2007b) found that student experiences tend to have greater variance across academic majors than across institutions. Based on the assumed two-level hierarchy in the UCUES data, this study utilized both student-level and department-level variables (Table 5 in Appendix 1 provides a list of coding schemes and descriptive statistics for all variables). Potential biases that may relate to the use of two-level hierarchies are discussed in the limitations section.

Student-Level Variables

Student-level variables included a dependent variable, two principal independent variables (i.e., student–faculty interaction measures), and 11 control variables (see Table 5 in Appendix 1). To examine the relationship between student–faculty interaction and student development, we employed cognitive skills as our student outcome measure. This measure is a three-item factor representing students’ self-assessment of their current cognitive abilities in three areas: analytical and critical thinking skills, ability to be clear and effective when writing, and ability to read and comprehend academic material (Chronbach’s α = .80). College impact research has continuously demonstrated that students’ growth in intellectual and cognitive skills is a function of their interaction with faculty (Astin 1993; Endo and Harpel 1982; Kuh and Hu 2001; Pascarella and Terenzini 2005; Sax et al. 2005; Strauss and Terenzini 2007; Terenzini and Pascarella 1980; Volkwein et al. 1986). Further, using the 2006 UCUES data, Kim and Sax (2009) confirmed the positive effects of student–faculty interaction on students’ gains in critical thinking and communication skills.

Students’ frequency of interaction with faculty was measured by both a factor scale (general faculty contact) and a dichotomous variable (research engagement with faculty). General faculty contact is composed three items concerning how often a student was involved in the following activities in college: talking with faculty outside of class about course material, communicating with faculty by email or in person, and interacting with faculty during lecture class sessions (Chronbach’s α = .82). This composite measure was generated through an exploratory factor analysis using principal component factoring and Varimax rotation methods, and we computed the composite measure scores by summing scores for each item and dividing by the number of items in the factor scale. Research engagement with faculty was created based on three items concerning whether students assisted faculty with research as a volunteer, for course credit, or for pay. Due to somewhat low internal consistency of the items (Chronbach’s α = .60), we generated a dichotomous variable that captures students’ general research engagement with faculty (1 = yes, 0 = no).

Student-level control variables include a pretest for the dependent variable (students’ retrospective self-assessed cognitive abilities when entering college) as well as multiple, but carefully selected, variables that capture students’ demographic and background characteristics and college experiences. The pretest measure is a three-item factor representing students’ self-assessment of their cognitive abilities when they entered the college in the same three areas as the dependent variable: analytical and critical thinking skills, ability to be clear and effective when writing, and ability to read and comprehend academic material (Chronbach’s α = .80). By including the pretest measure as a control variable when predicting students’ current cognitive skills, this study aimed to assess students’ growth or development in their cognitive skills. Although literature suggests a number of control variables we might consider when examining the relationship between student–faculty interaction and college student outcomes (see those studies cited in this and earlier sections) to control for confounding effects and minimize the self-selection bias, we reduced them to a manageable number for HLM analyses through careful selection and creating factor scales. Thus, we used the following 11 student-level control variables for HLM analyses: pretest, gender (female), African American, Latino, Asian American, social class, high school GPA, collaborative work, development of scholarship, time employed, and academic engagement.

Department-Level Variables

Department-level variables were included in HLM analyses to model group-level (i.e., academic major- or department-level) effects. These variables include disciplinary typology variables (Holland’s classification of academic environments), department climate variables, and peer-group climate variables to indicate each academic major’s structural and cultural characteristics (refer to Table 5 in Appendix 1). Disciplinary typology variables are broader categories of academic majors, and this study used Holland’s (1985) six clusters of academic environments: Realistic (reference group), Investigative, Artistic, Social, Enterprising, and Conventional. However, our sample did not show any academic majors that fall into the category of Conventional; hence we did not use that category for analysis. Department climate variables are aggregate measures of students’ perceptions of their department’s academic environments (e.g., well-organized program), while peer-group climate variables are aggregate measures of student-level variables (e.g., percent female students). Both climate measures were generated by aggregating students’ individual scores by academic major on each item. Such department-level measures are considered more proximal to the daily lives of college students than institutional-level measures that are more distal in nature (Feldman et al. 2008; Laird et al. 2008; Smart 2010). The peer-group climate variables were chosen in this study to examine the college peer group effects at the department level. A large body of college impact research has suggested over the years that peer and reference groups have significant impact on college student development (e.g., Astin 1993; Dey 1996, 1997; Feldman and Newcomb 1969; Milem 1994, 1998; Pascarella et al. 1996; Pascarella and Terenzini 1991, 2005; Sax 1996). In this study, departmental mean general faculty contact and percent students who had research engagement with faculty were used to represent departments’ peer normative context where students change and/or maintain their behaviors, attitudes, and values through socialization process.

Analysis

This study employed HLM to examine whether and why the strength of association between student–faculty interaction and students’ cognitive skills development varies across academic majors. The HLM approach improves estimation of individual- and cross-level effects in studies that use hierarchically structured, multilevel data by concurrently modeling within- and between-group effects (Raudenbush and Bryk 2002). Like most behavioral and social science research, data on college students tend to have a multilevel, hierarchical structure where students are nested within academic majors/departments which are nested within universities. However, the majority of previous research on student–faculty interactions has used OLS techniques for their data analysis, which has generated analytical shortcomings such as aggregation bias, misestimated precision, and the unit of analysis problem (Raudenbush and Bryk 2002). Employing HLM, the current study attempts to overcome the shortcomings of OLS by addressing the hierarchical structure of higher education data. Given that the intercorrelation between the two student–faculty interaction measures (i.e., general faculty contact and research engagement with faculty) was not severe (r = .19), this study included both measures in a single model. Consequently, the following three stages of modeling were developed for the analysis, described further below: (1) random-effects ANOVA model, (2) random-coefficient regression model, and (3) intercepts- and slopes-as-outcomes model.

Random-Effects ANOVA Model

The analysis for this study began with a fully unconditional model. This is the simplest hierarchical linear model used in this study where a group mean (i.e., student mean cognitive skills within each academic major) and a random error predict the dependent variable (i.e., individual students’ cognitive skills) in the level-1 (student-level) model. Then, the intercept of the student-level model (i.e., each academic major’s mean cognitive skills) is predicted by a grand mean (i.e. student mean cognitive skills as a whole including all academic majors) and a random error in level-2 or department-level model. The random-effects ANOVA model was used in this study as a preliminary step to examine total variance in the student outcome measure in terms of variation between and within academic majors.

Random-Coefficient Regression Model

In this model, the unconditional model described above was expanded by incorporating 13 student-level predictors to the level-1 model. The student-level predictors include 11 control variables (i.e., six student input characteristics, four college experience variables, and a pretest measure) and two items of interest (i.e., general faculty contact and research engagement with faculty). Given that we were mainly interested in examining whether the relationship between student–faculty interaction and student cognitive skills development (i.e., slopes of two student–faculty interaction measures) significantly varies across academic majors, slopes of general faculty contact (β _12j ) and research engagement with faculty (β _13j ) were specified as random in level-2 model, while the slopes of other level-1 variables and the level-1 intercept were fixed across academic majors (see Eqs. 1, 2). In the random-coefficient regression model, all of the variables were centered on the grand mean, except for the dichotomous variables.

Level-1 for Random-Coefficient Regression Model

$$\begin{aligned} Y_{{ij}}&=\beta_{{0j}}+\beta_{{{\text{1}}j}}\left({{\text{Female}}}\right)+\beta_{{{\text{2}}j}}\left({{\text{African\;American}}}\right)+\beta_{{{\text{3}}j}}\left({{\text{Latino}}}\right)+\beta_{{{\text{4}}j}}\left({{\text{Asian\;American}}} \right)\\ &\quad+\beta_{{{\text{5}}j}}\left({{\text{Social class}}} \right)+ \beta _{{{\text{6}}j}} \left({{\text{High school GPA}}} \right)+ \beta _{{{\text{7}}j}} \left({{\text{Collaborative work}}} \right)\\ &\quad+ \beta _{{{\text{8}}j}} ({\text{Development of\;scholarship }})+ \beta _{{{\text{9}}j}} \left({{\text{Time\;employed}}} \right)+ \beta _{{{\text{10}}j}}\left({{\text{Academic\;engagement}}} \right)\\ &\quad+ \beta _{{{\text{11}}j}} \left( {{\text{Pretest}}} \right)+ \beta _{{{\text{12}}j}} \left({{\text{General faculty contact}}} \right)+ \beta _{{{\text{13}}j}} \left( {{\text{Research engagement with\;faculty}}} \right)+ r_{{ij}}\\ \end{aligned}$$

(1)

where i = 1, 2,…,n _j students in department j, and j = 1, 2,…,119 departments.

Level-2 for Random-Coefficient Regression Model

$$\begin{aligned}\beta_{{0j}} & =\gamma_{{00}}\\ \beta_{{{\text{1}}j}} & =\gamma_{{{\text{1}}0}}\\ &\quad: & \\ {\beta}_{{{\text{11}}j}} & = \gamma_{{{\text{11}}}}\\ \beta_{{{\text{12}}j}} & = \gamma_{{{\text{12}}0}}+u_{{{\text{12}}j}} \\ \beta_{{{\text{13}}j}} & = \gamma_{{{\text{13}}0}} + u_{{{\text{13}}j}} \end{aligned}$$

(2)

Intercepts- and Slopes-as-Outcomes Model

This is the most elaborated multilevel model of this study and addresses the second research question of the study: What aspects of departmental climate contribute to differences in the relationship between faculty interaction and cognitive skills development? In this model, the student-level model remained the same as in the random-coefficient regression model (i.e., Eq. 1). However, slopes of general faculty contact (β _12j ) and research engagement with faculty (β _13j ) of the student-level model were conditioned by a combination of department-level variables including disciplinary typology variables, department climate variables, and peer-group climate variables at level-2. Like the earlier random-coefficient regression model, all other level-1 parameters (β _1j –β _11j ) and the level-1 intercept (β _0j ) were specified as fixed across academic majors at level-2 model.

In this stage of modeling, before specifying the final exploratory model, we first tested a pilot model entitled “typology effects model” where the slopes for student–faculty interaction measures were predicted by only disciplinary typology variables (i.e., Holland’s classification of academic environments) in the department-level model. As mentioned earlier, because our sample did not show any academic majors that fall into the category of Conventional, we used four disciplinary typology variables—Investigative, Artistic, Social, and Enterprising—with a reference group of Realistic for analysis. This pilot model allowed us to get a broad sense of disciplinary differences in the effects of faculty interaction by comparing the strength of association between student–faculty interaction and students’ cognitive skills growth across six different clusters of academic majors (e.g., Realistic vs. Social). However, as will be discussed later in this article, results from the pilot model indicated that none of the disciplinary typology variables consistently predicted the slopes of both student–faculty interaction measures (refer to “Results” section); hence, the typology measures were excluded from the final explanatory model.

Consequently, in the final explanatory model, the slopes for both student–faculty interaction measures were predicted by a combination of department-level variables including three department climate variables and six peer-group climate variables along with a base effect and a random error at department-level model. This final model helped to reveal certain aspects of departmental climate that contribute to differences in the association of student–faculty interaction and students’ cognitive skills development across academic majors. The level-2 model for this final explanatory model can be represented as the following equation:

Level-2 for Final Explanatory Model

$$ \begin{aligned}\beta_{{0j}}&=\gamma_{{00}}\\ \beta_{{{\text{1}}j}} &=\gamma_{{{\text{1}}0}}\\ &\quad: & \\ {\beta}_{{{\text{11}}j}} & = \gamma_{{{\text{11}}}}\\ \beta_{{{\text{12}}j}}&=\gamma_{{{\text{120}}}}+\gamma_{{{\text{121}}}} \left({{\text{Well-organized program}}} \right) + \gamma_{122}\;\left( {\text{Positive faculty support}} \right)\\ &\quad+ \gamma_{123}\;({\text{Course}}\;{\text{requirement:}}{\text{ Critical thinking}}) + \gamma_{124} \left( {\% {\text{ Female}}} \right)\\ &\quad + \gamma_{125} \left( {\% {\text{ African American}}} \right) + \gamma_{126}\;(\% {\text{Latino}}) + \gamma_{127}\;\left( {\% {\text{ Asian American}}} \right)\\ &\quad+ \gamma_{128}\;\left({{\text{AVG:}}{\text{ General faculty\;contact}}} \right)+ \gamma_{129}\;({\text{AVG:}}\;{\text{Research\;engagement with faculty}}) + u_{1 2j}\\ \beta_{{{\text{13}}j}}&= \gamma_{{{\text{130}}}}+\gamma_{{{\text{131}}}}\;\left({{\text{Well-organized\;program}}} \right) + \gamma_{132}\;\left( {\text{Positive\;faculty support}} \right)\\ &\quad + \gamma_{133}\;({\text{Course}}\;{\text{requirement}}:{\text{ Critical thinking}}) + \gamma_{ 1 3 4} \left( {\% {\text{ Female}}} \right) \\ &\quad + \gamma_{135}\;\left( {\% {\text{ African American}}} \right) + \gamma_{136}\;(\% {\text{Latino}}) + \gamma_{137}\;\left( {\% {\text{ Asian American}}} \right) \\ &\quad + \gamma_{138}\;\left({{\text{AVG:}}\;{\text{General\;faculty contact}}} \right) + \gamma_{139}\;({\text{AVG:}}\;{\text{Research engagement with\;faculty}}) + u_{ 1 3j} \end{aligned} $$

(3)

Results

Random-Effects ANOVA Model: Fully Unconditional Model

To examine how much variation in students’ cognitive skills development exists within and between academic majors, we first estimated a random-effects ANOVA model that has no predictor variable. Table 2 presents the results from the fully unconditional model. The estimated intercept term (γ₀₀) shows that the predicted grand mean of students’ cognitive skills is 4.80 (on the continuum ranging from 1 = very poor to 6 = excellent). That is, regardless of their academic majors, the students in our sample tended to perceive that they have “very good” cognitive skills. The estimated variance of intercept for cognitive skills was statistically significant (u _0j  = .03193, χ² = 3181.74, df = 118, p < .001), indicating that students’ average cognitive skills significantly vary across different academic majors.²

Table 2 Results from the random-effects ANOVA model: fully unconditional model

The results of the random-effects ANOVA model also provide a useful parameter, the intraclass correlation coefficient (ICC), which indicates the proportion of variance in an outcome measure that is explained by differences among the level-2 units (Raudenbush and Bryk 2002). This ICC was computed by the following formula ρ = τ ₀₀/(τ ₀₀ + σ ²) = .03193/(.03193 + .44267) = .067, indicating that 6.7% of the variance in cognitive skills development is due to differences among academic majors.

Random-Coefficient Regression Model: Conditional Effects Model

Upon determining that students’ self-assessment of cognitive skills varies significantly by academic major, we then set out to determine whether the strength of association between student–faculty interaction and cognitive skills development also varies by major. Thus, we began to use level-1 predictors to examine the relationship between student–faculty interaction and student cognitive skills development within the 119 academic majors. In this random-coefficient regression model, students’ assessment of cognitive skills was regressed on 13 student-level variables at level-1, two of which were student–faculty interaction measures (i.e., general faculty contact and research engagement with faculty). Then, the parameters of our interest in level-1 (i.e., two student–faculty interaction measures) were conditioned to vary across academic majors as a function of a base effect and a random error at level-2 to see if the strength of association between student–faculty interaction and student cognitive skills development varies across academic majors. Table 3 displays the results of the random-coefficient regression model and provides the average regression equation within the 119 academic majors.

Table 3 Results from the random-coefficient regression model: conditional effects model

The results revealed that both student–faculty interaction measures are significantly and positively related to students’ cognitive skills. Students’ level of general faculty contact (i.e., talking, communicating, or interacting with faculty), on average, significantly and positively predicted their cognitive skills (γ₁₂₀ = .08, t = 22.58, p < .001). The results also show that the estimated variance of the slope of general faculty contact on cognitive skills is statistically significant (u _12j  = .00053, χ² = 235.56, p < .001). This indicates that the relationship between students’ general faculty contact and their cognitive skills growth does vary significantly across academic majors. We then considered just how much of a difference exists across major fields in terms of the impact of general faculty contact on cognitive skills development. Thus, we calculated the range of plausible values for each slope using the following formula suggested by Raudenbush and Bryk (2002):

$$ \gamma_{qo} \pm 1.96 \left( {\tau_{qq} } \right)^{1/2} $$

(4)

where q = 0,…,Q random coefficients in the level-1 model (in this case, Q = 13). Applying the formula for our results, the 95% plausible value range for the slope of general faculty contact on cognitive skills was

$$ .08 \pm 1.96\left( {.00053} \right)^{1/2} = \left( {.04, .12} \right). $$

This tells us that there is substantial variability in the effect of general faculty contact on students’ cognitive skills across different academic majors, in which the effect (i.e., slope) ranges from .04 to .12. In other words, the relationship between general faculty contact and cognitive skills is three times stronger in the most effective academic majors than in the least effective academic majors.

Turning now to research engagement with faculty, the results showed a similar pattern as general faculty contact, but demonstrated more considerable variability in the slope. Students’ experience of assisting faculty with research as a volunteer, for course credit, or for pay (i.e., research engagement with faculty) significantly related to higher level of cognitive skills (γ₁₃₀ = .05, t = 4.74, p < .001). The estimated variance of the slope also suggests that the relationship between students’ research experience with faculty and their cognitive skills substantially varies across different academic majors (u _13j  = .00733, χ² = 388.67, p < .001). Using Eq. 4, we also computed a plausible value range for the slope of research engagement with faculty on cognitive skills growth as follows:

$$ .05 \pm 1.96\left({.00733} \right)^{1/2} = \left( {-.10, .21} \right). $$

The plausible value range of −.10 to .21 indicates that there exists much greater variability in the effect of students’ research experience with faculty on their cognitive skills than general faculty contact, where the effect ranges from −.10 to .12. In other words, the positive association between research engagement with faculty and cognitive skills is not only stronger in some majors than others, but actually appears to be negative (although statistical significance needs to be further examined) in some academic majors.

Intercepts- and Slopes-as-Outcomes Model: Final Explanatory Model

The results presented above address our first research question and reveal that the effects of student–faculty interaction on students’ self-assessed cognitive skills do vary depending on students’ academic majors. We now address the second research question by examining why some academic majors have stronger association between student–faculty interaction and cognitive skills than other academic majors, and specifically whether such differences are due to aspects of departmental climate.

As described earlier, before specifying the final exploratory model, we first tested a pilot model using only disciplinary typology variables in the department-level model. Results of the pilot model showed that only a typology measure—Social fields—significantly predicted the slope of research engagement with faculty (coefficient = .12, p < .001). The result suggests that the effect of students’ research engagement with faculty on their cognitive skills tends to be greater (i.e., steeper slope) in Social field than in Realistic fields (reference group). In contrast to research engagement, none of the disciplinary typology variables predicted the slope of general faculty contact on cognitive skills. This demonstrates that the variation in the slope of this type of faculty interaction seems not to be explained by the disciplinary typology, but by other department-level characteristics (e.g., department climate, peer-group climate).

Finally, a final explanatory model was developed as the most elaborated model in this study, incorporating a full combination of department-level variables. However, given the aforementioned results from the pilot model that none of the disciplinary typology variables consistently predicted the slopes of both student–faculty interaction measures, the typology measures were excluded from the final model. Thus, in the final explanatory model, the slopes of general faculty contact and research engagement with faculty were predicted by three department climate variables and six peer-group climate variables.

Table 4 displays the results from the final explanatory models. Although these final explanatory models were developed to examine certain departmental characteristics that contribute to differences in the effects of student–faculty interaction on student cognitive skills development across academic majors, it is also noteworthy that the results show that across all academic majors both types of student–faculty interaction significantly related to growth in students’ cognitive skills even after controlling for both student- and department-level confounding effects (γ₁₂₀ = .08, t = 22.57, p < .001 for general faculty contact; γ₁₃₀ = .05, t = 5.64, p < .001 for research engagement with faculty).

Table 4 Results from the intercepts- and slopes-as-outcomes model: final explanatory model

Results for general faculty contact show that a departmental climate of positive faculty support significantly and positively predicted the slope of general faculty contact on cognitive skills (γ₁₂₂ = .07, t = 2.29, p < .05). This result suggests that academic majors with higher levels of positive faculty support tend to have larger slopes than academic majors with lower levels of positive faculty support. That is, the association between general faculty contact and students’ cognitive skills development is stronger in academic majors where students have more open channels of communication with faculty, are treated more equitably and fairly by faculty, and obtain more prompt and useful feedback on student work than in other majors which have more restricted channels of communication, less equitable treatment of students, and less prompt and useful feedback from faculty.

When it came to research engagement with faculty, two department-level variables significantly predicted the slope. The results show that effects of students’ research experience with faculty on their cognitive skills significantly varied across academic majors depending on the degree to which how well the program is organized (γ₁₃₁ = .25, t = 3.21, p < .01). That is, academic majors where the program requirements are well defined, description of the major in the catalogue is accurate, and department rules and policies are clearly communicated tend to show a stronger association between research engagement with faculty and students’ cognitive skills development. The degree to which academic majors foster critical thinking and reasoning abilities in their courses also positively related to the slope (γ₁₃₃ = .18, t = 2.77, p < .01). In other words, the effect of the research experience on cognitive skills development tends to be greater in academic majors where students are more frequently required in their courses to examine and consider other methods and conclusions, incorporate ideas from different courses, generate new ideas, and use facts and examples to support their view points. Consequently, the results for research engagement with faculty suggest that students’ cognitive abilities benefit most when they have the research experience within academic majors that have a better structured and organized program and that emphasize critical thinking and reasoning skills in their course.

Variance Explained by Department-Level Variables

The above findings suggest that majoring in fields that have a higher level of positive faculty support, have a better-organized program, and emphasize students’ critical thinking and reasoning skills in their courses can potentially amplify the educational effect of student–faculty interaction. To see how much the findings are substantive, we calculated proportion of variances in the average effect (i.e., slope) of general faculty contact and research engagement with faculty explained by the level-2 predictors, using an equation suggested by Raudenbush and Bryk (2002)—[τ _qq (unconditional) − τ _qq (conditional)]/τ _qq (unconditional)—and the estimated variance components for student–faculty interaction measures reported in the bottom panel of Tables 3 and 4. The statistics calculated in this study suggest that the department-level variables explained a substantial proportion of the variance (53.2%) in the average effect of research engagement with faculty. The contribution of department-level predictors to explaining the variance in the average effect of general faculty contact was relatively modest, but the level-2 variables still explained 13.2% of the variance.

For an illustration of the conditional effects derived from different academic environments across academic majors, we compared regression lines for lowest five and highest five academic majors in department level of positive faculty support. As shown in Fig. 1, the five highest academic majors did have a steeper slope (i.e., greater effect) for general faculty contact on cognitive skills (.20) than their lowest five counterparts (.15).

Fig. 1

An illustration of conditional effects: regression lines for lowest five and highest five academic majors in departmental level of positive faculty support

Limitations

This study is limited in several aspects. First, due to the small number of institutions (n = 9), this study hypothesized for the purpose of statistical analysis that the UCUES data have two-level hierarchies (i.e., student- and department-level) while the data are in fact clustered samples in three different levels (i.e., student-, department-, and institution-level). Although the nine institutions are in a single research university system and they have numerous structural and organizational similarities (see Chatman 2007b), it should be noted when interpreting findings that there may be biases derived from not accounting for institutional variance. Also, the use of a secondary dataset limited our selection of variables for the analysis. For example, there are other department-level variables that might capture the structural characteristics of academic majors, such as department size, gender and race composition of faculty, student–faculty ratio, etc., however, these variables were not available in the UCUES data. For the same reason, faculty climate within academic majors was measured by its proxy variables—i.e., students’ perceptions on faculty climate—rather than faculty members’ own perceptions. Another limitation is that the data are not longitudinal. Though we were able to compare students’ current and initial (retrospective) assessments of cognitive ability as a proxy for “change” or “growth,” these variables were both measured on the 2008 survey. For future studies, we will investigate whether linking the 2008 UCUES responses to those from the 2006 administration produces a sample large enough for this type of analysis. Because this study used data from a single university system, the findings may be unique to the system, thus limiting the generalizability of findings. Also, given that the system is composed of public research universities, the findings or implications of this study may not be applicable to other types of universities (e.g., 2-year, private, non doctoral/research universities). Finally, it should be noted that departmental climate measures used in this study were generated by aggregating students’ scores by academic major on each item and the aggregate measures may be biased to the extent that survey respondents do or do not represent all students in a department. It is also important to acknowledge that our measures of departmental climate were generated at a single point in time, and do not account for the dynamic nature of departmental culture as students change majors during college. Smart et al. (2000) underscore the importance of recognizing the fluidity of departmental characteristics.

Summary and Discussion

College impact research in recent decades has improved our understanding of the role that faculty play in contributing to student development during college. However, the literature has lacked information on how the impact of faculty varies by academic discipline. As stated earlier, when considering the increasing size of higher education institutions and diversity of academic disciplines, studying the potential influence of the academic sub-environment on college student learning is critical to understanding how college experiences affect student learning and development during college. This study attempted to improve our understanding of the roles played by academic majors in explaining the effects of student–faculty interaction on students’ growth in cognitive skills.

First, although this study is primarily interested in examining conditional effects of faculty–student interaction, it does confirm the net general positive effects of student–faculty interaction. That is, both types of student–faculty interaction (general faculty contact and research engagement with faculty) significantly related to growth in students’ cognitive skills even after taking both student- and development-level confounding effects into account.

The results of this study also suggest that the impact of students’ interaction with faculty varies by academic major. For example, the benefits of general faculty contact were significantly greater in fields such as Chinese Language/Literature, Microbiology, and Nutrition Sciences than in fields such Information Science/Studies, Cell/Cellular Biology, and minor areas of Social Sciences. Similarly, students’ cognitive skill development appeared to benefit most from research engagement with faculty in the fields of Hispanic-American/Chicano Studies, Geography, and Microbiology and least within Mass Communication/Media Studies, Biotechnology, and minor areas of Social Sciences. Such results naturally beg the question of why faculty influence would vary across fields, and our findings do shed light on this question.

Specifically, the relationship between general faculty contact and cognitive skill development tends to be greater in academic majors with higher levels of positive faculty support (i.e., where students have more open channels of communication with faculty, are treated more equitably and fairly by faculty, and obtain more prompt and useful feedback on student work by faculty). These findings are consistent with prior research (Kuh et al. 1997; Pike and Killian 2001) showing that supportiveness of academic environments positively relates to student learning and intellectual development, although the positive relationship in other studies was observed in a single-level structure (i.e., student-level only) rather than in a cross-level frame.

Further, the benefits of research engagement with faculty tend to be maximized in departments where the program requirements, rules, and policies are better defined and communicated. Finally, the positive effects of research engagement with faculty seem to be strongest in majors that more frequently require students in their classes to examine and consider other methods and conclusions, incorporate ideas from different courses, generate new ideas, and use facts and examples to support their view points.

The aforementioned conditional effects suggest that attending academic majors where faculty provide a higher level of positive support, students more frequently participate in critical thinking and reasoning activities in their courses, and the programs are better organized can potentially magnify the effects produced by students’ interaction with faculty. To explore this further, we conducted supplemental descriptive analyses revealing that the departments where the benefits of student–faculty interaction are greater are generally the ones that have more of those beneficial departmental characteristics. For example, Chinese Language/Literature was in the top 13th percentile in departmental average level of positive faculty support, and Hispanic-American/Chicano Studies was in the top 3rd percentile in departmental average level of using critical thinking and reasoning abilities in courses. Further, Geography and Hispanic-American/Chicano Studies were in the top 2nd and 8th percentile, respectively, of average level of students’ perception on how well the program is organized. The extent to which these conditions are prevalent and beneficial across a range of majors will require further research with more rigorous statistical analyses and/or case study design.

Implications for Research and Practice

This study not only reveals the existence of group-level conditional effects of student–faculty interaction depending on students’ academic majors but also identifies the origins of the conditional effects. Taken together, the findings provide significant implications for both higher education research and practice.

First, our findings support the arguments of past and more recent higher education literature emphasizing the significance of academic disciplines’ contributions to differences in college students’ experience and their development. Through their extensive literature review of college impact research, Feldman and Newcomb (1969) and Pascarella and Terenzini (1991, 2005) emphasized the need to understand the potential influence of academic sub-environments on college student outcomes. More recent studies also provide evidence of disciplinary differences in student learning within institutions and suggest that disparate academic environments should be taken into account when we investigate how college experience affects students’ learning and development (Feldman et al. 1999; Pike and Killian 2001; Smart and Umbach 2007; Umbach and Milem; 2004). In this study, the effects of both general faculty contact and research engagement with faculty varied considerably depending on academic major.

In a methodological sense, these results justify the examination of cross-level effects (i.e., group-level conditional effects) using multilevel modeling. As stated earlier, college student data tend to have a hierarchical, multilevel structure where individual students are nested within academic majors, which are in turn nested within institutions. This hierarchical nature of college student data raises questions about cross-level effects—i.e., how the effects occurring at one level are affected by variables measured at another level—and these questions should be adequately addressed in college impact research to fully understand how college environments and experiences shape student development during college. Although it is common practice in social science research to use random sampling models (e.g., linear regression models) for clustered samples (Garson 2008), the findings of this study suggest that the use of multilevel models (e.g., HLM) improves our understanding of group-level conditional effects of college, providing statistics that traditional OLS techniques do not allow, thereby improving the accuracy of estimation (Kish 1992; Korn and Graubard 1995; Raudenbush and Bryk 2002).

It is also noteworthy that the impact of disciplinary typology variables (i.e., broad clusters of academic majors) is meager whereas that of department climate variables is relatively substantial. The finding seems to indicate that disciplinary typology effect is mostly mitigated by individual department-level factors such as department climate measures and aggregate peer-group measures. This suggests that at least in research on student–faculty interaction, higher education research ought to deal with each academic major or department as a unique sub-environment rather than grouping academic majors into a few broad categories for the sake of parsimony. Future research will need to address whether a department-level focus is prudent across other college experiences and outcomes.

The findings of this study also provide important implications for higher education practice. The current study does reveal the existence of conditional effects of student–faculty interaction across academic majors, which in turn means that students in some academic majors benefit more from their interaction with faculty (even with the same kind and same rate of faculty contact) than their peers in other academic majors. This finding suggests that simply increasing the frequency of student–faculty interaction may not be the best strategy to produce desired student outcomes. Rather, academic institutions and individual departments should not only encourage students’ engagement with faculty but also foster academic environments which maximize the effects of the positive college experience on student learning and development.

What characterizes these effective environments? First, the more supportive faculty members are in an academic major, the stronger the relationship between general faculty contact (i.e., talking, communicating, and interacting with faculty) and student cognitive skills development. This suggests that academic departments and institutions as a whole should nurture a supportive and favorable faculty climate—e.g., where students have open channels of communication with faculty, are treated equitably and fairly by faculty, and obtain prompt and useful feedback on their work from faculty—to amplify the effects of students’ general faculty contact on their gains in cognitive skills development. The importance of promoting these positive behaviors ought to be emphasized in graduate school training, and should continue throughout all stages of faculty development.

Departments’ organizational and structural climate is another factor that conditions the strength of the relationship between student–faculty interaction and student cognitive skills development. Specifically, departmental level of how well the program requirements, rules, and policies are defined and communicated significantly determines the extent of the influence of a student’s individual level of research engagement with faculty. This finding seems to show how the benefits of an individual student experience are maximized when the experience is supported by a department’s sound organization of curricula, teaching, and administrative regulations as well as its supportive and favorable faculty climate. Students may benefit from the well-organized structure of an academic program by being exposed to the overall quality of their academic environments beyond simply experiencing a more coordinated administrative or academic affairs structure.

Higher education institutions also need to develop curricula that facilitate students’ integration of their positive college experiences (in this case, research engagement with faculty). Chickering (1974) and Tinto (1987, 1993) argued that college students’ learning and development is a function of both involvement in educationally purposeful college experiences and integration of these experiences. Several empirical studies in higher education also support their argument, highlighting the importance of integration in the relationship with college experience and outcomes (Davis and Murrell 1993, Pike 1995, 1999, 2000; Pike and Killian 2001; Pike et al. 2003). Likewise, our findings indicate that students’ experience of assisting faculty with research produces greater effects on their cognitive skills development when their academic majors encourage critical thinking and reasoning in their courses. Thus, this study suggests that universities and their departments should provide diverse course activities that help students to integrate their research experiences with faculty into their cognitive skills development in a more meaningful and efficient way.

Conclusion and Future Directions

This study reveals departmental differences in the effects of student–faculty interaction on students’ cognitive skills development. The effects of both types of faculty interaction measures—i.e., general faculty contact and research engagement with faculty—are conditioned by major field, though the effect of research engagement with faculty is more conditional (i.e., has much greater variability) than that of general faculty contact. Some departmental factors explain the variability of the effects, which also suggests that students in some academic majors benefit more or less than their peers in other academic majors. Specifically, our findings demonstrate that students in academic majors which have a higher level of positive faculty support, emphasize students’ critical thinking and reasoning skills in their courses, and have a better organized program benefit more from student–faculty interaction for their growth in cognitive skills than their peers in other academic majors.

While the current study provides some explanations of the context where the individual level effects of student–faculty interaction vary depending on academic majors, the estimated variances of the slopes of both types of student–faculty interaction in the final explanatory model of the study suggest that future research is needed to understand what accounts for the unexplained variance. To illuminate the unknown portion, future research should incorporate additional department-level variables into the HLM models such as gender and racial composition of faculty members, student–faculty ratio, and faculty attitudes and values. Recent studies of Smart and his associates (Smart 2010; Smart et al. 2009) also suggest that level of environmental consistency (i.e., consistent versus inconsistent academic environments) should be an important discipline-level variable for future research using Holland’s theory. Use of three-level HLM—where students are nested within academic majors, which in turn are nested within institutions—may be another useful method to investigate the effects of student–faculty interaction in a more comprehensive perspective. That is, the three-level approach may improve our understanding of how individual-, department-, and institutional-level effects of student–faculty interaction interplay with each other to shape individual students’ unique experience of and benefits from faculty interaction. Furthermore, given that student–faculty interaction has a positive relationship with a broad range of college student outcomes, we should also examine how the disciplinary differences in the effects of faculty interaction vary across disparate outcome measures. Finally, given the increasing diversity of the college student population, it is important to consider the role played by students’ race, social class, and gender in conditioning the impact of student–faculty interaction within different fields.

Appendix 1

See Table 5.

Table 5 Coding schemes and descriptive statistics of variables