Introduction

Whereas the use of standardized measures of vocational orientations is common for educational and early-stage career choices, they are rarely used for ‘high-stakes’ decision-making. Likewise, much of the literature on career exploration and vocational orientations focuses on high school students (Pike, 2006). However, research has shown that career decision-making is also challenging for many highly educated professionals (Mangematin, 2000), and many are undecided about which career trajectory to pursue (e.g., academia, private industry; Sauermann & Roach, 2016). Thus far, only one study has taken an in-depth look at an academia orientation (i.e., ‘taste for science’; Roach & Sauermann, 2010). Yet, little is known about the motivational and ability-related characteristics regarding an aspiration for non-academic careers.

Our primary goal is to introduce the newly developed concept of application orientation (AO) to the higher education literature. We outline and empirically test its distinct components, which go beyond what has been suggested in previous conceptualizations. Our study advances the literature on vocational orientations regarding a specific occupational field (i.e., applied work) in a specific target population (i.e., early career academics). This ‘diversification’ of vocational orientations research has been frequently demanded (Hansen, 2019; Wille et al., 2015) but has rarely been pursued (Roach & Sauermann, 2010). Further, we suggest a self-report approach to make it measurable for both research and counseling purposes. Considering the relatively great indecisiveness of young scholars regarding whether to pursue a career inside or outside academia (Hancock & Walsh, 2016; Mangematin, 2000), the demand for vocational guidance is high (Neumann & Tan, 2011). Policymakers in higher education, however, have criticized that career counseling for young scholars is often mainly concerned with academic aspirations (Borrell-Damian et al., 2010; National Academies of Sciences, Engineering, & Medicine, 2018). Closing the knowledge gap concerning those who aspire toward “careers outside of the ivory tower” (Gemme & Gingras, 2012, p. 680) is not only relevant theoretically but is also of practical importance for career counseling in higher education, as we should help young scholars to better understand their vocational aspirations and to improve informed career decision-making.

We conceptualize and validate the new construct of AO and define it as a vocational orientation concerning the interest for certain principles, values, and activities that are common for university graduates working in applied (i.e., industrial) fields. Our studies are largely based on samples of highly educated individuals from STEM disciplines (i.e., science, technology, engineering, and mathematics) as we suggest AO to be particularly characteristic for the work tasks these professionals face when entering industrial employment settings. Conducting four studies with different designs, each comprising a different sample, we conceptualize (Study 1) and validate (Studies 2–4) the new construct of AO. In Study 1, in a qualitative content analysis with employees holding a university degree and working in an industrial field (N = 102), we identified facets of applied work by categorizing open-format statements about what made their work an applied one (construct conceptualization). In Study 2, in a sample of university students and professionals (N = 200), based on self-report survey items, we analyzed similarities and differences (convergent and discriminant validity) of AO facets in relation to other concepts of vocational orientations (i.e., Realistic, Investigative, Artistic, Social, Enterprising, and Conventional [RIASEC] dimensions; Holland, 1997). In Study 3, we investigated whether an AO scale adequately distinguishes between scores of highly educated professionals working in industry (N = 108) or academia (N = 40) by deploying a known-groups validation approach. Lastly, in Study 4, in a sample of early career scientists (N = 128), we used the AO facet product focus to predict actual applied work behavior as demonstrated in an assessment center one year later (i.e., criterion-related validity).

Vocational Orientations of Early Career Academics

Most research on vocational orientations is conducted within the framework of person–environment fit. The investigation of vocational orientations has a long tradition, with the RIASEC typology (Holland, 1997) constituting the most prominent and widely used framework for research and counseling purposes (Nauta, 2010). Despite its unequivocal usefulness for examining broad interest types (e.g., social type, artistic type), scholars have recommended taking a ‘fidelity’ instead of a ‘bandwidth’ approach (see Cronbach & Gleser, 1957) to not ignore variance within broad types (Armstrong et al., 2008). For instance, if a person has a high score on the social dimension, this indicates a general interest to work with people but gives little information about specific social interactions, such as in customer service. Measures of vocational orientations that are domain-specific are expected to lead to incremental validity beyond the broad RIASEC types in predicting specific criteria (see Wille et al., 2015). Hence, ‘fidelity’ measures contribute to the advancement of the field of vocational orientations.

Research has revealed that fewer than half of PhD students are aiming for an academic career (Neumann & Tan, 2011). For a disciplinary perspective, STEM PhD students are less interested in an academic career than PhD students from other fields (e.g., humanities, social sciences; Haley et al., 2014). Likewise, the share of graduates from science and engineering working in academia is comparably low (Lin & Chiu, 2016). Despite the increasing employment rate of graduates in private industry (National Science Board, 2016), young scholars have been suggested to not receive sufficient industry-related career guidance during graduate school (Borrell-Damian et al., 2010; Main et al., 2022). Careers in private industry are no longer a backup plan for many but the preferred path. Etmanski (2019) concluded that higher education should provide young scholars with “optional initiatives geared towards career exploration beyond academia” (p. 355).

Application Orientation: Conceptual Construct Definition

The starting point of our process of construct development and validation is a conceptual definition. The theoretical context of our newly proposed construct is the literature on vocational orientations (Schermer, 2016), relating to an individual’s preferences for specific career-related choices and activities reflecting certain positions (“what they care about and what they are looking for in a job”; Roach & Sauermann, 2010, p. 424). Vocational orientations are seen as dispositional trait-like factors that were found to be relatively stable across the lifespan (Low et al., 2005). We define AO as a vocational orientation consisting of preferring tasks that are most commonly found in applied working fields. These tasks are typical for industrial work settings, which many STEM graduates enter. We further define AO as an aggregate multifaceted trait complex of vocational preferences that are functional for applied work. Moreover, we suggest AO to be an additive construct, consisting of facets that are tied together by their relevance to applied work settings (see MacKenzie et al., 2011).

We acknowledge that a substantial number of faculty members conduct both basic and applied research (Bentley et al., 2015). Further, also higher education science managers ‘apply’ their knowledge for the benefit and success of universities and research institutions. Still, from the perspective of institutional culture differences (de Wit-de Vries et al., 2019; D’Este & Perkmann, 2011; Lee, 1996), applied research and work is prototypically associated with industry (i.e., the private sector/for-profit businesses as compared to academia/higher education, government, and non-profit organizations; Stephan, 1996). In industry the paramount objective is the generation of products and services that users can apply in real life (Agarwal & Ohyama, 2013). As put by Perkmann and Walsh (2009, p. 1046), “appliedness” is mostly conceptualized as the “proximity to market” and in that sense as the occupational goal of generating “market-ready products or services”. Accordingly, applied research commonly relates to market-oriented product development and technological innovation (Bentley et al., 2015; Lee, 1996). That is why we conceptualize AO as a vocational orientation consisting of preferring tasks that are most commonly found in industrial settings that represent a typical employment environment for STEM graduates.

But, of course, applied research does not only take place in disciplines which commonly engage with the generation of tangible products or technologies that are relevant to users (e.g., engineering). It is also found in, for instance, the social sciences (e.g., improving educational programs; Bentley et al., 2015). Further, graduates of very different disciplinary fields (e.g., social sciences, medicine, law etc.) who leave academia to become practitioners (e.g., working as a counselor, physician or lawyer) often do so to work towards the goal of direct and concrete practical utility. An analysis on employment sectors of doctorate recipients, however, indicated major differences between disciplines: whereas industry was the largest employment sector for graduates from the STEM disciplines, the majority of graduates from the humanities, arts, education, psychology, and social sciences were employed in academia (National Center for Science & Engineering Statistics, 2021). Therefore, we define AO as a vocational orientation that is most relevant to professionals from the STEM disciplines.

Study 1: Initial Explorative Content Analysis on Applied Work

In addition to defining the focal construct in reference to the relevant literature, MacKenzie et al. (2011) advise consulting practitioners and/or subject matter experts, thereby identifying “key aspects (or attributes) of the construct’s domain” (p. 298). Hence, we conducted a qualitative content analysis with practitioners from applied work fields. We inductively explored the aspects that practitioners working in applied fields reported as representing their applied work. The qualitative analysis served the conceptualization of the construct.

Materials and Methods

We focused on professionals in industry who met the formal requirement to enter different career options (e.g., academia, industry). Therefore, the sample consisted of N = 102 practitioners (4.9% doctoral degree holders) working in private industry (30.4% female, Mage = 37.5, SD = 9.8) and having an academic degree (96.1% in STEM fields). Invitation emails were sent directly to potential study participants, and invitations were posted on (work-related) social media platforms. We asked participants to state comprehensively what made their work an applied one (open-format questionnaire). We evaluated their statements based on a content analysis approach involving two steps (Prior, 2014). First, three raters with extensive knowledge in I/O psychology discussed approximately half of the statements. Based on converging content, the statements were structured into facets of applied work. This structuring process was continued until no further facets were identified. Facet definitions were derived based on shared content of the statements. Next, the raters individually elaborated on the remaining statements. In the process, no additional facets were derived; thus, all remaining statements could be assigned to the existing facets. Because we believe that more pertinent facets would be mentioned more often, we calculated frequencies indicating the number of statements placed in each category.

Results

As expected, the qualitative content analysis revealed applied work in industrial settings to be characterized by a heterogeneous set of facets. In sum, 16 facets of applied work were derived from the practitioners’ statements (see Table 1). The four most frequently mentioned facets were business organization, customer focus, product focus, and quality control, with each of the four facets accounting for more than 10% of all statements.

Table 1 Facets, Frequencies, and Definitions of Applied Work Facets (Study 1)
Full size table

Discussion

While some of the facets of applied work that we found in the content analysis clearly are central to industrial settings, such as a focus on economic issues and profitability (van der Sijde et al., 2014), others seem less specific (e.g., interdisciplinary teamwork). Since broad occupational fields (e.g., academia, industry) consist of various (heterogeneous) domains, it is necessary to focus on the most central ones. Therefore, for further analyses, we focused on the four AO facets that were mentioned most frequently in the qualitative content analysis (i.e., business organization, customer focus, product focus, quality control), in addition to the facet economic focus. Although economic focus was not mentioned as frequently as the other four facets, it can clearly be considered as particularly central to applied work (e.g., van der Sijde et al., 2014). Business logics is primarily concerned with financial profit, and scientists are employed in industry to transform scientific knowledge into economic gain (Agarwal & Ohyama, 2013). The need to work towards economic profitability might have been self-evident to many of our participants working in industry and therefore not worth mentioning. Accordingly, due to its pivotal role in industry, we included this facet for further analyses. Of the total practitioner statements from the content analysis, nearly 70% were attributed to one of these five facets. In the following section, we discuss to what extent these AO facets resemble other motivational variables. We link them to theoretical frameworks and empirical findings, thus underlining their relevance for applied fields of work.

A focus on the various processes that are relevant to businesses is captured by the facet business organization. The applied work facet quality control can be viewed as a component in the business organization strategy, focusing on evaluation and controlling of processes. A total of 36.0% of participant statements were categorized as either business organization or quality control. The two facets can be subsumed under the term of process orientation, which comprises an operational management perspective (e.g., planning, implementing, controlling; see Chen et al., 2009). Process orientation is crucial in industry, given that work units (e.g., R&D, manufacturing) consist of different personnel (e.g., researchers, manufacturers) and that places of work are spatially divided (e.g., laboratory, factory). To work efficiently, businesses must be systematically managed and coordinated. A systematic process orientation is less relevant in academia, where teaching and research are typically not as divided into different work units.

Next, 15.4% of participant statements were categorized as customer focus. A focus on customers’ wishes and needs manifests itself in an emphasis on interpersonal relationships and social interactions, putting effort into conveying information comprehensibly and a commitment to customer satisfaction. Customer orientation is a centerpiece of market-oriented businesses (Blocker et al., 2011). In contrast, even though in higher education students are sometimes compared to customers, customer orientation is not seen as a defining feature of academic work settings in higher education.

The third most commonly mentioned domain of applied work (14.1%) was product focus (aspiration to generate products serving a concrete and practical utility). This facet essentially characterizes market-oriented businesses (Brettel et al., 2011). The desire to produce something tangible is part of the Realistic dimension of the RIASEC model (Holland, 1997), which was found to be associated with occupations relevant for industry (e.g., manufacturing engineering; Prediger & Vansickle, 1992). Opposite to the goal of producing something with practical utility, outputs of academic work often are “measured in great part through prestige, legitimacy, and status and to a lesser degree through more tangible economic metrics” (Mendoza et al., 2018, p. 2).

In addition to the four facets of applied work that were mentioned most frequently, we suggest the pursuit of cost effectiveness and commercial success — economic focus — to be of crucial relevance to applied work fields. Per definition, industry’s chief aim is the generation of financial profits. In contrast, the primary goal of other fields of work (e.g., academia, government) is not necessarily related to commercial success. As expectations about work attributes shape career preferences (Roach & Sauermann, 2010) and as industrial work settings are prototypically associated with economic reasoning (van der Sijde et al., 2014), we included this facet in our analyses.

In sum, the qualitative content analysis revealed applied work to consist of different facets. When investigating broad occupational fields (here, private industry), it is necessary to narrow down to the principal construct dimensions (Roach & Sauermann, 2010). Thus, we zeroed in on a few domains of applied work for our analyses. Based on the initial findings from the construct conceptualization (Study 1), we next focused on the most central AO facets and investigated these facets with three validity studies, including convergent and discriminant validity, known-groups validity, and predictive validity.

Measures of Application Orientation

Our next goal was the development of self-report measures. Item generation was conducted as follows (see MacKenzie et al., 2011): (1) newly developed items were generated by experts in the field of I/O psychology with extensive knowledge of the literature; (2) we formulated items that represented the different facet definitions, which we derived from the empirical content analysis; and (3) when available, we drew upon and were inspired by related measures. Items for process orientation were developed partly building on the Process Orientation scale by Chen et al. (2009). For customer focus, we were inspired by the Market Orientation scales (Deshpandé & Farley, 1998) and the Sociability scale of the Business-Focused Inventory of Personality (Hossiep & Paschen, 2003). The item generation of product focus was inspired by the Creativity Styles Questionnaire (Kumar et al., 1997). Lastly, for the items of the facet economic focus, we drew upon the Cost Orientation scale (Chen et al., 2009) and the Preference Inventory (Amabile et al., 1994). The items represented each facet’s conceptual domain, which was outlined in the content analysis (see Table 1). Although the above scales bear some resemblance to AO, none of them fully captures the different aspects of a vocational orientation towards applied work fields. Hence, our measures exhibit incremental validity over the existing scales, given that they represent the conceptual breadth of the new construct. Since we conducted different validity studies with different samples, items had slightly distinct content, and precise item wording partly varied, even though the central content of the facets was consistently assessed.

Study 2: Convergent and Discriminant Validity in Relation to RIASEC

We investigated AO in relation to other constructs of vocational orientations, and we expected our AO facets to be related to but still be distinguishable from other relevant constructs. Here, we examined convergent and discriminant validity of specific AO facets in relation to the RIASEC dimensions (Holland, 1997). Each RIASEC dimension is characterized by specific personality-related attributes (e.g., interests, values, beliefs) as well as environmental demands and opportunities (e.g., incentives, working conditions).

First, we propose that the Enterprising (E) dimension resembles AO in several ways. E-type individuals are motivated by economic achievements and value materialistic outcomes, which corresponds to economic focus with its emphasis on cost-effectiveness and financial success. Industrially working professionals care particularly much for their level of income in comparison to academics (Erez & Shneorson, 1980). E-type individuals are also suggested to often work in sales, where they can be expected to commonly work with tangible and practically useful products that they sell to others. An aspiration to sell practical objects resembles the AO facet product focus. Moreover, E-type individuals commonly possess persuasive competencies that are beneficial whenever others need to be motivated to purchase products. This links the E-dimension to the AO facet customer focus.

Hypothesis

1a: The Enterprising dimension is positively associated with the AO facets economic focus, product focus, and customer focus.

Second, Realistic (R) type individuals typically value concrete things and practical work outcomes, which closely resembles the AO facet product focus. In addition, in an R-environment, people are rewarded for the display of “traditional values”, such as a concern for money (Holland, 1997, p. 43). They are expected to strive for material rewards and to focus on the economic outcomes of their work. Similarly, economic focus emphasizes the striving for profitability.

Hypothesis

1b: The Realistic dimension is positively associated with the AO facets product focus and economic focus.

Third, Conventional (C) environments are suggested to reward people for the display of “conventional ... values”, such as a focus on monetary issues (Holland, 1997, p. 47). C-type individuals are expected to frequently work as accountants or bank employees, that is, in jobs related to finance. They are assumed to care about materialistic outcomes and are described as efficient, linking the C-type to the AO facet economic focus. Furthermore, they prefer systematic work flows and aim to control work activities. This resembles the AO facet process orientation and its focus on systematically organized work processes.

Hypothesis

1c: The Conventional dimension is positively associated with the AO facets economic focus and process orientation.

Fourth, Social (S) individuals are inclined to work with people, for instance, in jobs where they can inform and advise or take the perspective of others. They are described as cooperative, persuasive, and helpful. Professionals in industry have been found to exhibit a higher preference for persuading others in comparison to non-industrially working professionals (Erez & Shneorson, 1980). As the S-dimension consists of heterogeneous aspects relating to various interpersonal interactions, it entails more than a focus on customer interaction. However, regarding the persuasion and advisory components of the S-dimension, it can be suggested to converge with customer focus.

Hypothesis

1d: The Social dimension is positively associated with the AO facet customer focus.

In comparison to academicians, professionals working in industry have been shown to exhibit lower artistic preferences (Erez & Shneorson, 1980), and artistic individuals are described as impractical and as lacking business competencies (Holland, 1997). This differs from the AO facets.

Hypothesis

1e: The Artistic dimension is not or negatively associated with the AO facets.

Investigative (I) persons are expected to seek intellectual instead of practical work and are described as being frustrated when having to persuade others (Holland, 1997). Moreover, academicians care for autonomy of work processes and exhibit comparably low interest in income (Burk & Wiese, 2018). This indicates AO to be different from the I-type.

Hypothesis

1f: The Investigative dimension is not or negatively associated with the AO facets.

In addition, we investigated the relation between the AO facets as well as the RIASEC dimensions and the preference for applied work settings.

Hypothesis 2

More variance of the preference for applied work settings is explained by the AO facets than by the RIASEC dimensions.

To examine discriminant validity, we calculated each facet’s average variance extracted (AVE). We then compared AVE coefficients with the variance explained by the RIASEC factors (squared factor correlations). If the variance explained by the respective factor is higher than the one explained by any other factor, this would indicate discriminant validity of the latent construct in relation to the other constructs (Fornell & Larcker, 1981).

Materials and Methods

The sample of this study (online questionnaire) consisted of N = 200 individuals (67.0% female, Mage = 27.8, SD = 9.9). Given that in this study we investigated general personality-related tendencies and occupational preferences, we did not solely focus on professionals. The sample consisted of undergraduate students (65.5%) and professionals with an academic degree (34.5%). Participants had backgrounds in different disciplines (46.5% law, economics, social sciences; 31.0% STEM; 15.5% humanities; 20.0% other; participants could indicate more than one discipline). Professionals worked in different fields, with private industry being the main occupational field. Participants were recruited via email and announcements on (work-related) social media platforms.

Based on structural equations modeling, we conducted a latent correlation analysis and a latent regression analysis. Considering that each latent variable consisted of only three items, and as lower numbers of indicators go along with lower internal consistency, reliability coefficients of the latent variables can be interpreted as adequate (see Cortina, 1993; Table 2). The items for process orientation were as follows: Responsibilities of work processes should be clearly assigned; The target compliance of work processes should be monitored; Work processes should be precisely defined for employees to have a clear understanding thereof. The items for customer focus were as follows: I enjoy advising others; I do not really like to explain things to others, to advise them, or to make recommendations (inverted); I am not very excited about ‘selling’ my ideas to others (inverted). The items for product focus were as follows: My work should have tangible utility and should not solely help find the truth; In my view, my work should always have a direct practical utility instead of solely serving fundamental research; I am particularly motivated if I develop a concrete product. The items for economic focus were as follows: I care about the financial profit my work contributes to the organization I am working for; It is important to me that my work translates into economic success; Employees should have an overview of what their work contributes financially to their organization. Responses had to be given on a six-point scale from 1 = I do not agree at all to 6 = I totally agree. For the RIASEC items, we used a six-point scale (1 = Does not interest me at all, 6 = Interests me very much) of an established German version of a RIASEC inventory (Bergmann & Eder, 1992), with three indicators for each dimension.

Prior to hypothesis testing, we investigated the factor structure of our empirical data. We examined whether a one-factor model fitted the data better than a primary-factor model, testing the suggested multidimensional nature of AO. Comparing fit indices of a one-factor CFA (χ2 = 288.40, df = 54, RMSEA = 0.15, CFI = 0.46, SRMR = 0.11) to a primary-factor CFA (factors were allowed to correlate) consisting of the four AO facets (χ2 = 81.83, df = 48, RMSEA = 0.06, CFI = 0.92, SRMR = 0.06), we found the primary-factor model to exhibit a much better fit. This supported our expectations about the multidimensional nature of AO. Accordingly, for our analyses, we deployed the four AO facets separately.

Results

For Hypotheses 1a–1f, correlational analyses largely supported our assumptions (see Table 2). Of the eight associations between the AO facets and the RIASEC dimensions that we expected to be significant and positive, seven supported our hypotheses. Only the association between product focus and the R-dimension was not significant. Moreover, as expected, the A- and I-dimensions were unrelated or negatively related to the AO facets. Concerning Hypothesis 2, we conducted a latent regression analysis predicting the preference for applied work settings. Of the AO facets and the RIASEC dimensions, only product focus significantly predicted the preference for applied work settings (βstrd = 0.60, p < .01).

Evaluating discriminant validity, in nearly all cases the variance of the AO indicators explained by their respective factor (AVE) was higher than the variance explained by the other factors. Only for customer focus, more variance was accounted for by the Social (62.4%) and the Enterprising dimensions (46.2%) than by the latent construct customer focus itself (40.6%).

Table 2 Correlations between Factors of AO Facets and RIASEC Dimensions (Study 2)
Full size table

Discussion

We found the results to largely support our hypotheses. Corroborating convergent validity, nearly all AO facets correlated with the RIASEC dimensions in the expected manner. Only the hypothesized association between product focus and the R-dimension was not significant. This result was unexpected, as R-type individuals are assumed to strive towards the production of tangible products. However, the R-type is also defined by preferences for technical and mechanical activities. This component is not necessarily related to the desire to produce something tangible and might explain why product focus did not significantly correlate with the R-dimension. In general, however, the study supports our multifaceted conceptualization of AO, with each facet representing a distinct aspect. The multifaceted approach captures AO more accurately than a typological approach and allows revealing facet-specific variance that would not have been possible if we had relied on the RIASEC types (Armstrong et al., 2008). This study contributes to the literature by heeding recommendations to investigate specific domains of vocational orientations (Wille et al., 2015), which are functional for predicting specific work-related criteria.

The latent regression analysis revealed product focus to be the factor leading to the highest prediction of the preference for applied work settings. Thus, the interest to produce something tangible with practical utility that can be sold is indicated to be of particular relevance for applied work interests. This result is supported by empirical findings, suggesting a focus on sellable products with concrete customer value as one of the defining characteristics of market-oriented (i.e., industrial) organizations (e.g., Brettel et al., 2011). In addition, product focus might have been the only significant predictor of the preference for applied work settings, because the emphasis on product manufacturing links product focus to work environments where tangible goods are generated (i.e., industry). The other three AO facets, in contrast, are not as exclusive to private industry. Moreover, the broad applicability of the RIASEC dimensions, which are not tailored for one specific work field, might explain why none of these dimensions predicted the preference for applied work.

Supporting discriminant validity assumptions, the variance explained by the respective latent constructs was higher than the variance explained by any other latent construct. Only for customer focus, more variance was explained by the S- and E-dimensions. This reveals the AO facet to possess a substantial amount of variance that is not unique to the latent construct but to other constructs. Although in the initial content analysis we found a focus on interacting with customers to be relevant to industrially working individuals, the focus on customers might be less distinct from social interactions taking place in other professional settings than we had expected.

Study 3: Known-Groups Validity in an Applied vs. Academic Sample

If our conceptualization of AO represents applied fields of work, scores of AO should differ between university graduates working in an industrial setting compared to those not working in an applied setting (i.e., academia). This form of validity is termed ‘known-groups validation’ (e.g., Cronbach & Meehl, 1955).

Hypothesis 3

Scores on AO facets are higher in applied working professionals in comparison to non-applied working professionals.

Materials and Methods

The sample of the study (online questionnaire) consisted of N = 166 professionals with an academic degree (30.1% female, Mage = 37.8, SD = 12.0). Participants had backgrounds in different STEM fields: 59.6% engineering, 25.9% sciences, 7.8% computer science, 4.2% mathematics, and 6.0% other STEM field (participants could indicate more than one discipline). Participants were recruited via email or responded to announcements posted on (work-related) social media platforms. While N = 108 participants were employed in an applied field, N = 40 were employed in academia (the remaining sample reported other fields). We used a short AO questionnaire with three items per facet. Process orientation: Work processes should be precisely defined for employees to have a clear understanding thereof; The target compliance of work processes should be monitored; Occupational success should be measured based on specific sub-goals for work processes. Customer focus: It is important to closely follow customer’s needs; I focus on regularly exchanging with customers; If the customer is satisfied with my work, I am satisfied, too. Product focus: My work should have tangible utility and should not solely help find the truth; In my view, my work should always have a direct practical utility instead of solely serving fundamental research; I am particularly motivated to work towards a sellable product. Economic focus: It is important to me that my work translates into economic success; When making career decisions, I am keenly aware of the income goals I have for myself; I need to feel that what I do is financially worthwhile. Responses had to be given on a six-point scale from 1 = I do not agree at all to 6 = I totally agree.

For hypothesis testing, we split the total sample into two subsamples (industry employment N = 108, academia employment N = 40). This led to relatively small sizes of subsamples (especially regarding the academia subsample). Therefore, latent analyses could not be executed. Hence, we conducted one-way analyses of variance testing group differences between industrially vs. academically employed individuals with manifest means of the four AO facets as dependent variables, controlling for age and gender. Internal consistencies (Omega) of the AO facets were satisfactory (process orientation = 0.75, customer focus = 0.68, product focus = 0.74, economic focus = 0.77).

Results

As expected, we found scores on the AO facets to differ between industrially vs. academically employed individuals. On all four AO facets, scores were higher in the industry compared to the academia subsample (see Table 3). The effect size (Cohen’s d) for customer focus can be interpreted as small, for process orientation as medium, and for product focus and economic focus as high.

Table 3 Analyses of Variance comparing Mean Scores of AO Facets in Professionals working in Industry vs. Academia (Study 3)
Full size table

Discussion

Deploying a known-groups validation approach, we were able to confirm that scores on AO are higher in individuals working in applied settings compared with individuals working in academia. Concerns may arise regarding the origin of these group differences. Theories on career choice suggest individuals to enter specific fields based on vocational orientations (e.g., Holland, 1997), but one might also argue that characteristics of the work environment and work-related personality reinforce each other. In other words, both (self-)selection and socialization processes take place. To test whether AO scores are influenced by field-specific work experiences, we conducted an additional correlation analysis, which only consisted of participants consistently working in industry (n = 70). If vocational orientations are the result of organizational socialization and preferences for one’s current job, professionals with more work experiences (as measured here with the proxy variable of age) should exhibit higher scores in AO compared to less experienced professionals. The empirical findings, however, revealed that field-specific work experience was not associated with scores on AO in professionals consistently working in industry. This suggests that even though work-related personality dimensions may become more firmly established over time, vocational orientations are not merely the result of organizational socialization processes but start forming earlier and get more stable gradually (see Low et al., 2005).

A limitation of this study concerns the relatively small sample size, particularly regarding academically employed individuals. Nonetheless, we found significant grouping effects (industry vs. academia employment) on all four AO facets, with particularly large differences in product focus and economic focus. Thus, the results clearly underline the validity of the AO facets.

Study 4: Predictive Validity for Actual Behavior

As vocational orientations should influence real-life outcomes, we tested the criterion-related validity of one particularly relevant AO facet by investigating its predictive power for actual performance in an applied work task. As the AO facet product focus was by far the strongest predictor of the preference of applied work settings in the previously conducted latent regression analysis (Study 2), due to parsimony and practical reasons, we only measured product focus as a predictor in Study 4. We assessed this AO facet by self-report at the first measurement point and conducted an on-site assessment center at the second measurement point. In the assessment center, we collected behavioral data by actively confronting participants with specific environmental cues as part of a behavioral simulation task that they were required to perform. It is widely acknowledged that for the assessment of specific observable phenomena, behavioral measures as implemented in an assessment center are particularly suitable.

Hypothesis 4

The AO facet product focus predicts individuals’ behavior in applied work interactions.

Materials and Methods

The sample for this study was drawn from a larger longitudinal survey project with more than 3,500 early career scientists from the STEM fields (for further details on the project, see Alisic & Wiese, 2020; Burk & Wiese, 2018; Claus et al., 2020; Lerche et al., 2022; Noppeney et al., 2021). Participants of the larger project were invited to take part in an on-site assessment center. A total of N = 128 individuals participated (77.6% doctoral students, 22.4% doctorate holders, 49.2% female, Mage = 30.6, SD = 4.4; STEM fields: 40.6% sciences, 39.8% engineering, 7.0% computer science, 3.9% mathematics, 8.7% other). The assessment center served the purpose of helping young scientists learn about personal strengths and potential for improvement as well as vocational preferences. As an incentive, participants received individual performance feedback.

Data for the analysis of predictive validity were collected at two measurement points, with an average interval of 12.8 months in between (SD = 2.4): (T1) three-item self-report measure on product focus, and (T2) behavioral simulation at an assessment center. Due to space restrictions in the T1 survey, we focused on only one AO facet. The three-item measure for product focus revealed satisfactory internal consistency (ω = 0.71). The items for product focus were as follows: My work should have tangible utility and should not solely help find the truth; My work makes particular sense to me if I can produce something that can be sold or is valued amongst individuals outside my subject area; In my view, my work should always have a direct practical utility instead of solely serving fundamental research. Responses had to be given on a six-point scale from 1 = I do not agree at all to 6 = I totally agree.

The behavioral simulation consisted of a face-to-face interaction in which we assessed two dimensions indicative of applied work: (1) conveying and communicating an understanding of industry-related activities, and (2) enthusiasm for business collaboration. The first indicator focuses on a cognitive component, that is, participants’ understanding of and insight into industrial working conditions as well as goals and principles as to be observed in an interaction with a client. The second indicator puts emphasis on enthusiasm for business collaboration, thus on a dimension with a distinct motivational and behavioral focus. To be rated as enthusiastic for business collaboration, participants needed to clearly demonstrate that they actively promote and are even keen on gaining practical industry experience and collaborating with industry partners.

Based on their personal interests, participants could choose one out of two scenarios (i.e., civil engineering scenario or electrical/mechanical engineering scenario). In the role play, participants had to act like a young engineer working at an academic institute. They had to represent the institute in a negotiation with a female business representative. Her enterprise wanted to launch a newly developed product. The enterprise approached the institute to gain positive marketing effects out of an independent scientific product evaluation. The interaction partner was a professional actor (four different actors in total), who was trained to consistently set certain behavioral prompts. They played a profit-oriented, competitive, self-confident, and provocative business representative with good negotiation skills, aiming for quick yet satisfying results produced at minimal cost. Prior to the actual role play, participants received a message from the head of the institute, outlining the importance of high standards of scientific work (e.g., need for basic research, methodological soundness). Contextualization of this message (i.e., embeddedness in an authentic framework) was achieved by a richness of detail and a realistic setting.

The assessment center tasks were videotaped and rated by three psychologists and students majoring in psychology. Assessors were provided with a behaviorally anchored written coding system and received coding training. Codings had to be made on six-point scales, indicating the extent of participants’ applied behavior. Mean scores were computed based on individual assessor ratings. Inter-rater reliability was estimated using a one-way random, average-measures intra-class correlation, indicating acceptable reliability (> 0.40; Cicchetti, 1994).

To predict actual applied work behavior, we conducted a latent regression analysis in a structural equations framework. Prior to hypothesis testing, we examined several control variables that might affect our central outcome variable. First, we controlled for career level, expecting PhD students to exhibit more interest in applied work in contrast to PhD holders who already decided to continue working in academia after graduating. In addition, we controlled for STEM field, as we suggested engineering and computer scientists to show a stronger preference for applied work in comparison to other STEM scientists (e.g., mathematicians). Furthermore, regarding the behavioral simulation, we controlled for the performance of the different actors, participants’ experience in negotiation situations, and observers’ overall positive impression of participants (with generally more skilled participants potentially receiving more favorable ratings).

Results

Because two of the control variables significantly correlated with the outcome (i.e., observers’ overall positive impression of participants, r = .70, p < .01; participants’ experience in negation situations, r = .22, p < .05), they were included as additional predictors in the regression analysis (χ2 = 18.12, df = 12, RMSEA = 0.06, CFI = 0.97, SRMR = 0.05). Over and above the control variables, product focus, which we assessed approximately one year before conducting the assessment center, significantly predicted participants’ applied work behavior (β = 0.25, p = .01).

Discussion

In the regression analysis, we clearly demonstrated the predictive validity of the short measure of product focus. It predicted how people behaved in an applied work role play regarding an affinity for and understanding of industry-related tasks and their enthusiasm for business collaboration. Past research has shown that associations of the same variable assessed with different measurement formats (e.g., self-report questionnaire vs. behavioral simulation) are comparably low (Bowler & Woehr, 2006). The more impressive is the size of the time-lagged association between the short scale of product focus and the behavioral criterion.

General discussion

The primary aim of this study was to conceptualize and validate a new concept of AO. Using a multi-study design consisting of an open-format questionnaire, standardized questionnaires, and an assessment center study with behavioral data, our validation approach led to convincing results. We investigated various validity aspects of the new construct (i.e., factorial validity, convergent and discriminant validity, known-groups validity, and predictive validity) with different samples, thereby strengthening the validation approach (MacKenzie et al., 2011). Together, this study corroborated specific validity assumptions of the construct. Therefore, we consider AO to be a valid contribution to the literature on specific vocational orientations of early career academics.

In the content analysis (Study 1), professionals working in industry indicated applied work to be multifaceted. Based on theoretical frameworks and the empirical results of the content analysis, we found four facets of applied work to be particularly relevant: process orientation, customer focus, product focus, and economic focus. The multifaceted nature was supported by a CFA, revealing a model with separate factors of the AO facets to fit the empirical data better than a one-factor model (Study 2). A latent regression analysis (Study 2) revealed the preference for applied work settings to be most strongly predicted by product focus. Investigating correlations between the AO facets and the RIASEC dimensions (Study 2), nearly all postulated correlations were significant. We further found support for discriminant validity assumptions, as in most cases the variances of the indicators of the AO facets were explained more strongly by their respective factor than by the other factors. In a known-groups validity approach (Study 3), AO scores were higher in applied working individuals than in individuals working in academia. Lastly, we found a self-report measure of product focus to predict actual behavioral performance in an applied work task one year later (Study 4).

Strengths, Limitations, and Future Research Prospects

The conceptualization and validation approach for our newly developed construct had several strengths. Clearly, the complex multi-study mixed methods design is the greatest strength. Concerning measurement, we collected not just written-format questionnaire data (self-report) but also behavioral data in a role play (external rater judgments), thereby strengthening validity assumptions and avoiding common method bias (Podsakoff et al., 2003). Relating to sampling, we used different samples for the various validation studies, thereby further strengthening validity evidence (see DeVellis, 1991). Moreover, the assessment center study was a longitudinal one. Another strength is the primary focus on highly educated professionals, investigating vocational orientations in a group that is pivotal to both academia and industry (National Academies of Sciences, Engineering, & Medicine, 2018).

As in any research, there were also limitations to our study. First, we did not have data on actual career transitions or career success. Second, we conducted a behavioral simulation that depicted only one specific work-related situation. Third, for the different validity studies, which were partly conducted simultaneously, we used different samples and somewhat varying AO items. We focused primarily on the initial conceptualization and validation of the construct of AO, not on a rigorous scale development. The latter would entail many different steps (e.g., item analysis, cross-validation, development of norms). But as pointed out by Nunnally and Bernstein (1994), though in the process of a new construct development various steps need to be considered, often not all steps can be realized at once. Since our primary goal was an initial conceptualization and validation of the construct of AO, we needed to prioritize which steps were most relevant, economic, and practical for our principal research aim (MacKenzie et al., 2011). We consider our study to be an important starting point, which next should foster a comprehensive psychometric scale development. For an AO scale to exhibit practical utility in higher education counseling, some of the items need to be tailored more specifically toward the circumstances and conditions that students and graduates face. Customer focus, for instance, should pertain not only to customers in sales but also to funding agencies and third-party sponsors. However, although the AO items were not developed exclusively for PhD students and holders, we propose that even in their current form they are suitable to be used in a higher education counseling context and can contribute to better informed career decision-making of early career scholars.

Clearly, applied work is multifaceted. In our analyses, we focused on four AO aspects. Further studies should investigate a fuller range of AO facets to draw a more complete picture of applied work. With respect to sampling, we mainly focused on early career academics from the STEM fields. Future research must show whether AO plays a comparable role in non-STEM disciplines (e.g., social or economic sciences) and whether the concept of AO must be modified to be more applicable for non-STEM disciplines. Finally, the interest in applied work should be more explicitly contrasted with the interest in academic work (see Roach & Sauermann, 2010) to ultimately come up with a full-range vocational orientation model in higher education.