1 Introduction

Career guidance benefit from the advancements in emerging technologies. The widespread adoption of information technology and communication systems has dramatically facilitated providing student career guidance services (Gati & Asulin-Peretz, 2011; Pordelan & Hosseinian, 2022; Supriyanto et al., 2019). Online career services and career information systems are being developed to support students in the career decision process. These technological tools serve multiple purposes, including career decision-making, intervention, and planning (Sampson & Osborn, 2015; Zainudin et al., 2020). Career information systems becomes particularly crucial in countries with a significant young population, such as Türkiye. In Türkiye, the higher education entrance central system examination is administered by the Evaluation, Selection and Placement Centre (Ölçme, Seçme ve Yerleştirme Merkezi [ÖSYM]). Based on the examination results, students must make critical decisions about their future careers, including selecting a university and department. According to current statistics, 3 million 527 thousand candidates applied to the 2023 examination (ÖSYM, 2023). After the exams, face-to-face university preference counselling services are provided by career counsellors to support students in their transition to higher education institutions (Republic of Türkiye Ministry of National Education, 2023). The most critical mission expected from career counsellors is to carry out guidance activities for students' choices, decisions, and preparations for their future occupations and careers (Yeşilyaprak, 2019). The general purpose of this service is to guide students in transitioning to a higher education institution according to their abilities, interests, needs, personality traits, and living conditions. The services provided in this context are processed and recorded in the e-Guidance System. Consequently, students should receive career guidance services to select a department and university, a meaningful career step that aligns with their preferences and competencies. With the diversification of career choices, it has become more complex for each individual to determine their own path. Moreover, opting for a suitable university and department significantly influences an individual's career and job satisfaction, which, in turn, impacts the overall productivity level of society (Turan & Kayıkçı, 2019). This situation highlights the need for tailored and effective guidance services that enable students to make informed careers decisions and help them assess their career choices and abilities.

Moreover, emerging guidance practices yield significant benefits for individuals and public welfare (Barnes et al., 2020). As countries strive to promote evidence-based career guidance, the expanding use of data analytics and digital technologies for monitoring and evaluating the effectiveness of career guidance services becomes increasingly prevalent (Harrison et al., 2022). Career information systems are expected to enable evidence-based effective career development interventions. However, current systems prioritize accessibility for everyone rather than providing tailored applications for individualized career guidance, termed career guidance 1.0. Advancing from these resource-centric systems to personalized, counselor-involved designs is critical.

Career information systems are particularly important for bringing labour market information into career guidance processes (McKinlay, 1975). The labour market is an important factor that shapes many career theories and influences an individual's career decision (Baruch & Sullivan, 2022; McKinlay, 1975). Students' earning expectations also play a crucial role in their preference for a university and department (Jiménez & Salas-Velasco, 2000). Soutar and Turner (2002) conducted research with 259 high school seniors, where students expressed a preference for good job opportunities over average ones, indicating that their satisfaction with university education depended on various factors such as career preparation, job opportunities, and employer interest (Alves & Raposo, 2007; Webb, 1993). These indicators are critical for making informed career decisions considering labour market performance (Baruch, 2015). Employment rates and job opportunities are crucial for students and parents to make conscious career choices (Tong, 2019). The use of labour market information as a basic resource for an effective career guidance process occurs through career information systems (Patton & McMahon, 2014; Pordelan & Hosseinian, 2022). Existing career information systems need proper alignment between labour market information and specific professions or higher education programs. It is essential to note that higher education plays a significant role as the first step toward a professional career. When individuals choose a university and department, they select a profession. For all countries, higher education institutions have a crucial impact on shaping labour market performance. Therefore, uncovering universities' labour market performance profiles and incorporating this data into career counselling processes to assess their effectiveness becomes essential. In line with, when current systems are examined in more depth, it is seen that they have a common design architecture. Navigation within the systems is primarily left to individuals in their career exploration. Understanding labour market information requires strong navigation skills, encompassing data on current labour market conditions, projections, job search duration, wages, occupational distribution, and sector knowledge. Career counsellors bridge labour market information and students, interpreting and using it to inform, advise, and guide learners. However, Individuals are expected to independently understand, combine, and make career decisions based on labour market information derived from indicator panels, demanding high data literacy and highlighting weak system designs.

Another noteworthy aspect is that the development of career information systems is typically supported by governments, indicating their policy-level consideration. Discrepancies in the organization, coordination, funding, delivery, and structure of career guidance have also been reported (Harrison et al., 2022). This indicates the need for a more consistent, effective, and comprehensive approach to career guidance. Additionally, having reliable evidence based on the scope and effectiveness of career guidance interventions is a prerequisite for justifying public investments in career guidance services (Harrison et al., 2022). Consequently, while there is a need for a robust experimental evidence base in career guidance, many countries require improvements in monitoring input, outcomes, and impact of career guidance processes (Barnes et al., 2020).

This research aims to design a web-based career information system integrated with the labour market performance of universities in the Turkish context. Digitizing universities' labour market profiles and utilizing them in career guidance processes is an innovative way to effectively place talent on a national scale. This research presents the results of the first cycle of design-based research (DBR).

1.1 Background and related works

A career is a life-long dynamic process that reflects progress along a career path (Baruch, 2015;

Esbroeck et al., 2005). A career decision is an important step on the career-planning path, which should be suitable for individuals (Bimrose & Mulvey, 2015; Lent & Brown, 2020). Making an appropriate career decision impacts an individual's life (Haneef et al., 2020). Individual's lifestyle, economic and social status, sense of productivity (Gati & Tal, 2008), career satisfaction (Gati et al., 2006), well-being (Creed et al., 2005), and living standards (Sabates et al., 2017) are influenced by their career choices. This progress encompasses several multidimensional concepts, including an individual's interests, attitudes, values, and professional decisions (Niles & Harris-Bowlsbey, 2013; Holt, 1989), all critical elements in the career development process. In addition, career development comprises a comprehensive range of psychological, educational, economic, and sociological factors (Herr, 2001; Song & Park, 2005) and includes interventions and practices to promote more effective career decision-making (Spokane, 1991).

Individuals require necessary guidance throughout their career journeys (Super, 1980). Career guidance is crucial in guiding individuals toward effective career planning and increasing awareness about their academic and professional careers (Eun et al., 2013; Gysbers, 2001), especially for students who need guidance in exploring job fields and opportunities (Haneef et al., 2020). Career guidance encompasses a comprehensive array of services designed to effectively bridge the divide between an individual's expectations, interests, qualifications, and skills and the broader societal or global expectations (European Center for the Development of Vocational Training (CEDEFOP) et al., 2019). In addition, previous research highlights the importance of career counselling in helping individuals access information about existing occupations and opportunities (Eun et al., 2013; Gati & Levin, 2015), especially for students who may face difficulties in making career decisions without sufficient knowledge (Ashari et al., 2019; Haneef et al., 2020). This process extends throughout a person's lifetime and is punctuated by significant milestones that influence the type of support provided at each stage. For example, career guidance involves.

  • offering valuable assistance to individuals when making pivotal decisions, including educational planning or transitions,

  • mapping out vocational education paths,

  • aligning personal goals with the ever-evolving demands of the labour market.

Effective career guidance is paramount in enabling individuals to achieve their maximum potential, fostering enhanced productivity within national economies, and ensuring efficient management of human resources (CEDEFOP et al., 2019). The key figures responsible for career and academic counselling, such as guidance specialists, career counsellors, and psychological counsellors, play a pivotal role in this process (Zhang et al., 2018). When seeking career guidance and support from counsellors, practitioners need to utilize relevant data and statistical information about university and department options, occupations, and job fields. Such data should be employed judiciously and customized to cater to individual needs. The significance of up-to-date, timely, personalized, and accurate career information and support is crucial due to the majority of individuals facing critical career decisions. Precisely tailored interventions in career guidance have far-reaching effects beyond the individual level; they hold the potential to advance society as a whole by aligning talent with contemporary demands and expectations, effectively matching human competencies with inherent capabilities.

Career information systems are among the most cost-effective approaches to providing current career information and supporting career exploration and planning (Garcia et al., 2021). These systems can potentially reach many participants, which is valuable when a career counselor must serve many students (Garcia et al., 2021). In such cases, addressing students' individual needs regarding career choices can become challenging. Therefore, digital career counselling tools can offer more comprehensive career guidance to individuals. Additionally, digitalization in career guidance can reduce geographical, psychological, physical, and financial barriers to accessing services, thus promoting equal opportunities (Mallen et al., 2005).

The pioneering research on career information systems was conducted by McKinlay (1975), and a career information system was developed to support individuals who were about to make career choices. In this system, occupational descriptions formed the core of the career information system. This initiative echoes continue today; many countries have moved towards digitalization in career guidance. Various career information systems have been developed and used by different countries. The government of Canada created an interactive tool presenting labour market outcomes for college and university graduates, including employment income based on age group, gender, education, geography, and field of study. In the U.S., the College Scorecard by the Department of Education lists universities and departments based on location and graduate earnings, providing information on graduation rates and student quotas. Other examples are Australia-the National Career Information System, Denmark-eGuidance, and Netherland-Kiesmbo. O*NET OnLine is an occupational information platform that provides information on around 1000 occupations. The platform was established under the coordination of the U.S. Department of Labor/Employment and Training Administration. The platform contains information on occupational salaries, future forecasts, job families, and job opportunities. The data is collected on a large population of workers from each occupation and updated frequently. Information is provided on job descriptions, technological skills required, work-related activities, work context (e.g., physical conditions, working hours), areas of interest, personality traits of people in this occupational group, hiring rates and wages, the level of education, experience, and training required to become a member of this occupational group, as well as internship opportunities. The platform's target audience is job seekers, students, and career seekers. Bundesagentur für Arbeit platform provides detailed information and guidance about job and education opportunities and the resources offered by the state for unemployed individuals, people with disabilities, migrants, parents, adolescents, and young adults by the German Federal Employment Agency. Individuals are guided according to the choices they make in the statements, "I know exactly what I want to be.", "I know roughly what I want to be.", "I'm leaning towards education, but I don't know which one." In this direction, information is provided about psychological tests, occupational lists, and types of education so that individuals can discover their interests, strengths, and weaknesses. This information is also supported by videos. Europass was launched in 2018 in line with the European Union's policies to support lifelong learning and improve the quality and productivity of the workforce. It is managed by the European Commission. The old digital platform managed by CEDEFOP has been closed and the new Europass Platform is now available. The platform provides information on European work and learning opportunities, qualifications and qualifications systems, processes and procedures for lifelong learning in European countries, recognition and validation of diplomas and certificates, institutions and organizations providing guidance for international mobility and career management, information on skills needs in occupations and sectors, data, analysis and forecasts, information on living, studying, and finding a job in Europe (Europass, 2024). eVejledning In 2011, the Danish Ministry of Children and Education established "eVejledning", a digital national advice center hosted by the National Agency for ICT and Learning. Based on some feedback, the ministry created this platform with the aim of establishing an integrated tool for all age groups in the country to assess career possibilities. Apart from education and labour market information, the online platform provides individual career advice to individuals through phone calls and e-mails and group career advice through web-based seminars and live chats. The fact that the overwhelming majority of career information systems with active users are designed with a focus on occupations, that universities do not have labour market profiles, that they are open-access systems that only offer resources rather than personalized career guidance, that career decisions are left to the individual and individuals are bombarded with valuable but intensive information, that the systems demand high data literacy, and that there is no common design architecture are seen as aspects open to improvement. In addition, there are no reports on the effectiveness of current information systems. This situation points to the need for systematic design research context of career information systems.

Although studies on online career guidance have been conducted and widely used career information systems have been developed nationally, it is known that there is a need for new research on the effectiveness of online career guidance. Pordelan and Hosseinian (2022) designed a web-based career information system to support university students' entry into the labour market. The research contains valuable information about the positive impact of online career counselling on career guidance processes. In this study, it is stated that it is important for web-based career information systems to ensure data confidentiality for clients, to have digital literacy skills of career counsellors, and to produce outputs based on log data for the counselling process. In addition, useful features of career information systems, such as reaching more students, easy follow-up and ensuring continuity in support, are reported. According to the online career intervention study by Nota et al. (2016) with 200 secondary school students in Italy, the computer-assisted career advice system can effectively prepare for career objectives and enhance job flexibility. Additionally, research has shown that digital expert systems are valuable tools in career guidance and students' academic major selection (Nota et al., 2016). This information strengthens the research in designing more effective career information systems.

1.2 Current research

This research presents the design process of a career information systems, CCIS, designed to evaluate graduates' labour market performance and enable these evaluations to be used in career counselling activities in Türkiye. For this purpose: 1) to improve and expand labour market information in career guidance, 2) to provide career counsellors with a "single entry point" that gathers all labour market information in one centralized place, 3) to add and evaluate the impact of mentor support, which is lacking in existing systems, 4) to match complex labour market information with higher education and create profiles that can be integrated into career guidance services in the simplest form, 5) to develop a tool that makes career counselling processes more personalized.

Overall, this study is original and valuable for the community of human resources professionals, career counsellors, and policy-makers interested in digital technology (and strategy) and labour market information used in the context of career guidance and is motivated by the following question:

  • How should a web-based career information system, integrated with universities' labour market performances, be designed to support career guidance processes?

The main objectives of the research are:

  1. 1

    Exploring the design features of a career information system for career counsellors providing labour market performance information

  2. 2

    Design and develop an information system that incorporates the defined design features

  3. 3

    Implementation to evaluate the use of career information systems in career guidance processes

  4. 4

    Create a framework for the design of a web-based career information system integrated with labour market performance indicators and higher education to support career guidance processes

2 Methods

2.1 Research design

This research is structured as a DBR approach. This approach involves mixed research methods, emphasizes collaboration among researchers, and prioritizes the practical effects of the intervention (Anderson & Shattuck, 2012). DBR is a systematic and flexible methodology that aims to continuously improve real-world settings practices through iterative analysis, design, development, and implementation of educational practices (Wang & Hannafin, 2005). It is a real-world practical issue to explore how universities can innovatively design a career information system integrated with labour market performance indicators to support individuals' career choice processes. DBR guided a methodological framework for identifying design features, development, implementation, and evaluation of the career information system. Due to the flexibility of the DBR, a contextually sensitive methodological framework was adopted to design a career information system. Figure 1 presents the design process and shows each sub-process's action steps and outputs. The output of each action is the input for the following action to optimize the design. Each process includes a unique method. A complex system design and diversity of procedures are other factors that make using DBR necessary. In accordance with the DBR approach, a dynamic and iterative cyclic design process has emerged. The design process is explained in detail in the following sections.

Fig. 1
figure 1

CCIS design process

Full size image

The research aims to reveal a design framework for a web-based career information system integrated with universities' labour market performance indicators to support career guidance processes. This design framework was shaped in an iterative process focused on career counsellors and implemented in the real world with 18,444 career counsellors. The research is not limited to this but also includes the interaction patterns of career counsellors in the system. Due to these patterns, DBR contributes to the advancement of theory in the career guidance processes. DBR is structured around five main steps: Analysis, Design, Development, Implementation, and Evaluation (Richey & Klein, 2005). Detailed information is presented in Table 1.

Table 1 Research Steps, Actions, and Outcomes
Full size table

In the analysis stage, relevant literature, examples, and similar applications were reviewed, and expert opinions were solicited to determine expectations, needs, and indicators. In the design step, we decided and designed the CCIS prototype and its components. Experts stated opinions on these designs, and the system was optimized based on feedback. The development stage followed, leading to the implementation of the system for real users in the subsequent implementation stage. Finally, the evaluation stage was conducted, which involved examining the indicator tendencies and opinions of career counsellors, making further improvements and developments to the system, and optimizing it accordingly.

2.2 Participants

This study involves a collaborative effort with a diverse group of experts to facilitate the development of CCIS. Experts consist of instructional technologies, software development, guidance counselling, career advisement, measurement and assessment, statistics, and human resources (detailed information in Table 2).

Table 2 Participants of the Research Process
Full size table

Domain experts, guidance, career counsellors, and human resources experts, provided consultation to determine the system needs and indicators in the analysis phase. The design step added instructional technologies, measurement and assessment, statistics, and software developers. Experts in measurement, assessment, and statistics contributed to calculating and presenting the indicators. The instructional technologies and software developers worked on the design and develop of the system and its components. Career counsellors provided support in the implementation of the system. 14,884 career counsellors in the implementation step consist of people who actively use the system and work as career counsellors in public schools.

2.3 Data collection tools and data analysis

This research consists of different data collection processes and tools in accordance with the nature of DBR. Semi-structured interview forms and log data are the data collection tools. Semi-structured interview forms were used in the analysis, design, and evaluation stages. All interviews were conducted in person.

The form used to define labour market performance indicators in the analysis process consists of six open-ended questions. These questions include 12 sub-questions in total. For example, "Evaluate the functionality of the qualification mismatch indicator in career counselling processes." and "Evaluate the appropriateness of rating the qualification mismatch groups. These groups are (1) no qualification mismatch, (2) low-level qualification mismatch, (3) medium-level qualification mismatch, and (4) high-level qualification mismatch."

During the design process, career counsellors were consulted about the design of CCIS components. The semi-structured form used in these interviews included ten open-ended questions about the design of CCIS components. For example, "Are the graphs of the firm size indicator produced by the reporting module understandable?" and "Is the functionality of the criteria module design adequate?" and "How understandable is the information presented to you?".

In the evaluation process, semi-structured interviews were conducted with 40 career counsellors using the system to examine the CCIS experiences of career counsellors and to enrich the log data. The interview form included seven questions aimed at understanding career counsellors' CCIS experiences, such as "What are the benefits of CCIS for career guidance services?", "What would be the most important reasons for continuing to use CCIS?" "What would you like to change in CCIS?" and "What aspects of CCIS need to be improved?". Additional questions were posed to the career counsellors in accordance with the context of the interview.

In the evaluation process, log data obtained during the implementation process were used to determine the tendency of career counsellors to use the indicators. Log data refers to the digital traces of users who provide career guidance services using CCIS. The log data consists of the total number of logins, number of unique users, number of university-department profile clicks, number of university-department profile reviews, number of comparisons, number of registered students, and number of career counsellors that enrol students. A sample dataset with log data is presented in Fig. 2.

Fig. 2
figure 2

Overview of log data

Full size image

The PageName column in Fig. 2 shows the actions performed by career counsellors. These actions consist of University and department and compare. University and department actions refer to career counsellors' examining the labour market performance profile of a university and a department based on indicators. Compare action involves evaluating the labour market performance profile of different universities and departments based on indicators. FactorId reports the indicators used. Indicator coding is as follows: 1- employment in the public sector, 2- sectoral distribution, 3- firm size, 4- entry level wage distribution, 5- duration until first employment, and 6- qualification mismatch. UniverstyId and DepartmentId represent the universities and departments investigated. In addition, career counsellors' Session ID and Student ID information are kept by matching them with Advisor ID. This enabled the profiling studies presented in the rest of the study.

Descriptive findings were presented according to log data. In addition to descriptive statistics, adjusted residuals analysis and association rules were conducted to examine the associations with log data. On the other hand, content analysis is used for interviews. Detailed explanations about the analyses conducted within the scope of this research are given under this heading.

Residual approaches play an important role in assessing the significance of cell tests. These approaches are the standardized and adjusted residuals are handled in two ways. MacDonald and Gardner (2000) suggest using the adjusted residual approach instead of the standardized residual approach because it better meets the assumption of normal distribution in evaluating cell contribution to significance. In this study, adjusted residual analysis was used to determine the usage tendencies of indicators. On the other hand, association rules analysis revealed the joint of indicators. Detailed information on adjusted residuals and association rules is provided in the findings section.

Content analysis creates a systematic picture based on qualitative data and answers research questions (Hsieh & Shannon, 2005). This research's six-stage model proposed by Creswell (2007) was as follows: 1) transcription, 2) selecting the reference interview for coding, 3) pre-processing, 4) coding, 5) theorizing and 6) analysis. First, transcripts of the interviews were created. Each researcher carefully read all the interviews and took notes where considered important. Then, with the joint decision of the researchers, the interview containing the richest data for the study was selected. Discussions were held on the selected interview, and explanatory words and/or phrases were placed on these statements. After the preprocessing stage, the coding process began. At this stage, the statements made by the participants were categorized meaningfully, and codes representing these statements were created. A list of codes was prepared, related codes were grouped, and unnecessary codes were removed. After the code list was created, the other interviews were analyzed according to the code list.

3 Findings

3.1 Findings of analysis phase

Labour market performance indicators, universities, departments, and the performance profiles of universities and departments were determined. Information on individuals who graduated from an undergraduate department between 2014 and 2019 was used. Departments were categorized based on their curriculum, resulting in 81 departments. The analysis outcome yielded indicators, university-department classifications, and labour market performance profiles of universities and departments, which served as inputs for the subsequent design process.

3.1.1 Determination of labour market performance ındicators

The data set contained information such as employment and job leaving dates, employment sector, wage information, and firm size of active workers. Five indicators that provided the most information were selected and used as indicators in the study. In addition, a new indicator called qualification mismatch was developed after an extensive literature review. Three focus group interviews were conducted with career counsellors to validate the indicators further, resulting in a consensus on the accuracy of the selected indicators for the career determination process. The content of the indicators is as follows:

  1. i.

    Entry Level Wage Distribution (ELWD): This indicator depicts the distribution of individuals based on four income groups, determined by their first job after graduation.

  2. ii.

    Duration Until First Employment (DUFE): This indicator illustrates the duration between graduation and the commencement of an individual's first employment. The time between graduation and the start of the first job is categorized into four periods: (1) before graduation, (2) 0–6 months, (3) 6–12 months, (4) 12 months and above.

  3. iii.

    Qualification Mismatch (QM): This indicator assesses whether graduates are employed in jobs that are appropriate for their qualifications. A value close to or equal to zero indicates a good match between a graduate's qualifications and their employment. A value approaching three suggests that the individual is working in a job that requires significantly less qualifications than they possess. The International Labour Organization (ILO) has developed a methodology based on standard occupation and skill sets classification to calculate the qualification mismatch using the level of education and field of study (ILO, 2018). The qualification mismatch of graduates in their first job is reported in four groups: (1) no qualification mismatch, (2) low-level qualification mismatch, (3) medium-level qualification mismatch, and (4) high-level qualification mismatch.

  4. iv.

    Employment in the Public Sector (EPS): This indicator measures the percentage of graduates who enter their first job in the public sector.

  5. v.

    Sectoral Distribution (SD): This indicator displays the distribution of sectors in which graduates are employed in their first job. The sectoral distribution is determined using Nomenclature of Economic Activities (NACE) codes, which define and compare economic activities according to international standards. The NACE (2022) code includes 21 different sectors, but for this study, 19 sectors in the employee information data set were considered as references, including Manufacturing, Education, Construction, and others.

  6. vi.

    Firm Size (FS): This indicator displays the distribution of graduates' employment by firm size. Firm sizes are classified into four groups: micro (1–9 employees), small (10–49 employees), medium (50–249 employees), and large (250 + employees).

3.1.2 Determination of university-department classification

In order to assess the labour market performance of undergraduate departments across 159 universities, a classification process was conducted. University-department combinations with at least 20 employed graduates were included in the analysis. Departments were subsequently grouped based on their curriculum and focus group interviews were conducted with career counsellors to determine the suitability of the classification. Based on the experts' feedback, updates were made, and the classification was finalized, resulting in 81 departments. Statistical analyses were conducted to determine the labour market performance of these departments, which will be discussed in the next step.

3.2 Findings of design phase

The system architecture is presented in Fig. 3.

Fig. 3
figure 3

System architecture

Full size image

CCIS has five components: Reporting Module, Criteria Module, Calculation Engine, Indicators, and User Interface. Users can select the university and department based on specific criteria in the Criteria Module. The Calculation Engine receives the criteria from the Criteria Module, processes the data, and transmits it to the indicators for analysis. After the calculations are completed, the Calculation Engine sends the results to the Reporting Module, where the information is visualized and presented to the User Interface.

CCIS comprises two distinct processing centers: the Home Page and the CCIS Guide. The former serves as a platform for disseminating general information about CCIS to the public while containing a wealth of career resources for career counsellors. The Home Page is replete with various informative materials, including FAQs, tutoring resources, and contact details. Meanwhile, the CCIS Guide is dedicated to counselee registration and regulation procedures and reviews of universities and departments based on predefined indicators. Notably, access to this section is restricted to authorized career counsellors only.

3.3 Findings of development phase

CCIS is composed of two distinct panels, namely the Admin and User panels. Individuals with CCIS administrator authority have the ability to access CCIS usage statistics and generate reports based on these statistics. On the other hand, users who have been authorized as career counsellors can utilize the CCIS Guide to obtain in-depth information about the system. Figure 4 shows the CCIS Home Page (http://dabis.cbiko.gov.tr).

Fig. 4
figure 4

CCIS home page

Full size image

The home page provides general information about the system that is accessible to all individuals. Authorized users can access the CCIS Guide by logging in through this page.

Figure 5 illustrates the CCIS Guide, where career counsellors can perform various tasks.

Fig. 5
figure 5

CCIS Guide

Full size image

One of its key features is that career counsellors can schedule online meetings with students through the CAM video conferencing application developed by the Human Resources Office of the Presidency of the Republic of Türkiye.

Figure 6 displays a screenshot of the University/Department profile, which is based on a filtering system that enables users to filter results based on score type, university, department, and indicator.

Fig. 6
figure 6

CCIS Guide- university/department profile

Full size image

Figure 7 demonstrates the Comparisons section, where users have the ability to perform multi-criteria comparisons based on the type of score and combinations of universities and departments.

Fig. 7
figure 7

CCIS Guide-comparisons

Full size image

After the development of the CCIS platform, 30 career counsellors provided feedback. During the interview process, the system was presented in detail, and experts found it to be user-friendly, consistent, and highly informative, especially with regards to the indicators supporting career development. Upon careful consideration of the feedback received from the career counsellors, it was deemed appropriate to evaluate it at the end of the development process in the subsequent stages of the system development. During this evaluation, the suggestions were re-evaluated to determine their potential to enhance the system's suitability for its intended purpose and the quality of data it provides. As a result, the recommended improvements were implemented, and the system was thoroughly tested to ensure that it was error-free and ready for implementation.

3.4 Findings of ımplementation phase

During this phase, career counsellors and university career center managers were authorized, and their username and password information was sent via email. Table 3 displays the number of users and login statistics.

Table 3 User and Login Statistics
Full size table

The group with the highest login rate was the guidance teachers working in secondary education institutions with a rate of 54.46%. Statistics on the number of students enrolled in CCIS are presented in Table 4 and usage statistics of career counsellors are presented in Table 5.

Table 4 Student enrollment statistics
Full size table
Table 5 System Usage Statistics
Full size table

3.5 Findings of evaluation phase

The evaluation process played a crucial role in supporting all stages of the research. Experts were consulted at every stage, and real users accessing the CCIS provided feedback during the implementation process.

3.5.1 Career counsellors' tendencies to use indicators

The usage of indicators by career counsellors, according to the CCIS Guide navigations, has been reported. Out of the 13,000 individuals designated to provide counselling for students making university and department choices, 7889 accessed CCIS. The career counsellors who accessed CCIS performed 41,710 actions in the CCIS Guide. Of these actions, 38,233 (92%) were indicator usages, and the breakdown of these usages according to the university-department profiles and comparisons is provided in Table 6.

Table 6 CCIS Guide log data by indicator use
Full size table

56% is attributed to university-department, while 44% is attributed to guide interactions in the comparisons section. The fact that indicator usage accounts for nearly all of the CCIS guide movements (92%) can be considered as a significant finding in terms of the effectiveness of this section. Indicator distributions regarding the frequency of each indicator's usage are presented in Table 7.

Table 7 Indicator distributions according to CCIS Guide
Full size table

The research examined the movements of career counsellors' CCIS Guide in two distinct groups: University-Department (U-D) and Comparisons (C), as presented in Table 7. In the University-Department evaluations, the most frequently utilized indicators, in descending order, were the employment in the public sector (37%) and duration until first employment (29%). Subsequently, the indicators of entry level wage distribution (17%), qualification mismatch (9%), sector distribution (5%), and firm size (3%) followed this ranking. Similarly, a comparable pattern emerged when analysing indicator usage in the Comparisons section. In this section, the leading indicators were the employment in the public sector (32%) and duration until first employment (27%), with entry level wage (18%), qualification mismatch (12%), sector distribution (6%), and firm size (6%) subsequently following suit. Comparing the two sections, it becomes apparent that the university-department reviews involved a higher number of indicator usages (N = 113,910) compared to the comparisons section (N = 84,351). Furthermore, Table 7 provides the presentation order of the indicators through the system order column. When analysing these findings in light of the sequence of indicator order, it becomes evident that a linear navigation is absent. Instead, a deliberate tendency towards specific indicators is apparent.

Table 8 presents a comprehensive analysis of the frequency of indicators used in both session-based university-department reviews and the comparisons section, independent of a specific indicator.

Table 8 Use of indicators in combination
Full size table

The findings from Table 8 reveal that in the University-Department reviews, the predominant usage of indicators was focused on a single indicator (26%). Subsequently, there was a decreasing trend in the use of indicators, with four indicators (19%), two indicators (17%), three and six indicators (14%), and five indicators (9%) following in sequence. Conversely, in the Comparisons section, the most prevalent usage was observed with six indicators (36%), followed by five indicators (17%), one indicator (15%), four indicators (14%), three indicators (10%), and two indicators (8%). These results highlight a notable disparity in the utilization of indicators between the University-Department evaluations and the Comparisons section. Specifically, it was noted that career counsellors tend to rely on a single indicator in the University-Department evaluations while favoring the utilization of six indicators in the Comparisons section. A corrected residual approach was employed to assess these tendencies' statistical significance. This approach compares the remaining cell values to the critical two-tailed z-value at the predetermined significance level, adjusted through Bonferroni correction. A statistically significant deviation is indicated when the absolute value of the residual exceeds the critical z-value, providing valuable insights for profiling purposes (Terzi et al., 2023). The results are presented in Table 9.

Table 9 Users' indicator tendencies (z-values)
Full size table

The analysis of Table 9 reveals significant deviations from expected values (α bonferroni = 0.00; z-value = 2.87) in six cells. These corrected residual values indicate observations that differ significantly from what was anticipated. The direction of the residuals, whether positive or negative, signifies whether the observations surpass or fall below expectations. Positive residuals indicate higher-than-expected values, while negative residuals indicate lower-than-expected values. Specifically, the findings from Table 4 demonstrate a stronger inclination toward the usage of 5 and 6 indicators in the Comparisons section, whereas in the University-Department evaluations, a greater tendency is observed for the usage of 1, 2, 3, and 4 indicators.

Further analyses were conducted to explore the relationships between these indicators using the apriori algorithm. Association rules are widely used in data mining to uncover non-linear patterns within a dataset. Evaluation metrics specific to the algorithm are employed to filter the rules, ensuring the selection of relevant rules based on specific criteria. This study filtered the association rules obtained using commonly used metrics such as support and confidence. A minimum confidence value of 0.8 and a minimum support value of 0.2 were set. The resulting rules may involve one or multiple antecedents and consequents. These rules are represented in the form of x—> y, where the confidence metric provides information about the likelihood of the items on the y side given the occurrence of the item or items on the x side, and the support metric indicates the occurrence probability of this relationship within the dataset. The items on the left side of the expression are referred to as antecedents (x), while the items on the right side are known as consequents (y). A total of 10 association rules were generated from the apriori algorithm, which is presented in Table 10.

Table 10 A set of association rules in the use of indicators
Full size table

Notably, the rules containing the duration until the first employment indicator attract immediate attention. Notably, the duration until first employment is not an antecedent for other indicators. This finding implies that, despite being ranked first in the system, the utilization of the duration until the first employment indicator is contingent upon using other indicators. It becomes evident that various combinations exist. For instance, examining the first rule reveals that individuals utilizing the industry distribution, entry-level wage, and employment in the public sector indicators all include the duration until the first employment indicator (confidence value).

Additionally, this rule covers 45% of the dataset (support value). Similarly, it is observed that the rate of public employment indicator solely acts as an antecedent for the duration until first employment and entry-level wage indicators. Notably, except for the sixth and seventh rules, the employment in the public sector indicator is an antecedent for all other rules. This finding suggests that employment in the public sector holds a critical position among the indicators. The fourth rule indicates a pattern of combined indicator usage, as individuals incorporating the sector distribution, firm size, entry-level wage, qualification mismatch, and employment in the public sector indicators also include the duration until the first employment indicator (confidence value). This rule encompasses 30% of the dataset. In conjunction with the sector distribution, entry-level wage, and employment in the public sector indicators, the qualification mismatch indicator exclusively serves as an antecedent in the second and fourth rules.

3.5.2 Results of content analysis

The analysis findings indicated that the counsellors emphasized the significance of CCIS in the career counselling process and provided valuable recommendations for its further development. They commended the system for its ability to enhance the effectiveness of counselling services by incorporating the performance of universities in the labour market as a new parameter. Furthermore, the counsellors underscored the value of CCIS in empowering students to make informed decisions about their career paths based on reliable data.

The statements of career counsellors regarding the contribution of CCIS are presented below:

  • P1: “CCIS provides the opportunity to guide students based on data in the career selection process.”

  • P2 : “CCIS is a good example of caring about the future, youth and employment…”

  • P3 : “A nice, well-equipped system; if the data is kept up-to-date, it will be a guide for us and the students in the counselling process.”

Based on the feedback provided by the career counsellors who actively use CCIS, it was identified that there were certain areas for improvement. In addition to highlighting the positive impact of CCIS on the career counselling process, the counsellors emphasized the need for certain features to be added to the system. The opinions of the counsellors on the areas for improvement were analysed, and the results were presented in Table 6 along with sample statements.

4 Dıscussıon and conclusıon

This research involves the process of designing a web-based career information system. DBR methodology was used for the research problem. Interface design, information design (data visualisation, etc.) and interaction design were handled holistically and systematically to reveal a career information system design model suitable for the needs. The research has enabled universities to identify design features for developing career information systems integrated with labour market performance indicators. CCIS can be used on a national scale with a high level of real users and provides a basis as a best practice for developing career information systems based on labour market performance indicators, especially for developing countries and countries in digitalization. One of the strengths of the research is that it has been tested in a real-world setting and in a real context (career guidance processes) and with a wide range of users, which the DBR also emphasizes. An important limitation of DBR research is that the implementation process is carried out with limited participants and in a controlled setting (Dermentzi et al., 2022; Sandanayake et al., 2021). As a result, CCIS has a database that allows the 4000 + universities and departments’ profiles reviews and 17 million comparisons based on six indicators. This database continues to enrich with log data from user interactions. CCIS, which aims to support young people to make more informed career choices with labour market data, was cited as a best practice in the OECD, 2023 Türkiye Economic Survey for contributing to Türkiye's human resources reforms (OECD, 2023).

Ginzberg (1952) characterizes career choice as a largely irreversible decision. To create an ideal career path, individuals must balance career decisions with labour market information (Parsons, 1909). CCIS was conceived to support young people's decision-making processes with objective data, aligning with Parsons' call for timely, accurate, up-to-date, and easily accessible guidance. Unlike other career information systems, CCIS includes mentor support and underscores the role of career counsellors in decision-making.

In order to create an ideal career path, it is important for individuals to be able to find a balance in career decisions by supporting their self-knowledge with labour market information (Parsons, 1909). At this point, the idea of CCIS, which is the subject of this research, was born in order to support the decision processes of young people with objective data. As Parsons (1909) stated, every individual needs help in making the most important decision of his/her life and this help should be timely, accurate, up to date and easily accessible. In this direction, unlike other existing career information systems, mentor support was added to CCIS and the necessity of career counsellors to take an active role in career planning processes were once again revealed in the resulting decision table.

Labour market information is central in the delivery of effective career guidance (Gysbers et al., 2014) therefore in every career information system. Parsons (1909) described it as “a knowledge of the requirements and conditions of success, advantages and disadvantages, compensation, opportunities, and prospects in different lines of work” (p. 5). Defining labour market indicators remains uncertain (Kumar & Arulmani, 2014), but our research demonstrates how six indicators enhance career guidance. CCIS's usage statistics show its effectiveness in facilitating career decisions and university and department selection. Moreover, usage statistics demonstrate a steady increase in the number of users, students, university-department profile reviews, and comparisons within the CCIS. This suggests that CCIS is meeting a need in the field of career guidance, particularly in terms of the use of labour market information in the university and department selection process. Using labour market information in selecting higher education programs is crucial in promoting effective employment management and individual's career growth.

Prior research underscores guided career decision-making's significance (Eun et al., 2013). Choosing a university and department is critical career decision in an individual's career path and career counsellor assistance is vital in facilitating and improving individuals' decision-making process for a career choice that aligns well with their aptitudes, interests, personality traits and labour market (Eun et al., 2013). CCIS is only accessible to professionals in the field, such as career counsellors, due to the critical nature of career decision-making and the potential risks associated with misinterpreting labour market performances. CCIS aims to support students in this process by providing career counsellors with relevant labour market data to guide students in making informed decisions. The integrated presentation of information, such as employment in the public sector and firm size of universities and departments, helps facilitate effective career management.

On the other hand, system interactions show that there is no linear movement between indicators and that there is a deliberate tendency towards indicators. According to the research results, it is seen that employability is important in career determination processes. Another striking point is that there is a statistically significant trend in the use of five and six indicators in the comparisons section. This reveals the need for comprehensive labour market indicators when there is more than one career choice. In addition to this result, based on the system interactions, questions such as what the ideal number of indicators should be in a career information system and which indicators should be included, what are the factors affecting the use of indicators (indicator use according to universities and departments, etc.) and what are the decision-making indicators will be interesting research topics.

CCIS is a system continuously undergoing development and improvements based on feedback from stakeholders. We present the initial results from the DBR first cycle and invite researchers to explore the potential of career information systems. The main areas identified for improvement include for second cycle: a) adding of career counsellor sign-up feature, b) incorporating associate degree programs into the system, c) adding a reporting function, d) adding of student sign-up feature, and e) diversifying the range of universities and departments. In addition, usage statistics have highlighted the need to ensure that users effectively utilize the system's indicators and comparison sections.

The career counsellors have recognized CCIS as a tool that facilitates effective employment and human resource management, going beyond its function as a career choice facilitator. The development and evaluation of CCIS involved input from experts who emphasized the importance of the indicators in supporting the career decision-making process. Especially, based on log data, research findings suggest that considering multiple parameters, such as those presented in CCIS, is more appropriate to make informed career choices than relying solely on a single factor, such as duration until first employment. By incorporating a multidimensional decision-making process that includes six indicators, CCIS strengthens the career decision process, ultimately benefiting individuals and promoting better employment outcomes.

Staunton and Rogosic (2021) argue that career information systems developed without a scientific basis cannot support career decision-making processes. In accordance with the iterative and formative nature of DBR, the output of each cycle provides a source for the following design research, in other words, new cycles. Therefore, the first cycle outputs obtained in this study have a robust scientific base as input for subsequent research. CCIS is also a precursor to a human resources ecosystem that recognizes the importance of early career intervention in career decision-making. CCIS has the potential to provide valuable information for top stakeholders such as policymakers, career counsellors, curriculum developers, and university administrators. The most important milestones of this DBR are that the universities have reached a sufficient number of graduates who have entered employment (a limit of 20 graduates was for this research), employment information is digitally recorded, universities and programs are classified in a way that overlaps with employment, data preprocessing processes are carefully carried out to combine data from different institutions, it is long-term and dynamic process, and since a national scale digital transformation is aimed in the context of career guidance, it is necessary to work with large masses and an interdisciplinary team. In this research, the career information system was developed in collaboration with experts, career counsellors and researchers. In future research, individuals at the career decision-making stage can be included. Moreover, the following research should examine the specific role of these indicators in the career determination process in detail. The findings of this study are expected to open up new avenues for further research in the field of career guidance.