The Sport Mental Training Questionnaire: Development and Validation

Abstract

Research in sports psychology has revealed that mental training is essential in the development of successful athletes. This project, comprised of five studies (a total of N = 2015) resulted in a novel approach towards the measurement of mental skills in sport. In Study 1 (N = 797) and Study 2 (N = 405) we used principal component analysis to identify and confirm the structure of a wide range of identified sport mental skills. Study 3 (N = 429) and Study 5 (N = 330) confirmed factorial structure of Polish and English versions. Study 4 documented how both versions of the scale were developed, and how these two versions converge supporting their cross-cultural validity. The final version contains 20 items grouped into 5 subscales (foundational skills, performance skills, interpersonal skills, self-talk and mental imagery). The Sport Mental Training Questionnaire is a reliable and valid brief questionnaire that will facilitate psychological evaluation of mental preparation among athletes.

Introduction

Most sport coaches observe that the victory against an opponent with similar physical abilities depends in ca. 50% on psychological preparation (Weinberg and Gould 2015). Research in sports psychology has revealed that mental training facilitates successful performance and enhances athletes’ personal well-being (Vealey 2007). Therefore most athletes make additional mental efforts to enhance their performance as a supplement to physical training. Mental training interventions are used to facilitate specific positive outcomes such as passes efficacy in soccer (Thelwell et al. 2010) or forehand drive efficacy in tennis (Hatzigeorgiadis et al. 2008).

Mental training has been founded on the assumption that psychological factors enhance or inhibit physical performance (Vealey 2007). For instance, excessive levels of psychological stress disturb cognitive focus on the task and increase the focus on the self resulting in a lower level of sport efficacy (Hill et al. 2010; Mesagno et al. 2012). Therefore regulation of excessive stress and negative emotions promotes better performance (Anshel and Anderson 2002; Lane et al. 2011). This is the general theoretical rationale for the use of mental training that provides athletes with new skills that are effective in building a stronger mindset, e.g., with new mental resources that reduce negative states (e.g., distress through positive self-talk (Hatzigeorgiadis et al. 2009) or enhance positive states (e.g., enhanced focus relaxation routines) (Lutz et al. 2009).

Several components of mental training have been validated, e.g., mental imagery techniques (Gentili et al. 2010), self-talk techniques (Hatzigeorgiadis et al. 2004), and preperformance routines (Velentzas et al. 2011). Yet a list of active mental ingredients in these interventions as well as their structure have not been established. Research within the psychological interventions theory has indicated that numerous components of interventions used in applied settings can be reduced to a lower number of components that are sufficient to explain the intervention outcomes (Michie et al. 2013). This is an important aspect of applied studies, because such reduction helps to separate active ingredients (that generate desirable outcomes) from inactive intervention ingredients (that are irrelevant to outcomes). In statistical terms, it is important to test whether each identified ingredient is independent (first-order factors), and secondly, whether specific ingredients group together (second-order factors). From an applied perspective, such structure testing is important to examine whether levels of specific skills are likely to depend on other skills or techniques. Furthermore, there are conceptual inconsistencies within the literature regarding the meaning of mental skills and mental techniques that are often used interchangeably (Hardy et al. 2010). However, theorists suggest that these terms refer to two groups of mental training elements (Birrer et al. 2012; Vealey 2007). A mental skill is the learned capacity or ability to carry out a specific training task (goal), e.g., attentional focus or coping with stress. Whereas a technique is a specific procedure used to achieve mental training goals, e.g., mental imagery or self-talk. This distinction between skills and techniques is important conceptually and it is also important for the applied science because athletes and coaches need to set adequate goals and use efficacious techniques to achieve these goals. From the research perspective, it is vital to further identify what key components are uniquely related to performance success in order to improve assessment and interventions throughout mental training for athletes.

One of the models that account for the differentiation between mental skills and mental techniques is a comprehensive model of mental training developed by Vealey (2007). This model suggests that mental training is nested within several broader contexts. From the most general perspective, mental training is nested within a specific cultural context of society. Within this broad societal context, specific individual training philosophy or approach (e.g., educational vs clinical or program-centered vs athlete-centered approach) proposed by the coach or training consultant determines the choice of a preferred model of training. Models of training (e.g., self regulatory models or cognitive-behavioral models) are overarching thematic frameworks that comprise a functional background for more specific interventions. Finally, within this broad contexts specific techniques (eg. imagery, self-talk) are used in sport psychology interventions to influence the development of targeted mental skills such as performance skills (eg. emotion regulation) or foundation skills (eg. self-confidence), within well planned and adjusted to physical preparation implementation model.

Noteworthy, this model emphasizes the need for cross-cultural research in sports psychology in the pursuit of high efficacy of mental training intervention. This is likely to facilitate cross-cultural research that is currently sparse and still needed in sports psychology after decades of calling for more systematic work on culture in sport (Duda and Allison 1990).

Mental Training Assessment

Psychological assessment in sports identifies psychological factors that enhance successful performance (Hardy et al. 2010). As presented in Table 1, several instruments have been created to measure personality constructs related to sports outcomes (Tutko et al. 1969; Nideffer 1976),general sport-related behaviours (Durand-Bush et al. 2001; Hardy et al. 2010; Loehr 1986; Smith et al. 1995; Thomas et al. 1999), specific mental skills (Williams and Cumming 2011; Zourbanos et al. 2009), or target specific disciplines (Albrecht and Feltz 1987; McAuley 1985). As with all psychometric questionnaires, it is essential to meet methodological standards such as established construct validity and reliability that contribute to the utility of these instruments (Tkachuk et al. 2003; Beckmann and Kellmann 2003). Given the abundance of present perspectives on the measurement of mental training components, it is imperative to perform integrative work that is likely to improve efficiency in psychometric measurements. For instance, using shorter questionnaires that address only the most important components make repeated measurements less burdensome (and thus more likely to occur). It may facilitate better tracking of changes in training effects. Secondly, a questionnaire with the most robust components which are universal across multiple sport disciplines may be more useful at early stages of mental training, during group assessments, in pre-screening, or in situations when the time resources of a coach are limited. Such integrative analyses are also likely to corroborate theoretical unity between different approaches present in the literature.

Table 1 Review of mental training frameworks

The focus of the present research project is on the development of a brief integrative tool that assesses viable components in mental training. Given a dynamic character of the sport domain, the most valuable measurement tool should be ecologically valid and able to assess current state of mental preparation of athletes in a relatively short time (e.g., several minutes). In line with these objectives, we focused on creating a brief questionnaire targeting mental training that could be used in the sport field. Additionally, we followed theoretical suggestion that mental training questionnaires should account for distinctions between skills and techniques (Birrer et al. 2012; Vealey 2007).

Our questionnaire development process followed a recommendation that testing the reliability and validity of measures should be based on four stages (Netemeyer et al. 2003). Psychometric work should begin with the definition of the construct and content domain that would serve as the basis for the generated items pool. In study 1 we examined the structure of the questionnaire by principal component analysis (PCA). As a step toward content validity, six experts judged the reduced item pool. In Study 2 we examined the factor structure identified in the study 1 by PCA. Then we used Structural Equation Modelling (SEM) to validate findings from Study 2. Next, we developed the English version of the instrument. Finally, we used SEM to validate the instrument for international athletes.

Study 1

In line with the four-stages approach towards development of psychometric instruments (Netemeyer et al. 2003) our first aim was to define construct and content domain of mental training, generate an items pool, and examine the structure of the resulting questionnaire.

Material and Methods

Participants

Participants were 797 athletes who completed questionnaires in groups of 10–20 individuals (N = 421) recruited from local sport clubs and university students from the Faculty of Physical Education, or via a web-based survey (N = 376) (For details, see Table 2). Following suggestions from Tabachnick and Fidell (2007) that 10:1 ratio would be suitable for PCA, the criterion was met with the total of 797 cases. Written informed consent was obtained from each participant.

Table 2 Participant characteristics for Studies 1 to 5

Measures

Baseline Information

Participants reported their age, gender, sport played, competitive level (recreational, club, national, international), duration of participation in sport, duration of weekly physicaland mental training, and mental preparation efficacy beliefs, i.e.,to what extent mental preparation helps them achieve the best possible result during the competition in their discipline (using a rating scale 0–100%).

Mental Training

Using Sport Mental Training Questionnaire (SMTQ) participants reported their mental training routines. This was a66-item inventory which yielded an overall mental training score as well as scores for the 4 subscales of foundational skills, performance skills, personal development skills, and mental techniques. Participants responded using a 5-point Likert scale (1 = “strongly disagree”, 5 = “strongly agree).

Procedure

Given the number of already existing assessment tools and available theoretical propositions targeted salient mental skills and techniques (Table 1), we explored their structures and conceptualized our questionnaire on the basis of Vealey’s (2007) framework. Based on the source materials (e.g., quotes) available in qualitative studies (Gould et al. 2002; Gould et al. 1999) and items from measures of mental training (Table, 1), new items were developed in line with guidelines for items wording. Each item was worded so that athletes of 14 years of age and older would understand it. Furthermore, two sport psychologists (the second and fourth author of this paper) reduced the initial pool of 84 items to 66 items excluding redundant or ambiguous items.

Based on the theoretical framework by Vealey (2007), the items were grouped into two conceptually different subscales: mental skills (47 items) and mental techniques (19 items). The disproportion between the number of items in each category results from the model that accounts for thirteen basic mental skills (e.g., attentional focus) and four basic mental techniques (e.g., self-talk). This produced the initial pool of items in line with the general rule that there should be at least three items for one latent factor (Worthington and Whittaker 2006). Furthermore, based on the qualitative content analysis of the items we discussed the groups of similar items until a consensus was reached. This provided a mental skills scale that comprised three subscales: foundational skills (i.e., intrapersonal resources that are necessary to achieve success in sport), performance skills (i.e., mental abilities critical to the execution of specific physical skills during sport performance), and personal development skills (i.e., significant maturational markers of personal development which allow for high psychological functioning through clarity of self-concept, feelings of well-being, and a sense of relatedness to others). Mental techniques were reflected by items accounting for goal-setting, relaxation techniques, mental imagery, self-talk, and preparatory routines. Furthermore, to ensure that the tool is discipline-independent, we excluded skills that were exclusive to team sport disciplines (e.g., team confidence).

The reduced item pool was sent to six experts in mental training in sport: sport psychologists and sport psychology scholars. Each expert (a sport psychologist) received the definitions of the four factors and a table with the items pool. Based upon the procedure from prior studies (Dunn et al. 1999),the experts identified the subscale to which each item corresponded most strongly, or indicated that an item did not correspond to any subscale substantially. Additionally, experts reported potential problems, e.g., item length or reading level difficulty. Based on this input items were retained, modified, or deleted. An agreement of 80% among the investigators was required to retain an item in its original form.

Analysis

We performed principal component analysis (PCA) using SPSS 23.0 (Armonk, NY) with an direct oblimin rotation (Osborne and Costello 2009). To ensure minimal ambiguity between factors, criteria for an acceptable factor solution were: minimum eigenvalue of one and the minimum of three items loading on each factor (Tabachnick and Fidell 2007). The criteria for item exclusion were: a loading below .50 and cross-loading greater than .32 on any other factor (Osborne and Costello 2009). Listwise deletion was used for missing data.

To support the content validity we analyzed the experts’ answers. To establish the reliability of agreement between raters we calculated Fleiss’κ (Fleiss et al. 1969). This is a statistical measure for a fixed number of raters and categorical ratings with an inter-rater reliability coefficient κ > .80 interpreted as an almost perfect agreement (Fleiss et al. 2013).

Results and Discussion

The analysis identified 13 principal components that accounted for 55.7% of the variance. However, the scale failed to provide a rotated solution suggesting that 13 factors were unnecessary (Tabachnick and Fidell 2007). Scree plot inspection revealed four meaningful components in line with the conceptual framework. Direct oblimin rotation produced a 4-factor solution with 25 items, accounting for 47.71% of the variance. The items had moderate communalities ranging from .32–.65. The mean item score was 3.73 with higher scores representing greater mental training intensity.

The 24-itemquestionnaire showed satisfactory internal consistency (Cronbach’s α = .86) with item-total correlations ranging from .316 to .539.Based on items that remained in the instrument we relabelled the third subscale as interpersonal skills (previous personal development skills). The interpersonal skills were defined as the ability to interact effectively with others (Vealey 2007). Cronbach’s αs showed acceptable internal consistency for each of the four subscales (Kline 2005):foundations skills (5 items)α = .73, performance skills(7 items) α = .81, interpersonal skills(5 items)α = .76, and mental techniques(7 items)α = .80.

Content Validity

Experts’ agreement supported the four-factor model, κ = .81 (95% CI, .74 to .87),p < .001.Twentythree items received unanimous endorsement from the six reviewers, i.e., more than 80% of experts believed the item tapped the intended construct, did not correspond to another subscale, and was worded clearly. However, experts reported problems with several items in the mental techniques subscale. Some of the items included statements about using mental techniques during performance. We removed “performance statements” and retained items with the more general meaning. For instance, “During the performance I talk to myself to regulate my own thoughts, emotions, or arousal” was modified into “I talk to myself to regulate my own thoughts, emotions, and arousal.”Finally, we removed one item from the mental techniques scale which did not meet the prior requirements.

Study2

The purpose of Study 2was to re-examine the factor structure of the 23-item SMTQ developed in Study 1using another sample.

Material and Methods

Participants

Participants were 405 athletes who completed questionnaires in groups of 10–20 individuals (N = 250) recruited from local sport clubs and University students from the Faculty of Physical Education, or via a web-based survey (N = 155) (For details, see Table 2). Written informed consent was obtained from each participant.

Measures

The 23-item inventory for mental training was used with the same response format. Other measures were identical to those used in Study 1.

Results and Discussion

Analytical strategy was the same as in Study 1. Direct oblimin rotation produced a 5-componet solution with 23 items accounting for 54.9% of the variance. Based on the previously adopted criteria, three items were removed from the subsequent analysis. The items had moderate communalities ranging from .43 to .75. The mean item score was 3.70. Based on PCA results we divided mental techniques into two subscales: self-talk (inner dialogue)and mental imagery (using images for situation rehearsal). It supported previous findings which have shown that these particular skills are effective and frequently used (Hatzigeorgiadis et al. 2014; MacIntyre et al. 2013).

The 20-item SMTQ was internally consistent with Cronbach’s α = .83andacceptable internal consistency for subscales: foundations skills(4 items) α = .73, performance skills (6 items)α = .79, interpersonal skills (4 items) α = .75, mental imagery(3 items) α = .70, and self-talk (3 items) α = .72. The final 20 items and their loadings are reported in Table 3.

Table 3 SMTQ items and factor loadings for five-factor solution (Study 2)

Study 3

The aim of Study 3was to use structural equation modeling (SEM) to validate findings from Study 2. We hypothesized that athletes with a higher competitive level would display higher mental preparation level compared with those competing at a lower level. To evaluate the test-retest reliability, the SMTQ was administered over a 3-weeks period.

Material and Methods

Participants

Participants were 429 athletes who completed questionnaires in groups of 10–20 individuals (N = 220) recruited from local sport clubs and University students from the Faculty of Physical Education, or via a web-based survey (N = 209) (Table 2). Written informed consent was obtained from each participant.

Measures and Procedure

The 20-item inventory for mental training was used. Other measures and procedures were identical to those in Study 1.

Analysis

We performed structural equation modeling with mPlus 7.2 with the MLM estimator (Muthén and Muthén 2012). Following guidelines for evaluation of model fit (Hu and Bentler 1999), we calculated Standardized Root Mean Squared Residual (SRMR, values < .08 indicating good fit) supplementing it with Room Mean Squared Error of Approximation (RMSEA, values < .06 indicating good fit).An item may be removed if the fit statistics are inadequate and the item fails to meet one or more of the following criteria: large standardized residuals (>2.00),modification indices suggesting that the error term of an item correlated with that of another item, low factor loading (<.40; Mullan et al. 1997), or cross-loading on an unintended latent factor (Hair 2010).

Next, we examined the reliability of the SMTQ in two ways: Cronbach’s α and test-rest reliability. Interclass correlation coefficients were calculated using a two-way mixed effect model to establish test–retest reliability.

To test the construct and concurrent validity the samples from Study 2 and Study 3 were combined, totalling the number of 835 participants. To compare differences between the high and low-level, female and male athletes, t tests for independent samples were performed on each of the five factors and the overall score. For the purpose of criterion validity analyses, the participants were grouped into the following categories:

Competition Level

Among the participants, 252 high-level and 427 low-level athletes were identified. In line with previous research (Eton et al. 1998; Roberts et al. 2008), the high-level athletes were defined as those who participated in their sport at a national and international level, and low-level athletes were those who participated in their sport at a recreational or club level. To ensure that each sport was represented equally in each group, participants were matched for sport type across the two groups. Where sports were not matched, the respective data were deleted from the sample. This resulted in a sample of 326 sport-matched participants, 163 per group.

Gender

In the sample, 368 female and 466 male athletes were identified. To ensure that each gender was represented equally in each group, participants were matched for sport type and competition level across the male and female groups. Where competition level and sport discipline were not matched, the respective data were deleted from the sample. This resulted in a sample of 376 sport-matched participants, 188 per group.

Pearson’s r correlations were used to examine the second hypothesis regarding evidence for concurrent validity.

Results and Discussion

Structural Equation Modeling

Due to of the partial conceptual overlap between mental skills and mental techniques, a five-factor model was tested (Fig. 1). The hypothesized second-order solution with two scales: mental skills (foundation, performance, and interpersonal skills) and mental techniques (self-talk and mental imagery) demonstrated an adequate fit to the data, SRMR = .048, RMSEA = .042, 90% CI [.034; .050]. Inspection of the standardized factor loadings (from .48 to.86), modification indices, and standardized residuals revealed all values were within acceptable limits (Hair 2010). Consequently, each item meaningfully contributed to its intended subscale.

Fig. 1

The hierarchical structure of mental training in sports. Confirmatory factor analysis with standardized coefficients. Note. Numbers next to the shortcut of the scales correspond to order each items within the scales. Numbers in parenthesis correspond to order each items within the questionnaire

Reliability

Adequate internal reliability was demonstrated for the model (Cr = .86) and all five subscales with Cronbach’s αvalues (foundations skillsα = .75, performance skillsα = .81, interpersonal skillsα = .78, mental imageryα = .75, self-talk α = .76) with item-total correlations ranging from .30 to .55. All results were above the acceptable cut off criterion (α ≥ .70; Kline 2005).

After three weeks, an adequate test-retest reliability was demonstrated for the model (α = .89) and all five subscales with αfoundations skillsα = .80, performance skillsα = .83, interpersonal skillsα = .84, mental imagery α = .81, self-talk α = .72). All results were above the acceptable cut off criterion (α ≥ .70; Kline 2005). These results demonstrated test-retest reliability of the SMTQ over a 3-week period.

Criterion Validity

High-level athletes had higher levels of foundation, performance, interpersonal skills and scored higher on mental imagery and self-talk scales compared to low-level athletes (Table 4). Male athletes scored higher on foundations skills, performance skills, and overall score compared to female athletes (Table 4).

Table 4 Criterion validity on the 20-item SMTQ-PL& SMTQ-ENG

The purpose of construct validity analyses was to see if the SMTQ was able to distinguish between groups of athletes based on previous research suggesting that certain athlete characteristics such as competitive level or gender influence mental training components (Durand-Bush et al. 2001; Gould et al. 2002; Roberts et al. 2008; Thomas et al. 1999; Zourbanos et al. 2009). It was hypothesized that athletes of a higher competitive level would display higher mental training levels compared to lower competitive level athletes. Our findings supported this hypothesis. We also observed meaningful differences between male and female athletes.

Study 4

The aim of this study was to develop the English version of the SMTQ. First, the scale was translated and evaluated qualitatively. Second, we established the convergence between the translation and the original version.

Material and Methods

Participants

Participants were 54 English philology students (Table 2).Written informed consent was obtained from each participant.

Measures

The measures were identical to those used in Study 1.Sport Mental Training Questionnaire-Polish and English (SMTQ-PL & SMTQ-ENG) were used.

Procedures and Analysis

The original scale was independently translated into English by mental training experts fluent in English. The four versions were discussed and merged into one in line with the consensual approach. A back translation was prepared by an independent translator. The back translation compared against the original further supported validity of the translation. Subsequently, we used the bilingual answers method for testing the convergence between translation and the original. Half of the participants completed the SMTQ-ENG as the first scale. The r-Person correlation of the total scores for both scales and Student’s t test were computed with SPSS 23.00 (Armonk, NY).

Results and Discussion

Both language version had comparable internal consistency: SMTQ-PL withα = .85 and SMTQ-ENG with α = .84. Supporting the scales convergence, there was no difference between the overall score on the SMTQ-ENG (M = 67.38, SD = 9.93) and the SMTQ-PL (M = 67.14,SD = 10.29), t (106) = 0.12, p = .90, d = 0.02, and none of the scales: foundation skills, t (106) = −0.22, p = .83, d = −0.04; performance skills, t (106) = −.10, p = .92, d = −0.02,;interpersonal skills, t (106) = .48, p = .63, d = 0.09,; self-talk, t (106) = −.09, p = .92, d = 0.02,; and mental imagery, t (106) = .49, p = .62, d = 0.09. The convergence of both questionnaires was excellent for the overall score r = .97, p < .001,foundation skills, r = .96, p < .001; performance skills, r = .96, p < .001,interpersonal skills, r = .96, p < .001; self-talk, r = .92, p < .001; and mental imagery, r = .92, p < .001. The obtained result showed that the adaptation was satisfactory.

Study 5

Study 5 was conducted to confirm the construct validity of English version of the Questionnaire by testing its factorial structure against the underlying theory

Material and Methods

Participants

Participants were 330 athletes from 48 countries who completed questionnaires in groups of 10–20 individuals (N = 162) recruited from local sport clubs and local Universities (international exchange students who studied in English), or via a web-based survey (N = 168) (Table 2).As language skills in this group vary substantially, we excluded students who reported in a control question that they had language problems while completing the survey.

Written informed consent was obtained from each participant.

Measures and Procedures

The measures and procedures were the same as in Study2. The English version of the SMTQ was used. We also added a question about the country of origin.

Analysis

Statistical strategy was identical to that used in study 3.For the purpose of criterion validity, participants were grouped by category based on their a) competition level(from this sample, 45 high-level and 273 low-level athletes were identified, following the same procedure as in study 3 the sample resulted in 74 sport-matched participants, 37 per group), b) gender(from this sample, 172 female and 157 male athletes were identified; following the same procedure as in study 3 the sample resulted in 82 sport-matched participants, i.e., 41 per group).

Results and Discussion

Factorial Structure

We excluded four students who reported in a control question that they had language problems.

The hypothesized model identified in Study 3 demonstrated an adequate fit to the data in Study 5, SRMR = .049, RMSEA = .042, 90% CI [.032; .052].Inspection of the standardized factor loadings (ranging from .48 to .86), modification indices, and standardized residuals revealed all values were within acceptable limits and no offending estimates existed (Hair 2010).

Reliability

Adequate reliability was demonstrated for the model (Cr = .84) and performance skillsα = .78, interpersonal skillsα = .72, mental imagery α = .70, and self-talk α = .85) with item-total correlations ranging from .23 to .52. Cronbach’sα reliability analysis was lower for foundations skills (α = .60).However, based on previous findings, we propose that the foundations skills subscale should retain for further examination in future studies.

Forty participants of Study 5 (Table 2) were asked to complete the SMTQ for the second time after three weeks. Test-retest reliability was demonstrated for the model (Cr = .83) and all five subscales with Cronbach’s αvalues (foundations skills α = .64, performance skills α = .75, interpersonal skills α = .73, mental imagery α = .67, self-talk = α .86).Consequently, results demonstrate test-retest reliability of the SMTQ over a 3-weeks period.

Criterion Validity

We found that high level athletes had higher foundation skills, performance skills, and the overall level of mental training level compare to low level athletes (Table 4). There were no significant differences for interpersonal skills, self-talk, and mental imagery. Furthermore, female athletes scored higher on interpersonal skills, self-talk, and mental imagery and lower on foundational and performance skills. These analyses partly replicated findings from Study 3.

General Discussion

Training of specific mental skills and techniques facilitates success in sports (Gould et al. 2002).Despite several studies that examined mental skills and techniques (see Gardner and Moore 2006 for a review), there is still little consensus regarding the most robust and parsimonious structure of mental training components. Understanding core elements of mental training is critical for successful psychological evaluation and preparation to athletic performance. Building upon prior models of mental training in sport (Vealey 2007) and available psychometric instruments we developed an integrative multidimensional model of mental skills and techniques used in sports. The conceptual and psychometric analyses revealed the components that are essential to mental training as well as their most parsimonious structure. More specifically, we identified the most viable factors, i.e., foundation skills, performance skills, interpersonal skills and mental techniques. This corroborates and extends previous conceptualizations (Vealey 2007). Furthermore, we provided new evidence that self-talk and mental imagery can be considered the most distinguished mental techniques used by athletes (Hatzigeorgiadis et al. 2014; MacIntyre et al. 2013).Finally, our analyses supported a hierarchical structure of mental skills and techniques with two overarching separate factors comprising skills (foundation, performance, and interpersonal) and techniques (self-talk and mental imagery).This hierarchical order is helpful to emphasize that skills and techniques are two separate targets that should be addressed in psychological preparation for an athletic performance.

Our multistep work resulted in a new psychometric instrument that advances the measurement of mental training components. This work provided a brief questionnaire with strong psychometric properties such as high internal consistency, high temporal stability, supported construct validity, and equivalence of two language versions of the questionnaire. Moreover, we provided evidence for criterion validity of the SMTQ scores. For instance, more experienced athletes had significantly higher mental training skills and techniques compared to lower-level athletes; a finding in line with previous studies (Gould et al. 2002; Thomas et al. 1999). This is because high-level athletes engage in more psychologically demanding practice and performance with more frequent opportunities to craft greater mental preparation. Gender differences were also observed. Male athletes scored significantly higher on the foundation and performance scale in line with previous research (McGrane et al. 2016; Smirniotou et al. 2009; Hammermeister and Burton 2004). These results indicate specific weaker points that should be targeted to minimize gender differences.

Using questionnaires that provide maximum information and place minimal burden on the athletes and at the same time are valid and reliable is critical in sports science so that the assessment does not interfere with the training process. Thus, our empirical findings suggest that the final versions of SMTQ might be an optimal choice for researchers and practitioners that are interested in obtaining brief data that cover the most critical aspects of mental training. Yet, some other instruments can be considered a better choice to pinpoint specific aspects of mental training.

One strength of this project is that it resulted in a questionnaire that is already available in two language versions: English and a Central European language. This is likely to facilitate cross-cultural research that is currently sparse and still needed in sports psychology after decades of calling for more systematic work on culture in sport (Duda and Allison 1990). For instance, the population of Poland compared to USA is less individualistic and indulging but more prone to uncertainty avoidance (Hofstede et al. 2010). It would be a worthwhile to research how such cultural differences moderate mental training preferences, practice, and outcomes. Cross-cultural perspectives in sports psychology has been still considered a new line of research focused on a contextualized approach towards humans and their performance (Blodgett et al. 2015). Moreover, within the cross-cultural approach, there is little research focused on Central and Eastern European populations with the majority of studies focused on other race or ethnic issues reflecting cultural identity.

Several study limitations require consideration. First, some conceptual problems with the measurement of components of mental training emerged. The results of Study 1 confirmed some of the problems identified by other authors (Birrer et al. 2012; Vealey 2007), i.e., the distinction between mental skills and mental techniques. Second, SMTQ can fall short for some more sophisticated research questions or practical applications. For example, the instrument does not differentiate between athletes’ use of attentional focus as well as motivational and instructional self-talk (Zourbanos et al. 2009). Neither does it distinguish between the use of internal, external, or kinaesthetic imagery (Roberts et al. 2008). As a result of adopted inclusion criteria, SMTQ also lacks items that concern goal-setting, which has been identified by some researchers as one of the most popular mental technique used by athletes (Brewer 2009).Moreover, we have not compared our measure with any already existing questionnaire seeking to understand whether this new questionnaire accounts for additional variance in sport-related outcomes. Finally, the results are based on cross-sectional and self-report data. Further studies might use prospective designs and observational measures to ascertain that SMTQ predicts real-life preparatory actions and sport performance outcomes.