Keywords

1 Introduction

Universal design – accessible, inclusive, usable for all solutions and environments [1] – has the potential too lessen democratic, economical and accessibility issues [2]. The desire for inclusive societies has prompted a focus on universally designed ICT. Norway has thus legislated that all public ICT systems must be “universal designed” [3]. As Norway legislated the Anti-Discrimination and Accessibility Act in 2008 [3], any web-based ICT solution aimed at the general public must be universally designed. The regulations that legislates universal design of ICT systems in Norway came into effect in 2014 [4]. This means all new web-based services and solutions in Norway must adhere to accessibility and inclusion regulations. Further, existing solutions must adhere to the law and regulations within 2021. At least a minimum level of accessibility and inclusiveness quality assurance must be present in design and development.

The legislations have triggered increased UD awareness and focus both in the general public and in ICT fields, such as front-end development, visual design, IxD and UX. Raised awareness has inspired beyond legislated criteria; focusing on achieving awards, securing company reputations and ensuring good UX for all users, on all devices, in different contexts of use. Professionals in the field of interaction design and related design disciplines are invested in ensuring digitalized solutions are indeed meeting the criteria for universal design [5]. In relation to ICT systems the criteria are vaguely defined beyond WCAG 2.0 AA level compliance [4], and best practices appear fuzzy. An understanding of current practice is important in order to advance design research on UD aiming to develop new knowledge to support industry, the education of professionals and communicate to stakeholders what UD entails [6].

In order to better inform professionals and stakeholders on what ensuring universal design of ICT systems and eService solutions entails, more insight is needed into how universal design and accessibility work is performed in practice. This paper studies the practices of profiled Norwegian professionals on universal design of ICT systems and the main factors influencing their approach choices; methodological stances [7,8,9,10,11,12], methods used, and key reasons for methodological choices. Factors influencing usage choices are looked into, including correlations between methods, design approaches and methodological stances. Factors unrelated to individual preferences and opinions, such as external constraints (e.g. time, budget, competence and project goals) and external influences (e.g. company culture, team members’ wishes and stakeholder interests) are also studied.

2 Universal Design as a User-Centered Methodology

Universal design methodologies reported in literature are largely human-centric [13], inclusive and iterative. Examples of methodological approaches used in universal design are inclusive design [14], user sensitive inclusive design [11], collaborative [15] and participatory design [16]. The British Standard 8878 Web Accessibility Code of Practice suggests a user-centered approach to producing web products that are accessible to a range of users [17]. Though universal design may be viewed as an extension of user-centered approaches, there are variations in recommended approaches as well as degrees of user sensitivity, user contact and user involvement [8,9,10,11,12, 14, 18].

Paradigm stances and worldviews influences on methodological choices could be categorized as epistemological beliefs. Two different cultural stances appear present in the field. The first is focused on technological solutions, universal design checklists and standards to be used in automatic tests and expert inspections [9, 19], and seems to be influenced by classic (post) positivist research. This culture is sometimes referred to as taking a “just tell me what to do” approach to universal design, and may view universal design demands negatively – as placing additional demands on and limiting the freedom of the developer or designer.

The second cultural stance, in contrast, holds a more positive attitude to universal design. It is focused on users and user experience, and seems more aligned with critical and interpretive paradigm stances. This stance is reflected participatory, inclusive and user-sensitive approaches [10, 16].

Professionals are likely to face deadlines, limited budgets, politics, agendas and disagreeing stakeholders. Facing these challenges, several reasonable design approach worldviews may be utilized. If a designer chooses an expert-driven low-contact process, this may be viewed as a mechanical approach [7, 18, 20]. One could say an interventionist (or ethical) approach is utilized if the designer is actively attempting to influence constraints. If the designer’s focus is on facilitating dialog and keeping stakeholders in agreement, a romantic approach is taken, in line with soft system thinking [18] and postdesign attitudes [21].

Previous data analysis shows there is an acquiescence response to items on agreement with paradigm stances and worldviews among Norwegian professionals [20] – indicating the sample may hold tacit, nuanced or pragmatic views. They largely agree with all the three different types of worldviews, though only the two non-mechanical views correlate (moderately at 0.468, Sig. 0.016). However, two different overall methodological approaches are identified; one characterized by user-involvement and direct user contact, and the other by a no/low-contact approach. The two approaches may be viewed as opposing, as agreement with any user involved design strategy correlates strongly to another – and negatively to a no-contact approach. However, they are both user-centered. The no-contact approach is linked more to mechanical worldviews (with correlations indicating a quantitative data preference and stronger agreement with positivist stances), the user-involved approach seems to not be linked to any specific overarching epistemologies and methodological stances. As such, there are some indications of opposing methodological cultures both in literature and among the Norwegian professionals, but it is unknown if these are related to method selection.

The aim of the study is to support awareness on approaches and methods usage in the field, including reasons for profiled professionals choosing these approaches, as a step in design research on universal design aimed at better universal design quality control. The underlying assumption is that methodology influences the quality of the resulting solution.

3 Method

In order to reach a larger number of informants, an online survey is the chosen approach for data collection. Due to the possible sensitive nature of some questions, no personally identifiable information such as browser type and version, IP address, operating system or e-mail, are saved along with the answer, even though anonymous participation limits further investigation and clarification of individual respondents. The survey link was distributed via e-mail containing an introductory letter to inform and establish credibility. The Norwegian Social Science Data Services (NSD) approved the study (project 44702).

3.1 Survey Sample

The population “expert professionals in universal design of IT” is not easily defined. This study defined “universal design” broadly in relation to IT, based on NCSU [22] and UN [1] definitions. Work contributing to accessible, inclusive and usable for all ICT solutions (including specialized design and inclusive design for/with marginalized users) is defined as “universal design of ICT”. “Expertise” is defined on visibility over specific degrees or titles. It may be argued that the survey was primarily tailored to experts from design, interaction design (IxD) and user experience (UX) disciplines and also front-end/web accessibility, as these are the fields of the researcher and the survey pilot testers.

The study use the following approach to collect a target sample (non-probabilistic purposive expert sampling); (1) members of the “Norwegian network focusing on Universal Design and ICT”, (2) recognized universal design professionals identified through online search; websites, twitter, blogs, presentations, etc. in IT companies sponsoring Oslo Interaction Design Association (IxDA), and (3) professionals being referred by already identified professionals (snowball). The goal was to identify 30–50 expert professionals. The approach resulted in a final list of 71 professionals from 14 enterprises. All but one referred informant was already included through previous steps, indicating that the selection approach [23] was sufficiently broad.

The first item in the questionnaire is a filtering-question asking for years of experience in the field. One informant withdrew from the sample due to lacking experience within IT, leaving 70 professionals. A multi-step contact approach was taken to increase response rate, by two times reminding and encouraging non-respondents via to reply. Only completed surveys were accepted. 26 professionals answered.

3.2 Survey Design

The survey mixes open-ended and close-ended questions, designed to be non-biased and easily understood, avoiding double-barred questions and negative wording. Collegial reviews and pilot testing by a handful interaction designers familiar with universal design (but not in the sample) was used. Background variables measured were number of years of experience in the field, age (categorized), gender, job title, academic background and area of expertise within UD of ICT (multiple choice with option of specifying other background/expertise than listed).

Three items explored UD definitions; UD of ICT (open answer), UD terms usage (multiple choice on overlapping terms) and disability (agreement with disability model views on 4 point Likert scale) [24]. Two items explored user focus; a matrix (5 point scale frequency on pre-defined user groups) and an open item [24]. Five items assessed methodological approaches (agreement with design approach strategies (A) no/low-contact UCD, (B) direct contact UCD, (C) user-involved UCD, (D) participatory design and (E) empathic design, on a 4 point Likert scale), worldviews (agreement with polarized mechanical, romantic and interventionist views, on a 4 point scale), paradigm stances (agreement with 3 positivist and 3 interpretive statements, on a 4 point scale), epistemological relativism (single select between relativist statement and opposing stance) and quantitative/qualitative preference (single select) [20].

Four items measured typical work process, factors influencing method selection and method usage. This article focuses on analyzing these items. The first asked the respondents to describe how they work to achieve universal design – e.g. typical processes or projects. The goal was to get more insight into how the professionals, in their own words, view their work. An open item asked for factors influencing choices, in order to obtain reflections on method selection and any influencing limitations (external factors, such as external influences or external constraints).

Two items mapped methods usage. A matrix presented 20 pre-defined specific methods and techniques common in UCD, based on literature (e.g. [25,26,27,28,29]). In this study, the term “method” may include both specific techniques (such as personas) and more general methods (such as interview). 7 of the 20 methods/techniques were suspected more common to a “mechanical” no-contact approach: eye-tracking, expert evaluations, surveys, market research, statistical analysis, summative assessment testing and lab testing. These methods are quantitative or quantified, have low or no degree of user contact, target generalizable information, and assumed influenced by classic positivistic aspects such as validity, reliability and generalization.

Likewise, 7 methods were assumed more frequently used in user-involved approaches and more in line with constructivist or critical paradigms: interview, observation, workshops, formative (exploratory) user testing, contextual real-life testing, informal user feedback and storyboard visualizations. These are typically qualitative and exploratory in nature, focus on in-depth understanding and/or visualizations and have a higher degree of direct user contact. 6 methods were viewed as general or cross-stance: personas, scenarios, user journeys, service design, sketching and prototyping. The matrix asked if, and how often, each method was used. Frequency of usage was measured on a 5-point Likert scale. Finally, an open item asked respondents to identify if, and how often, any additional methods were used.

3.3 Data Analysis

One may categorize methodological approaches based on the methods selection, combination or overall approach attributes. Previous theoretical top-down data analysis shows theory-based cultures are not clearly connected to specific methodological practices in the sample [20]. This study therefore focuses on a bottom-up, inductive and emergent approach for analyzing methods usage and methodological approaches.

Spearman’s rank correlation coefficient (rho) is used to look for connections. The analysis starts with mapping method popularity and the breath of method usage. Next, reasons for method selection are investigated. The researcher also categorizes methods and explores the assumed split between no-contact and user-involved preferences. Correlations are investigated to see if profiles emerge based on methodological beliefs and stances.

4 Results

37% responded to the survey (26 respondents), of which 10 women and 16 men. Half of the respondents are 30–39 years of age, while 3 are younger, 9 are 40–49 years and one is above 50 years. Many are highly experienced relative to their age, as years of experience range from 2 to 25 with an arithmetic mean of 7.73 years and a median of 7. There are 21 unique job titles across the 26 respondents. Only two professionals have titles specifying expertise within universal design, see Table 1.

Table 1. Categorized work areas reflected in job titles
Full size table

Interaction design is the most common area of work within UD of ICT is (22 professionals). Next come technical and programming expertise (17), visual design (17) and content production (13). 5 work with service design and 2 in management, while single respondents work with ergonomics, counseling, standardization and supervision. The categories are non-exclusive, and only 5 respondents work within one area. 2 work in two of the above areas of expertise, but most work in three or more areas (10 in 3 areas, 4 in 4, 3 in 5 and 2 in 6 areas).

Likewise, the academic backgrounds are diverse, ranging from pedagogics, law and journalism to more traditional development and design disciplines. Most backgrounds are categorized within informatics (73%) or design/UX (54%), and these are often combined. 3 have media or marketing backgrounds, while 3 report other backgrounds. Table 2 presents a cross-tabulated overview, showing about half of the respondents have cross-disciplinary backgrounds.

Table 2. Cross-tabulation of academic backgrounds
Full size table

4.1 Methods Used

The expert professionals utilize a broad spectrum of methods (Fig. 1). Prototyping, Sketching and Workshops are the most frequently used in the sample. No methods are always used by all, or never used by any. All of the methods are used quite often - even the least used method is reported used at least occasionally by more than 30% of the respondents. A few mention additional methods, such as using automatic test tools and testing with assistive technologies.

Fig. 1.
figure 1

Methods employed by Norwegian professionals in universal design of IT

Full size image

4.2 Method Categories

In order to investigate links between methods, the frequencies of usage are re-categorized (re-coded) into 3 categories: seldom, occasional and often, as not all five possible frequency categories are being used for all methods. Spearman’s rho shows high inter-methods correlation; all methods correlate at least moderately (see Table 3) with at least one other. There are no clear groups of methods, as all methods are indirectly linked to any other through a 1-step removed correlation. For example, lab testing does not correlate with observation, however formative testing correlates strongly with both lab testing and observation.

Table 3. Correlation value strength interpretation
Full size table

In order to get a better overview of the bi-variable correlations, only the moderate to very strong (highly significant) bi-variable correlations are selected. All moderate, strong and very strong highly significant correlations are identified for each method. Using these, methods are classified into groups using a bottom-up categorization.

The categorization process indicates three method groups, dividing methods across pre-perceived theoretical traits. G1 consists of methods that are associated with a “user-involved” style (assumed romantic, constructivist or critical). An exclusion criterion for G1 is correlation to G3 methods. G2 is comprised of methods theoretically belong to either G1 or G3, but correlating with the opposite group – indicating a more pragmatic method nature than assumed. Also in G2 are methods that are not assumed belonging to any specific approach, and that correlate only with other methods in G2 or with methods from both G1 and G3. G3 has methods associated with a mechanical view and positivistic values, which do not correlate with G1.

Figure 2 presents the three emerging groups and bi-variable correlations. 9 methods fall into Group 1 (G1), another 9 in Group 2 (G2). All 18 methods are user-centered, and methods theoretically assumed belonging to a “user involved” style are evenly split between the two groups. There is no evidence that G1 methods have a higher degree of user contact. Only 2 methods fit Group 3 (G3).

Fig. 2.
figure 2

Method categories (Group 1–3) based on usage correlations (Coeff. ≥ 0.39 ≈ 0.4)

Full size image

Figure 2 shows ‘Sketching’ and ‘Prototyping’ are very strongly correlated (0.85). The data indicates these two very frequently used methods (see Fig. 1) are used together. They are both among the three methods used the most. Other strong correlations are ‘Storyboarding’ and ‘Scenarios’ (0.65), ‘User Journey’ and ‘Service Design’ (0.73), ‘Statistical analysis’ and ‘Surveys’ (0.67) and ‘Statistical analysis’ and ‘Eye-tracking’ (0.60). It seems likely that these method pairs are used together, within the same processes and phases, though respondents may also alternate between them.

‘Interviews’ correlate to all user test methods (strongly to ‘Formative (exploratory)’, ‘Real-life (contextual)’ and ‘Summative (assessment)’, and moderately to ‘Lab’ and ‘Eye tracking’). When usability testing, both screening/pre-test and debrief/probing/post-test interviews are common, regardless of type of test [28, 29].

Depending on approach, one may talk to informants when observing, for example using participatory observation in a case study, or probing after observing a test/usage scenario. This may explained why ‘Observation’ correlates with ‘Interview’ as well as with most of the user test methods (though not eye tracking and laboratory testing).

‘Eye tracking’ is usually executed in a ‘Laboratory test’ set up, so the moderate correlation between these (0.534, Sig. 0.009) makes sense. ‘Formative testing’ correlates strongly with both ‘Real-life (contextual)’ and ‘Lab testing’, indicating exploratory approaches are conducted both in laboratories and in real-life scenarios. ‘Summative (assessment)’ tests have a stronger correlation to ‘Real-life (contextual)’ than ‘Lab’ testing. This may be interpreted as professionals typically assessing the system solutions in real-life situations, using for example the users personal assistive technology equipment, in the users typical contexts of use.

Apart from eye tracking, data for statistical analysis may be obtained from questionnaire surveys and marked research. ‘Statistical analysis’ correlates on p < 0,01 with ‘Lab testing’ (moderate; 0,526), ‘Eye tracking’ (strong: 0,608) and questionnaire ‘Survey’ (strong: 0,674), and on p < 0,05 with ‘Marked research’ (moderately: 0,423). This also fits pre-study method combination assumptions.

Some strong correlations are surprising, such as the one between ‘Workshop’ and ‘Eye Tracking’ (0.523, Sig. 0.006). The methods themselves was theoretically pre-perceived as representing different methodological styles, with workshops assumed a formative, exploratory and highly user involved method, while eye tracking assumed summative, assessing and with low user contact. Some expected method correlations are also missing; it was for example assumed that those using a service design approach would also use observation. No correlation is found here.

4.3 Factors Influencing Method Selection

When asked what factors influence methodological choices, 14 say Budget and 13 say Time constraints influencing methodology. Complexity and type of project are mentioned by 9, as are the Purpose and area of use, while 7 state Type of target user group is influential. 6 introduce access to Human resources (including users), 5 highlight degree of UD focus in a project (the team or costumer’s willingness to focus on marginalized users and ensure inclusiveness) and another 5 point to the project/process Phase as a factor. Finally, 3 respondents say previous Experience is influential.

The factors can be categorized as external and personal. Resource constraints are viewed as external factors. Three types of external constraints are identified; Human, Time and Budget resources. Sufficient access to one or several of these is influential for 17 professionals (65%). Further, UD focus is interpreted as an external value influence, and adding this a total of 20 respondents (77%) mention external factors.

A similar amount of respondents (18, 69%) mention personal factors. These relate more to the professionals’ personal (internal) reflections on methodological fit to the problem at hand. Purpose or Type of user are given as influential factor by 12 (46%), pointing to the end-goal being important for choosing methods and approach. Complexity and previous Experience also seem important, and are mentioned by 11 (42%). One significant (0.05 level, 2-tailed) Spearman’s rho factor correlation is identified, moderate at 0.463 between Time and Budget resource constraints.

Comparing Influencing Factors to Method Usage.

Cross-tabulating factors to method usage frequencies reveal some correlations. Only two correlations are directly linked to external constraints; both negative, moderately weak and linked to the usage of ‘Informal (user) feedback’. It seems professionals skip getting informal feedback when confronted with Budget or Time resource constraints. However, the external value UD focus correlates to 6 methods. As Table 4 shows, ‘Observation’, ‘Storyboard’, ‘Marked research’, ‘Summative (assessment) testing’ and ‘Eye tracking’ are used less by professionals reporting degree of ‘UD focus’ is influencing their method selection. All 5 methods have moderate significances. ‘Expert inspection’ (including accessibility check) is used more, but the influence is weak (0,39).

Table 4. Spearman rank correlations on factors influencing method selection
Full size table

Correlations show one personal factor significantly affect method usage. Type of target user correlates significantly with 8 methods, and all correlations are moderate or strong. The 69% factoring in Type of user tend to use ‘Observation’, ‘Interview’, ‘Personas’, ‘Scenarios’, ‘Storyboard’ and ‘Summative (assessment)’, ‘Formative (exploratory)’ and ‘Real-life (contextual)’ user testing more than the remaining third.

4.4 Epistemological Beliefs

Previous analyses revealed the sample typically agree with several paradigm stances (positivist, constructive and critical) and several worldviews (mechanical, romantic and interventionist) simultaneously [20]. It seems there is an acquiescence response to items on agreement with paradigm stances and worldviews, indicating the sample may hold nuanced or pragmatic views. About one quarter agree with the relativist statement, and a majority (61.5%) prefer qualitative methods over quantitative [20].

Comparing Epistemological Beliefs to Method Usage.

No relationships are identified between epistemological relativism and methods usage, but two of three assumed non-positivist paradigm stances somewhat influence method usage (original frequency categories): Emphasis on mutual understanding indicates less frequent use of observation (0.546, p = 0.004), questionnaire (0.53, p = 0.005), marked research (0.507, p = 0.008), formative explorative user testing (0.496, p = 0.01) and real-life contextual testing (0.595, p = 0.001), while emphasis on end-user involvement indicates increased use of observation (−0.566, p = 0.003), personas (−0.51, p = 0.008), scenarios/user stories (−0.522, p = 0.006) and interview (−0.599, p = 0.001).

One of three likely positivist stances [20] correlates with one method; with stronger emphasis on objectiveness, interviews are used slightly less (0.556, p = 0.003). A romantic worldview emphasizing communication and negotiations indicates more use of questionnaires (−0.391, p = 0.048) however, in addition to user stories (−0.497, p = 0.01), storyboard (−0.414, p = 0.035) and service design (−0.398, p = 0.044). This is the only worldview which agreement with influences method usage frequencies.

Comparing Quantitative/Qualitative Preference to Method Usage.

For a few methods there is a connection between usage and quantitative/qualitative preference. Spearman’s rank correlation finds significances between quantitative/qualitative preference and utilization of ‘Formative (exploratory)’ and ‘Real-life (contextual)’ user tests, and ‘Interviews’. Spearman’s rho ties qualitative preference to increased use of ‘Real life (contextual) testing’ (strong at 0.65, p = 0.000 for original method frequency categories and −0.63, p = 0.001 for re-categorized (re-coded) frequency categories, see 4.2), ‘Formative testing’ (moderately at 0.46, p = 0.019 for original and −0.50, p = 0.09 for re-categorized) and ‘Interviews’ (moderate at 0.43, p = 0.027 original and −0.42, p = 0.03 re-categorized). Professionals preferring qualitative methods use these three methods more frequency compared to those preferring quantitative methods. For example, 11 of the 16 professionals preferring qualitative data use interviews often, while only 2 of the 10 preferring quantitative do.

4.5 Methodological Approaches

Previous data analysis indicate four methodological design approaches in the sample [20]: an expert-driven no-contact approach (Strategy A), a generic some-contact user-centered approach (Strategy B), a user-involved medium/high-contact approach (Strategies C + D) and finally a generic empathic design approach (Strategy E). Two of these are opposing each other in the sample (A versus C + D), while the other two are agreed upon by a majority of the sample (73% agree with B, and 88% with E).

Comparing Methodological Approaches to Method Usage.

When exploring connections between methods usage and the items on design strategy approaches, respondents fully agreeing with a user-involved strategy have particularly high frequencies of use in ‘Observation’, ‘Interview’, ‘Sketching’, ‘Informal feedback’ from users and ‘Formative (exploratory) testing’. Correlation calculations using Spearman’s rho confirms this, detecting correlations (for original method frequency categories) between agreeing with user-involved Strategy C and using more ‘Observations’ (strong: −0.692, p = 0.000), ‘Interviews’ (moderate: −0.596, p = 0.001), ‘Formative testing’ (strong: −0.637, p = 0.000), as well as ‘Summative testing’ (moderate: −0.452, p = 0.021), ‘Storyboards’ (moderate: −0.473, p = 0.015), ‘Scenarios’ (strong: −0.695, p = 0.000) and ‘Personas’ (strong: −0.691, p = 0.000).

Likewise, there are correlations between user-involved Strategy D and using more ‘Observations’ (moderate: −0.445, p = 0.023), ‘Interviews’ (moderate: −0.463, p = 0.017), ‘Informal feedback’ (moderate: −0.561, p = 0.003), ‘Formative testing’ (moderate: −0.437, p = 0.025), ‘Summative testing’ (moderate: −0.555, p = 0.003), ‘Real-life testing’ (moderate: −0.458, p = 0.019), ‘Storyboards’ (moderate: −0.429, p = 0.029), ‘Scenarios’ (moderate: −0.579, p = 0.002) and ‘Personas’ (moderate: −0.579, p = 0.002).

Opposite correlations are identified between most of these methods and a no-contact Strategy A; for ‘Observations’ (moderate: 0.415, p = 0.035), ‘Interviews’ (moderate: 0.558, p = 0.003), ‘Formative testing’ (strong: 0.629, p = 0.001), ‘Real-life testing’ (moderate: 0.456, p = 0.019), ‘Expert inspections’ (weak: 0.392, p = 0.048), ‘Scenarios’ (moderate: 0.586, p = 0.002) and ‘Personas’ (moderate: 0.437, p = 0.025).

Only 3 expert professionals fully agree with a no-contact style. These 3 use a limited range of methods. Non-ethnographic user research methods (‘Marked research’, ‘Statistics’ and ‘Surveys’) are not influenced by any approach strategy. The most frequently used methods - prototyping, sketching and workshops - are not influenced by either the professionals’ approach nor other identified external or personal factors.

5 Discussion

The data provide an overview of method usage among profiled ‘expert’ professionals in Norway, and indicates a breadth of method use and approaches. A quite broad spectrum of methods is utilized by the sample, in a varied and cross-method manner.

5.1 Personal Factors Influencing Methodological Approach

Epistemologies.

Norwegian professionals are not successfully classified into theory-based methodological styles based on epistemologies, paradigm stances and worldviews. The sample agrees with several theoretically opposing paradigm stances and worldviews simultaneously, indicating the sample has nuanced and/or pragmatic epistemological views. Looking at methodological and epistemological beliefs and stances as methodology indicators, there is little to no systematic influence on method usage based on the samples paradigm stances and worldviews. Instead, the professionals appear inter-disciplinary and capable of holding and utilizing many epistemological perspectives. This may be interpreted as strength among experts, showing a broad methodological competence and ability to adapt in relation to the problem at hand. It may also be viewed as a weakness, either in professionals’ awareness of their own tacit knowledge, or the ability of the survey to measure items or staying relevant.

Method Combinations.

Results indicate that a theoretical classification of methods based on pre-perceived traits do not fit well with the data from this sample, i.e. may not be a viable approach to identifying different methodological approaches. Instead, a bottom-up correlation categorization using highly significant moderate to very strong bi-variable methods correlations was used to look for methodological approach styles. The categorization shows methods theoretically assumed belonging to a “user involved” style are in fact evenly split between G1 and G2. Instead of seeing theoretically pre-perceived “opposite” methods being grouped in different methodological style approaches, it seems expert professionals combine methods in a pragmatic and diverse manner.

Design Approach Strategies.

Correlating design approach strategy agreement to method usage frequencies, opposing methodological approaches are indicated in the sample. Overall, the assumed “mechanical” expert-driven no-contact approach is not well reflected in the empirical data. However, it seems user involvement adherence in design approaches influence methods usage. There are correlations identified between method usage and adhering to a user-involved (Strategies C and D) versus agreeing more with an expert-driven no-contact (Strategy A) approaches. A higher adherence to user-involved approaches increase use of the ethnographic methods ‘Observation’ (0.692, p = 0.000) and ‘Interview’ (0.596, p = 0.001)). Further, it increases user testing, particularly ‘Informal’ (0.561, p = 0.003) and ‘Formative (exploratory)’ (0.637, p = 0.000) testing. It also influences the usage frequency of user-centered specification techniques, such as ‘Personas’ (0.691, p = 0.000) and ‘Scenarios’ (0.695, p = 0.000).

The correlations between specific methods and design approach support the assumption on user-involved emphasis as a methodological design style opposing the second expert driven no-contact approach. Further, the assumption on empathic and some-contact user-centered approaches (Strategies E and B) being acquiescent/social desirability factors that do not indicate specific styles in the sample is strengthened.

The findings may indicate personal influencing methodological factors are not necessarily linked to epistemologies or meta-level methodological stances, but rather to the design approaches one agrees with and personal quantitative/qualitative preferences. The new findings provide more insights into these two styles. Furthermore, it shows that adherence to user-involved versus no-contact approaches indeed impacts the frequency of use for (a) ethnographic methods such as observation and interview, (b) user testing, especially informal user feedback and formative exploratory testing, and (c) user-centered specification techniques such as scenarios/user stories and personas. The different styles do not affect (d) non-ethnographic user research techniques such as marked research, statistics and surveys, and (e) prototyping techniques, including sketching and workshops.

Type of Target User.

Correlations show that which user is being targeted affects the professionals’ method selection process, influencing the use of (a) ethnographic methods (moderately to both ‘Interview’ and ‘Observation’), (b) user testing techniques (moderately to ‘Summative (assessment) testing’ and ‘Real-life (contextual) testing’, and strongly to ‘Formative (exploratory) testing’), and (c) user-centered specification techniques (moderately to ‘Storyboard’, ‘Scenarios’ and ‘Personas’). This is interpreted as the professionals being oriented towards considering method fit in relation to both aim and constraints; how to best solve the specific problem at hand for the aimed at users in order to reach the goal. Again, (d) non-ethnographic user research techniques (marked research, statistics and surveys), and (e) prototyping techniques, including sketching and workshops are not influenced. As such, there may be some types of methods that are more easily influenced than others. It may be hypothesized that non-ethnographic user research techniques, or method types belonging to Group 3, are more specifically applied in certain circumstances. Thus, their specific usage makes them more robust to influences. Further, it may be hypothesized that the most frequently used methods will usually be applied to a universal design process, regardless of influencing factors, as they are so commonly used. Thus, their popularity may make them more robust to influences.

5.2 External Factors Influencing Methodological Approach

Though the most frequently mentioned factors influencing method selection are external Time and Budget constraints, the correlations show they are less influential than expected. The most influential external factor seems to be the willingness to focus on marginalized users and ensure inclusiveness (“UD focus”), not constraints. The ‘UD focus’ influence is interpreted as the experts adhering to and being influenced by external (normative) values. For example, whether a project is aimed at achieving minimum inclusion criteria or at competing for a universal design award. Thus, this factor may be even more influential in the cases where the designer is yet not an expert, but rather a junior. It could be argued that the degrees of UD focus in a project, and thus the legitimacy of spending resources on marginalized users’ needs, is interlinked to resource constraints.

I hypothesize that Budget and Time constraints are hygiene, and not key, factors. A hygiene factor is something that must be sufficiently present in order to not have a negative influence, but once sufficiently present does not lead to further positive influences when increased. Hertzberg coined the term [30] in his two-factor theory of motivation. A minimum of Budget and Time must be present in order for experts to choose appropriate approaches, but are not key influencers in themselves. Instead, the findings show that the degree of emphasis on user-involvement in the project process, combined with qualitative or quantitative method preferences and considerations regarding the type of user, are key influencers for selecting a methodological approach.

5.3 Limitations of the Study

This article looks into survey items on universal design methodology amongst experts. The data must be interpreted with care due to the fairly low response rate and the nonprobability expert sample, and results are only indicative based on a limited and local sample. However, no biases are identified in the respondents compared to the sample and the gender distribution between the target group and the sample is considered equal (39% versus 37% women and 61% versus 63% men respectively).

The level of confidence in the sample is considered sufficient for seeking insights over generalizable results. The sample includes interaction- and UX- designers as well as developers, project managers and other practitioners within the domain. Looking at data on how they interpret “universal design” and their background variables, the impression is the sample is quite varied. The results indicate Norwegian professionals on UD of ICT are interdisciplinary, and work within several different areas simultaneously. Combined with a low N, this may explain why no clear sub-populations appear in the data set.

Looking at literature, user-centered approaches are common in UD approaches worldwide. UCD, UX and UD methodology overlap. Historically, participatory and high-contact user-involved may have been viewed as Scandinavian approaches, however international research papers describing inclusive and universal design approaches show these are now common approaches. From this perspective, the methodological stances and practices among Norwegian experts are likely not unique. However, the new emphasis on accessibility and universal usability in the Norwegian ICT industry since the Anti-Discrimination and Accessibility Act may make the Norwegian data somewhat unique compared to international practices.

6 Conclusion

This study addresses a need for a deeper understanding of appropriate practices for ensuring UD in IT systems. Engineering design research aims to support industry through improving the understanding of current design practices and based on this develop new knowledge – such as what criteria should be used to judge success, what influences success, guidelines, methods, models, tools and so forth [6]. This research approach is considered beneficial for studying the field of UD methodology. The underlying assumption is that methodology used influences the quality of the resulting solution. This paper reports methodological stances and factors influencing method selection from a Norwegian non-probabilistic expert sample of profiled professionals.

The Norwegian experts employ a variety of methods in user-centered approaches. Methods used, stances, worldviews and approaches in UD of ICT systems highly overlap with “mainstream” UCD methodology. The findings point to methodological approach differences linked to how much the experts emphasize user involvement. Diverging design cultures are identified between emphasizing user-involvement (frequent direct contact, collaborative and participatory approaches) versus agreeing with a no/low-contact approach (expert driven, minimal direct user contact). The user-involved approach dominates the sample.

Comparing method correlation to mentioned influencing factors; personal factors are more influential than external, though external resource constraints are more frequently mentioned. The factors identified as the most influential on method selection are: (1) perceived fit for target user/problem (affecting (a) ethnographic methods, (b) user testing methods and (c) user-centered specification techniques), (2) personal qualitative/quantitative preference (affecting use of interviews and formative/contextual testing) and (3) degree of user-involvement emphasis in design approach strategy. Degree of adherence to user-involved design increase the use of (a) ethnographic methods observation and interview, (b) user testing methods, particularly informal and exploratory techniques and (c) user-centered specification techniques such as personas and scenarios/user stories. The results support a hypothesis of resources being “hygiene” factors on methodological choices – limiting when not present, but not key effectors on approach choices once present to a sufficient degree.

Methodological approach do not affect (d) non-ethnographic user research techniques such as marked research, statistics and surveys. Further, prototyping, sketching and workshops are the most frequently used methods in the sample. Usage frequencies for these top three methods are not influenced by the experts’ methodological approaches or any other identified personal or external factors.

Overall, universal design methodology in Norway appears varied, cross-method and overall user-centered, with personal factors (including adherence to external values) influence methodology. The study points to which types of factors are important influences on approach, in a field using cross-method interdisciplinary universal design methodology with varying degrees of user involvement.

6.1 Further Research

Insights have been made into the methodological space and variety. This knowledge may be communicated as is to stakeholders and professionals in an effort to de-mystify universal design work, as the approaches and methods used are highly overlapping with mainstream IxD/UX work. Many methods and techniques are shared among universal design, user-centered design. However, it is unclear whether some are more frequently used in UD approaches.

The data does not specify whether the methods are used differently within UD – for example how one may involve blind users in prototyping and testing. This study may be used as a design research starting point in relation to UD usage variants of mainstream methods. Investigations focusing on how the experts conduct the reported methods within UD, and whether or not this deviates from mainstream UX, would clarify UD methodology.

It could also be interesting to look into in what phases which methods are being utilized, for example if workshops are more used for ideation or eye tracking for redesign, and if correlating methods are usually combined within the same project, or if the professionals reporting experience with these methods alternate between them.

The study focuses on identifying profiled UD professionals and mapping their practices and stances. Whether the inter-disciplinarity and simultaneous holding of different methodological stances in the sample is a strength (pointing at capabilities of utilizing different approaches based on need) or a weakness (lack of awareness) needs to be further studied. It is unknown whether the diversity of target solutions and process attributes call for a pragmatic methodological approach, or if specific methodological approaches are beneficial for ensuring UD quality.

It can be argued that there is not enough evidence suggesting the approaches, choices and views of the study sample are in fact the most appropriate. In order to explore more robust recommendations for UD methodology, it is interesting to study cases that have been successful in ensuring UD. Investigations here are already ongoing, looking at industry project successes with regards to universal design quality [5].

In order to go beyond merely methodological best practices, the factors tied to industry success cases are also being researched. Rather than focusing solely on best practices, the aim is mapping out factors believed to increase the likelihood of achieving and ensuring high UD quality – both hygiene and key factors. Preliminary results [5] highlight tentative success factors, dividing factors into promoting and obstructing factors, categorized at three levels: personal, project and organizational factors.

Going forward, it may also be beneficial to study specific types of professionals in more detail, for example professionals specialized on web accessibility and front-end development, in order to investigate methodology linked to specific phases, tasks or challenges in more detail.