The Next Generation Tester: Meeting the Challenges of a Changing IT World

If we stop and look back at events that are shaping IT, the pace of change is amazing. Things are so different now, even if we just consider the last 5 years. Projects using, for example, artiﬁcial intelligence or Internet of Things are no longer “over the horizon”; they are reality. Testing needs to keep moving with the times even more than in the past. It needs to face the challenges posed by a changing IT landscape and continue to add value. To do that testers must make sure they are providing the right competencies to projects, regardless of the software development lifecycle used or the particular type of project: It’s time to shape what the “Next Generation (NG)” Tester should be.


Introduction
If we stop and look back at events that are shaping IT, the pace of change is amazing. Things are so different now, even if we just consider the last 5 years. Projects using, for example, artificial intelligence or Internet of Things are no longer "over the horizon"; they are reality.
Testing needs to keep moving with the times even more than in the past. It needs to face the challenges posed by a changing IT landscape and continue to add value. To do that testers must make sure they are providing the right competencies to projects, regardless of the software development lifecycle used or the particular type of project: It's time to shape what the "Next Generation (NG)" Tester should be.
For two decades the software testing industry has established the tester as a role. Organizations such as the International Software Testing Qualifications Board (ISTQB) and the International Software Quality Institute (iSQI) have done much to establish this role and have also helped to define more specific roles such as Test Manager and Test Analyst. Those roles are now well established, which is a credit to those organizations, and a great improvement on the "dark" days of the past where being a tester was not always considered to be a particularly attractive and challenging career to pursue. But does this role-based approach still serve the IT industry well?
Looking at the title of this article, you might be forgiven for thinking that the Next Generation (NG) Tester is just another attempt to define a new role in testing. In fact, the NG Tester is more a conceptual idea than a new role. The NG Tester may be considered as someone with a particular set of testing competencies that suits an organization or project. The word "competency" is of particular relevance here. A competency: • Defines something that a tester can do (e.g. design effective test cases for AI applications) • Is something that a tester can deliver in real life projects • Can be evaluated within the context of a skills framework This chapter first considers some of the key factors that influence businesses today and how the IT industry has responded. Based on those factors, the chapter considers some of the principal testing competencies required of an NG Tester. These are described within the conceptual model of a professional development skills framework which provides both a structure for organizing testing competencies and an approach to defining and evaluating them. Using the skills framework, organizations and projects will be able to flexibly define what competencies "their NG Testers" need.

Forces That Drive Businesses
Fundamentally, the following three forces influence business today: 1. A continuous drive for cost-effectiveness 2. The need to deliver new products and services quickly, flexibly, and frequently 3. The ability to adapt (i.e. agility) Quality? Yes, but achieving quality must be in balance with these key forces.

Solutions in IT
In response to the forces facing business, several new solutions in IT have emerged, all of which impact the competencies required for testing. These include: • Highly connected applications -Applications using the concept of the Internet of Things (IoT) -Mobile applications The list is, of course, not exhaustive. There are many other IT trends that could also be listed, but those mentioned are considered to have the most impact on software testing in general.

Practices in IT
The way in which software projects are conducted continues to be influenced by the following trends: • A more collaborative way of working between project stakeholders -Agile -DevOps • Less focus on specific testing roles • Increased levels of automation -Continuous integration (CI) -Integrated Test Automation Frameworks (TAF) -Modelling for test automation (MBT) • More integration into projects of particular engineering disciplines -Design engineering for IoT -Usability engineering

What Competencies Will Be Needed in Testing?
Considering the trends in IT mentioned above, testing professionals will clearly need a wide range of testing competencies if they are to be considered valuable members of project teams. Providing an answer to the high-level question of "what competencies" is therefore best addressed by using a framework model which enables the different competencies to be organized. Initially, the well-known T-Model will be used, but this will later be extend to a more comprehensive and usable form.
The T-Model, as the name suggests, organizes competencies into two categories: 1. Broad-based, general competencies which may be applied to a wide range of projects. These competencies, such as being able to communicate effectively to a wide range of stakeholders, are represented by the horizontal part of the T. Testers should try to achieve all of these competencies. 2. Specialized competencies, such as being able to set up a test environment for mobile application testing, are represented by the vertical part of the T. Testers may acquire any number of these competencies as required.
A selection of typical competencies from each of these categories is given below.

General Competencies
Some of the more important general competencies are suggested below. It would be expected that different people might want to put other competencies on this list. Defining the NG Tester using a skills framework allows for this flexibility.
Having said that, it would be unusual to not find the following competencies within a particular NG Tester definition.

Testing Essentials
The fundamental skills of a tester will always be required, whatever the project or context. These are the skills which enable testers to find defects more effectively and efficiently than those without those skills. Competencies required of the NG Tester: • Design tests using the "traditional" black-box techniques. This includes the ability to design tests using equivalence partitioning, boundary value analysis, decision tables, state transition diagrams and combinatorial techniques. • Perform effective exploratory testing. • Take on various tasks in the testing process (e.g. analysis of user stories, design and execution of appropriate tests).

Risk-Based Testing
As an established and well-understood approach, applying risk-based testing is an essential general competency. Competencies required of the NG Tester.
• Identify and categorize different types of risks.
• Prioritize testing according to risk levels.

System Architectures
The architecture of a system has an impact on how it is tested. Competencies required of the NG Tester: • Understand and analyse a specific type of system architecture (e.g. web-based, micro-services, Model-View-Controller, client-server). • Discuss architectures with system designers to identify risk areas and target testing.
Remember, the competencies here may be focused on one or more particular types of architecture which a given project applies. It is uncommon to require competencies in all possible architectures.

Communication
Effective communication within projects is essential to ensure correct understanding and give stakeholders the required information.
Competencies required of the NG Tester: • Communicate effectively in teams.
• Talk to stakeholders at their level.
• Judge how much communication is enough.

Automation
Competencies in automating tests are no longer considered a speciality; they are an essential skill, especially where agile and DevOps approaches are used in a project. Competencies required of the NG Tester: • Achieve efficiencies by using test automation tools.
• Program test automation scripts which are maintainable and efficient.

Adapting to Different Lifecycle Approaches
A tester must have competencies in working with different lifecycle models, including sequential (e.g. V-Model) and iterative (e.g. agile). Competencies required of the NG Tester: • Place testing activities within the particular lifecycle model. • Apply communication and automation competencies as required by the lifecycle model.

Summary of the General Competences
The NG Tester can: • Practice the essentials of testing in order to detect defects early • Apply risk-based testing to prioritize testing •

Specialist Competencies
A wide spectrum of different specialist competencies may be acquired by the professional tester. In this section, we consider the competencies needed for testing applications which use Big Data, connected devices and artificial intelligence. These three subjects illustrate the typical challenges facing the tester and the competencies they require to deal with them (further specialist subjects are covered, for example, by the ISTQB in their Specialist stream of certification syllabi www.istqb.org).

Testing Big Data (BD) Applications
Big Data applications enable organizations to extract value from the vast amount of data provided by sensors, monitors and various other sources of business information.
Challenges for testing • A wide range of functional and non-functional quality attributes must be considered, such as: -Performance of data analysis to provide business with "instant" results -Scalability of applications to enable potentially huge volumes of data to be acquired and integrated into the existing system data -Functional correctness of analysis methods and algorithms applied to the data • Data currency, backup and recovery capabilities must be tested to ensure the high quality of data. In many cases this data may represent a "single point of truth" for the entire business.
Competencies required of the NG Tester: • Provide stakeholders with an assessment of product risks which consider the key quality attributes of BD applications. • Agree testing goals with a wide variety of business stakeholders. • Conduct exploratory testing sessions with business stakeholders to evaluate the value and accuracy of results provided by the Big Data application.

Testing Applications Using Connected Devices
Applications which use connected devices fall into two broad categories: • Apps which run on hand-held mobile devices such as a smart phone • Applications built on the concept of the "Internet of Things". In this case, the connected device might be a car, a refrigerator or any other form of device which can be wirelessly connected via the internet Challenges for Testing The wide range of quality characteristics to be considered presents the main challenge for testing. In addition to functionality, these include the following nonfunctional characteristics and associated answers to be provided by testing: • Usability: -Can the user effectively and efficiently use the application? -Is there a positive experience for the user? -Can users with disabilities also access the applications?
• Performance -Does the system respond to user requests in a timely manner? -Can the system deal with the loads imposed by many users?
• Security -Are communications protected from security threats (e.g. man in the middle attacks)? -Can a mobile device be misused by unauthorized users (e.g. distributed denial of service attacks)?
• Interoperability -Can we be sure our application runs on the devices and versions intended? -Can the required different operating systems, browsers and platforms be supported? -Can we easily upgrade or swap different hardware and software components?
• Reliability -Can the system handle a wide range of different exception conditions, such as loss of connection, intermittent signals and hardware or software failures in particular system components?
Just consider the testing effort that would be required to test all of these individual characteristics and provide answers to all those questions. Although it might be desirable to cover everything, this is rarely a practical proposition in terms of available budget and timescales. The challenge for the tester therefore comes in helping to decide how to prioritize testing.
Competencies required of the NG Tester: • Identify the principal risks associated with applications and systems using connected devices. • Determine a test strategy to mitigate a wide range of technical risks. • Design and perform tests which focus on the particular non-functional quality characteristics of applications and systems which use connected devices • Apply appropriate tools (e.g. simulators and emulators) to testing applications which use connected devices.

Testing Artificial Intelligence (AI) Applications
Testing AI applications focusses principally on the ability of the application's algorithms to match ("fit") requests for information with the application's available knowledge base. Testing focusses on two main aspects: • Detection of situations where the application cannot match information requests at all and therefore provides no response (underfitting). This is an indication that the matching algorithm may be too precise.
• Detection of situations where too many matches are found and potentially incorrect responses are provided to requests (overfitting). This is an indication that matching algorithms are too general and provide unwanted responses.

Challenges for testing
• Any test needs expected results to enable detection of defects. Because AI applications are continuously "learning" and updating their knowledge bases, the definition of expected results is difficult. • Testers must involve business users in the evaluation of results and in any decision to raise a defect report.
Competencies required of the NG Tester: • Make stakeholders aware of the key risks of AI applications.
• Coordinate and test specific aspects of AI applications in cooperation with business stakeholders. • Understand AI application architectures and the specific purpose of the application. • Set up different sets of data in order to identify issues with underfitting and overfitting.

The Skills Framework
So far we have considered the general competencies to be provided by the "next generation" tester and a selection of the specialist competencies. Whereas ideally all of the general competencies should be available, testers, employers and projects need to decide on which specialist competencies are required and where skills development should take place. Given the wide range of possible testing competencies, these decisions are best supported by a skills framework which goes well beyond the simple T-Model. A skills framework captures the following elements: • Competency descriptions • Competency levels • A method for assessing competencies • Profiles

Competency Descriptions
Competency descriptions are provided of general and specialist competencies. At high level, these would be similar to those outlined above, but may be further broken down as required.

Competency Levels
Each competency in the skills framework is assigned one of the following suggested levels: • Initial level: At this level a person can demonstrate a particular competency in a project if supported by others with higher competency levels. • Full level: At this level a person can fully demonstrate the competency in a project without support • Consulting level: At this level, a person is sufficiently competent to provide consultation services to people, projects and organizations on the particular subject. The person can take the initiative in identifying ways of improving the competencies of others.

Assessing Competencies
A skills framework must describe the conditions for achieving individual competencies at different levels. Given the three levels described above, the following conditions may be defined for their assessment. Initial level • Proof of understanding is required: Evidence for this may be provided by a certification provided by internationally recognized authorities such as iSQI and ISTQB.
Full level • Proof of practical experience over a minimum period (e.g. 18 months) is required. A project portfolio is submitted and a reasonably complex predetermined practical task completed as evidence of achieving the competency. An independent evaluation of the portfolio and the practical task is performed.

Consulting level
• The criteria to be achieved are similar to those described for the full level, except that the length of the experience is longer, the portfolio is more extensive and the predetermined practical task more complex.
Note that the individual levels mentioned above should not be formally bound to each other by pre-conditions. For example, if a person can show evidence of achieving competencies at consulting level, they should not first need to be assessed at initial and full levels.

Profiles
A key element of a skills framework is its ability to be easily and flexibly applied to the practical needs of people, projects and organizations. Profiles provide a way to define collections of specific competencies at particular levels which match particular demands. For example: • An organization which creates mobile applications which make use of AI may define a profile which includes all general testing competencies to initial level plus specialist competencies at full level in testing applications using connected mobile devices and AI. • An individual may construct a profile which enables them to define a goal for their personal development. They may, for example, set the goal of being competent at full level in certain general competencies and competent to initial level in a particular speciality.
Note that profiles are similar to but are not the same as roles. Profiles are more flexible in their definition and relate more directly to a specific need. They are constructed from the competencies provided in the skills framework and do not suffer from the inflexible "one size fits all" problem which is inherent in traditional role definitions.

Conclusion
Role-based approaches in testing have shown their value and are still in use today. However, projects (in particular those using agile approaches) are increasingly moving away from fixed roles and towards a more flexible competency-based approach. Adopting this type of approach enables projects to define their precise competency needs and to align better with the needs of business.
A skills framework supports the overall concept of a competency-based approach. They provide the skills definitions that enable people, projects and organizations to set up their own flexible competency profiles. By including levels of competencies and a scheme for assessing them, the skills framework becomes a flexible tool with which to measure skill levels and promote their development.
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.