1 Introduction

Information and communication technology (ICT) has been a key driving force in development and diffusion for knowledge-based economies [1]. As competition in ICT innovation among nations becomes more intense in the theoretical and social paradigms of a national innovation system, there are growing demands for improved judgment, evaluation, and prediction of ICT capacity in order to improve a nation’s competitiveness [2–4].

Technological competitiveness is important for national economic performance, particularly with national introductions of stronger intellectual property rights, regulatory harmonization and standardization, and the worldwide spread of emerging ICT [5, 6]. Technological capabilities are the conceptual-level elements that capture technological competitiveness in operation. A technological level can be defined as the relative technological capability to use technological knowledge efficiently and refers to the extent to which technological knowledge is accumulated, invested in, produced, and innovated. A technological level refers to the present status of technological accumulation and accomplishment based on past technological activities [4, 7]. Therefore, a technological-level evaluation (TLE) can be defined statistically by estimating and assessing the technological status in terms of technological performance and capacity at the time of the evaluation.

Several new models for national TLE have recently been developed, with the research being enabled by a certain theoretical and empirical consensus about the nature of technology [8]. There is significant recent research about the constitution of composite statistics for TLEs at a national level. Policy analysts and academic researchers both need new and improved measures at the technological level regarding the performance of nations in understanding technological, economic, and social developments.

For a long time, the research domain of technological innovation and strategy has suffered from a lack of proper statistical data, forcing most of the earlier studies to focus on conceptual and/or qualitative approaches [9]. Patent statistics have recently been used by technicians, researchers, and economists in the field of technological innovation to analyze current directions of, and to forecast, technological development in terms of well-defined patent databases (DBs). Patents are regarded as an essential source of commercial knowledge, and almost 80 % of all technological information can be found in patent-application data [10]. Furthermore, according to statistics released by the World Intellectual Property Organization (WIPO), 90–95 % of economically valuable human-innovation results are retained in patent DBs [11]. In an increasing number and variety of studies, patent statistics have been used to evaluate research and development (R&D) performance and production. Tools based on patent statistics are used to analyze R&D production and are increasingly being used for R&D performance evaluation.

Mobile communications can be considered to be the fastest growing industry of all over the past decade. Mobile telephones have become daily necessities for billions of people, and they are increasingly being used in both developed and developing nations. There were 2.21 billion mobile subscribers worldwide in 2005, with 6.84 billion expected by 2013 [12].

In this paper, we evaluate the technological level of mobile communications using patent statistics analysis at a national level. In particular, we perform a TLE of 3G, 3G transitional, and 4G mobile communications technologies for five nations; namely the US, EU, Japan, China, and Korea, and we consider the EU a single nation to simplify the evaluation.

The paper is organized as follows. Section 2 outlines research approaches, reviewing the literature on TLE and patent analysis. Section 3 describes the research model for TLE using patent statistics. An empirical analysis and the results of a TLE for mobile communications are presented in Sect. 4. Section 5 contains a discussion and conclusions.

2 Research approaches and literature review

2.1 The demands for data-based TLEs

Various international organizations and entities have made announcements on worldwide technological and economic competitiveness. The International Institute of Management Development (IMD), the Research And Development Corporation (RAND), the United Nations Development Program (UNDP), the United Nations Industrial Development Organization (UNIDO), and the World Economic Forum (WEF) release national competitiveness reports periodically that measure national capacities to improve economic development and living standards. These reports deal with technological competitiveness, and most use patent statistics as a solid basis for national innovative capacity [13]. In addition, major developed and developing nations conduct TLEs when establishing strategies for national R&D programs and monitoring technological trends. Examples of these reports include the ‘National Core Technology Report (US, 2005)’, the ‘Survey on 20-year Technology Predictions for the Future (China, 2008)’, and the ‘Survey on Research and Development Levels (Japan, 2011)’.

In Korea, government agencies and institutes have conducted studies involving TLE, as shown in Table 1. These involved peer reviews and expert surveys at a qualitative technological level. In general, it is recognized that peer or expert surveys have various limitations and shortcomings. In particular, the outcomes of a peer review process may contain errors nearly 50 % because of human chance and randomness [14–16]. However, patents can be regarded as realizations of technologies, and patent statistics would therefore be sufficient for TLEs [17, 18]. Moreover, several new measures of national technological capability using patent statistics have been developed recently, such as ‘ArCo’ [19] and the ‘Patent Asset Index’ [20].

Table 1 Recent Korean reports involving TLEs
Full size table

With this background, we propose a TLE method based on composite and quantitative patent statistics. TLEs based on patent statistics would be expected to apply as supplementary tools in the peer-review process and even to begin to replace the peer-review method.

2.2 Patent statistics for TLE

A patent DB can offer valuable information and knowledge for technological strategy planning and R&D funding. It is also constructed from patent statistics that involve the analysis of successful applications in high-tech industries such as ICT and biotechnology.

Patent statistics analysis techniques have also been used to reflect technological innovation levels in a nation, an industry, or a firm [24]. Within many economics and technology management disciplines, patent statistics have become well defined and established as a proxy factor for the measure of technological capacity and accepted as (albeit imperfect) statistics for innovation and R&D production in the absence of more robust measures [25–27]. Patent application and registration is also a system that companies and institutes use to perfect their technological innovations, and for legal reasons, patent-filing information is systematically registered by national government offices [28].

Much research has shown a positive relationship between patenting and R&D institute or company performance, provided that patent applications are weighted according to their quantity and quality [25, 29]. The patent statistics used in this research include the numbers, citations, registrations, and technological and international scope of the patent applications.

Table 2 summarizes an important set of statistics used to analyze technological strategy. The statistics were proposed for the evaluation of competitive positions in: number of patent applications (NP, PGPA, PCPA), patent citations (CI, CII, TS, TCT), number of families (NF), number of citations (NC), technology independence (TI), and revealed patent advantage (RPA) [30].

Table 2 Literature on patent statistics
Full size table

3 Research method

3.1 Research process

As shown in Fig. 1, our TLE based on patent statistics for mobile communications technologies involved six steps. First, we developed a model for the TLE and made an overall plan for its analysis. Second, we established a technological classification for mobile communications technologies, and we identified keywords and a search formula from the technological classification to draw out some target patent statistics. Third, we established a patent DB in mobile communications to analyze the TLE after screening out noise with the help of technical experts. Fourth, we identified the weights for patent evaluation statistics via a rational weighting method. Fifth, we computed the real TLE by using weighted patent statistics. Finally, we investigated validation of the TLE model and discussed the results, thereby suggesting how to use the results to create an R&D policy strategy.

Fig. 1
figure 1

A research process for TLE

Full size image

3.2 Patent statistics for TLE

We defined a Patent AIMCFootnote 1 model for a TLE based on quantitative and qualitative patent statistics. The patent statistics used in the Patent AIMC model are the PAI, PII, PMI, and PCI, as described in Table 3. The selected patent statistics are empirically derived from literature reviews and patent professionals survey which have shown a positive relationship between patenting and technological capability if patent applications are weighted according to their quantity and quality [25, 29]. The appropriate and weighted patent applications quantity statistics are patent activity and intensity index. Also, the appropriate and weighted patent applications quality statistics are patent market-power and citation index.

Table 3 Selected patent statistics for TLE
Full size table

A TLE based on the Patent AIMC model can be computed by Eq. (1).

$$\begin{aligned}&\hbox {Patent AIMC} = \hbox {W}\times \hbox {P}^{\prime } = ({\hbox {W}_{1} \times \hbox {PAI}})+ ({\hbox {W}_{2} \times \hbox {PII}})\nonumber \\&\qquad \qquad \qquad \qquad +\, ({\hbox {W}_{3} \times \hbox {PMI}})+ ({\hbox {W}_{4} \times \hbox {PCI}}),\nonumber \\&\hbox {where W}=({\hbox {W}_{1}, \hbox {W}_{2}, \hbox {W}_{3}, \hbox {W}_{4}}),\hbox {P}=({\hbox {PAI},\hbox {PII},\hbox {PMI},\hbox {PCI}}).\nonumber \\&\hbox {W is a weight vector},\hbox {and P is a patent statistics vector}. \end{aligned}$$
(1)

3.3 Determination of weights for patent statistics

As for any method involving nonlinear values and dimensionless processes, the weights of each statistic need to be determined, the goal being to reflect the function and significance of the various evaluation statistics in multiple-decision problems [31]. Methods for the determination of weights fall into two categories. One category involves identifying weights by data analysis, determining the weights via correlations or the variations between statistics. Examples include the mean squared method, factor analysis, and using average or equal means. This approach avoids deviations caused by human randomness factors, but fluctuations can depend on the data. The other category involves identifying weights by a related-expert survey, determining the weights mainly via comprehensive expert-consultation scores. This approach has some subjectivity but has been used in a wide variety of fields that have given rational solutions. Examples include the analytic hierarchy process (AHP), the Delphi Survey, and questionnaire investigations. To acquire generality and to avoid variability caused by giving weights that depend on specific data, we had to select a technique that distributes the weights for TLE statistics rationally.

Satty [32] originally developed a pairwise comparison method based on the analytic hierarchical structure of the target problem. An inverse pairwise comparison matrix was devised based on a numerical evaluation scale of 1–9 [32]. In various studies of uncertainty with vagueness, impression, and complexity, expert opinion can provide a rational method by which human knowledge for making decisions is incorporated in an adaptive and flexible manner [33–35]. Therefore, the weights for the TLE statistics could be rationally obtained by an AHP technique based on expert systems.

4 Results and empirical analysis

4.1 Patent statistics for TLE in mobile communications

We performed a TLE for the US, EU, Japan, China, and Korea using patent statistics for 3G, 3G transitional, and 4G mobile communications over the past decade. We analyzed patent data for public, notified, or registered patent applications filed with the national patent offices in the US, EU, Japan, China, and Korea from January 1, 1999 to December 31, 2011, with respect to the inventors’ nationalities. Table 4 shows an overview of the patent statistics analysis for the TLE among nations in mobile communications.

Table 4 Overview of patent statistics analysis
Full size table

The analysis targets for technology classification in mobile communications are 3G, 3G transitional, and 4G. The general definition of 3G systems was formally completed by the International Telecommunication Union Radiocommunication Sector (ITU-R) in 1997. To create a collaboration body for the various telecommunications associations, the 3G Partnership Project (3GPP) was established in 1998. 3G mobile communications are implemented by a generation of network standards for mobile phones and communications services fulfilling the International Mobile Telecommunications-2000 (IMT-2000) specifications. To meet the IMT-2000 standards defined by the ITU, a system is required to provide peak data rates of at least 0.2 Mbit/s[36–39]. The 3G transitional release, sometimes denoted 3.5G, 3.75G, or 3.9G, provides mobile broadband access of several

Mbit/s to smartphones and mobile modems in laptop computers and tablets [40–42]. The 4G mobile communications standard is the successor to the 3G and 3G transitional network standards [43]. The 4G system provides mobile ultrabroadband Internet access to laptop computers via universal serial bus (USB) wireless modems, to smartphones, and to other mobile devices [44–46, 57]. Two recent commercial 4G systems are the Mobile WiMax standard (firstly in Korea in 2006), and the first-release long-term evolution (LTE) standard in Scandinavia in 2009. In the US, Sprint Nextel deployed Mobile WiMax networks in 2008, and MetroPCS was the first operator to offer LTE services in 2010. USB wireless modems have been available since 4G’s inception, while WiMax smartphones have been available since 2010 and LTE smartphones since 2011.

Based on the published literature, Table 5 shows the technology classifications of mobile communications standards used to evaluate technological levels among nations [44].

Table 5 Technology classifications of mobile communications
Full size table

4.2 Weights for TLE statistics using patent statistics

We conducted a survey of selected experts and calculated weights for the TLE statistics based on the AHP technique with an expert survey using the ‘Expert Choice ver. 2000’ program [56]. The subjects of the AHP survey were 48 experts from professional companies, research institutes, patent agencies, and universities. They were surveyed online during the period November 22, 2012 – November 24, 2012. Table 6 shows the resulting weights for the patent statistics used in the TLE.

Table 6 Weights for the Patent AIMC model for TLE
Full size table

4.3 Empirical results for TLE in mobile communications

We used a keyword extraction method to identify those public, notified, and registered patents whose 25,623 applications had been filed with the patent offices in the US, EU, Japan, China, and Korea from January 1, 1999 to December 31, 2011 with respect to technology classifications in mobile communications. We analyzed and evaluated four patent statistics—namely, PAI, PII, PMI, and PCI—with respect to the inventors’ nationalities.

The results of evaluating the Patent AIMC in overall mobile communications technology are given in Tables 7 and 8, showing that the US is the leader (100.0 %), the EU is second (95.6 %), Korea is third (76.9 %), Japan is fourth (75.4 %), and China is fifth (60.9 %). Specifically, the US is stronger than other nations in patent activity, the EU is stronger than other nations in patent citation, Japan is stronger than other nations in patent market power, and China is stronger than other nations in patent intensity. In 3G mobile communications technology, the EU is the leader, but the US is the 3G transitional and 4G leader.

Table 7 Aggregate patent statistics in mobile communications, 1999–2011
Full size table
Table 8 TLE among nations, based on patent statistics for mobile communications, 1999–2011
Full size table

Korea had fourth position in 3G, but second position in 3G transitional and 4G mobile communications technologies according to the strategic investment of national and industrial R&D funding to mobile communications. In R&D strategy of Korea with the goal of having the highest mobile phone and higher mobile systems market share from the viewpoint of patent AIMC, we could suggest more amount of national R&D funding to beyond 4G or 5G mobile communications technologies to create and acquire higher patents market power and citation like US, EU, Japan.

4.4 Comparison and validation

We checked the consistency of the TLE measures, by performing an empirical validation study. We compared the Patent AIMC and an Expert Delphi Survey among nations given in Table 9. The Expert Delphi Survey included reallocated results for selected mobile communications technologies from KEIT’s ‘Research report on total industrial technology level in 2011’, and the strategic domain of next-generation mobile communications was surveyed by 495 researchers, professors, and industrial technicians in Korea [22]. Although we dealt with typical interval data, the small sample did not justify using a normal distribution. Therefore, our method of construction of the statistics allowed only comparison between the rankings, and not absolute values. Accordingly, we checked the similarity between the Patent AIMC and Expert Delphi Survey with respect to the rankings of nations. For this purpose, a rank correlation (Spearman statistic) was employed because it did not require a normal distribution of data.

Table 9 Relative score summary for the Patent AIMC and Expert Delphi Survey among nations in mobile communications, 1999–2011
Full size table

Table 10 shows the correlations among the rankings for the nations available for each pair of statistics. This allowed two considerations of all nations in the Patent AIMC–Expert Delphi Survey. As shown in Table 10, the Patent AIMC and Expert Delphi Survey were mutually interdependent, showing that the correlation (0.727) was significant at the 0.01 level (2-tailed). Therefore, although different TLE methodologies lead to somewhat different results in general, we found that the Expert Delphi Survey could be replaced rationally by the Patent AIMC.

Table 10 Spearman rank correlations for the Patent AIMC and Expert Delphi Survey with respect to mobile communications
Full size table

5 Conclusion and discussion

5.1 Limitation

The TLE method based on patent statistics described in this paper has some limitations. First, we did not thoroughly justify the patent statistics (Patent AIMC) because we considered only those quantitative and qualitative patent statistics that had simple, specific, distributed, and relative features described in the literature [45]. Second, we identified subjectively the weights of patent statistics by an AHP method via a selected-expert survey. Data-oriented methods such as the mean-square method and factor analysis avoid some variations caused by human randomness factors, but they also contain fluctuations that depend on the data. We therefore have to review and consider further the weighting of TLE statistics. Third, the patent data used our empirical research were incomplete for mobile communications technologies. Patent announcements usually appear two years after patent applications and are then only gathered and screened by keyword extraction methods using technology classifications for mobile communications technologies.

5.2 Conclusion

In this paper, we show that a TLE based on patent statistics can lead to a consistent relative evaluation of the attainment level of technology, and this evaluation can be used to identify the technological advancement among nations. Furthermore, these methodologies can provide a more objective TLE than subjective Expert Delphi, or peer-review systems. Our proposed method would be extended to not only mobile communications field but also other technical fields like robotics, software, semiconductors, etc. Our proposal involving synthetic TLE statistics (the Patent AIMC) using patent statistics at a national level could also be applied to researchers, institutions, companies, and technological domains.

Our empirical research results present a faithful and consistent picture on the national status of innovative activities and performance in mobile communications technologies among major nations of the world. In overall mobile communications technology, the US is a technological innovation leader, the EU is second, Korea is third, Japan is fourth, and China is in last place. In 3G mobile communications technology, the EU is the technological innovation leader, but the US is the 3G transitional and 4G leader. In particular, in the Asia zone, Japan was a leader in 3G, but now Korea has more innovative power in 3G transitional and 4G technologies than Japan and China.

5.3 Further study

The validity of the proposed TLE statistics and weights are not sufficiently checked by comparison with other studies. In future work, we plan to establish consolidated patent and bibliometric statistics for measuring national technological innovation and scientific advancement by more literature reviews and statistical data analyses in various domains.

We considered only a TLE based on patent statistics in mobile communications. We are eager to review and compare it with statistical analyses of levels of production, sales, market shares, trades, and domain sites in the mobile communications industry. Each nation follows a different path in science research and technology development because of differences in R&D policy, professional human resources, R&D funding, and infrastructure, etc. Therefore, a TLE should consider the characteristics of each nation’s science and technology and develop strategies that are tailored to each nation’s status in science and technology. The identification of technological and industrial features for use in a TLE would also be a practical and interesting topic for future research.