Keywords

1 Introduction

Studies of industry research labs show that institutions that collaborate with other institutions tend to produce more research and are generally more innovative [1]. Examining such collaboration and innovation patterns in institutional settings, we compared universities and industry research labs specializing in human-computer interaction (HCI). Research in HCI has greatly influenced everything from graphical user interface prototypes to the notions of ubiquitous computing, and the HCI field at large continues to dramatically influence trends in modern computing. We find that institutions doing research in HCI that actively collaborate with other institutions (both universities and industry research labs) have greater numbers of published research papers (productivity), and possibly higher impact (innovativeness).

1.1 Patterns of Collaboration and Innovation Within HCI

HCI’s origins can be traced back to the 1970s when prototype computer systems such as Xerox PARC’s Alto system started to emphasize human factors in computing. HCI research activity has seen steady growth over the past few decades and this is reflected in the increasing popularity of ACM’s Computer-Human Interaction conference and published papers in the field of HCI [2]. Major contributors to HCI research include Carnegie Mellon University, Georgia Institute for Technology, IBM Research, Microsoft Research and Stanford University. Many theories and concepts from these institutions have defined HCI and continue to influence computing today ranging from research in user interface software development to ethnographic techniques adopted for computing [2].

Top institutions are recognized for their research prominence by word of mouth reputation, citations, and so on; however, we attempt to show that these institutions are partially successful because of inter-institutional collaboration. Collaboration not only enhances innovation, but also can help an institution become more central and influential to a field. A number of studies show how centrality is created and why it is useful. Suarez and Utterback [3] discussed how alliances are formed between institutions when a dominant standard is established. At research universities, often the dominant players in a research area are hard to surpass and these key players often remain at the forefront of an established research area for decades. These key research universities often get the best faculty and students. In industry, Cusumano and Nobeoka [4] found that research and development (R&D) used to shape business strategy has a significant impact on overall financial performance of companies. For firms involved in commercial R&D, this means that R&D needs to be an integral part of product planning and strategy [1, 5].

2 Methodology

Network analysis and information flow visualization are useful techniques to help understand influence and centrality with respect to research activity. Polites and Watson [6] studied the influence of journals within the information systems discipline using social network analysis and information flow. Polites and Watson found Communications of the ACM to be one of the most influential and respected publications in the field and demonstrated that prestige does matter in the sense that core journals have substantial effects in influencing the research of many related journals in closely related fields. Therefore, centrality in network graph visualizations is a good indicator of influence.

To discover collaborative relationships within the domain of HCI, we used data from the Association of Computing Machinery’s online digital library (ACM DL) to obtain author and institution collaboration data by searching for the keyword “HCI” and grouping results by institution. From each institution, ACM DL lists the top 100 contributing authors for publications affiliated with that institution based on a keyword. Authors are listed in descending numerical ranking, with the authors associated with the most papers at the top of the list. Authors can be either a primary author or a co-author of a paper to show up in an author ranking associated with an institution. Using primary author and co-author institution affiliation data we linked institutions by collaborating authors. Authors may be affiliated with multiple institutions since one author may show up in different institution rankings. We used D3 for Data-Driven Documents, a data visualization JavaScript library to create our network visualizations. This tool is commonly used for data visualization of social networks, amongst other purposes [7]. D3’s primary use is for data visualization so deriving relationships between data is limited to spatial analysis and approximating the proximity of nodes in the graph. Because of how D3 animates graphics there is some variability in how graphs are rendered. Despite these limitations Weng and Menczer [8], Wu et al. [9] and others have used the tool for social network data visualization.

3 Analysis and Key Findings

This analysis examined the top 20 institutions contributing to HCI research analyzing the profiles of approximately 2000 authors and co-authors contributing to the field of HCI research. We compared the social network data visualizations to a ranked table of institutions publishing HCI research (Table 1) to provide further analysis of top HCI research institutions.

Table 1. Institutions and the number of corresponding HCI papers. 2014 ACM Digital Library
Full size table

We used author publication count data and author institution affiliation data to create social network visualizations. In a social network visualization, each node represents an author with the size of the node representing the number of papers an author published as a primary author or co-author. Authors, nodes in the network, are clustered together by institution. Institutions are connected to each other using co-author relationships. We created social network visualizations on the top 10 and 20 institutions publishing HCI papers. Collaboration trends and connections between institutions became visible through these visualizations.

3.1 Analysis of Institutional Collaboration

The author collaboration visualization by the top 10 HCI institutions (Fig. 1) captures collaboration and influence, showing the most collaborative institutions at the center of the graph. Each node in the social network graph visualization represents an author, where a node’s size is based on the number of papers an author has published. These nodes are grouped together by institution so each author is affiliated with a parent institution. Authors are affiliated with their institution and their co-author’s institution, creating connections between collaborative institutions. The top 10 institutions publishing HCI research show distinct patterns of collaboration and influence. The most prolific authors are the largest nodes in the graphs—an indicator of influence.

Fig. 1.
figure 1

Author collaboration amongst the top 10 institutions involved with HCI research

Full size image

There are high degrees of collaboration between Carnegie Mellon, Georgia Tech and Microsoft Research. Georgia Tech is the most central institution in this visualization, demonstrating that it is one of the most collaborative. Nokia collaborates a great deal with Eindhoven University of Technology in the Netherlands, perhaps indicating that geographic distance helps these two institutions collaborate a great deal. University of California, Irvine, is also fairly central in the graph, indicating it also does quite a bit of collaboration with other institutions. Finally, Stanford University has many prolific authors in the field of HCI, but there appears to be little collaboration at Stanford occurring with other institutions within the top 10 HCI publication contributors.

Expanding social network analysis to the top 20 institutions involved in HCI research (Fig. 2), some of the patterns of collaboration and influence from the top 10 institutions become clearer. The pattern of geographic collaboration within clusters of innovation becomes more evident. There are clear geographic patterns to collaboration. Institutions based in Europe, for example, collaborate with other European institutions more than they do with American based institutions. Even within the U.S., Pacific Northwest based institutions, such as Microsoft Research and the University of Washington, Seattle seem to collaborate more with each other than with other institutions elsewhere in the country. The University of Nottingham collaborates a great deal with University College London presumably because they are both in the U.K., and so on. Porter [10] found symbiotic, collaborative relationships evolve within many industries based on geography and economies that evolve within a region, and this pattern appears to hold true of institutions conducting HCI research.

Fig. 2.
figure 2

Author collaboration amongst the top 20 institutions involved with HCI research

Full size image

Some of the most influential institutions tend to be able to collaborate across geographic clusters. Carnegie Mellon and Georgia Tech are at the center of the social network graph, demonstrating their centrality and influence. Stanford and UC Berkeley, both in the San Francisco Bay area, also collaborate a great deal with each other, but appear to be some of the least collaborative institutions within the top 20 HCI researcher publishers. Because research tends to be very specialized even within the field of HCI, part of this could indicate that those who are in the center of the social network graph do the most interdisciplinary work or have the most diverse set of research interests. Stanford and UC Berkeley, for example, do a great deal of research in HCI with computer science centric topics, whereas UC Irvine focuses more on the social science aspect of HCI [2]. Given Georgia Tech and Carnegie Mellon are more central in the graph indicates that they are perhaps doing both computer science and social science research within HCI.

Another interpretation of centrality could have to do with the immediacy of impact of research. Institutions on the periphery of the social network graph with prolific researchers could indeed be conducting highly impactful research, but researchers at those institutions may be doing research that is many years away from having an impact [3]. Leonoard-Barton [11] has shown that while institutional specialization can increase impact, it can also lead to rigidities that can impede innovation.

3.2 Corporate Research in HCI

While it is common for research labs in private industry to collaborate with universities, firm-to-firm collaboration solely within industry is another matter. Intellectual property and proprietary research tends to be more difficult to disseminate externally due to more legal restrictions and perhaps the fear of losing a competitive edge [1, 5]. Not surprisingly, Microsoft Research in Cambridge, England collaborates a great deal more with Microsoft Research in Seattle, Washington than with any arm of IBM. Similarly, IBM researchers collaborate a great deal with their respective corporate labs.

The author collaboration visualization by the top three research labs (Fig. 3) examines collaboration amongst research labs in private industry and shows that this type of collaboration is more common than one would expect. In fact, avoiding these types of firm-to-firm research lab collaborations appears to put more independent labs outside of the purview of influence in the HCI domain, limiting the influence and depth of research understanding of the firm. Perhaps not surprisingly, research labs that are affiliated under the same corporate parent collaborate with each other a great deal.

Fig. 3.
figure 3

The top three corporate research labs contributing to HCI research

Full size image

Microsoft, IBM and Nokia rank near the top of corporate contributors that publish research in HCI. Google is below average in terms of its contributions to publishing within the HCI community. It ranks 65 in HCI publications with 111 publications. Nokia is one of the top HCI publication contributors, even surpassing Microsoft Research Cambridge. Figure 4 shows the top 10 corporate research labs conducting HCI research.

Fig. 4.
figure 4

The top 10 corporate research labs contributing to HCI research

Full size image

Corporations are taking advantage of clusters by positioning their research labs in areas where there is a great deal of technological innovation [10]. Silicon Valley, Seattle, New York and Cambridge, England are all thriving technology clusters. IBM, Microsoft and Nokia have all opened research labs in these clusters. While there are benefits from locating a research lab within an existing cluster, this does not necessarily mean that all the benefits of the cluster will benefit the research lab [10, 12]. Silicon Valley arguably has more companies, human capital, financial capital and social networks that are aimed at growing technology than anywhere else in the world. Corporate research labs based in technology-oriented clusters don’t necessarily seem to benefit from being in stronger clusters. IBM T.J. Watson Research Center in Yorktown Heights, New York had 216 publications versus IBM Almaden Research Center in San Jose, California with 109. Silicon Valley is typically viewed as the world’s center of innovation and clusters with strong ecosystems are supposed to foster innovation, yet IBM’s Silicon Valley research center is producing half the amount of HCI oriented research as its New York counterpart. Microsoft Research Cambridge, England produced 189 publications versus U.S. based Microsoft Research in Redmond, Washington that produced 387 publications. In the case of corporate research labs what seems to matter is not necessarily being in the best technology cluster, but rather proximity to corporate headquarters. IBM headquarters are in Armonk, New York. Microsoft is based in Redmond, Washington.

3.3 Company-University Collaborations and Research Productivity

One characteristic that becomes clear from a cursory analysis of collaboration between universities and research labs is that more collaboration is an indication of better corporate research lab performance in terms of HCI publications. More often than not firms are unwilling to share information when it is known how to apply the information in some advantageous way in the market [3, 13]. Typically, it is information with unknown value or seemingly insignificant value that is shared in collaborative endeavors. In other words, firms are only willing to share their lemons, so to speak [1]. Perhaps it is this mentality of conservatism and restriction of information intended to save a firm that ironically leads the firm to lose its competitive position by not being able to collaborate as effectively with partners due to a lack of useful information to share in a collaborative endeavor.

Cohen and Levinthal [14] argue that research and development can help increase a firm’s absorptive capacity—their ability to interpret and apply research. Clearly, IBM and Microsoft’s research in HCI have increased their absorptive capacity as much HCI related research coming out of the labs have been applied towards improving a product’s user experience or towards implementation of innovative user interfaces. Research within the broader HCI community has also influenced work at Microsoft and IBM. For example, research in social network analysis coming from the University of Michigan and Carnegie Mellon University has been applied to research at IBM and Microsoft. Both firms are prodigious producers of HCI related publications and rank at the top of the list of HCI research contributors. Nokia is also a top producer of HCI publications, but arguably, it’s innovation in its research and development labs have not positively affected the company’s bottom line in the mobile phone industry.

Nokia’s organization structure may not have the same direct connections to product that IBM and Microsoft have. IBM’s consulting model has some emphasis on making high value products that commercialize research done within R&D labs. For example, the company spent millions of dollars on R&D for its Watson technology that is able to answer domain specific questions. Watson is best known for competing with humans in Jeopardy and winning. However, the technology has since been commercialized and applied towards medical and financial domains to help answer complex questions [15]. Microsoft has done years of HCI research related to touch interfaces and computer vision. This work has been commercialized in its gaming division and implemented in its Xbox gaming consoles [16]. While long-term research with no commercial application is done at both IBM and Microsoft, there is a culture and directives in place that encourage researchers to find ways to create products that can be commercialized. Product groups are also encouraged to work with researchers to find commercial applications of research products. This sort of organizational structure does not seem to be in place at Nokia. Despite being a major contributor to HCI publications, the firm has not been able to apply this research to commercial products.

Pisano [1] showed that collaborative arrangements between companies can help improve research and development performance. These developments in R&D can help firms develop commercial products faster and efficiently. Firms tend to make these collaborative research arrangements in areas where their expertise is complimentary, for competitive reasons [11, 17]. While outside technology has a higher rate of failure than internal technology, the overall domain-specific knowledge a firm generates through collaborations can help extend expertise to many areas, which can benefit the firm overall. This model of research collaboration seems to work well with companies collaborating with universities.

An indicator of these collaborations between a university and company are co-authored papers and journal articles. Microsoft and Carnegie Mellon University co-authored 478 HCI related publications. IBM and Carnegie Mellon authored 292 HCI publications. Nokia only authored 88 HCI publications with Carnegie Mellon. Collaborations between those same companies and Georgia Tech numbered 255, 153 and 73 respectively. While Nokia is a strong producer of HCI publications in general, it is evident that its university collaborations are far less than Microsoft and IBM. Microsoft has the best track record of working with universities on HCI publications. These numbers may be an indication of the core capabilities and rigidities within each respective company.

Comparing collaboration with Georgia Tech on HCI papers to the total number of HCI papers, 36 % of Microsoft papers have university collaborators, 33 % at IBM and 27 % at Nokia. Similar numbers appear for collaboration with Carnegie Mellon University. As an overall percentage of its HCI publications Nokia does fewer university collaborations than Microsoft or IBM. Work in certain parts of an organization may not be valued as much as in other parts of an organization due to incentive systems and the like [11]. Nokia may not have an incentive system in place that values sharing of information and collaborating with outside universities. Firms that do not collaborate are literally on the edges of network visualizations, while those that do collaborate are in the center of the visualization, benefiting from knowledge coming from a variety of institutions that collaborate within the HCI research ecosystem.

4 Assessing the Strength of HCI Programs

Measuring the quality of HCI programs is in many ways a highly subjective process. Because the research foci of university labs may differ greatly from one to another, sometimes comparing HCI programs from one institution versus another is like comparing apples and oranges. However, a general indication of the research strength of a program can be demonstrated through the number of publications it produces. These data can be captured using data from the ACM DL. ACM DL provides publication data from the top 100 institutions that publish HCI research simply by searching for the term HCI and using institution as a criterion to narrow search results. Sorting by this list provides a rough ranking of research strength.

The top five contributors in the research area of HCI (from ACM Digital Library 2014) in order of rank are Carnegie Mellon University, Georgia Institute of Technology, Microsoft Research, Stanford University and University of Washington, Seattle. From 1977 through 2014, the top 100 contributors to HCI published 15,631 papers and journal articles pertaining to HCI. The top five contributors published 2,427 or 15.5 % of the total amount of publications. The next five contributors published 1,350 papers, or about 8.6 % of total contributions or 55.6 % of the amount of the top five contributors. Institutions on average have 156 publications and have a median of 131 publications.

Some institutions publish under different names though they are affiliated with a larger organization. For example, Microsoft publishes under Microsoft Research, Microsoft Research Cambridge and Microsoft. IBM also publishes under its various research labs including T.J. Watson Research Center and Almaden Research Center. MIT publications fall under the Massachusetts Institute of Technology and the MIT Media Laboratory. Combining total publications of affiliated institutions increases their ranks significantly. MIT, for example, rises from a rank of 21 to 5, replacing University of Washington, Seattle, when its publications are combined with MIT Media Lab (ranked 33). Microsoft and IBM move to 2 and 4 respectively when their affiliated institutions are combined, raising their rankings from 3 and 41 respectively.

By using the simple metric of publication ranking it becomes clear that some institutions are substantially more prolific than others. Few other institutions match the sheer quantity of publications coming from Carnegie Mellon and Microsoft. Georgia Tech and IBM come close, but would have to increase their output close to 30 % to compete with the top two contenders. Georgia Tech has 4,875 academic faculty and staff versus 1,442 at Carnegie Mellon. Similarly, IBM has four times the number of employees as Microsoft, 434,246 versus 100,932 respectively. Yet Microsoft has more than a third more publications than IBM. HCI research is prioritized at leading institutions, but certainly having nearly quadruple the faculty resources or employees should help Georgia Tech or IBM at least have similar output to Carnegie Mellon or Microsoft, though this is not the case.

5 Conclusion

We used quantitative data of academic paper citations and collaborations from the Association of Computing Machinery’s online digital library (ACM DL) to construct social network graph data visualizations. We used published conference and journal papers as a proxy to indicate the influence and magnitude of research conducted at the institution. Through these visualizations it became evident which institutions within the HCI community are most central, influential and collaborative; and similarly, which institutions tend to not be as influential and collaborative. These visualizations corroborate some opinions from expert researchers within the HCI community on which institutions are most influential and the type of work that is most important [2]. The social network graph visualizations show greater connections between institutions that collaborate more and these institutions appear closer to the center of the social network graph.

Visualized social network publication data based on quantity of publications and incidence of joint activities clearly shows that the most collaborative institutions, and those that are the most highly regarded in rankings, are at the center of social network visualizations. Visualization data show collaborative institutions at the center and non-collaborative institutions in the periphery. Collaboration occurs within clusters to some degree and geographic clusters evident in collaboration. Merely having highly prolific authors that publish a great deal and do not collaborate leads to less impact and influence, if influence is indicated as being closest to the center of a social network graph visualization. From these visualizations and data it is evident that there is a relationship between collaboration and how prolific an institution is in publishing papers related to HCI. In future research we hope to apply similar network analysis to other measures of influence such as paper citations. There are opportunities to apply this type of analysis to other fields within computer science to understand influential institutions in other areas of specialization.