1 Introduction

One of the mandates of science is to create generalizable results, which are meant to be universal, and this mandate is often conflated with the deletion of local contingency.

Star (1985, p. 413)

Studies of science and technology (Star 1985; Latoure and Woolgar 1986) have pointed to the fact that while final outcomes of any scientific work, be it scientific articles, functional technologies, or newspaper reports, inform about scientific facts and truths, a large number of procedural insights and local contingency are often filtered out. In particular, how scientists come about making sense of their data, images, or other type of information and what cognitive processes manifest themselves is rarely reported. The conduct of scientific research involves a varied set of cognitive processes and skills. Some of these are internal processes of the sort that have been the focus of the traditional cognitive science for decades, such as, categorization, reasoning, problem solving, and analogy formation. Others are processes that take place when information is propagated across different representational media, such as documents, papers, and other type of external representations (e.g., Hutchins 1995; Latour 1986; Sellen and Harper 2002). Researchers have recognized that cognition is a socially and culturally embedded phenomenon that is situated and distributed between people concerned (Engestrom and Middleton 1996; Hutchins 1995; Suchman 1987; Schmidt and Wagner 2002). Cognition is as much rooted in mental processes as it is in the external world of objects, artifacts, and social practices. In particular, the importance of external representations in reasoning and knowledge construction has been noted by many researchers seeking to understand the nature of the science (Latoure 1987; Lynch and Woolgar 1990; Hutchins 1995).

The aim of this article is to study visualization practices of scientists working in the tissue engineering research. As Lynch (2006, p. 27) describes, ‘Rather than being a discrete, well-bounded aspect of science, visualization is intertwined with observational and experimental practices, literary representations, and methods for disseminating scientific results. It might be said that an attention to the practices and documents through which researchers visualize phenomena is a way to gain perspective on the field of scientific practice instead of singling out a particular aspect or phase of it.’ Extracting how scientists generate knowledge and facts from the visual information that they work with is highly desirable to develop rich accounts for cognitive processes that are at work. In particular, we are interested in understanding how different visual representations play a role in facilitating scientists’ sense making. Being a multidisciplinary field of research that draws a lot of knowledge from biological and life sciences, tissue engineering is a highly visual field (Minger 2006). It deals with replacing or regenerating human cells, tissues, or organs to restore or establish normal function. Regenerative medicine (a broader term used for tissue engineering) offers unique opportunities for developing new therapeutic approaches to prevent and treat debilitating and life-threatening diseases such as cancer and HIV, and new ways to explore fundamental questions of biology (van Blitterswijk et al. 2008). The current optimism over potential stem cell therapies is driven by new understandings of genetics and developmental biology. Importantly, this field brings together professionals from a variety of disciplines, including biology, chemistry, material science, and engineering.

With an ethnographic orientation, we studied eight scientists working on cell culturing experiments by capturing their laboratory sessions and using semi-structured interviews. We studied reasoning behind these scientists’ visualization practices and were able to identify and track the semiotic structures that scientists bring into coordination with the images in the process of knowledge construction. We used insights from Edwin Hutchins’ Distributed Cognition framework to take into account the internal processes and external processes that help in the cognition (Hutchins 1995). We believe that visualization is a process grounded in bodily activities, where practitioners act, feel, and articulate the information that is presented either on the digital screens or on the microscopy displays. As Alač (2008) suggested, vision is an embodied process because acts of seeing are produced by the enactment of purely visual information in the world of embodied actions.

From our field study, we describe our results in two themes: (1) multiplicity and (2) physicality. The multiplicity theme refers to the nature of scientists’ visualization practices, where they utilize a varied set of scientific methods and tools for creating and analyzing multiple visual representations, to be able to make better sense of their work. This theme refers to the fact that visual representations ‘travel’ through multiple sources to allow scientists support their experimental reasoning and sense making. The physicality theme refers to a large set of practices where scientists’ bodily activities such as gestures and external representations such as charts or notes help in understanding the visual information. Such practices allow scientists to use their bodily actions and external objects as cognitive artifacts (Norman 1991) that help in better communication and collaborative cognition.

In the rest of the paper, we will provide a short background on tissue engineering research and briefly discuss related work on visualization practices in the scientific world. Next, we will describe our fieldwork settings, selection of participants, and our results. We will provide several examples of scientists’ visualization practices and show that multimodality of such practices plays an important role in scientific visualization. Overall, we aim to bring awareness and attention of the HCI practitioners to this underexplored line of research.

2 Tissues engineering

According to the U.S. Department of Health and Human Services (HHS 2012), annual US healthcare costs for 2011 were more than $2 trillion. Tissue loss and organ failure are considered to have played a major part in these costs with approximately 8 million surgical procedures being performed annually in the United States to treat these disorders. Tissue engineering is said to be an area of research that could bring better curing possibilities and decrease these expenses. According to the HHS, U.S. private sector has till date spent over $4 billion on regenerative medicine research. And, new initiatives such as Federal Initiative for Regenerative Medicine (FIRM) are expected to establish a global medicinal industry with an expected potential of $100–$500 billion worldwide by year 2020. With the potential of treating almost every tissue and organ of the human body, regenerative medicine is expected to solve problems related to major diseases of our current time: diabetes, heart disease, strokes, cancer, HIV, and so on.

Scientists working in regenerative medicine come from a variety of backgrounds: cell biology, material sciences, biochemistry, physics, chemistry, applied engineering, and many others. They facilitate each other’s work by utilizing each other’s expertise. In some cases, their practices change or evolve overtime. As the HCI practitioners, we have to understand these new, emerged practices and be able to design new tools to support these practices. Especially, since biology is a visually oriented field (see the next section), we need to provide implications for supporting these new visualization ‘techniques’ and ‘technologies.’

Although it would be outside the scope of this paper to provide a detailed introduction to tissue engineering research, we will briefly describe a specific aspect of tissue engineering called cell culturing, which is the main focus of this paper. For other aspects related to tissue engineering, we refer to a text book titled Tissue Engineering (van Blitterswijk et al. 2008). Cell culturing is a process by which cells are grown under controlled conditions, generally outside of their natural environment. It is one of the most basic steps in several of the tissue engineering experiments. Scientists normally start with a specific goal of research and a hypothesis related to cells or specific tissue that they want to grow using different procedures and techniques. Specific cells are selected to be used in vitro (outside the organism’s body) and external material is added to it, maintaining a specific environment by taking into account the timing, temperature, weight, and so on. In doing so, a scaffold surface is used where cells are supposed to be grown and biomaterials are used to focus on the specific parts of the cell that needs to be tested. Chemical membranes are often used that allow only specific materials to pass through cells. Biodegradable polymers are used as a carrier for inserting external materials inside the cell. Several versions of these cells are created with a variety of materials added into them to see which topology is best suited for scientists’ research. Based on different cases, different types of membrane and polymer techniques are used. For a cell to be cultured (grow or mutate), it could take from few hours to several days. Using different visualization and microscopic techniques, the cells are computationally processed and analysis is carried out at the end of it.

3 Scientific visualization in life sciences

Scientific visualizations are central to how natural objects are ‘revealed’ and made analyzable (Lynch and Woolgar 1990). They are not simple visual renderings of the object, but they show the object’s internal order and essential qualities (Lynch and Woolgar 1990), a characteristic which has to withstand questioning. Making the natural object visible, in this context, also means making it measurable (Lynch and Woolgar 1990), through representational means which, to be ‘objective,’ must retain stability, manipulability, and reproducibility.

…physiological function of the cell can be understood only in terms of the three-dimensional configuration of its elements……all biological phenomena, no matter what their complexity, can ultimately be accounted for in terms of conventional physical laws.

(Stent 1968)

Keller (2002) describes that all life scientists, to a large extent, have the main goal of understanding and explaining the biological development in individual organism, that is, ‘making sense of life.’ Life scientists use a wide range of models, metaphors, structures, and tools to explain this process. The visual and multidimensional representations related to human biology (e.g., images, models) provide great insights into understanding the complexities of different organisms. Moreover, during most experiments, life scientists have to observe spatiotemporal information related to reactions, growth, or mutations within different entities of organisms (ranging from muscles to micro-molecules). This information can be observed in an efficient way through visual means. It has also been claimed that nearly all activities at cellular and molecular levels depend on form, color, and physical structures (Altman 1998).

Historically, it has been shown that in order to better understand biology, life scientists need to think in a three-dimensional way. For example, Francoeur (2000, 2002) gave a detailed account on the role of materiality of molecular models and suggested that a three-dimensional representation of molecular structure can reveal several mechanical properties, support both qualitative and quantitative reasoning about the structure, and allow us to articulate theories and concepts about them. To him, textual and statistical results about biological information reduce many important phenomena about human biology, which can be better explained through a more physical approach. In addition, to deal with human biological problems, life scientists use different graphical structures related to cells, genes, DNAs, and molecules. Many of their decisions are based on the embodied information (such as shape, size, color, and behavior) of these structures that are used and explored during biological studies. See Fig. 1 for some frequently used visual information by biologists. Based on their specific field of research, life scientists use different computer-based or analogue tools (microscopes) to carry out visualization practices—sometimes even combining more than one device. They require dynamic and prolonged manipulation and articulations with these graphical structures.

Fig. 1
figure 1

Three different visual representations used frequently by life scientists. A 3D protein structure (left), human chromosomes (center), and chemical structure of cytosine (right)

Full size image

In the post-cognitive area, researchers have also realized that in addition to their visual characteristics, scientific visualization also relies on other external and multimodal aspects. The research of Alač (2005) on functional magnetic resonance imaging (fMRI) of humans has revealed that the movement of subjects and the scanner noise make such complex visual representation meaningful. She claims that such multimodal interaction between scientists and digital screens plays an important role in supporting human cognition. Her research also suggests that the coordination of various semiotic modalities such as images, talk, body movements, and gestures makes the scientific reading of brain images crucial. In her work to understand protein modeling, Myers (2008) shows that researchers use their bodies kinesthetically to manipulate and learn protein structures. Researchers’ bodywork enables modelers to animate their molecular mechanisms both on-screen and through elaborate gestures and affects. In this way, molecular mechanisms are enlivened as they are propagated between researchers in pedagogical and professional contexts. Monteiro (2010) has also shown that manipulation of digital images helps scientists make better sense of their visual representations.

4 The field study

Cell culturing (or tissue culturing) is a very fundamental step toward understanding the state-of-the-art research on tissue engineering. Using an ethnographic approach, we studied the visualization practices of eight scientists working toward supporting and engineering for cell culturing processes. Table 1 shows a list of our participants. It is important to note that within the multidisciplinary research of tissue engineering, one needs to understand working practices of not just the traditional biologists, who are involved in experiments but also the people who work closely with them—people who build biomaterials and membranes. Our attempt was to investigate people who are in one way or the other involved in the process of cell culture. We recruited these participants to have a variety in our data collection. However, we made sure that all our participants had done or were currently doing experiments that involved cell culturing.

Table 1 Information about the participants
Full size table

We mainly used two methods in our field study: semi-structured interviews and video recordings of experiments. In the interviews, we asked questions about scientists’ individual ways of looking at visual representations and how they understood these representations. We also asked questions about how they collaborated through these visuals, what type of tools they used to study representations, and what kind of methods and procedures they developed over the years. To be able to get a naturalistic insights into scientists’ work, we video recorded our participants’ laboratory sessions. We captured their laboratory experiments and simulation sessions, their reasoning processes, and their collaborative practices. Our aim here was to understand scientists’ enactment of creating, using, and interacting with different scientific visualizations.

This research calls upon the theoretical frameworks provided by distributed cognition (Hutchins 1995) and embodied cognition (Anderson 2003). Distributed cognition expands the unit of analysis for cognition beyond individual brains to include bodies, material structures, and social contexts of cognitive activity and provides a framework for examining the propagation of information through representational forms, such as spoken language, gesture, graphical models, text, and so on.

In our analysis, we began by inspecting the video recording and interview notes for themes related to scientists’ visualization practices. Our particular focus was on the multi-modality in visualization activities and how it helped scientists making sense of their work. Using an open coding technique (Glaser 1992), we developed two broader themes describing the characteristics of scientists’ visualization practice. In what follows we report the two themes that emerged through cross referencing the two sources of data and then discuss the mechanisms and consequences of visualization practices.

5 Results

During our field study, we observed a large set of visualization practices of tissue engineering researchers. In this part, we will show how utilizing different scientific and non-scientific techniques, our participants approached different visual representations. Our particular attention will be on the multimodal nature of such visualization practices. We divide our results into two broader themes that we observed from our analysis. These themes are multiplicity and physicality. In the following, we present our analysis via these two themes.

5.1 Multiplicity

Representations are essential to visualization and visual inference. Individual visual representations of biological objects or samples are static and momentary glimpses into complex biological structures (Gooding 2010). However, their multiplicity changes their nature from static inscriptions to dynamic and active participants in the visualization process, as we shall show in the section. This theme of multiplicity refers to a set of practices by which tissue engineering researchers develop a number of visual representations using different methods and techniques to make better sense of their scientific research.

Most often scientists make sure that they collect several samples to be able to quantify their final results and do statistical analyses. Often, due to the scientific nature of cell culturing research, scientists frequently collect images of their cell samples as they aim to observe changes in cell behavior over a given amount of time. Additionally, more than one procedure and techniques are followed on biological samples to explore their behaviors under different conditions. In a sense, a biological sample goes through multiple techniques and multiple microscopic devices to help scientists make better sense of their research. Scientists also use computer-based tools to quantify the raw data of cells that they have collected during the cell culturing process. These tools allow them to generate graphs, statistics and eventually make conclusions about their overall work. However, the visual information is considered the primary source as can be seen in the following comments by a scientist.

For us, the structures, the size, and the shape of a cell are very important. Also the relative dimension that is the ratio of the length and the breadth and sometimes in cases where I am using the FISH technique [Fluorescence in situ hybridization] then the amount of fluorescence that comes of out the cell is also important. There is also a post-processing involved from the graphics that we acquire in order to extract the information that we need.

From our field study results, we look at multiplicity from two angles: multiplicity of representations and multiplicity of scientists’ approach. In the following, we describe these two categories and discuss examples from our field study.

5.1.1 Multiplicity in representations

Visualization can be seen as an active process of manipulation and articulation that links sets of representations to their source data that need interpretation and to images that integrate the information the interpreted sources convey (Gooding 2010). By multiplicity in representations, we refer to a set of scientific practices by which scientists create multiple versions of visual representations, to be able to develop better understanding about their biological samples. Scientists ‘move’ between these representations to explore facts, generate knowledge, and communicate their reasoning and findings to others.

We start by describing an example where two scientists are performing histological analysis on cell culture. Histological analysis (McDonald et al. 1999) allows examination of cells and tissues by sectioning and staining, followed by examination using a range of microscopic techniques. Of the two scientists we observed, one was a senior level scientist who was training his junior colleague through this process. The process of staining was previously carried out by the senior scientist. In this excerpt, she is teaching histological analysis to her junior via a set of images that she captured during her staining experiments. In the following, we describe a small excerpt of their discussion about different visualization of their cell samples. The senior and junior scientists are referred to as S and J, respectively.

The above excerpt shows four different types of visual representations, generated from four different staining techniques, in order to verify whether the cells have cultured to a mouse ear scaffold in an appropriate manner. Figure 2a is a representation of cells using the Methylene Blue staining technique that indicates whether enough cells are attached to the scaffold so that further analysis can be carried out. Figure 2b shows growth of the cells and how they are interacting with each other. Figure 2c indicates the live and dead cells on the scaffold, using florescent microscopy. Figure 2d shows an image using the Phalloidin staining technique that can help the scientist to understand the detailed structure of the cells. These four procedures are typically applied in a linear fashion to be able to justify reasoning about a particular experiment. These different representations are used to support deductive reasoning in the experiment, that is, by ruling out undesired behaviors in cells, the scientists can warrant appropriate cell samples for their future needs.

Fig. 2
figure 2

Four different staining techniques applied to the artificial ear cells

Full size image

This example illustrates how different visualizations are produced to serve different purposes. Figure 2a provides rough estimation about the number of cells attached to the scaffold surface, and Fig. 2c allows the scientists to estimate live and dead cells on the surface. These two staining techniques help scientists to make a rough estimate about their cell samples and paved the way for carrying out further analysis. The understanding of color and color amount is enough for making such estimates. Whereas, Fig. 2b, d are the images where scientists can carry out precise calculations on the number of cells, their diameters, and their overall structures. These images are analyzed in a rigorous manner, and graphs and statistically significant results are generated. Hence, this excerpt provides an example of how different visualizations can allow rough as well as precise calculations. In scientists’ visualization practice, these staining techniques play a pivotal role. Staining techniques utilize different color schemes to highlight and/or differentiate between different entities within the cell culture that can be viewed with different types of microscopes. Representations generated through different staining techniques allow rough estimates that can be observed via different colors and in other cases support precise calculation of certain phenomena.

Manipulation of the original cell sample using external media (staining techniques) is central to this visualization exercise. Such a manipulation in a scientific discourse is composed of logical steps toward arriving at important understanding. Such manipulations of the ways in which certain aspects of the cell samples are displayed and highlighted present themselves as directly observable evidences that are not possible in other ways. These manipulations become appealing exactly because the cell samples in question are seen to materialize the dynamism and relationships expressed that are not observable in the original raw data. The handling of cell samples, which involves physically interacting with them, allows scientists to experience the expressive properties of the cell behaviors. And the fact that each ‘manipulation’ (using different staining processes) highlights a new behavior and adds to the previously observed behavior about the cells makes the whole visualization process dynamic. It is thus through only manipulating these cell samples that new evidence and new understandings of the processes in question are produced, and not through direct interaction or observation of the natural objects themselves.

To explore the communicative role of representations, the idea of ‘immutable mobiles’ is crucial to understanding how these cell samples are analyzed. According to Latour, the relevance of inscriptions in scientific practice is their ability to represent in unchanging and stable terms the central relationships of a natural object. As the excerpt showed, learning was an integral aspect of the discussion between the two scientists. While these cell samples incorporated different staining techniques and other theoretical assumptions, the captured images are used to support learning and allow discussion between the scientists. From a distributed cognition point of view, the excerpt shows how these cell images simulated the need of external representation that helped in supporting communication between the two scientists.

Here is another example of the multiplicity of representations. Figure 3 shows an image of a laboratory note from one of the scientists who participated in our study. The scientist has glued cell culture images from different stages of her experiment in her laboratory notebook with some description about the images. These images represented different topology and combinations of cell culture materials and were captured at different time-stamps. Her laboratory notes are full of such images and annotations. There are several reasons why this was done. In addition to keeping the record of all her activities in the laboratory and communicating results to her colleagues, such a spatial representation of these images played an important role in her cognitive processes. She comments:

Fig. 3
figure 3

A logbook page from our participant

Full size image

In my experiment, I have to collect cell images at specific times. This is actually the very basis of any cell culturing experiment. Collecting cell images at different times can tell us how cells behaved at what time and when they changed their behaviors. As you can see[referring to the image], this image sample with 6% gel density was captured at the 1 st hour [top left side] and this image [bottom right side] was taken at 4 th hour. You can easily see several black spots in the image taken at the 4 th hour. These are actually dead cells. This tells me that something happened between these two instances. By keeping such visual information in my notebook, I can get a quick overview of my experimental results and also inform my future experiments.

This example shows a very interesting practice of visualizing different images in a spatial order where scientists can easily move across different stages in the cell culturing exercise and be able to explore facts. Such a practice allows a scientist to go back to previously captured images of cell culture and inform their future decisions. This way, scientists’ ‘travel’ between these visual representations and their manipulation of different visualization practices become key here. As Merleau-Ponty (1962) argues that sensation and movement are intimately tied to visual understandings of form, it can be said that through the labor of constructing, manipulating, and navigating through different scientific images, scientists are literally able to come to grips with the form and function of cells, that is, being able to efficiently make sense of cell. The biological objects and models that are produced over an extensive period of time should not be seen only as marks or traces of information. Additionally, we believe that these visual representations have integrative power. The example in Fig. 3 shows that through such a practice, it is possible for scientists to integrate these images, at least cognitively, to be able to see the larger picture of their scientific experiment. The way these different representations, step-by-step, offered new insights by facilitating deductive reasoning shows the integrative nature of visual representations. Scientists’ moves between these images allowed them to see the convergence of these various representations and gain an overview of their work.

Overall, we observed that there were two reasons why scientists developed multiple visualizations and why their biological samples travelled through multiple machines and techniques. First, this multiplicity of visual representations helped them collect and present evidences of their scientific work, empirically testing some hypothesis or scientific models or using these images in their scientific papers and presentations. Second, these visualizations helped in reducing the complexity of the problem by lowering the cognitive load and helped them to make better sense of their work.

5.1.2 Multiplicity in scientists’ approach

Multiplicity in scientists’ approach refers to a set of practices by which scientists use different methods and tools to explore detailed facts about their biological samples. In order to carry out experiments and modeling, scientists use and deal with several heterogeneous media: physical as well as digital. Figure 4 shows a few devices that were commonly used by these scientists. Often they needed to use more than one device and perform different activities on biological samples that they were working on. Especially, scientists who work on understanding the biological processes used several simplification techniques such as Fluorescent In Situ Hybridization (FISH), Raman scattering, and so on. It was important to take into consideration various media used, not only to understand how scientists worked with different visual information related to cell culturing processes, but also for understanding how these distinct media engaged scientists’ bodies in different ways. Different machines, tools, and materials afforded different modes of interaction and manipulation and provided different kinds of insight into the cell culturing realm. We will focus on these bodily aspects in Sect. 5.2 where we will discuss several examples of such behaviors.

Fig. 4
figure 4

Images of different devices used in modeling and experiments. a Electron microscope, b scanning-electron microscope, c stereo lithography machine

Full size image

The following is a comment from a scientist who explained his views about using imaging techniques:

Images that we use bear a lot of importance in our work. These cell culturing images… they can be acquired by various modalities and techniques so there can be various ways in which we have to view these graphical images of cell culturing. We have fluorescent microscopic image, light microscopic image, electron microscopic image, atomic force microscopic images… All of these images bear different levels of importance and bear different kinds of information. These all depend on the aim of the research and method that we want to apply in our research.

Throughout the field study, it was observed that these scientists heavily used specialized microscopes, related imaging devices, and interactive computer graphics technology. These all required an ‘active’ and ‘prolonged’ handling and manipulation of the biological samples (experimental cell samples) and on-screen models throughout different cell culturing processes.

My work aims at forming bone cells from different materials in an in vitro setting. The final thing would be to bring it to the clinic, but before that there needs to be lots of testing done. Throughout my research project, I will have to interact with different samples of bones again and again. I have to analyze and understand the growth of bone cells using several types of microscopes and bio-reactors. If the growth is less than expected then I have to restart the whole procedure using different compounds and materials. Thinking about these cells and trying to understand the process of their growth is what I do most times while I am in the laboratory. Sometimes it makes me literally think as if I am inside the cell.

The way scientists interpreted their visual representations is also important to take into account. It was also observed that ‘getting the feeling’ about these cell functions was a habit among some expert workers. One of them said: ‘the quantified data provides us the final evidence about my work but I need to feel this object too (referring to a cell slide).’ Scientists always try to think ‘what is this structure saying.’ The cell chip or cell samples embody use of several different artifacts used for creating it. To these scientists, the resulting sample or simulated models are not just another outcome of their research, the labor these scientists have put in gives them a sense of feeling about how it will behave in a given situation. Here is a comment by one scientist:

I normally view these images at two levels: At Raw level, which is an unprocessed version of the cell behavior. And at Processed level that is computationally processed to generate detailed information about the cell behavior. This final data depends upon the raw data. So, I work iteratively on raw and final data. If the raw data is perfect then it can be a final data otherwise I need to capture the process again.

Interestingly, the cell culturing process was seen not limited only to a means of producing representations of cells, it was also an activity to train novice scientists’ ‘bodily-reasoning’ and ‘imaginations.’ To be able to think intelligently about these cell structures, they had to acquire skills to understand the function of a particular cell or the interactions between more than one cell type. Figure 5 illustrates an example of making a scaffold (skeleton model) of an ear of mouse, for the purpose of artificially generating a mouse ear. A scientist described the process of making such a scaffold as creating a building:

Fig. 5
figure 5

A metaphor of building (a) is used to model (b) and develop the scaffold for an ear (c)

Full size image

#: The way a building needs support from rods and pillars, I also think about designing a skeleton of an artificial organ. I use a micro plotter to create such a scaffold where cells will be seeded.

Scientists rarely just ‘look at’ patterns in their visualizations. They infer the existence of more complex regularities and causal mechanisms. Scientists manipulate what they are looking at to induce changes that generate new information about hidden structures and processes (as we saw in the example of Fig. 2). In this way, science extends knowledge by playing to human cognitive strengths and limitations, as well as by using cultural resources, social conventions, and techno-scientific systems. Scientists use a variety of images that visualize phenomena, visual representations of theories about phenomena, and models that display structure and connectivity. Such objects always combine visual and non-visual elements because scientific work requires representations that are hybrid (that combine verbal or symbolic expressions with visual and other sensory modalities) and plastic, enabling the meaning of an image, word, or symbol to be negotiated and fixed (Gooding 2010).

Visual representation becomes an important source of collaboration. The labor that scientists put into developing certain biological objects, structures, and models make scientists intimately close to these objects. One of the participants commented during an interview that on several occasions he had failed to explain certain things to his colleagues without the visuals. Here is a comment he made during our interview:

These images become a crucial communication tool. These images help us explain to them what we exactly want. By projecting these images on a large screen I can show my colleague by pointing to a specific portion of a cell.

5.2 Physicality

This theme is referred to as physicality, since it focuses on the ways in which our participants, while working with different images and communicating with others, used their bodily actions such as gestures and external representations such as charts and drawings to organize and make sense of their experimental data in a social setting. These physical practices involving gestures and external objects are used in combination with different visual representations for articulating the physical world of embodied social action.

A relevant aspect of the physicality theme is the notion of cognitive artifacts. Cognitive artifacts are tools used for aiding, enhancing, or improving human cognition (Norman 1991; Hutchins 1995; Suchman 2000). Here, an artifact can be any human constructed object of cultural significance, which can vary from human actions such as gestures to other externally created objects. Close investigation of artifacts and their use can provide insight into the meaningful cognitive practices of a culture. Often the nature of these practices is not crystallized, but they are dynamically adapted through time and circumstance. Hence, gestures, notes, or drawings can be seen as cognitive artifacts, as they are explicitly constructed and are used to aid human cognition.

Gestures are important semiotic resources in social interactions that focus on spatially organized phenomena. By incorporating gesture as a feature of our cognitive processes, we can view gesture as not only conveying meaning but gesture can also be seen as a link between distinct representations (Alac and Hutchins 2004). In other words, constructed gestures and other bodily expressions make visual representation meaningful not only to the self but also to other involved parties. Recently, there has been a growing interest in gesture studies (Kendon and Muller 2001), brought about in large part by a change in the theoretical foundations of many of the disciplines that traditionally viewed gesture as peripheral and incidental to spoken language. The gestures together with practitioners’ talk, gaze, and body orientation turn the physical space occupied by practitioners into a field of meaning production. In the context of laboratory practice, the multiple ‘semiotic fields’ (Goodwin 1994), such as the field of the digital screen and the one inhabited by material bodies, are superimposed and intertwined. The way in which the images are aligned with the gestures, body orientation, gaze, and talk suggests an action-oriented, publicly available, and intersubjective character of visualization practices.

Throughout our field study, we observed that gestures were used in almost all social interactions in laboratories as well as outside of it. Figure 6 shows a few examples of different gestures that were used to pin point specific areas on a digital screen, providing indications of size and structures as well as to animate or mimic certain complex procedures.

Fig. 6
figure 6

A set of gestures performed by our participant

Full size image

Near a spectro-microscope, two scientists are discussing their cell culture results using the TopoChip technique (Fig. 7). TopoChip (Unadkat et al. 2011) is a state-of-the-art technique to explore cell behavior on multiple scaffold structures, all at the same time. This technique has come into prominence due to the fact that physical characteristics of the scaffold surface (at both micro- and macro-level) determine the behavior of cells: whether they will attach themselves, grow, or in the case of stem cells, develop into the required cell type.

Fig. 7
figure 7

Two scientists discussing cell culturing image on a TopoChip

Full size image

In the following, we describe their conversation from our video recording, which illustrates how their gestures and bodily articulations help in making collaborative sense of the problem at hand. The senior scientist (on the left in Fig. 7a) is showing his junior counterpart how to look at the visual data. Here, both the scientists can look at the cells at the same time, since there is a common screen available that provides the exact view of the spectro-microscope. The conversation begins with the junior researcher (right on Fig. 7a) is showing his cell culture on TopoChip plate by positioning his plate on the microscope appropriately.

The senior and junior scientists are referred to as S and J, respectively.

Unlike the excerpt described in Fig. 2, where a senior scientist takes her junior counterpart through an already completed experiment using a set of images, the current example shows an ongoing experiment. There are several issues at work here. The above conversation is connected to a visual representation that is publicly available to both the parties, but can only be manipulated by one who is handling the microscope. One of the important aspects of physicality here is the bodily movements of both the scientists allow coordination in shared viewing of the visual representation. Both the scientists coordinated and analyzed their cell sample by positioning and orienting their bodies toward each other, toward the microscope and the screen. Throughout the discussion, both the scientists adjust the TopoChip plate to be able to show specific portions of the cell areas. By moving the plate on the microscope makes it possible for the scientists to see the complete picture of their TopoChip and importantly to observe how cells have attached to different surfaces on the TopoChip. To effectively do so, they both have to coordinate their actions to be able to see specific areas in the plate.

When S points to a certain area in the plate, he is not asking J to only look at it but to perceive and understand the structures in the image. Such an act results from S instructing J to position the plate in an appropriate area on the microscope so that J and S both can see the right area on the screen that is available for both of them to discuss, where they can easily orient them to each other and go back to the screen. Here is where S introduces his gesture (Fig. 7b) to emphasize the active nature of visualizing an image. The gesturing performance of S does not simply replicate what is already present on the screen (i.e., roundness in TopoChip). The performance is an interpretation of the visual representation shown on the screen, where S is trying to emphasize the non-blurring visuals around round shape. The gesture in this case functions as an aid to the visual representation. This gesture does not merely stand for the cell area present on the screen. Instead, S’s hand represents the solution of problem where he stressed on looking at the blurring and sharp edges. S’s gestures do not only function as indexical signs or transient inscriptions imposed on the screen to categorize and make it visible. They also participate in the process where the screen is taken in the construction of phenomenal objects so that the J can learn to spot the image artifact in respect to what should be done.

The gestures are enacted from a relative distance from the screen. Where the screen image provides a rich substrate for the action and allows for the potentially numerous interpretive paths, the gesture functions as an ‘eidetic’ mark that brings into relief the essential, synthetic, constant, veridical, and universally present aspects of the thing itself (Lynch and Woolgar 1990: 163). At the same time the scientists’ orientations toward the screen, their talk, the alignment with the screen images, and the general context of the practice constrain the interpretation of the gesture. Additionally, the gesture’s meaning is relative to the ongoing activity and the laboratory setting in which it is lodged. Here, however, the visibility of the sharing is generated through a physical coupling between the gestural enactment and the digital images. The activity of adjusting the plate on the microscope by S (Fig. 7c) marks the salient feature in the image, for example, touching and moving across the computer screen. However, its enactment does more than reveal features of the images to J.

The following excerpt (Fig. 8) is a continuation of the previous discussion, where S enacts a specific action to teach J a certain issues about the experiment.

Fig. 8
figure 8

A senior scientist explaining a problem

Full size image

In this case, the S places that TopoChip sample in a microscope and discusses it with his junior colleague. His gesture imitates painting-with-a-brush to refer to the spreading activity. This example shows an iconic gesture (hand as referent). By enacting such a gesture, S transforms a two-dimensional representation shown on the screen into a three-dimensional meta-representation (Norman 1991). Through his animated gesture, he is suggesting to his colleague to spread the cells over the complete surface without missing any area on the surface. The gesture performed in Fig. 8 is not applied directly in connection to any visual representation; rather, it is applied to support scientists’ verbal description. The gesture is used to describe the idea behind TopoChip technique and the process of cell seeding on a scaffold. The scientist’s hands are used as a symbol, when he says, ‘ You have to really spread them across the whole slide .’ The words ‘ across the whole slide ’ reference the co-speech gesture. At this point, speech provides the syntax needed to support a gesture-based lexical item.

In addition to gestures, other types of cognitive artifacts were also seen during our field study. In our study, we observed that several scientists carried specific charts, diagrams, and even hand-made drawings to guide their experiments. The following example describes a hand-made drawing used to follow specific procedures while preparing cell culture plates. A cell culture plate (Fig. 9a) is where cells with different combinations of materials are cultured and stored over a period of time depending on the experiment. To be able to remember and follow a predefined combination of cell cultures, one scientist created a hand-made drawing that she used while culturing the cells (Fig. 9b). Here is a comment from the scientist:

Fig. 9
figure 9

Plates that are used to carry cell culture (a) and a graphical representation created by a scientist to remember the topology inside the plate (b)

Full size image

The drawing was my idea. Our cell culturing process varies from a few hours to several days and even weeks. It would be really difficult to remember all the different combinations of cell densities and external medium that I have used in the cell culture plates. So, we now use diagrams of the plates, as we cannot label this transparent plates because we have to look through their tiny wells under microscope on a day to day basis. So we make plate’s diagram and label each well in diagram, which helps us to remember what all these well contain and at what date and time cells were cultured in it.

Alač and Hutchins (2004) also had similar findings, in their studies of Functional Magnetic Resonance Imaging (fMRI) technology. They found that scientists used hand-made drawings of the brain structure to map the digital images generated by the fMRI machine onto the paper-based drawing of a brain. They explored that diagram of the visual field provides an anchor for an imagined visual field object which is then squeezed with the hands to make its shape match the shape of the retinotopy space.

We also found other examples of hand-made drawings that scientists kept close by. Figure 10 shows a drawing of all 23 human chromosomes. This chart was not relevant for cell culturing research, but the scientist kept it in her office to answer any questions related to her cytogenetic research work. It was created several years back but she still uses it as a guide. She comments:

Fig. 10
figure 10

All 23 chromosomes. A practice note provided by one participant

Full size image

When I first joined this lab, I learnt all the 23 chromosomes by trying to remember their shape and formation and practiced by drawing them on my notes. I spent at least one day for each chromosome.

6 Discussion

The visualization practices of scientists working in the field of tissue engineering can be seen as much as embodied as they are visual. As we explored through our ethnographic field study, by exploiting the multiplicity of different visual representations and using gestures, external representations such as paper-based diagrams and specialized languages, these scientist aided their cognitive processes. We believe that these practices aid scientists’ visualization activities and are rarely given importance.

In our example (Fig. 2), we showed that scientists varied representations by making transformations from biological cell samples to different representations utilizing a set of staining techniques. These different representations provided specific explanations about how cells have behaved on the scaffold surface: Are enough cells attached? Are there enough living cells? How is the structure of these cells? In this case, the scientists used these visual representations in a linear fashion to support their deductive reasoning. We also explored how these visual representations allowed rough estimates that can be observed via different colors. In other cases, how they offered precise calculations about numbers of cells and details about their structures such as their diameters. Gruber (1974) argued that altering the modality of a representation is a means of discovering invariant properties. By moving ‘… from visual imagery, to sketches, to words and equations explaining (i.e., conveying the same meaning as) the thinker is pleased to discover that certain structures remain invariant under these transformations.’ (Gruber 1974, 1994, 410–411). We can say that scientists’ visual strategies are motivated and constrained by the capacities and limitations of biologically endowed cognitive capacities.

Importantly, these visual representations have an ‘integrative’ function. As we observed in our example (Fig. 2), each of the representations contained different information and ways of interpreting them also differed—where color, shapes, and diameter of cells played their parts. Each of these representations offered different cognitive demands. These differences were meaningful on a perceptual level (i.e., they are visible and perceptible as such) and on a conceptual level, as they referred to variables of scientific interest that helped them build theories and interpretations of nature that are ‘scientifically valid’ and ‘objective.’ On the one hand, each of these visual reorientations provided evidence, and they also served as source for the following representations. Hence, the collective nature of these visuals helped in better interpretations about the scientists’ experiment.

The theme of physicality treated scientists’ bodies and other external representations as a part of their distributed cognition. The use of gestures showed the ways in which scientists interacted with different visual representations and one another to describe how subjective bodies were involved in problem solving, learning, imagining, and how such processes, situated in the scientific laboratories, enact and shape such bodies. The study brings the physicality theme into the spotlight by identifying and describing processes of generation and interpretation of visual representations in tissue engineering. Gestures are visual displays. It is thought their visual nature, gestures achieve their power because they convey visual/spatial/dynamic information directly (McNeill 1995), unlike spoken language, which is symbolic and often bears arbitrary relationship to its referent. As talk-in-interaction is an important cognitive activity, and gesture is intimately involved in acts of spoken linguistic expression, then it is reasonable to look closely at gesture for the light it may cast on cognitive activity. The examples that we discussed in Sect. 5.2 showed that visual inference involves a series of moves that transform mental, material, and virtual objects (Gooding 2010).

It must be noted that these visualizing practices of cell culturing challenge the narrow conceptions of understanding structures as a rational activity. The physical and mental labor that is put into the whole research activity is recursive and iterative. Throughout different activities, scientists use craftwork, creativity, and use their own embodied knowledge to enhance their scientific reasoning. Cell structures and models, to a certain extent, are interactive objects since they demand participation and continual transformation from the scientists.

In summary, in this paper, we provided an account of the visual practices of scientists belonging to tissue engineering research. One of the main claims of this paper is that scientists’ visualization practices are affected by multimodal aspects that are at play. Overall, visual information supported by microscopic and computer generated images helped scientists in the following aspects:

  • They provided evidence and results of scientists’ research. As some participants expressed, these images are the outcomes of their research.

  • Moreover, these 2D (or 3D) visuals helped better understand what was happening at the micro- and molecular level.

  • They were also a collaboration tool for the colleagues and collaborators to understand what a scientist desired from them. This eventually improved the working process.

  • Visual insights were constructed through embodied practices of interpreting and ‘visualizing’ evidence; it was also through scientists’ multimodal practices that these objects became meaningful as evidence in themselves.

7 Conclusion

Through an ethnographic field study, we provided an account of scientists’ visualization practices. Our results showed that multiplicity and physicality played important role in the way scientists made sense of their visual representations. Our work provided several examples of distributed cognition at work, where in addition to internal and external representations; bodily activities of scientists played an important role. It was observed during this field study that multimodality plays a vital role in supporting visualization practices. The main reason to bring the visual culture of life scientists to the HCI community is because this field of research has not been well studied from a human factors point of view. Especially, when research programs on tissue engineering could bring cutting-edge solutions to life-threatening problems (HIV, cancer, etc.) in a less expensive way, we believe that more efforts from human factors research will only improve the current understandings of tissue engineering work.