Abstract
This paper tackles our ordinary engagements with social robots to focus on their materiality. In considering how an educational robot is encountered as a part of everyday practices in social robotics, the focus is on the robot’s materiality that is interactionally achieved. The attention to tactile exploration, spatial arrangements, and multimodal interactional aspects that characterize the encounter indicates how the robot is simultaneously enacted as a thing and as a social agent. The paper proposes that social agency of the robot is mutually constituted with its materiality and that to conceive of the robot’s social character its thing-like aspects need to be taken into account.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
As such, these machines are not only seen as having potential practical purposes, but also as models for human cognition (seen here as “embodied” and “social”).
I do so also because those who interact with robots do. As exemplified by a witnessed instance of a young child who, during her first encounter with a robot, explicitly compared her own arms and head to the robot’s, interactional participants themselves also recognize humanoid features of the robotic technology.
As I followed the design process of social robots in two university laboratories, I saw the efforts put in programming robots’ software, but I also witnessed the attention that went into assembling the visual and tactile appearances of these technologies. Even the meetings with in-house mechanical engineers or the moments when a robot’s readymade parts were delivered to the laboratory abounded in careful visual inspections and judicious touching directed at assessing the robot’s physical architecture.
Since this process of design and construction is meant to respond, at least in part, to the preschool visits, its contingencies do not only concern the work of the roboticists but also the classroom’s interactions between the children and their teachers.
Each of the robots can be mapped on a “project” (Lynch 1985: 53–80) as it participates in organizing laboratory practices within a temporal context. The appearance of the project’s unity and sequential organization is achieved through local production and in situ activities of obtaining funding, responding to grant cycles, writing up results, as well as designing and building physical instantiations of the robotic machine.
The robot is also equipped to display “songs.” When a song (e.g., “Nine Little Monkeys”) is played, the B screen shows cartoon illustrations that accompany the song. On those occasions, the user is asked to look at (but not touch) the screen. During the interaction reported below, the robot does not play any songs, so this feature will not be further discussed in the present text.
Since the F screen is also touch-sensitive, when the user touches it, the eyes move as if following the touching finger. During the interaction reported below, the participants do not touch the F screen, so this feature will not be further discussed in the present text.
This is something that roboticists interested in haptic design are generally reacting against, as it erases the complexity of human touch (reducing it to the touch of an inert glass-like surface).
On the relevance of spatial organization in the enactment of the robot’s agency, see also Alač et al. 2011.
It is possible that the lowering of the pitch and flattening of the intonation is there to make the content of the utterance appear less extraordinary. The mundanity of it is further performed when that what the participants have in front of them is turned into something that is not only encountered from a visual perspective but is handled.
The involvement of touch in interaction has been discussed by Aug Nishizaka in his research on activities at a Japanese midwife house. Nishizaka describes how touch features in a midwife’s work of palpating a pregnant woman’s abdomen to specifically focus on referential practices. In his 2007 paper, Nishizaka shows how the tactile reference to specific locations is shaped by the action sequences in which it is contingently embedded, while his 2009 paper (2011) discusses the accomplishment of reference through touch that is not accompanied by visual access. When the teacher in Line 3 touches the robot’s forearm, she, however, does not refer to a specific location of the robot’s body, nor does she tactilely differentiate the sensed location from any other section of the robot (except the screens, perhaps). That the teacher engages the robot’s arm (rather than the rest of its body) may simply have to do with the spatial configuration of the group at the specific moment in interaction: T1 engages with the robot’s forearm so that the child can access and touch the robot himself.
For this kind of engagement with the digital materiality, see Alač 2011.
This supposition is in accordance with the ideas implemented in the design of the machine. As mentioned earlier, the robot is designed with a goal to eventually recognize and provide its users with an individually targeted educational content.
That the teacher’s action implicates Irma in the orienting of the others toward the robot is also seen in Line 21 where Irma’s individual actions are highlighted so that they now participate in doing the orientation collaboratively. We see again that the teacher talks about Irma in the third person rather than addressing her directly (“Irma really likes he:r,”), which suggest that the utterance is once more designed for the overhearers (Goffman 1981), namely those located by the books. To them, it indicates that the direct engagements with the robot are the actions that need to be attended to. At this point of interaction, however, the teacher (who followed Brian) is located in the area of distance experience, so that, with this turn, she seems to also display her orientation toward the actions around the robot, despite her new location.
In my previous work (Alač et al. 2011), I highlighted the embeddedness of sociable robots in the laboratory setting. While the autonomy of sociable robots is imagined as a future possibility, they presently live in laboratories. Here, as I trace how the actions of the adults participate in achieving the robot’s social character, the prominence is given to teachers (rather than researchers). Nevertheless, it should not be forgotten that we are still in the laboratory. Not only does the robot encompass the laboratory voices in its design while its moves are coordinated with the researcher’s actions, the events featured here, as noted earlier, are a part of the iterative design cycle that researchers perform as they design the robot.
For example, between Irma and T2 in Line 24, or T1 and Brian in Line 22.
Despite the teacher’s positive assessment of her actions, the situation this time is also distinct from the one in Line 11 as Brian and the teacher are not located right next to her (and the robot) anymore. In Line 24, Irma starts to turn away.
See, for example, Tanaka et al. 2007.
It seems plausible to suppose that a person (instead of a thing) would not be touched as frequently.
The teachers may, for example, use the actions of others as models (Lines 9, 11, and 35)—turning those who display familiarity with the language game into associates co-participating in orienting attention toward the robot.
They enable the instruction to be performed “from the robot” where the technology is designed to be used by a single user (we also saw how, when Brian and Irma engage the robot at the same time, the design predisposes for that engagement to be done in parallel but not in concert). As an element of that individualized instruction (Alač in preparation), the roboticists and their educational consultants project the future development where the robot would not only “deliver” but also “collect information.” If children touch its screen, the robot would be in a position to collect and measure their performance on a learning task and, eventually, provide each child with a “targeted” treatment. The acquisition of vocabulary where words to be learned are framed in relationship to their reference to certain visual representations is an example of that kind of instruction.
Barthes talks about the smoothness produced through a perfect junction of the car’s components: “one keenly fingers the edges of the windows, one feels along the wide rubber grooves which link the back window to its metal surround.” (88).
Those gestures may also remind us that sociable robots are often imagined as primarily consumer goods. An example is Sony’s entertainment robot Aibo, modeled on a pet dog (and also adopted as a research platform, together with Sony’s QRIO robot). When in production (between 1999 and 2005), a consumer could open a catalogue of a luxury leather and clothing brand, Hermès, and on the glossy pages of the catalogue—displaying fur jackets, bags, jewelry, fragrances, cutlery and porcelain—find an advertisement for Aibo (e.g., Le Monde d’Hermès 2001, N 39, Volume II). Apparently, Hermès also produced bags specifically tailored for carrying the robot. Those bags, whose contours follow the shape of the robot’s body, allowed owners to exhibit the possession of the commodity (the bag, while being iconic to a medium-sized dog, is unmistakably distinct from a dog carrier, not to mention leashes and other pets’ accessories that a brand could produce), while functioning as an envelope that protects the precious object it transports.
Some of the previous versions of the robot discussed here could be also labeled as a touch-sensitive affective sociable robot even though they didn’t embed haptic technologies in the real sense of the word. In those versions, the robot—remotely controlled—would, for example, vocalize crying and retract if a child touched it aggressively.
For example, Tanaka et al. (2007) dealt with the problem of haptic interaction focused on a QRIO robot that—assisted by a human operator—responded to touch by giggling. The authors focused on how a single user touches the robot with a goal to show that over time children start to engage the robot as a peer. In other words, they aimed at indicating that the robot can be treated as a person.
It may also be mentioned that the teachers also explicitly attribute value to this material aspect of the robot. For example, after the robot’s first visits to the preschool, the roboticists’ requested feedback from the teachers. They responded by asking for the movements of the robot to be limited, at least initially. In interviews, they repeated that children should see the robot as a thing, similar to toys and educational materials encountered in the classroom.
References
Alač M (2009) Moving android: on social robots and body-in-interaction. Soc Stud Sci 39(4):491–528
Alač M (2011) Handling digital brains. MIT Press, Cambridge
Alač, M (In preparation) On everyday encounters with an educational robot and the quest for individualized instruction
Alač M, Movellan J, Tanaka F (2011) When a robot is social: enacting a social robot through spatial arrangements and multimodal semiotic engagement in robotics practice. Soc Stud Sci 41(6):126–159
Alač M, Movellan J, Malmir M (2014) Grounding a sociable robot’s movements in multimodal, situational engagements. In: Nakano Y et al. (ed) JSAI-isAI2013 Lecture notes in artificial intelligence 8417. Springer-Verlag, Heidelberg, p 261–281
Bailey DW (2014) Touch and the cheirotic apprehension of prehistoric figurines. In: Dent P (ed) Sculpture and touch. Ashgate, London, pp 27–43
Barthes R (1957/1972) Mythologies. Vintage, London
Breazeal C (2002) Designing sociable robots. MIT Press, Cambridge
Castaneda C, Suchman L (2014) Robot visions. Soc Stud Sci 44(3):315–341
Collins HM (1990) Artificial experts: social knowledge and intelligent machines. MIT Press, Cambridge
Dreyfus H (1972) What computers can’t do. MIT Press, New York
Dreyfus H (1992) What computers still can’t do. MIT Press, New York
Dreyfus H, Dreyfus S (1986) Mind over machine: the power of human intuition and expertise in the era of the computer. Blackwell, Oxford
Faucconier G, Turner M (2002) The way we think. Basic Books, New York, Conceptual Blending and the Mind’s Hidden Complexities
Fitzi, G, Matsuzaki, H, Lindermann, G (2014) Going beyond the laboratory—ethical and societal challenges for robotics In: International conference, Institute for advanced study Delmenhorst/Germany, 13–15 Feb 2014
Garfinkel H (1967/1984) Studies in ethomethodology. Polity Press, Englewood Cliffs
Garfinkel H (2002) Ethomethodology’s program. Rowman and Littlefield, Lanham
Garfinkel H, Lynch M, Livingston E (1981) The work of a discovering science construed with materials from the optically discovered pulsar. Philos Sociol Sci 11:131–158
Goffman E (1981) Footing in forms of talk. University of Pennsylvania, Philadelphia, pp 124–159
Goodwin C (1994) Professional vision. Am Anthropol 96(3):606–633
Goodwin C (2000) Action and embodiment within situated human interaction. J Pragmat 32(10):1489–1522
Gurwitch A (1964) The field of consciousness. Duquesne UP, New York
Heath C, Hindmarsh J In, May T (eds) (2002) Analyzing interaction video ethnography and situated conduct. Qualitative Research in Action. Sage, London, pp 99–121
Henare A, Holbraad M, Wastell S (eds) (2006) Thinking through things: theorizing artefacts ethnographically. Routledge and Kegan Paul, London
Hertenstein MJ (2002) Touch: its communicative functions in infancy. Hum Dev 45:70–94
Hetherington K (2002) The unsightly: touching the parthenon frieze. Theory Cult Soc 19(5–6):187–205
Hutchins E, Palen L (1998) Constructing Meaning from Space, Gesture and Speech. In: Resnick L, Saljo R, Pontecorvo C, Burge B (eds) Discourse, tools, and reasoning: situated cognition and technologically supported environments. Springer-Verlag, Heidelberg
Ingold T (2013) Making: anthrolpology. Art and Architecture. Routledge and Kegan Paul, London
Jefferson G (2004) Glossary of Transcript Symbols with an Introduction. In: Lerner GH (ed) Conversation analysis: studies from the first generation. John Benjamins, New, York, pp 13–31
Kidwell M, Zimmerman D (2006) “Observability” in the interactions of very young children. Commun Monogr 73(1):1–28
Kidwell M, Zimmerman D (2007) Joint attention in action. Pragmatics 39(3):592–611
Knappett C, Malafouris L (eds) (2008) Material agency: towards a non-anthropocentric approach. Springer, New York
Knorr-Cetina K (1981) The manufacture of knowledge: An essay on the constructivist and contextual nature of science. Pergamon Press, Oxford
Kobayashi Y, Yamazaki K, Yamazaki A, Gyoda M, Tabata T, Kuno Y, Seki Y (2012) Care robot able to show the order of service provision through bodily actions in multi-party settings. Extended Abstracts of ACM CHI'12 Conference on Human Factors in Computing Systems, May 2012, vol 2, pp 1889–1894
Koschman T, LeBaron C, Goodwin C, Zemel A, Dunnington G (2007) Formulating the triangle of doom. Gesture 7:97–118
Latour B (1987) Science in action. Harvard UP, Cambridge
Latour B, Woolgar S (1979) Laboratory life: The social construction of scientific facts. Sage, London and Beverly Hills
Latour B (1988) Mixing humans with non-humans: sociology of a door-closer. Soc Probl 35:298–310
Lynch M (1985) Art and artifact in laboratory science: A study of shop work and shop talk in a research laboratory. Routledge and Kegan Paul, London
Mead GH (1932) The Physical Thing, Supplementary Essay 2. In: Murphy AE (ed) The philosophy of the present. Open Court, LaSalle, pp 119–139
Meister M (2014) When is a robot really social? an outline of the robot sociologicus. Sci Technol Innov Stud 10(1):107–134
Merleau-Ponty M (1968) The visible and the invisible. Northwestern University Press, Evanston
Michalowski M, Šabanović S, DiSalvo C, Busquets D, Hiatt LM, Melchior NA, Simmons R (2007) Socially distributed perception. Auton Robots 22(4):385–397
Miller D (ed) (2005) Materiality. Duke UP, Durham
Mol Annemarie (2002) The body multiple. Duke UP, Durham
Mondada L (2007) Multimodal resources for turn-taking: pointing and the emergence of possible next speaker. Discourse Stud 9(2):194–225
Montagu A (1986) Touching: the human significance of the skin. Harper and Row, London
Montessori M (1948) The discovery of the child. Kalkshetra Publications Press, Madras
Nishizaka A (2007) Hand touching hand: referential practice at a Japanese Midwife house. Hum Stud 30(3):199–217
Nishizaka A (2011) Touch without vision: referential practice in a non-technological environment. J Pragmat 43:504–520
Ochs E, Gonzales P, Jacoby S In, Ochs E, Schegloff E, Thomson S (eds) (1996) When I come down I’m in a domain state: talk, gesture, and graphic representation in the interpretative activity of physicists: interaction and grammar. Cambridge University Press, Cambridge, pp 328–369
Paterson Mark (2009) Haptic geographies: ethnography, haptic knowledges and sensuous dispositions. Prog Hum Geogr 33(6):766–788
Pitsch, K, Kuzuoka, H, Suzuki, Y, Sussenbach, L, Luff, P, Heath, C (2009) “The first five seconds”: Contingent stepwise entry into an interaction as a means to secure sustained engagement. IEEE international symposium on robot and human interactive communication, 985–991
Robertson J (2010) Gendering humanoid robots: robo-sexism in Japan. Body Soc 16(2):1–36
Robins B, Dautenhahn K, Dickerson P (2012) Embodiment and cognitive learning—Can a humanoid robot help children with autism to learn about tactile social behaviour? lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics) Vol. 7621 LNAI, 66–75
Šabanović S (2010) Robots in society, society in robots: mutual shaping of society and technology as framework for social robot design. Int J Social Robot 2(4):439–450
Sacks H (1992) Lectures on conversation, vol 2. Blackwell Publishing, Oxford
Scassellati, B (2005) How social robots will help us to diagnose, treat, and understand autism.In: 12th international symposium of robotics research (ISRR) San Francisco CA Oct 2005
Schutz A (1945) On multiple realities. Res 5(4):533–576
Schutz A (1967) The Phenomenology of the Social World. Northwestern University Press, Evanston
Searle JR (1980) Minds brains, and programs. Behav Brain Sci 3(3):417–457
Shibata T, Wada K, Ikeda Y, Šabanović S (2009) Cross-cultural studies on subjective evaluation of seal robot. Adv Robot 23(4):443–458
Streeck J (2009) Gesturecraft: the manufacture of meaning. John Benjamins, Amsterdam and Philadelphia
Suchman L (2000) Embodied practices of engineering work. Mind Cult Act 7(1–2):4–18
Suchman L (2007) Human-machine reconfigurations: plans and situated actions, 2nd edn. Cambridge University Press, New York
Tanaka F, Cicourel A, Movellan J (2007) Socialization between toddlers and robots at an early childhood education center. Proc Natl Acad Sci 104(46):17954–17958
Turkle, S (2006) A nascent robotics culture: new complicities for companionship. AAAI technical report series Jul 2006
Verran H (2001) Science and an African logic. University of Chicago Press, Chicago
Vinkhuyzen, E [tokyovink] (2013) Some challenges for social robotics. https://www.youtube.com/watch?v=gRIAmxkEAGA&list=UU9xdGYoEanXIXBvEOZTBt6g Accessed 23 Aug 2013
Vom Lehn D (2014) Harold garfinkel: the creation and development of ethnomehodology. Left Coast Press, Walnut Creek
Wilson E (2010) Affect and artificial intelligence. University of Washington Press, Seattle
Wittgenstein L (1953) Philosophical investigations. Basil Blackwell, Oxford
Acknowledgments
I would like to thank Lucy Barbizzi, Fernando Dominguez Rubio, Don Everhart, Deborah Forster, Yelena Gluzman, Rebecca Hardesty, Sarah Klein, Maurizio Marchetti, Moshen Malmir, Susanna Messier, Javier Movellan, Ben Sheredos, and the participants in the ethnographic study for their contribution to this paper.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Alač, M. Social robots: Things or agents?. AI & Soc 31, 519–535 (2016). https://doi.org/10.1007/s00146-015-0631-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00146-015-0631-6