Lauren Hall, Elizabeth Peel & Saul Albert
Download transcripts | Keywords: Agency, Virtual Assistants, Participation, Footing, Technology
How should we understand participation and agency in talk involving voice interfaces? ‘Conversational technologies’ are proliferating both in everyday life and, by extension, in conversation analytic research (Stokoe et al., 2024). Interactional studies involving a wide range of technologies have tended to simply transpose conversation analytic frameworks and methods onto the analysis of talk with machines (Mlynář et al., 2024). As Suchman’s (2023, p. 4) critique points out, uncritically adopting the framing of these technologies as social actors risks contributing to the mystification and rhetorical reproduction of ‘AI’ as a “singular and autonomous agent”. This paper contributes to debates emerging within ethnomethodological and conversation analytic (EM/CA) approaches to AI that interrogate our conceptual frameworks and analytic vocabularies for describing these supposed interactions and their ostensible ‘participants’ using examples of the multimodal organisation of talk with and around voice user interfaces.
A range of critical perspectives within EM/CA take issue with a key paradigm in human-computer interaction (HCI) that assumes users treat ‘Computers As Social Actors’ (CASA) (Nass et al., 1994). For example, CASA explains anthropomorphic use of politeness markers such as “please” and “thank you” as ‘mindless’ spill-overs of social rules into talk with machines (Nass & Moon, 2000). Reeves and Porcheron (2023) argue that the CASA paradigm has led to the misplaced ‘agentification’ of machines, misleading reifications of ‘conversation’ in design (Reeves, 2019), and analytic misrepresentations of ‘AI’ that obscure how sense-making is achieved by the situated, interactional work of human users. Habscheid et al., (2023) respecify the ontological concept of agency in HCI as an interactional achievement whereby users’ interactions place machines on a spectrum of being treated as more or less agentic. Similarly, Pelikan et al., (2022) track how robots can be interactionally constructed as having sequentially and pragmatically differentiated types of ‘autonomous’, ‘hybrid’, ‘ascribed’, ‘potential’, or ‘non’-agency.
This paper contributes to EM/CA respecifications of participation and agency in talk involving machines by building on Goodwin’s (2007) concept of interactive footing. Goodwin extended Goffman’s (1981) dramaturgic taxonomy of agentic roles or ‘footing’ in talk by highlighting the co-constructed, embodied nature of social interaction. Interactive footing provides a framework for the study of participation that develops from a structural typology of participants (e.g., speaker, hearer, animator, principal, etc.) embedded in speech, to shift focus onto the temporal unfolding of joint actions. Drawing on a corpus of video-recordings collected by users of smart speakers in their homes, we focus on practices through which users invoke, implicate, or may exclude virtual assistants from talk and embodied action, both solo and with co-present others. For example, in Extract 1 we see Annabelle, who lives alone, rummaging in her sewing box after chastising her virtual assistant Alexa for not playing a particular song.
Extract 1:
On line 4, Annabelle first invokes Alexa in third person, as an absent other, musing on having been ‘rude’ to her, then topicalizes her ‘programming’ as a source of trouble in lines 7-8. In line 10, she then shifts footing from talking about Alexa to using her name, though not to activate the device or solicit a response. Here Annabelle invokes Alexa, ‘off the record’, turning away from the device, insulting her as a “silly old woman” before shifting back to ‘self-talk’ about Alexa and musing on her troubles (lines 10-13).
This paper proposes virtual footing as a way of capturing the multimodality and dynamism of Goodwin’s (2007) extension of Goffman’s participation frameworks, while highlighting users’ clearly ‘mindful’ practices for implicating virtual assistants in talk. Our analysis shows how users themselves can deflate the ‘uncontroversial thingness’ of AI, as Suchman (2023) puts it, by reflexively exploiting and satirising the limitations of such technologies as a resource for participation and distributed agency in interaction.
Bibliography
Alač, M., Gluzman, Y., Aflatoun, T., Bari, A., Jing, B., & Mozqueda, G. (2020). Talking to a toaster: How everyday interactions with digital voice assistants resist a return to the individual. Evental Aesthetics, 9(1), 51.
Albert, S., & Hall, L. (2024). Distributed agency in smart homecare interactions. Discourse & Communication, 18(6), 905–917.
Albert, S., & Hamann, M. (2021). Putting wake words to bed: We speak wake words with systematically varied prosody, but CUIs don’t listen. Proceedings of the 3rd Conference on Conversational User Interfaces, 1–5.
Castells, M. (2010). The rise of the network society (2nd ed.). Wiley-Blackwell.
Deleuze, G. (1994). Difference and repetition (P. Patton, Trans.). Columbia University Press.
Edwards, D. (1994). Imitation and artifice in apes, humans, and machines. American Behavioral Scientist, 37(6), 754–771.
Emirbayer, M., & Mische, A. (1998). What is agency? American Journal of Sociology, 103(4), 962–1023.
Enfield, N. J., & Kockelman, P. (Eds.). (2017). Distributed agency. Oxford University Press.
Garfinkel, H., & Wieder, D. L. (1992). Two incommensurable, asymmetrically alternate technologies of social analysis. In G. Watson & R. M. Seiler (Eds.), Text in context (pp. 175–206). Sage.
Goffman, E. (1978). Response cries. Language, 54(4), 787–815.
Goffman, E. (1981). Footing. In Forms of talk (pp. 124–157). University of Pennsylvania Press.
Goodwin, C. (2004). A competent speaker who can’t speak: The social life of aphasia. Journal of Linguistic Anthropology, 14(2), 151–170.
Goodwin, C. (2007). Interactive footing. In E. Holt & R. Clift (Eds.), Reporting talk (pp. 16–46). Cambridge University Press.
Habscheid, S., Hector, T., & Hrncal, C. (2023). Human and non-human agency as practical accomplishment: Interactional occasions for ascription and withdrawal of (graduated) agency in the use of smart speaker technology. Social Interaction. Video-Based Studies of Human Sociality, 6(1), Article 1. https://doi.org/10.7146/si.v6i1.137378
Hall, L. (2025). Virtual assistants in smart homecare [Doctoral dissertation, Loughborough University].
Hall, L., Albert, S., & Peel, E. (2024). Doing virtual companionship with Alexa. Social Interaction, 7(3).
Hutchins, E. (1995). Cognition in the wild. MIT Press.
Latour, B. (2005). Reassembling the social: An introduction to actor-network-theory. Oxford University Press.
Mlynář, J., de Rijk, L., Liesenfeld, A., Stommel, W., & Albert, S. (2024). AI in situated action: A scoping review of ethnomethodological and conversation analytic studies. AI & Society, 40, 1497–1527. https://doi.org/10.1007/s00146-024-01919-x
Nass, C., & Moon, Y. (2000). Machines and mindlessness: Social responses to computers. Journal of Social Issues, 56(1), 81–103. https://doi.org/10.1111/0022-4537.00153
Nass, C., Steuer, J., & Tauber, E. R. (1994). Computers are social actors. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 72–78. https://doi.org/10.1145/191666.191703
Pelikan, H., Broth, M., & Keevallik, L. (2022). When a robot comes to life: The interactional achievement of agency as a transient phenomenon. Social Interaction. Video-Based Studies of Human Sociality, 5(3), Article 3. https://doi.org/10.7146/si.v5i3.129915
Porcheron, M., Fischer, J. E., Reeves, S., & Sharples, S. (2018). Voice interfaces in everyday life. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Paper 640, pp. 1–12). Association for Computing Machinery.
Reeves, S. (2019). Conversation considered harmful? Proceedings of the 1st International Conference on Conversational User Interfaces, 1–3. https://doi.org/10.1145/3342775.3342796
Reeves, S., & Porcheron, M. (2023). Conversational AI: Respecifying participation as regulation. In W. Housley, A. Edwards, R. Beneito-Montagut, & R. Fitzgerald (Eds.), The SAGE handbook of digital society (pp. 573–592). Sage. https://doi.org/10.4135/9781529783193.n32
Sacks, H. (1984). On doing “being ordinary”. In J. M. Atkinson & J. Heritage (Eds.), Structures of social action: Studies in conversation analysis (pp. 413–429). Cambridge University Press.
Sidnell, J. (2022). Reframing ‘footing’. In M. H. Jacobsen & G. Smith (Eds.), The Routledge international handbook of Goffman studies (pp. 131–142). Routledge.
Stokoe, E., Albert, S., Buschmeier, H., & Stommel, W. (2024). Conversation analysis and conversational technologies: Finding the common ground between academia and industry. Discourse & Communication, 18(6), 837–847. https://doi.org/10.1177/17504813241267118
Suchman, L. (2023). The uncontroversial ‘thingness’ of AI. Big Data & Society, 10(2), Article 20539517231206794. https://doi.org/10.1177/20539517231206794echnologies. Discourse & Communication, 18(6).
Suchman, L. (2023). The uncontroversial ‘thingness’ of AI. Big Data & Society, 10(2).