Navigating a multimodal ensemble: Learners mediating verbal and non-verbal turns in online interaction tasks

Janine Knight, Melinda Dooly, Elena Barberà

Research output: Contribution to journalArticleResearchpeer-review

1 Citation (Scopus)


Research into the multimodal aspects of language is increasingly important as communication through a screen plays a greater role in modern society than ever before (Liou, 2011). Multimodality has been explored from a number of angles relating to computer-mediated communication (CMC), such as its affordances and impact on language learners, highlighting its relevance and importance in the field of second language acquisition (SLA). Because CMC scenarios require attending to both peers and the screen, learners can be seen as positioned as semiotic initiators and responders (Coffin & Donohue, 2014). Increasingly, researchers are highlighting a need for a methodological turn to analyse this scenario from a language focus to a more holistic understanding of the interactions (Flewitt, 2008; Hampel & Hauck, 2006; Kress & van Leeuwen, 2001; Lamy, 2006). Along these lines, this case study explores how the action of task completion is mediated between six dyads (and individuals within the dyads) during an online peer-to-peer audioconferencing event. Drawing on notions from multimodal (inter)actional analysis (Norris, 2004, 2006) and the notion of semiotic initiators and responders, it investigates semiotic mediation with screen-based resources through analysis of audio recordings, screenshots, log files, task simulation and reconstruction. Results highlight oral and screen-based initiations and responses that take place during task completion, which is presented as a framework.

Original languageEnglish
Pages (from-to)25-46
Number of pages22
Issue number1
Publication statusPublished - 1 Jan 2020


  • agency
  • multimodal turn-taking
  • screen-based resources
  • semiotic mediation
  • spoken interaction
  • task-based synchronous communication (TB-SCMC)


Dive into the research topics of 'Navigating a multimodal ensemble: Learners mediating verbal and non-verbal turns in online interaction tasks'. Together they form a unique fingerprint.

Cite this