Multimodal Communication in Instructional Settings: An Investigation of the Functional Roles of Gestures and Arrows

Authors

  • Cengiz Acarturk Middle East Technical University & Jagiellonian University, Poland http://orcid.org/0000-0002-5443-6868
  • Melda Coskun Middle East Technical University, Turkey & McMaster University
  • Serap Emil Middle East Technical University

Keywords:

Modality, deixis, arrows, multimodality

Abstract

In daily communication and instructional settings, arrows and spontaneous gestures usually convey similar spatial information to interlocutors. They can visualize spatial information in 2D or 3D space based on shared conceptual and spatial representations. The present study investigates the relationship between spontaneous gestures and arrows from the perspective of generation and comprehension. We introduce a corpus of arrows in written multimodal text and a corpus of gestures generated during communication through multimodal instructional material. We also report an experimental investigation that used corpus material to study the impact of gestures on comprehension learning. A comparison of the corpora reveals functional similarities and differences between arrows and gestures and their complementary role in multimodal communication. The findings suggest that arrows and gestures may share a common conceptual space during communication, having a significant impact on comprehension and learning.

Author Biographies

Cengiz Acarturk, Middle East Technical University & Jagiellonian University, Poland

Associate Professor in Cognitive Science

Melda Coskun, Middle East Technical University, Turkey & McMaster University

PhD Student at the Cognitive Science of Language Department

Serap Emil, Middle East Technical University

Associate Professor at the Educational Administration and Planning Department

Downloads

Published

2022-01-05

How to Cite

Acarturk, C., Coskun, M., & Emil, S. (2022). Multimodal Communication in Instructional Settings: An Investigation of the Functional Roles of Gestures and Arrows. Revista Signos. Estudios De Lingüística, 54(107). Retrieved from https://revistasignos.cl/index.php/signos/article/view/816

Issue

Vol. 54 No. 107 (2021): December

Section

Multimodalidad y Multimedios

License

Copyright agreement:

Authors who have a manuscript accepted for publication in this journal agree to the following terms:

  1. Authors will retain their copyright and grant the journal the right of first publication of their work by means of this copyright agreement document, which is subject to the Creative Commons Acknowledgment License that allows third parties to share the work provided that its author and first publication in this journal are indicated.

  2. Authors may adopt other non-exclusive license agreements for distribution of the published version of the work (e.g., depositing it in an institutional repository or publishing it in a monographic volume) as long as the initial publication in this journal is indicated.

  3. Authors are allowed and encouraged to disseminate their work via the internet (e.g., in institutional publications or on their website) before and during the submission process, which can lead to interesting exchanges and increase citations of the published work (read more here).