Publication:
Quantifying the speech-gesture relation with massive multimodal datasets: informativity in time expressions

Simple item page
dc.contributor.authorPagán Cánovas, Cristóbal
dc.contributor.authorValenzuela Manzanares, Javier
dc.contributor.authorOlza, Inés
dc.contributor.authorRamscar, Michael
dc.contributor.authorAlcaraz Carrión, Daniel
dc.contributor.departmentFilología Inglesa
dc.contributor.otherFacultades de la UMU::Facultad de Letras
dc.date.accessioned2026-02-12T18:48:59Z
dc.date.available2026-02-12T18:48:59Z
dc.date.copyright© 2020 Pagán Cánovas et al.
dc.date.issued2020-06-02
dc.description.abstractThe development of large-scale corpora has led to a quantum leap in our understanding of speech in recent years. By contrast, the analysis of massive datasets has so far had a limited impact on the study of gesture and other visual communicative behaviors. We utilized the UCLA-Red Hen Lab multi-billion-word repository of video recordings, all of them showing communicative behavior that was not elicited in a lab, to quantify speech-gesture co-occurrence frequency for a subset of linguistic expressions in American English. First, we objectively establish a systematic relationship in the high degree of co-occurrence between gesture and speech in our subset of expressions, which consists of temporal phrases. Second, we show that there is a systematic alignment between the informativity of co-speech gestures and that of the verbal expressions with which they co-occur. By exposing deep, systematic relations between the modalities of gesture and speech, our results pave the way for the data-driven integration of multimodal behavior into our understanding of human communication.
dc.formatapplication/pdf
dc.format.extent18
dc.identifier.citationPLoS ONE, 2020, Vol. 15(6): e0233892
dc.identifier.doihttps://doi.org/10.1371/journal.pone.0233892
dc.identifier.eissn1932-6203
dc.identifier.urihttp://hdl.handle.net/10201/204642
dc.languageeng
dc.publisherPublic Library of Science
dc.relationFunding support was provided by two I + D Knowledge Generation Grants from Spain’s Ministry of Science and Innovation and FEDER/UE funds, one to C.P.C. and J.V. (ref. PGC2018-097658-B-I00) and another to I.O. (ref. PGC2018-095703-B-I00); a EURIAS Fellowship from NetIAS and the Netherlands Institute for Advanced Study (C.P.C.); a Ramón y Cajal grant (C.P.C.); an Arts and Humanities Research Council doctoral scheme scholarship (D.A.C.); a fellowship from the SRUK On the move postdoctoral research program (D.A.C.).
dc.relation.publisherversionhttps://journals.plos.org/plosone/article?id=10.1371/journal.pone.0233892
dc.rightsAttribution 4.0 International*
dc.rights.accessRightsinfo:eu-repo/semantics/openAccess
dc.rights.urihttp://creativecommons.org/licenses/by/4.0/*
dc.subject.odsNo relacionado con ningún objetivo de desarrollo sostenible
dc.titleQuantifying the speech-gesture relation with massive multimodal datasets: informativity in time expressions
dc.typeinfo:eu-repo/semantics/article
dc.type.versioninfo:eu-repo/semantics/publishedVersion
dspace.entity.typePublicationes
relation.isAuthorOfPublicationcd2110ad-a57f-479f-90bf-f13163f6a48c
relation.isAuthorOfPublicationb36862c7-e0d9-4364-a3e4-0b75c004bbc2
relation.isAuthorOfPublication194306fe-a744-454a-9041-686afe94694f
relation.isAuthorOfPublication.latestForDiscoverycd2110ad-a57f-479f-90bf-f13163f6a48c
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
Quantifying the speech-gesture relation.pdf
Size:
1.26 MB
Format:
Adobe Portable Document Format
Description:
Download
License bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
license.txt
Size:
1.37 KB
Format:
Item-specific license agreed upon to submission
Description:
Download
Collections
Artículos