Utility of gestural cues in indexing semantic miscommunication

Masashi Inoue, Mitsunori Ogihara, Ryoko Hanada, Nobuhiro Furuyama

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Scopus citations

Abstract

In multimedia data analysis, automated indexing of conversational video is an emerging topic. One challenging problem in this topic is the recognition of higher-level concepts, such as miscommunications in conversations. While detecting miscommunications is generally easy for speakers as well as observers, it is not currently understood which cues contribute to their detection and to what extent. To make use of the knowledge on gestural cues in multimedia systems, the applicability of machine learning is investigated as a means of detecting miscommunication from gestural patterns observed in psychotherapeutic face-to-face conversations. Various features are taken from gesture data, and both simple and complex classifiers are constructed using these features. Both short-term and long-term effects are tested using different time window sizes. Also, two types of gestures, communicative and non-communicative, are considered. The experimental results suggest that there is no single gestural feature that can explain the occurrence of semantic miscommunication. Another interesting finding is that gestural cues correlate more with long-term gestural patterns than with short-term ones.

Original languageEnglish (US)
Title of host publication2010 5th International Conference on Future Information Technology, FutureTech 2010 - Proceedings
DOIs
StatePublished - Jul 14 2010
Event5th International Conference on Future Information Technology, FutureTech 2010 - Busan, Korea, Republic of
Duration: May 20 2010May 24 2010

Publication series

Name2010 5th International Conference on Future Information Technology, FutureTech 2010 - Proceedings

Other

Other5th International Conference on Future Information Technology, FutureTech 2010
CountryKorea, Republic of
CityBusan
Period5/20/105/24/10

Keywords

  • Face-toface
  • Gesture
  • Psychotherapy
  • Semantic indexing

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Information Systems

Fingerprint Dive into the research topics of 'Utility of gestural cues in indexing semantic miscommunication'. Together they form a unique fingerprint.

  • Cite this

    Inoue, M., Ogihara, M., Hanada, R., & Furuyama, N. (2010). Utility of gestural cues in indexing semantic miscommunication. In 2010 5th International Conference on Future Information Technology, FutureTech 2010 - Proceedings [5482653] (2010 5th International Conference on Future Information Technology, FutureTech 2010 - Proceedings). https://doi.org/10.1109/FUTURETECH.2010.5482653