WG3 Details

TP of complex, multisensory events

In an attempt to investigate TP, early research studies focused on the use of simple,low-informational content stimuli. While such studies have successfully put TP at the forefront of multisensory processing and identified a number of key factors modulating it, it is now appropriate to start investigating TP of more complex, ecologically-valid stimuli (e.g., speech, musical, or action stimuli).

The last five years a number of researchers have started more vigorously investigating TP of more complex stimuli. Although the use of complex stimuli is essential for answering fundamental questions regarding TP, some unavoidable problems exist. Specifically, the: a) quality of the stimuli used is frequently put in question since processing power and recording limitations lead to time inaccuracies; b) use of musical stimuli is still lagging, making it’s necessity greater (given that musical research can lead to perceptual comparisons with speech and action data); c) use of complex stimuli for imaging research is often difficult due to the nature of the stimuli and the technique’s limitations; and d) results from studies using complex stimuli have not been systematically used for the development of real-world applications (e.g., identifying reading impairments based on rhythm perception, refining the International Telecommunication Union (ITU) recommendations in terms of audiovisual timing for broadcasting). The problems associated with the investigation of TP using complex stimuli are challenging. A coordinated Action, such as TIMELY, targeting those problems should offer the opportunity for better controlled stimuli and comparable results.

Possible outcomes:

  1. In-depth analysis and review of the problems associated with complex stimulus use in TP.
  2. Development of standardized methodology/analysis for measuring TP for complex stimuli.
  3. Definition of strategies for using current/future results for the development of real-world applications.
  4. Development of standardized methodology for generating complex stimuli and system requirements for time-controlled stimulus presentation.
  5. Development of a stimulus database. This database will include speech, action, gesture, and musical stimuli.