|
|
|
-- Kitajima, M., Dinet, J., & Toyota, M. (2019)Kitajima, M., Dinet, J., & Toyota, M. (2019). Multimodal Interactions Viewed as Dual Process on Multi-Dimensional Memory Frames under Weak Synchronization. COGNITIVE 2019 : The Eleventh International Conference on Advanced Cognitive Technologies and Applications, 44-51.
Multimodal Interactions Viewed as Dual Process on Multi-Dimensional Memory Frames under Weak SynchronizationBehavior of users interacting with multimodal interfaces looks complex because the degrees of freedom of sensory input and motor output is large. This paper suggests that this complexity can be alleviated by applying the Simon's ant metaphor to the multimodal interaction situations, i.e., "What users do is simple. 1) they use perceptual input to generate a mirror image of the real world surrounding the self to be shared in conscious and unconscious processes, 2) they select next actions by consciously planning ahead and unconsciously tuning motor movements for the event to happen, and after performing the action, they unconsciously modify the participated neural network and consciously reflect on the result of action, and 3) they perform 1 and 2 in synchronous with the ever-changing external environment, which this paper calls "weak synchronization." A cognitive architecture, Model Human Processor with Realtime Constraints (MHP/RT) and its associated memory structure, Multi-Dimensional Memory Frames, developed by the authors is briefly introduced considering the situation of users interacting with multimodal environment. Then, the above three items are derived as the essential principles for organizing user's behavior in multimodal interaction environment. Future work on designing mixed reality multimodal interaction environment is introduced that has its basis on the perspectives for multimodal interactions this paper claims. Download
|