TACIT KNOWLEDGE IN COMPLEX MIND-ENVIRONMENT SYSTEMS
- Cross-modal temporal behavior modeled with artificial neural network societies
A multidisciplinary project supported by the Information Research Programme of the Academy of Finland
Tacit knowledge means procedural knowledge that a being needs to act and react in its environment, for example to play a musical instrument. It is unformalized, related to intuitions, feelings and emotions. We study tacit knowledge in the context of musical behavior, and as a means of communication in virtual environments, from perspectives of cognitive science.
We propose that tacit knowledge can be accessed by simulating cross-modal bodily actions (i.e interactions between different senses like vision, hearing, and motion) with artificial neural networks (ANNs). Our hypothesis is that a temporal self-organizing ANN learns to communicate when embedded in a society of similar agents, and collective intelligence emerges. Human-like behavior results only when physical constraints of a virtual body are modeled and the agent communicates with a human through sensory-motor devices.
Specific research goals towards the general aim are to:
Potential applications of computer systems using tacit knowledge include adaptive user interfaces, substitute actors in networked virtual environments and adaptive music synthesizers.
- demonstrate emergent timing by an ANN that generates rhythmic patterns without an explicit timer
- model temporal behavior based on expectation and anticipation
- model temporal behavior with a system of hierarchical time-frames, capable of recognizing and generating musical patterns
- develop models for coordination of sound and motion, applicable to tap-dance or percussion music
- develop mind-environment coordination models that take the physical constraints into account
- develop models for coordinated action of multiple agents in musical tasks, for example a group of mutually synchronizing players
- explore the possibility to construct ANN models, which simulate emotional arousal based on the violation of expectations
- simulate a band of musicians, or a theater company in virtual reality, as a demonstration platform for the above topics
The project is a continuation of previous research done by the involved teams. Each team has previously conducted research relevant to this proposal and has been supported by the Academy of Finland. This project arises from the recognition of shared interests in artificial neural networks and temporal behavior of complex systems.
The proposal addresses the Information Research Programme in three ways: (1) complex non-linear systems are studied both within agents (ANNs) and as societies formed by agents, (2) medialisation techniques (virtual reality) are extensively used, and (3) man as processor of information is the main objective of study.
Keywords: tacit knowledge, artificial neural networks, connectionism, virtual reality, artificial life, agent systems, mind-environment systems, ecologism, music perception and performance, music generation, future-oriented temporality, cross-modality, emotions, complex nonlinear systems
Tapio.Takala@hut.fi 26 Nov 1998