- Cross-modal temporal behavior modeled with artificial neural network societies

A multidisciplinary project supported by the Information Research Programme of the Academy of Finland


Tacit knowledge means procedural knowledge that a being needs to act and react in its environment, for example to play a musical instrument. It is unformalized, related to intuitions, feelings and emotions. We study tacit knowledge in the context of musical behavior, and as a means of communication in virtual environments, from perspectives of cognitive science.

We propose that tacit knowledge can be accessed by simulating cross-modal bodily actions (i.e interactions between different senses like vision, hearing, and motion) with artificial neural networks (ANNs). Our hypothesis is that a temporal self-organizing ANN learns to communicate when embedded in a society of similar agents, and collective intelligence emerges. Human-like behavior results only when physical constraints of a virtual body are modeled and the agent communicates with a human through sensory-motor devices.

Specific research goals towards the general aim are to:

Potential applications of computer systems using tacit knowledge include adaptive user interfaces, substitute actors in networked virtual environments and adaptive music synthesizers.

The project is a continuation of previous research done by the involved teams. Each team has previously conducted research relevant to this proposal and has been supported by the Academy of Finland. This project arises from the recognition of shared interests in artificial neural networks and temporal behavior of complex systems.

The proposal addresses the Information Research Programme in three ways: (1) complex non-linear systems are studied both within agents (ANNs) and as societies formed by agents, (2) medialisation techniques (virtual reality) are extensively used, and (3) man as processor of information is the main objective of study.

Keywords: tacit knowledge, artificial neural networks, connectionism, virtual reality, artificial life, agent systems, mind-environment systems, ecologism, music perception and performance, music generation, future-oriented temporality, cross-modality, emotions, complex nonlinear systems

Project group


26 Nov 1998