Home page for the BIMM (Brain-Inspired Memory Model) neural net of Kary FRÄMLING

Brain-Inspired Memory Model is the current working name for a neural reinforcement learning model that uses notions of short- and long-term working memory for performing animal- and human-like trial-and-error learning. This page gives access to demonstration programs using this technique.

Background

This artificial neural net (ANN) was born from the idea to develop a model that would do problem solving and learning in similar ways as humans and animals do. The model would also correspond to some very rough-level ideas and knowledge about how the brain operates, i.e. activations and connections between different areas of the brain and notions of short- and long term working memory.

Animal problem solving mainly seems to be based on trial and learning. The success or failure of a trial modifies behavior in the "right" direction after some number of trials, where "some number" is in the range one (e.g. learning how to turn on the radio from "power" button) to infinity (e.g. learning how to grab things, which is a life-long adaptation procedure).

Such behavior is currently studied mainly in the scientific research area called reinforcement learning (RL). RL methods have been successfully applied to many problems where more "conventional" methods are difficult to use due to factors like lacking data about the environment, which forces the neural net to explore its environment and learn interactively. Exploring is a procedure where the agent has to take actions without a priori knowledge about how good or bad the action is, which may be known only much later when the goal is reached or when the task failed.

Maze solver

This application has been/will be used in the following publications:

[1] FRÄMLING, Kary. Reducing state space exploration in reinforcement learning problems by rapid identification of initial solutions and progressive improvement of them. To appear in Proceedings of 3rd WSES International Conference on Neural Networks and Applications (NNA'02) . Interlaken, Switzerland, 11-15 February 2002.

Maze route finding is commonly used in studies of animal learning and behavior. Animals have to explore the maze and construct an internal model of the maze in order to reach the goal. The more maze runs the animal performs, the quicker it goes to the goal since solutions get better memorized.

Maze route finding is not a very complicated problem to solve with many existing methods like depth-first and breadth-first search or reinforcement learning methods like temporal difference (TD) or Q-learning. Ordinary search methods usually require a model of the problem being solved and they are not good at handling unstatic problems. Many reinforcement learning methods do not require a model of the problem space and they are also usually capable of handling changes in the problem space. The main problem of currently existing reinforcement learning methods is their need for exhaustive exploration of the entire problem space, which often means very long learning times. As the number of possible states grows, current methods become unusable. Big mazes are one example of such problems, where state space reduction by generalisation and function approximation is not possible.

BIMM's goal is to reduce the state space exploration to a minimum so that a "usable" solution is identified very rapidly (often found with only one episode). Then better solutions can be searched for when there "is time for it" by balancing between old knowledge stored in long-term working memory and random exploration favored by short-term working memory.

Last updated 30 April 2003.