Proposer: Christine Tanner, Christine.Tanner@kcl.ac.uk
Suggested Supervisors: John Hallam, Jessica Gaines, David Willshaw
Principal goal of the project: To investigate the dynamic behaviour of self-organising neural networks built on Marr's Theory of Neocortex
Description:
Marr suggested that the neocortex learns by self-organisation to extract the structure from the patterns of activity incident upon it (Marr, 1970). He proposed a feed-forward neural network with three layers of cells: input cells which project to codon cells, which in turn project to an output layer of pyramidal cells. The codon cells are intended to extract the salient features from the input pattern. Their wiring to the input cells depends on the precise structure of the input information and is determined in an pre-training phase. From then onwards it remains fixed while the connections to the output cells are modified by a mechanism of competitive learning.
Marr addressed separately the problem of how to set the fixed input-to-codon connections and the variable condon-to-output connections. An analysis of the second problem is given in (Willshaw et al., 1997) which provides necessary conditions under which desirable stable states exit but not sufficient conditions that such states are reachable. This project attempts to answer this question by analysing the dynamic behaviour of the learning rule.
The weights between codon cells and an output cell and the used threshold are calculated by a recursive stochastic estimation procedure. Such systems are studied by control engineers, for whom questions of convergence are important. It can be shown that the learning rule's dynamic behaviour is equivalent to that of a certain continuous non-linear dynamic system. The values of weights and threshold reachable by the learning rule are the limit points (if any) of that dynamical system. Further analysis of this equivalent dynamical system to this case shall be carried out during this project.
Software based on MATLAB simulating the Marr competitive learning rule shall be built in order to compare empirical and theoretical behaviour.
Resources Required: Workstation, MATLAB, access to Marr's learning algorithm
Degree of Difficulty: Might be very hard to prove, open ended analysis
Background Needed: CC1 module, Probability module
Degree Programmes Suitable: AI/M4
References: