Research

The problems with existing Hopfield-type neural networks are their inability to (a) learn online, (b) self stabilize, (c) have few spurious memories and (d) learn non-binary stimuli. Consequently, it is difficult for them to be considered as good cognitive models. Over the past years, several types of unsupervised models were proposed that have progressively overcome all of the above-mentioned limitations of standard Hopfield-type networks. First, a local online learning rule that directly incorporated output feedback from the transmission was proposed. That model was followed by a modification of the output rule that resulted in a spurious memory reduction while increasing its performance. More recently, an unsupervised recurrent neural network model was developed that is able to store non-binary memories and massively reduce the amount of spurious memories while having a better noise tolerance and generalization performance compared to other Hopfield type models. Finally, the model was generalized to bidirectional associative memories (BAM). BAM can associate two different pairs of stimuli. Therefore, they are useful for unsupervised and supervised learning. In addition, it was shown that BAM can perform multi-step pattern recognition (i.e. procedural learning) without the need for a special learning algorithm and with the capacity to learn more than two series of temporal patterns. The model can also learn temporal series of patterns of different lengths and, unlike previous models, the stimuli can be coded using real values. Finally, an extra autoassociative layer was added so that the model can accomplish one-to-many associations, a task that was specific to feedforward networks. However, in most dynamic-system models, stable point attractors are used to store and retrieve information. Such models are challenged by both psychological and neuroscience data. These data suggest that, in real neural systems, information is stored and retrieved via both stable and dynamic orbits (e.g. chaotic) attractors. Recently, a model that can show such behaviours has been introduced. Results indicate that the attractor basins are still well defined in the chaos regime making possible the extraction of the correct output. Thus, the model exhibits the kind of properties described within the general nonlinear dynamic system perspective found in neuroscience.


Research
Bidirectional Associative Memories

The problems with existing Hopfield-type neural networks are their inability to (a) learn online, (b) self stabilize, (c) have few spurious memories and (d) learn non-binary stimuli. Consequently, it is difficult for them to be considered as good cognitive models. Over the past years, several types of unsupervised models were proposed that have progressively overcome all of the above-mentioned limitations of standard Hopfield-type networks. First, a local online learning rule that directly incorporated output feedback from the transmission was proposed. That model was followed by a modification of the output rule that resulted in a spurious memory reduction while increasing its performance. More recently, an unsupervised recurrent neural network model was developed that is able to store non-binary memories and massively reduce the amount of spurious memories while having a better noise tolerance and generalization performance compared to other Hopfield type models. Finally, the model was generalized to bidirectional associative memories (BAM). BAM can associate two different pairs of stimuli. Therefore, they are useful for unsupervised and supervised learning. In addition, it was shown that BAM can perform multi-step pattern recognition (i.e. procedural learning) without the need for a special learning algorithm and with the capacity to learn more than two series of temporal patterns. The model can also learn temporal series of patterns of different lengths and, unlike previous models, the stimuli can be coded using real values. Finally, an extra autoassociative layer was added so that the model can accomplish one-to-many associations, a task that was specific to feedforward networks. However, in most dynamic-system models, stable point attractors are used to store and retrieve information. Such models are challenged by both psychological and neuroscience data. These data suggest that, in real neural systems, information is stored and retrieved via both stable and dynamic orbits (e.g. chaotic) attractors. Recently, a model that can show such behaviours has been introduced. Results indicate that the attractor basins are still well defined in the chaos regime making possible the extraction of the correct output. Thus, the model exhibits the kind of properties described within the general nonlinear dynamic system perspective found in neuroscience.

Learning in Noisy Environments
The development of artificial neural networks allows learning in noisy environments. In neural networks, categorization is generally achieved by learning from prototypes. However, in a natural setting the categories should emerge from learning a set of exemplars instead. The vigilance procedure has been proposed for hard competitive networks that enable them to learn a given stimulus banks by regrouping them as a function of a given resemblance function. In addition they are able to perform learning in a noisy environment. The procedure was adapted in the case of a recurrent autoassociative memory model. As a consequence, the network was able to learn categories from the presentation of the corresponding exemplars and the number of the resulting categories varied as a function of the value of the vigilance parameter. The prototypes developed were the theoretical mean of a given category and they were sensitive to the category frequency presentation. Moreover, it was shown that hard competitive learners are unable to use environmental biases while recurrent autoassociative memories use frequency of exemplars and categories independently, which is in agreement with the growing body of evidence that now suggests that humans are using base rate information. Recently, another approach was studied. Using the previously developed BAM, the model was modified in such a way that it could behave like a principal component analysis network and thus developed invariant features in a noisy environment. Results have shown that FEBAM (Feature-Extracting Bidirectional Associative Memory) could performs feature extraction, image compression, blind source separation, cluster development and reorganization, and learning in the presence of noise.

Neural Networks Applications
Over the years, several neural networks applications were developed. A hybrid model composes of a competitive layer and a RAM was developed that reproduces the attentional blink effect during a rapid serial visual presentation task. In addition, a simple RAM was adapted to perform an edge detection task directly from grey level images. The model can detect only the significant edges and it can generalize to other images. Also, a pulse coupled neural network was used to filter impulse noise due to hardware failure and eye blink in an eye tracker system. Recently, focus is given to the development of a neural network model that will discriminate between oculomotor patterns from sexual offenders and those from non-offenders.