A. Nyamapfene (UK) and K. Ahmad (Ireland)
Hebbian-linked self-organising maps, Multimodal, Crossmodal, Neural Networks
We present two separate algorithms for unsupervised multimodal processing. Our first proposal, the single pass Hebbian linked self-organising map network, significantly reduces the training of Hebbian-linked self organising maps by computing in a single epoch the weights of the links associating the separate modal maps. Our second proposal, based on the counterpropagation network algorithm, implements multimodal processing on a single self-organising map, thereby eliminating the network complexity associated with Hebbian linked self organising maps. When assessed on two bimodal datasets, an audio-acoustic speech utterance dataset and a phonological-semantics child utterance dataset, both approaches achieve smaller computation times and lower crossmodal mean squared errors than traditional Hebbian linked self-organising maps. In addition, the modified counterpropagation network leads to higher crossmodal classification percentages than either of the two Hebbian linked self-organising map approaches.
Important Links:
Go Back
