R. Kamimura (Japan)
mutual information maximization, competitive learning, forced information, information loss, winner-take-all
In this paper, we propose a new type of computational method to accelerate a process of information maximiza tion and a new technique to extract important features in in put patterns by a concept of information loss. Information theoretic competitive learning has been proposed to solve the fundamental problems of competitive learning such as the dead neuron problem with many practical applica tions. However, one of the major problems in information theoretic competitive learning is slow in increasing infor mation in competitive units, depending upon given prob lems. To overcome this shortcoming, we propose a new computational method in which maximum information is supposed to be already achieved before learning. By this computational method, we force networks to converge much faster. In addition, information loss is proposed in which difference in formation between an original network and network without an input unit is measured. If the in formation loss for the unit is large, the input unit should a very important role. By forced information with the infor mation loss, information-theoretic competitive learning is expected to be applied to large-scale practical problems.
Important Links:
Go Back
