R. Kamimura (Japan)
: Competitive learning, information maxi mization, dead neurons
In this paper, we propose a new information theoretic competitive learning. In realizing competition, neither the winner-all-take algorithm nor lateral inhibition are used. Instead, the new method is based upon mutual information maximization between input patterns and competitive units. In maximizing mutual information, the entropy of competitive units is increased as much as possible. This means that all competitive units must equally be used in our framework. Thus, no under-utilized neurons or dead neurons are generated. We applied our method to a simple artificial data problem and an actual road classification problem. In both cases, experimental results confirmed that the new method can produce the final solutions almost independently of initial conditions and classification performance is significantly improved.
Important Links:
Go Back
