TY - JOUR
T1 - Dynamics of learning in linear feature-discovery networks
AU - Leen, Tood K.
N1 - Funding Information:
The author thanks Professor Dan Aammerstrom and Dr Bill Baird for lively discussion. David Rw helped automate the bifurcation calculations and Vince Weatherill worked on the figures. The reviewers and several of my colleagues have provided valuable comments on the manuscript. This work was supported by the Office of Naval Research under contracts N0001488-K-0329 and N0001490-1349 and by DARPA grant MDA 972-68-3-1004.
PY - 1991
Y1 - 1991
N2 - In this paper I addresses the dynamics of learning in unsupervised neural feature-discovery networks. The models introduced incorporate feedforward connections modified by a Hebb law, and recurrent lateral connections modified by an anti-Hebb law. Conditions for stability of equilibria are derived, and bifurcation theory is used to explore the behaviour near loss of stability. Stability of the equilibria is shown to depend on the learning rates in the system, and on the statistics of the input signal. The bifurcation analyses reveal previously overlooked behaviours, including equilibria that consist of mixtures of the principal eigenvectors of the input autocorrelation, as well as limit cycles. The results provide a more complete picture of adaptation in Hebbian feature-discovery networks.
AB - In this paper I addresses the dynamics of learning in unsupervised neural feature-discovery networks. The models introduced incorporate feedforward connections modified by a Hebb law, and recurrent lateral connections modified by an anti-Hebb law. Conditions for stability of equilibria are derived, and bifurcation theory is used to explore the behaviour near loss of stability. Stability of the equilibria is shown to depend on the learning rates in the system, and on the statistics of the input signal. The bifurcation analyses reveal previously overlooked behaviours, including equilibria that consist of mixtures of the principal eigenvectors of the input autocorrelation, as well as limit cycles. The results provide a more complete picture of adaptation in Hebbian feature-discovery networks.
UR - http://www.scopus.com/inward/record.url?scp=0010226653&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=0010226653&partnerID=8YFLogxK
U2 - 10.1088/0954-898X_2_1_005
DO - 10.1088/0954-898X_2_1_005
M3 - Article
AN - SCOPUS:0010226653
SN - 0954-898X
VL - 2
SP - 85
EP - 105
JO - Network: Computation in Neural Systems
JF - Network: Computation in Neural Systems
IS - 1
ER -