Neural Network Design and the Complexity of Learning

Using the tools of complexity theory, Stephen Judd develops a formal description ofassociative learning in connectionist networks. He rigorously exposes the computational difficultiesin training neural networks and explores how certain design principles will or will not make theproblems easier.Judd looks beyond the scope of any one particular learning rule, at a level abovethe details of neurons. There he finds new issues that arise when great numbers of neurons areemployed and he offers fresh insights into design principles that could guide the construction ofartificial and biological neural networks.The first part of the book describes the motivations andgoals of the study and relates them to current scientific theory. It provides an overview of themajor ideas, formulates the general learning problem with an eye to the computational complexity ofthe task, reviews current theory on learning, relates the book's model of learning to other modelsoutside the connectionist paradigm, and sets out to examine scale-up issues in connectionistlearning.Later chapters prove the intractability of the general case of memorizing in networks,elaborate on implications of this intractability and point out several corollaries applying tovarious special subcases. Judd refines the distinctive characteristics of the difficulties withfamilies of shallow networks, addresses concerns about the ability of neural networks to generalize,and summarizes the results, implications, and possible extensions of the work.J. Stephen Judd isVisiting Assistant Professor of Computer Science at The California Institute of Technology. NeuralNetwork Design and the Complexity of Learning is included in the Network Modeling and Connectionismseries edited by Jeffrey Elman.