Implementation Techniques

Note: Supplemental materials are not guaranteed with Rental or Used book purchases.

ISBN: 9780124438637 | 0124438636
Cover: Hardcover
Copyright: 11/13/1997

Buy New

Print on Demand: 2-4 Weeks. This item cannot be cancelled or returned.

$121.17
eBook

Available Instantly

Online: 1825 Days

Downloadable: Lifetime Access

$132.75

This volume covers practical and effective implementation techniques, including recurrent methods, Boltzmann machines, constructive learning with methods for the reduction of complexity in neural network systems, modular systems, associative memory, neural network design based on the concept of the Inductive Logic Unit, and a comprehensive treatment of implementations in the area of data classification. Numerous examples enhance the text. Practitioners, researchers,and students in engineering and computer science will find Implementation Techniques a comprehensive and powerful reference.

Contributors

xiii

(2)

Preface

Recurrent Neural Networks: Identification and Other System Theoretic Properties

(50)

Francesca Albertini

Paolo Dai Pra

I. Introduction

(3)

II. Recurrent Neural Networks

(13)

A. The Model

(1)

B. Examples of Admissible Activation Functions

(2)

C. State Observability

(4)

D. Identifiability and Minimality

(2)

E. Controllability and Forward Accessibility

(4)

III. Mixed Networks

(29)

A. The Model

(1)

B. State Observability

(6)

C. Proofs on Observability

(7)

D. Identifiability and Minimality

(3)

E. Proofs on Identifiability and Minimality

(13)

IV. Some Open Problems

(1)

References

(4)

Boltzmann Machines: Statistical Associations and Algorithms for Training

(40)

N. H. Anderson

D. M. Titterington

I. Introduction

(2)

II. Relationship with Markov Chain Monte Carlo Methods

(2)

III. Deterministic Origins of Boltzmann Machines

(1)

IV. Hidden Units

(1)

V. Training a Boltzmann Machine

(10)

A. The Case without Hidden Units

(1)

B. The Case with Hidden Units

(1)

C. Relationship with the EM Algorithm

(1)

D. Discussion of the Case with Training Data

(1)

E. Information Geometry of Boltzmann Machines

(1)

F. Training Boltzmann Machines by Alternating Minimization

(1)

G. Parameter Estimation Using Mean-Field Approximations

(1)

H. The Method of Gelfand and Carlin

(1)

I. Discussion

(1)

VI. An Example with No Hidden Unit

(6)

A. Exact Likelihood Estimation

(2)

B. The Method of Geyer and Thompson

(1)

C. Alternating Minimization

(1)

D. Conclusions from the Example

(1)

VII. Examples with Hidden Units

(8)

A. Description of the Examples

(1)

B. The Gradient-Descent Method

(3)

C. Direct Maximization

(1)

D. Application of the EM Algorithm

(5)

VIII. Variations on the Basic Boltzmann Machine

(5)

A. The Boltzmann Perceptron

(1)

B. Polytomous Boltzmann Machines

(2)

C. Neal's Approach Based on Belief Networks

(2)

IX. The Future Prospects for Boltzmann Machines

(1)

References

(4)

Constructive Learning Techniques for Designing Neural Network Systems

(56)

Colin Campbell

I. Introduction

(4)

II. Classification

(16)

A. Introduction

(1)

B. The Pocket Algorithm

(1)

C. Tower and Cascade Architectures

(4)

D. Tree Architectures: The Upstart Algorithm

101

(2)

E. Constructing Tree and Cascade Architectures by Using Dichotomies

103

(6)

F. Constructing Neural Networks with a Single Hidden Layer

109

(2)

G. Summary

111

(1)

III. Regression Problems

111

(13)

A. Introduction

111

(1)

B. The Cascade Correlation Algorithm

111

(5)

C. Node Creation and Node-Splitting Algorithms

116

(3)

D. Constructing Radial Basis Function Networks

119

(4)

E. Summary

123

(1)

IV. Constructing Modular Architectures

124

(5)

A. Introduction

124

(1)

B. Neural Decision Trees

124

(3)

C. Reducing Tree Complexity

127

(1)

D. Other Approaches to Growing Modular Networks

128

(1)

V. Reducing Network Complexity

129

(5)

A. Introduction

129

(1)

B. Weight Pruning

129

(3)

C. Node Pruning

132

(1)

D. Summary

133

(1)

VI. Conclusion

134

(1)

VII. Appendix: Algorithms for Single-Node Learning

135

(4)

A. Binary-Valued Nodes

135

(3)

B. Continuously-Valued Nodes

138

(1)

References

139

(8)

Modular Neural Networks

147

(36)

Kishan Mehrotra

Chilukuri K. Mohan

I. Introduction

147

(2)

II. Why Modular Networks?

149

(3)

A. Computational Requirements

149

(1)

B. Conflicts in Training

150

(1)

C. Generalization

150

(1)

D. Representation

151

(1)

E. Hardware Constraints

151

(1)

III. Modular Network Architectures

152

(1)

IV. Input Decomposition

152

(2)

A. Neocognitron

153

(1)

B. Data Fusion

154

(1)

V. Output Decomposition

154

(3)

VI. Hierarchical Decomposition

157

(2)

VII. Combining Outputs of Expert Modules

159

(8)

A. Maximum Likelihood Method

160

(1)

B. Mixture of Experts Networks

161

(1)

C. Mixture of Linear Regression Model and Gaussian Densities

162

(2)

D. The EM Algorithm

164

(3)

VIII. Adaptive Modular Networks

167

(10)

A. Attentive Modular Construction and Training

169

(3)

B. Adaptive Hierarchical Networks

172

(1)

C. Adaptive Multimodule Approximation Network

173

(3)

D. Blockstart Algorithm

176

(1)

IX. Conclusions

177

(1)

References

178

(5)

Associative Memories

183

(76)

Zong-Ben Xu

Chung-Ping Kwong

I. Introduction

183

(9)

A. Associative Memory Problems

184

(1)

B. Principles of Neural Network Associative Memories

185

(6)

C. Organization of the Chapter

191

(1)

II. Point Attractor Associative Memories

192

(21)

A. Neural Network Models

193

(5)

B. Neurodynamics of PAAMs

198

(7)

C. Encoding Strategies

205

(8)

III. Continuous PAAM: Competitive Associative Memories

213

(19)

A. The Model and Its Significance

214

(3)

B. Competition-Winning Principle

217

(5)

C. Competitive Pattern Recognition Example

222

(6)

D. Application: Perfectly Reacted PAAM

228

(4)

IV. Discrete PAAM: Asymmetric Hopfield-Type Networks

232

(18)

A. Neural Network Models

193

(5)

B. General Convergence Principles

234

(5)

C. Classification Theory for Energy Functions

239

(6)

D. Application: Complete-Correspondence Encoding

245

(5)

V. Summary and Concluding Remarks

250

(3)

References

253

(6)

A Logical Basis for Neural Network Design

259

(50)

Robert L. Fry

Raymond M. Sova

I. Motivation

259

(3)

II. Overview

262

(11)

III. Logic, Probability, and Bearing

273

(5)

IV. Principle of Maximized Bearing and ILU Architecture

278

(15)

V. Optimized Transmission

293

(3)

VI. Optimized Transduction

296

(4)

VII. ILU Computational Structure

300

(2)

VIII. ILU Testing

302

(3)

IX. Summary

305

(1)

Appendix: Significant Marginal and Conditional ILU Distributions

305

(2)

References

307

(2)

Neural Networks Applied to Data Analysis

309

(62)

Aarnoud Hoekstra

Robert P. W. Duin

Martin A. Kraaijveld

I. Introduction

309

(2)

II. Data Complexity

311

(5)

A. Prototype Vectors

311

(2)

B. Subspace or Projection Methods

313

(1)

C. Density Estimation Methods

314

(2)

III. Data Separability

316

(11)

A. Mapping Algorithms

318

(3)

B. Visualizing the Data Space

321

(1)

C. Experiments

322

(5)

IV. Classifier Selection

327

(14)

A. Confidence Values

328

(1)

B. Confidence Estimators

329

(5)

C. Validation Sets

334

(1)

D. Experiments

334

(7)

V. Classifier Nonlinearity

341

(13)

A. A Nonlinearity Measure

341

(3)

B. Experiments

344

(10)

VI. Classifier Stability

354

(12)

A. An Estimation Procedure for Stability

354

(4)

B. Stabilizers

358

(5)

C. Stability Experiment

363

(3)

VII. Conclusions and Discussion

366

(1)

References

367

(4)

Multimode Single-Neuron Arithmetics

371

(26)

Chang N. Zhang

Meng Wang

I. Introduction

371

(3)

II. Defining Neuronal Arithmetics

374

(4)

A. A Brief History of Study

374

(1)

B. Polarization Configuration

375

(1)

C. Composite Operation and Clean Operation

376

(2)

D. Developedness of Arithmetical Modes

378

(1)

III. Phase Space of Neuronal Arithmetics

378

(9)

A. Prototypes of Clean Arithmetics

378

(2)

B. Domain Range

380

(4)

C. Dynamic Range

384

(2)

D. Phase Diagram

386

(1)

IV. Multimode Neuronal Arithmetic Unit

387

(5)

A. Developedness of Arithmetic Operations

387

(2)

B. Multimode Arithmetic Unit

389

(1)

C. Triggering and Transition of Arithmetic Modes

390

(2)

V. Toward a Computing Neuron

392

(3)

VI. Summary

395

(1)

References

395

(2)

Index

397

Please wait while the item is added to your bag...

FREE SHIPPING

Amazon no longer offers textbook rentals. We do!

Implementation Techniques

Summary

Table of Contents

Supplemental Materials