Connectionism A Hands-on Approach

CONNNECTIONISM is a "hands on" introduction to connectionist modeling. Three different types of connectionist architectures distributed associative memory, perceptron, and multilayer perceptron are explored. In an accessible style, Dawson provides a brief overview of each architecture, a detailed introduction on how to use a program to explore this network, and a series of practical exercises that are designed to highlight the advantages, and disadvantages, of each and to provide a "road map" to the field of cognitive modeling. This book is designed to be used as a stand-alone volume, or alongside Minds and Machines: Connectionism and Psychological Modeling (Blackwell Publishing, 2004). An accompanying website is available at www.bcp.psych.ualberta.ca/~mike/Book3/ and includes practice exercises and software, as well as the files and blank exercise sheets that are required for performing the exercises.

Michael R. W. Dawson is a member of the Department of Psychology and the Biological Computation Project at the University of Alberta, Canada. He is the author of Understanding Cognitive Science (Blackwell , 1998) and Minds and Machines (Blackwell, 2004).

Hands-On Connectionism

1

(4)

Connectionism in Principle and in Practice

1

(1)

The Organization of This Book

2

(3)

The Distributed Associative Memory

5

(4)

The Paired Associate Task

5

(1)

The Standard Pattern Associator

6

(2)

Exploring the Distributed Associative Memory

8

(1)

The James Program

9

(13)

Introduction

9

(1)

Installing the Program

9

(1)

Teaching a Distributed Memory

10

(5)

Testing What the Memory Has Learned

15

(6)

Using the Program

21

(1)

Introducing Hebb Learning

22

(8)

Overview of the Exercises

22

(1)

Hebb Learning of Basis Vectors

22

(2)

Hebb Learning of Orthonormal, Non-Basis Vectors

24

(6)

Appendix -- Creating mutually orthogonal vectors with Maple

26

(4)

Limitations of Hebb Learning

30

(7)

Introduction

30

(1)

The Effect of Repetition

30

(1)

The Effect of Correlation

31

(6)

Appendix -- Creating the linearly independent set of vectors

33

(4)

Introducing the Delta Rule

37

(4)

Introduction

37

(1)

The Delta Rule

37

(2)

The Delta Rule and the Effect of Repetition

39

(1)

The Delta Rule and the Effect of Correlation

39

(2)

Distributed Networks and Human Memory

41

(5)

Background on the Paired Associate Paradigm

41

(2)

The Effect of Similarity on the Distributed Associative Memory

43

(3)

Limitations of Delta Rule Learning

46

(2)

Introduction

46

(1)

The Delta Rule and Linear Dependency

46

(2)

The Perceptron

48

(10)

Introduction

48

(1)

The Limits of Distributed Associative Memories, and Beyond

48

(4)

Properties of the Perceptron

52

(5)

What Comes Next

57

(1)

The Rosenblatt Program

58

(14)

Introduction

58

(1)

Installing the Program

58

(1)

Training a Perceptron

59

(5)

Testing What the Memory Has Learned

64

(8)

Perceptrons and Logic Gates

72

(9)

Introduction

72

(1)

Boolean Algebra

72

(4)

Perceptrons and Two-Valued Algebra

76

(5)

Performing More Logic With Perceptrons

81

(5)

Two-Valued Algebra and Pattern Spaces

81

(2)

Perceptrons and Linear Separability

83

(3)

Appendix -- The DawsonJots Font

85

(1)

Value Units and Linear Nonseparability

86

(5)

Linear Separability and Its Implications

86

(1)

Value Units and the Exclusive-Or Relation

87

(1)

Value Units and Connectedness

88

(3)

Network By Problem Type Interactions

91

(3)

All Networks Were Not Created Equally

91

(1)

Value Units and the Two-Valued Algebra

92

(2)

Perceptrons and Generalization

94

(5)

Background

94

(1)

Generalization and Savings for the 9-Majority Problem

94

(5)

Animal Learning Theory and Perceptrons

99

(9)

Discrimination Learning

99

(3)

Linearly Separable Versions of Patterning

102

(6)

The Multilayer Perceptron

108

(6)

Creating Sequences of Logical Operations

108

(2)

Multilayer Perceptrons and the Credit Assignment Problem

110

(2)

The Implications of the Generalized Delta Rule

112

(2)

The Rumelhart Program

114

(15)

Introduction

114

(1)

Installing the Program

114

(1)

Training a Multilayer Perceptron

115

(5)

Testing What the Network Has Learned

120

(9)

Beyond the Perceptron's Limits

129

(4)

Introduction

129

(1)

The Generalized Delta Rule and Exclusive-Or

130

(3)

Symmetry as a Second Case Study

133

(4)

Background

133

(1)

Solving Symmetry Problems With Multilayer Perceptrons

133

(4)

How Many Hidden Units?

137

(8)

Background

137

(3)

How Many Hidden Value Units Are Required for 5-Bit Parity?

140

(5)

Scaling Up With the Parity Problem

145

(6)

Overview of the Exercises

145

(1)

Background

145

(1)

Exploring the Parity Problem

146

(5)

Selectionism and Parity

151

(6)

Background

151

(2)

From Connectionism to Selectionism

153

(4)

Interpreting a Small Network

157

(6)

Background

157

(2)

A Small Network

159

(2)

Interpreting This Small Network

161

(2)

Interpreting Networks of Value Units

163

(11)

Background

163

(6)

Banding in the First Monks Problem

169

(3)

Definite Features in the First Monks Problem

172

(2)

Interpreting Distributed Representations

174

(9)

Background

174

(4)

Interpreting a 5-Parity Network

178

(5)

Creating Your Own Training Sets

183

(5)

Background

183

(1)

Designing and Building a Training Set

183

(5)

References

188

(7)

Index of Names

195

(3)

Subject Index

198

What is included with this book?

The New copy of this book will include any supplemental materials advertised. Please check the title of the book to determine if it should include any access cards, study guides, lab manuals, CDs, etc.

The Used, Rental and eBook copies of this book are not guaranteed to include any supplemental materials. Typically, only the book itself is included. This is true even if the title states it includes any access cards, study guides, lab manuals, CDs, etc.

FREE SHIPPING

Amazon no longer offers textbook rentals. We do!

Connectionism A Hands-on Approach

Summary

Author Biography

Table of Contents

Supplemental Materials