Language Testing The Social Dimension

Winner of the SAGE/ILTA Award for Best Book on Language Testing 2009This volume focuses on the social aspects of language testing, including assessment of socially situated language use and societal consequences of language tests. The authors argue that traditional approaches to ensuring social fairness in tests go some way to addressing social concerns, but a broader perspective is necessary to examine the functions of tests on a societal scale. Considers these issues in relation to language assessment in oral proficiency interviews, and to the assessment of second language pragmatics. Argues that traditional approaches to ensuring social fairness in tests go some way to addressing social concerns, but a broader perspective is necessary if we are to fully understand the social dimension of language assessment.

Tim McNamara is Professor of Applied Linguistics at The University of Melbourne, where he gained his Ph.D. in Applied Linguistics, and where he helped establish the Language Testing Research Centre, of which he was subsequently Director. His research interests include language testing, language teaching, and language and identity. His recent work has focused on the relevance of contemporary social theory to concerns in language assessment and other areas of applied linguistics, including the role of language tests in the processing of the claims of asylum seekers and in citizenship procedures. He is the author of Language Testing (2000) and has published widely in applied linguistics journals.

Carsten Roever is a Lecturer in Applied Linguistics at the University of Melbourne. He has a Ph.D. in Second Language Studies from the University of Hawai‘i at Manoa. His main research interest is the assessment of second language pragmatics. Some of his favorite topics are acquisition of pragmatic knowledge, second language aptitude, web-based language testing, and research design. He has published in Language Testing, Language Learning & Technology, and various edited collections.

Series Editor's Foreword

xi

Acknowledgments

xv

Introduction

1

(8)

Structure of This Volume

4

(5)

Validity and the Social Dimension of Language Testing

9

(34)

Cronbach

9

(3)

Messick

12

(12)

Messick on Construct Validity

15

(4)

Construct Definition and Validation: Mislevy

19

(5)

Kane's Approach to Test Score Validation

24

(6)

The Social Dimension of Validity in Language Testing

30

(7)

The Broader Social Context: The Limits of Validity

37

(3)

Conclusion

40

(3)

Notes

42

(1)

The Social Dimension of Proficiency: How Testable Is It?

43

(38)

Assessing Face-to-Face Interaction

44

(10)

Assessing Second Language Pragmatics

54

(9)

Second Language Pragmatics

54

(2)

Tests of Second Language Pragmatics

56

(1)

Testing Sociopragmatics

56

(2)

Testing Implicature

58

(2)

Testing Pragmalinguistics

60

(1)

Apologies and Requests for Chinese EFL Learners

61

(2)

The Tension in Testing Pragmatics: Keeping It Social But Practical

63

(2)

DCT Items in Pragmatics Tests: A Case Study in Limiting the Construct

65

(5)

Methodological Issues in DCTs

67

(3)

Testing Sociopragmatics: Role Plays and Appropriateness Judgments

70

(3)

Foreigner-Specific Norms?

73

(2)

Assessing Pragmatic Aptitude

75

(2)

The Way(s) Forward

77

(1)

Conclusion

78

(3)

Notes

78

(3)

Psychometric Approaches to Fairness: Bias and DIF

81

(48)

Bias and Differential Item Functioning: Definition and History

82

(9)

Bias

82

(1)

Differential Item Functioning

83

(2)

History

85

(6)

The Where and Wherefore of DIF

91

(2)

Methods for Detecting DIF

93

(1)

Early DIF Techniques: Different Item Difficulties for Different Groups

94

(4)

The Delta Plot in Language Testing DIF Studies

95

(3)

Contingency Table Methods and Other Nonparametric Approaches

98

(6)

Item Difficulty and Matched Groups: The Standardization Procedure

101

(1)

Contingency Table Methods in Language Testing

101

(3)

DIF and Item-Response Theory

104

(9)

Calculating DIF in IRT Models

108

(3)

IRT and Language Testing DIF Research

111

(2)

Comparison of DIF Techniques

113

(2)

Other Approaches

115

(7)

Logistic Regression

115

(1)

Logistic Regression in Language Testing DIF Research

116

(2)

Generalizability Theory

118

(1)

Multifaceted Rasch Measurement

119

(3)

Problems with DIF Analyses

122

(2)

Dealing With DIF and Bias: A Judgment Call

124

(3)

Conclusion

127

(2)

Notes

128

(1)

Fairness Reviews and Codes of Ethics

129

(20)

Preempting DIF: Fairness Reviews

129

(8)

University of Cambridge ESOL Examinations

130

(1)

Educational Testing Service

130

(7)

Fairness as a Professional Obligation: Codes of Ethics and Practice

137

(3)

Codes of Ethics and Practice

138

(2)

Codes of Ethics and Practice in Language Testing

140

(3)

The ILTA Codes of Ethics and Practice

141

(1)

The ALTE Code of Practice and Quality Management Checklists

142

(1)

Codes of Ethics in Related Areas

143

(3)

Future Directions

146

(1)

Summary

147

(2)

Notes

148

(1)

Language Tests and Social Identity

149

(54)

Shibboleth Tests

150

(9)

Language Tests, Identity, and the Denial of Rights

159

(6)

Language Tests and the Claims of Asylum Seekers

165

(8)

Language Tests and Immigration

173

(6)

Language Tests and Citizenship

179

(4)

Canada: Language Tests and the Official Languages Policy

183

(13)

Conclusion: Language Tests and Subjectivity

196

(7)

Notes

199

(4)

Language Assessments at School: Social Values and Policy

203

(44)

Values and Policy Dimensions of Test Constructs

205

(8)

The Japanese Senior High School Entrance Examination in English

205

(5)

The Common European Framework of Reference for Languages

210

(3)

Policy Implementation Through Standards

213

(3)

Standards: Format and Structure

216

(1)

Standards: Conceptual Organization (the Construct)

217

(7)

Competing Views of the Construct: Literacy

219

(3)

Catering for Second/Additional Language Learners in Policy-Driven Literacy Assessments

222

(2)

Standards: The Process of Formulation

224

(1)

Standards: Gathering Evidence---Teacher Assessment and Formal Tests

225

(10)

Teacher Assessment

226

(1)

Formal Tests

227

(2)

Formal Tests and the Evaluation of Policy Initiatives

229

(1)

The NALSAS Evaluation Project

230

(3)

Policy-Related International Comparative Assessments: PISA

233

(2)

Standards and Impact: The No Child Left Behind Act and ESL Learners

235

(9)

The NCLB Itself

236

(2)

The Law's Impact: Consequences Intended and Unintended

238

(2)

ESL Learners in the Larger Student Population

240

(4)

Conclusion

244

(3)

Notes

245

(2)

Where to From Here? Implications for Research and Training

247

(10)

Directions for Research

250

(4)

Implications for Training

254

(3)

References

257

(28)

Index

285

What is included with this book?

The New copy of this book will include any supplemental materials advertised. Please check the title of the book to determine if it should include any access cards, study guides, lab manuals, CDs, etc.

The Used, Rental and eBook copies of this book are not guaranteed to include any supplemental materials. Typically, only the book itself is included. This is true even if the title states it includes any access cards, study guides, lab manuals, CDs, etc.

FREE SHIPPING

Amazon no longer offers textbook rentals. We do!

Language Testing The Social Dimension

Summary

Author Biography

Table of Contents

Supplemental Materials