Assessing Teacher Dispositions : Five Standards-Based Steps to Valid Measurement Using the DAATS Model
, by Judy R. Wilkerson- ISBN: 9781412953689 | 1412953685
- Cover: Paperback
- Copyright: 5/16/2007
List of Tables, Figures, and Boxes | p. x |
List of Abbreviations and Acronyms | p. xv |
Disposition Assessments Aligned With Teacher Standards (DAATS) Steps and Worksheets | p. xvi |
Foreword | p. xix |
Preface | p. xxi |
Acknowledgments | p. xxv |
About the Authors | p. xxvi |
What Are Dispositions, and Why Should We Measure Them? | p. 1 |
The Importance of Measuring Dispositions | p. 2 |
The Challenge | p. 3 |
What Are Standards-Based Dispositions? | p. 4 |
Hierarchical Relationships Among Knowledge, Skills, and Dispositions | p. 6 |
Remembering Bloom | p. 7 |
Dispositions and Accreditation: Requirements and Definitions | p. 7 |
Measuring Dispositions: Sources of Confusion | p. 10 |
Measuring Dispositions: Morals, Ethics, or Standards Based? | p. 11 |
Different Construct, Different Assessments, Similar Assessment Design Process | p. 14 |
Wrap-Up | p. 15 |
Questions for Exploration | p. 16 |
What Have You Noticed? | p. 17 |
Assessment Belief Scale-Beliefs About Assessment | p. 18 |
Cognitive, Affective, and Psychomotor Objectives and Assessments | p. 20 |
Methods for Assessing Dispositions | p. 21 |
A Conceptual Framework for Measuring Dispositions | p. 21 |
Measuring Teacher Dispositions: The State of the Art | p. 23 |
Back to Basics: Bloom and Krathwohl | p. 24 |
Available Methods for Measuring Dispositions or Affect | p. 26 |
Selected-Response Methods | p. 27 |
Constructed-Response Methods | p. 28 |
Observed Performance | p. 30 |
Projective Techniques | p. 31 |
The Importance of Inference in Measuring Dispositions | p. 31 |
Wrap-Up | p. 34 |
Questions for Exploration | p. 35 |
Bloom and the INTASC Principles | p. 36 |
Field Work | p. 37 |
Review Your Feelings | p. 38 |
Where We Have Been So Far | p. 40 |
DAATS Step 1-Assessment Design Inputs | p. 41 |
Why Are Purpose, Use, Propositions, and Content So Important? | p. 43 |
Define the Purpose(s) and Use(s) of the System | p. 43 |
Define the Propositions or Principles That Guide the System | p. 46 |
Define the Conceptual Framework or Content of the System | p. 48 |
Review Local Factors That Impact the System | p. 52 |
Wrap-Up | p. 53 |
Purpose, Use, Propositions, Content, and Context Checksheet | p. 55 |
Purpose, Use, and Content Draft | p. 57 |
Propositions | p. 58 |
Contextual Analysis | p. 59 |
Where We Have Been So Far | p. 60 |
DAATS Step 2-Planning With a Continuing Eye on Valid Assessment Decisions | p. 61 |
Analyze Standards and Indicators | p. 62 |
All Those Indicators | p. 62 |
Why Bother? | p. 65 |
Visualize the Teacher Demonstrating the Affective Targets | p. 66 |
Select Assessment Methods at Different Levels of Inference | p. 69 |
Build an Assessment Framework Correlating Standards and Methods | p. 71 |
Wrap-Up | p. 74 |
Organizing for Alignment (Version 1) | p. 76 |
Organizing for Alignment (Version 2) | p. 77 |
Visualizing the Dispositional Statements | p. 78 |
Selecting Assessment Methods for INTASC Indicators | p. 79 |
Assessment Methods for INTASC Indicators: Blueprint | p. 80 |
Cost-Benefit and Coverage Analysis of Assessment Methods | p. 81 |
Where We Have Been So Far | p. 82 |
DAATS Step 3-Instrument Development | p. 83 |
Draft Items and Directions for Each Instrument | p. 84 |
Thurstone Agreement Scales | p. 84 |
Questionnaires, Interviews, and Focus Groups | p. 91 |
Observed Performance | p. 96 |
Thematic Apperception Tests or Situation Reflection Assessment | p. 100 |
Review Items for Applicability to Values, Domain Coverage, and Job Relevance | p. 103 |
Wrap-Up | p. 106 |
Creating Scales | p. 108 |
Creating Questionnaires, Interviews, or K-12 Focus Group Protocols | p. 109 |
Creating an Affective Behavior Checklist | p. 110 |
Creating an Affective Behavior Rating Scale | p. 111 |
Creating a Tally Sheet for Affective Observation | p. 111 |
Checklist for Reviewing Scale Drafts | p. 113 |
Review Sheets for Questionnaires and Interviews | p. 114 |
Review Sheets for K-12 Focus Group Protocols | p. 115 |
Checklist for Reviewing Observations and Behavior Checklists | p. 116 |
Coverage Check | p. 117 |
Rating Form for Stakeholder Review | p. 118 |
Where We Have Been So Far | p. 120 |
DAATS Step 4-Decision Making and Data Management | p. 121 |
Develop Scoring Rubrics | p. 122 |
Dichotomous Response Scoring Keys | p. 122 |
Rating Scale Rubrics | p. 123 |
Determine How Data Will Be Combined and Used | p. 127 |
Need for Shared Data | p. 127 |
Data Storage | p. 127 |
Data Aggregation | p. 128 |
Develop Implementation Procedures and Materials | p. 134 |
Preponderance of the Evidence Versus Cut Scores | p. 134 |
Advising and Due Process | p. 135 |
Scoring Procedures | p. 137 |
Implementation | p. 138 |
Wrap-Up | p. 140 |
Explanation of Dichotomous Scoring Decisions | p. 142 |
Rubric Design | p. 143 |
Sample Format for Candidate/Teacher Tracking Form | p. 144 |
Format for Data Aggregation | p. 145 |
Sample Disposition Event Report | p. 146 |
Management Plan | p. 147 |
Where We Have Been So Far | p. 148 |
DAATS Step 5-Credible Data | p. 149 |
What Is Psychometric Integrity, and Why Do We Have to Worry About It? | p. 150 |
Create a Plan to Provide Evidence of Validity, Reliability, Fairness, and Utility | p. 151 |
Elements of a Plan | p. 151 |
Purpose and Use | p. 152 |
Construct Measured | p. 153 |
Interpretation and Reporting of Scores | p. 154 |
Assessment Specifications and Content Map | p. 155 |
Assessor/Rater Selection and Training Procedures | p. 156 |
Analysis Methodology | p. 156 |
External Review Personnel and Methodology | p. 157 |
Evidence of Validity, Reliability, and Fairness (VRF) | p. 157 |
Implement the Plan Conscientiously | p. 173 |
Wrap-Up | p. 174 |
Assessment Specifications | p. 176 |
Analysis of Appropriateness of Decisions for Teacher Failures | p. 177 |
Analysis of Rehire Data | p. 178 |
Program Improvement Record | p. 179 |
Expert Rescoring | p. 180 |
Fairness Review | p. 181 |
Analysis of Remediation Efforts and Equal Opportunity (EO) Impact | p. 182 |
Psychometric Plan Format | p. 183 |
Logistic Ruler for Content Validity | p. 184 |
Computation of the Lawshe (1975) Content Validity Ratio (CVR) | p. 186 |
Disparate-Impact Analysis | p. 187 |
Computation of Cohen's (1960) Kappa for Inter-Rater Reliability | p. 190 |
Two Pearson Correlation Coefficients and Scatterplots: Disposition Scores Correlated With Praxis and Portfolio Scores | p. 192 |
Spearman Correlation Coefficient and Scatterplot: Disposition Scores Correlated With Principal Ratings | p. 195 |
Correlation Matrix and Scatterplots for Knowledge, Impact, Dispositions, Skills (KIDS) | p. 197 |
t-Test Comparing Dispositions of Mathematics and Science Teachers | p. 199 |
DIP Analysis for Programs | p. 201 |
Using Teacher Scores for Continuous Improvement | p. 203 |
Reasons Why We Use the Rasch Model | p. 204 |
The Classical Approach | p. 206 |
A Quick Overview of Where Rasch Fits Into the Grand Scheme of IRT Models | p. 208 |
Rasch: The Basics | p. 208 |
Getting Started | p. 210 |
Differences That Item Writers Make | p. 211 |
Guttman Scaling | p. 211 |
A Sample Rasch Ruler | p. 213 |
From Pictures to Numbers | p. 214 |
The Fit Statistic | p. 219 |
Gain Scores-Real or Imagined? | p. 221 |
Ratings and Raters | p. 221 |
Learning More About Rasch | p. 227 |
Wrap-Up | p. 228 |
A Decision-Making Tool for Measurement | p. 229 |
Legal and Psychometric Issues-The Return of the Pied Piper | p. 231 |
Why Not Portfolios? | p. 232 |
Why the Pied Piper? | p. 232 |
What If?? A Legal Scenario: Mary Beth JoAnne Sues XYZ University | p. 233 |
MBJ Helps Us to Understand the Convergence of Psychometrics and Legal Requirements | p. 233 |
Background "Facts" | p. 234 |
Scenario #1 | p. 236 |
Scenarios #2, #3, and #4 | p. 237 |
Psychometric Issues and Legal Challenges in the Real World | p. 238 |
Legal Issues and Precedents | p. 240 |
Three Landmark Dispositions Cases in Two Years | p. 241 |
Tide Changing in NCATE | p. 242 |
Standards Are the Vanguard! | p. 243 |
MBJ Revisited | p. 244 |
End Note | p. 245 |
INTASC Principles and Disposition Indicators | p. 247 |
Glossary | p. 251 |
References | p. 261 |
Index | p. 267 |
Table of Contents provided by Ingram. All Rights Reserved. |
The New copy of this book will include any supplemental materials advertised. Please check the title of the book to determine if it should include any access cards, study guides, lab manuals, CDs, etc.
The Used, Rental and eBook copies of this book are not guaranteed to include any supplemental materials. Typically, only the book itself is included. This is true even if the title states it includes any access cards, study guides, lab manuals, CDs, etc.
Digital License
You are licensing a digital product for a set duration. Durations are set forth in the product description, with "Lifetime" typically meaning five (5) years of online access and permanent download to a supported device. All licenses are non-transferable.
More details can be found here.