Improving Surveys with Paradata Analytic Uses of Process Information

FRAUKE KREUTER is Associate Professor in the Joint Program in Survey Methodology at the University of Maryland; Professor of Statistics at Ludwig Maximilian University of Munich, Germany; and head of the Statistical Methods Research Department at the Institute for Employment Research (IAB) in Nuremberg, Germany.

1 Improving Surveys with Paradata: Introduction 1
Frauke Kreuter

1.1 Introduction 1
1.2 Paradata and Metadata 3

1.3 Auxiliary Data and Paradata 4

1.4 Paradata in the Total Survey Error Framework 4

1.5 Paradata in Survey Production 5

1.6 Special Challenges in the Collection and Use of Paradata 7

1.7 Future of Paradata 8

PART I PARADATA AND SURVEY ERRORS

2 Paradata for Nonresponse Error Investigation 3
Frauke Kreuter and Kristen Olson

2.1 Introduction 3

2.2 Sources of Paradata 4

2.3 Nonresponse Rates and Nonresponse Bias 10

2.4 Paradata and Responsive Designs 20

2.5 Paradata and Nonresponse Adjustment 21

2.6 Issues in Practice 22

2.7 Summary and Take Home Messages 24

3 Collecting Paradata for Measurement Error Evaluations 33
Kristen Olson and Bryan Parkhurst

3.1 Introduction 33

3.2 Paradata and Measurement Error 34

3.3 Types of paradata 38

3.4 Differences in Paradata by Modes 45

3.5 Turning paradata into data sets 51

3.6 Summary 55

4 Analyzing Paradata to Investigate Measurement Error 63
Ting Yan and Kristen Olson

4.1 Introduction 63

4.2 Review of Empirical Literature on the Use of Paradata for Measurement Error Investigation 64

4.3 Analyzing paradata 66

4.4 Four empirical examples 73

4.5 Cautions 81

4.6 Concluding Remarks 82

5 Paradata for Coverage Research 89
Stephanie Eckman

5.1 Introduction 89

5.2 Housing Unit Frames 93

5.3 Telephone Number Frames 101

5.4 Household Rosters 103

5.5 Population Registers 105

5.6 Subpopulation Frames 106

5.7 Web Surveys 106

5.8 Conclusion 107

PART II PARADATA IN SURVEY PRODUCTION

6 Design and Management Strategies for Paradata-Driven Responsive Design 117
Nicole G. Kirgis and James M. Lepkowski

6.1 Introduction 117

6.2 From Repeated Cross-Section to Continuous Design 118

6.3 Paradata Design 123

6.4 Key Design Change 1: A New Employment Model 128

6.5 Key Design Change 2: Field Efficient Sample Design 130

6.6 Key Design Change 3: Replicate Sample Design 131

6.7 Key Design Change 4: Responsive Design Sampling of Nonrespondents in a Second Phase 132

6.8 Key Design Change 5: Active Responsive Design Interventions 134

6.9 Concluding Remarks 135

7 Using Paradata-Driven Models to Improve Contact Rates 141
James Wagner

7.1 Introduction 141

7.2 Background 142

7.3 The Survey Setting 144

7.4 Experiments: Data and Methods 145

7.5 Experiments: Results 157

7.6 Discussion 162

8 Using Paradata to Study Response to Within-Survey Requests 169
Joseph W. Sakshaug

8.1 Introduction 169

8.2 Consent to Link Survey and Administrative Records 173

8.3 Consent to Collect Biomeasures in Population-Based Surveys 177

8.4 Switching Data Collection Modes 179

8.5 Income Item Nonresponse and Quality of Income Reports 181

8.6 Summary 185

9 Managing Data Quality Indicators with Paradata-Based Statistical Quality Control Tools 191
Matt Jans, Robyn Sirkis and David Morgan

9.1 Introduction 191

9.2 Defining and Choosing Key Performance Indicators (KPIs) 193

9.3 KPI Displays and the Enduring Insight of Walter Shewhart 201

9.4 Implementation Steps for Survey Analytic Quality Control with Paradata Control Charts 212

9.5 A Method for Improving Measurement Process Quality Indicators 214

9.6 Reections on SPC, Visual Data Displays, and Challenges to Quality Control 221

9.7 Some Advice on Using Charts 223

 Appendix 225

10 Paradata as Input to Monitoring Representativeness and Measurement Profiles 233
Barry Schouten and Melania Calinescu

10.1 Introduction 233

10.2 Measurement profiles 235

10.3 Tools for monitoring nonresponse and measurement profiles 238

10.4 Monitoring and improving response: a demonstration using the LFS 243

10.5 Including paradata observations on households and persons 254

10.6 General discussion 256

10.7 Take home messages 257

PART III SPECIAL CHALLENGES

11 Paradata in Web Surveys 263
Mario Callegaro

11.1 Survey data types 263

11.2 Collection of paradata 264

11.3 Typology of paradata in web surveys 265

11.4 Using paradata to change the survey in real time: adaptive scripting 273

11.5 Paradata in online panels 274

11.6 Software to collect paradata 274

11.7 Analysis of paradata: levels of aggregation 275

11.8 Privacy and ethical issues in collecting web survey paradata 276

11.9 Summary and conclusions on paradata in web surveys 277

12 Modeling Call Record Data: Examples from Cross-Sectional and Longitudinal Surveys 283
Gabriele B. Durrant, Julia D'Arrigo and Gerrit Müller

12.1 Introduction 283

12.2 Call record data 285

12.3 Modeling approaches 287

12.4 Illustration of call record data analysis using two example datasets 294

12.5 Summary 305

13 Bayesian Penalized Spline Models for Statistical Process Monitoring of Survey Paradata Quality Indicators 311
Joseph L. Schafer

13.1 Introduction 311

13.2 Overview of splines 316

13.3 Penalized splines as linear mixed models 323

13.4 Bayesian methods 327

13.5 Extensions 330

14 The Quality of Paradata: A Literature Review 341
Brady T. West and Jennifer Sinibaldi

14.1 Introduction 341

14.2 Existing Studies Examining the Quality of Paradata 342

14.3 Possible Mechanisms Leading to Error in Paradata 354

14.4 Take Home Messages 357

15 The Effects of Errors in Paradata on Weighting Class Adjustments: A Simulation Study 363
Brady T. West

15.1 Introduction 363

15.2 Design of Simulation Studies 367

15.3 Simulation Results 372

15.4 Take Home Messages 386

15.5 Future Research 388

Topic Index 393

The New copy of this book will include any supplemental materials advertised. Please check the title of the book to determine if it should include any access cards, study guides, lab manuals, CDs, etc.

The Used, Rental and eBook copies of this book are not guaranteed to include any supplemental materials. Typically, only the book itself is included. This is true even if the title states it includes any access cards, study guides, lab manuals, CDs, etc.