Statistical and Methodological Myths and Urban Legends: Doctrine, Verity and Fable in Organizational and Social Sciences
, by Lance; Charles E.Note: Supplemental materials are not guaranteed with Rental or Used book purchases.
- ISBN: 9780805862379 | 0805862374
- Cover: Hardcover
- Copyright: 10/3/2008
The objective of this book is to provide an up-to-date review of commonly undertaken methodological and statistical practices that are sustained, in part, upon sound rationale and justification and, in part, upon unfounded lore. The practices themselves are not necessarily intrinscially faulty. Rather, it is often the reasoning why or rationalization used to justify the practices that is questionable. In this book, a group of scholars look at statistical and urban myths and legends and suggest what the state of practice should be. This book meets an important need and will be of interest to researchers, students and scholars in the fields of organizational and social sciences. Book jacket.
Preface | p. xv |
About the Editors | p. xvii |
Acknowledgments | p. xix |
Introduction | p. 1 |
Statistical Issues | |
Missing Data Techniques and Low Response Rates: The Role of Systematic Nonresponse Parameters | p. 7 |
Organization of the Chapter | p. 8 |
Levels, Problems, and Mechanisms of Missing Data | p. 8 |
Three Levels of Missing Data | p. 9 |
Two Problems Caused by Missing Data (External Validity and Statistical Power) | p. 9 |
Missingness Mechanisms (MCAR, MAR, and MNAR) | p. 9 |
Missing Data Treatments | p. 11 |
A Fundamental Principle of Missing Data Analysis | p. 11 |
Missing Data Techniques (Listwise and Pairwise Deletion, ML, and MI) | p. 13 |
Systematic Nonresponse Parameters (d[subscript miss] and f[superscript 2 subscript miss]) | p. 14 |
Theory of Survey Nonresponse | p. 17 |
Missing Data Legends | p. 21 |
"Low Response Rates Invalidate Results" | p. 21 |
"When in Doubt, Use Listwise or Pairwise Deletion" | p. 24 |
Applications | p. 26 |
Longitudinal Modeling | p. 26 |
Within-Group Agreement Estimation | p. 27 |
Meta-analysis | p. 27 |
Social Network Analysis | p. 28 |
Moderated Regression | p. 29 |
Conclusions | p. 29 |
Future Research on d[subscript miss] and f[superscript 2 subscript miss] | p. 30 |
Missing Data Techniques | p. 31 |
References | p. 31 |
Appendix | p. 35 |
Derivation of Response Rate Bias for the Correlation (Used to Generate Figure 1.1c) | p. 35 |
The Partial Revival of a Dead Horse? Comparing Classical Test Theory and Item Response Theory | p. 37 |
Basic Statement of the Two Theories | p. 38 |
Classical Test Theory | p. 38 |
Item Response Theory | p. 40 |
Criticisms and Limitations of CTT | p. 44 |
Lack of Population Invariance | p. 44 |
Person and Item Parameters on Different Scales | p. 45 |
Correlations Between Item Parameters | p. 46 |
Reliability as a Monolithic Concept | p. 47 |
Criticisms and Limitations of IRT | p. 48 |
Large Sample Sizes | p. 48 |
Strong Assumptions | p. 49 |
Complicated Programs | p. 50 |
Times to Use CTT | p. 50 |
Small Sample Sizes | p. 50 |
Multidimensional Data? | p. 51 |
CTT Supports Other Methodologies | p. 52 |
Times to Use IRT | p. 53 |
Focus on Particular Range of Construct | p. 53 |
Conduct Goodness-of-Fit Studies | p. 53 |
IRT Supports Many Psychometric Tools | p. 55 |
Conclusions | p. 56 |
References | p. 57 |
Four Common Misconceptions in Exploratory Factor Analysis | p. 61 |
The Choice Between Component and Common Factor Analysis Is Inconsequential | p. 62 |
The Component Versus Common Factor Debate: Methodological Arguments | p. 66 |
The Component Versus Common Factor Debate: Philosophical Arguments | p. 68 |
Differences in Results from Component and Common Factor Analysis | p. 69 |
Orthogonal Rotation Results in Better Simple Structure Than Oblique Rotation | p. 71 |
Oblique or Orthogonal Rotation? | p. 71 |
Do Orthogonal Rotations Result in Better Simple Structure? | p. 72 |
The Minimum Sample Size Needed for Factor Analysis Is... (Insert Your Favorite Guideline) | p. 74 |
New Sample Size Guidelines | p. 76 |
The "Eigenvalues Greater Than One" Rule Is the Best Way of Choosing the Number of Factors | p. 79 |
Discussion | p. 83 |
References | p. 85 |
Dr. StrangeLOVE, or: How I Learned to Stop Worrying and Love Omitted Variables | p. 89 |
Theoretical and Mathematical Definition of the Omitted Variables Problem | p. 91 |
Violated Assumptions | p. 96 |
More Complex Models | p. 97 |
Path Coefficient Bias Versus Significance Testing | p. 100 |
Minimizing the Risk of LOVE | p. 102 |
Experimental Control | p. 102 |
More Inclusive Models | p. 103 |
Use Previous Research to Justify Assumptions | p. 103 |
Consideration of Research Purpose | p. 104 |
References | p. 105 |
The Truth(s) on Testing for Mediation in the Social and Organizational Sciences | p. 107 |
Baron and Kenny's (1986) Four-Step Test of Mediation | p. 110 |
Condition/Step 1 | p. 111 |
Condition/Step 2 | p. 111 |
Condition/Step 3 | p. 111 |
Condition/Step 4 | p. 112 |
The Urban Legend: Baron and Kenny's Four-Step Test Is an Optimal and Sufficient Test for Mediation Hypotheses | p. 113 |
The Kernel of Truth About the Urban Legends | p. 113 |
Debunking the Legends | p. 116 |
A Test of a Mediation Hypothesis Should Consist of the Four Steps Articulated by Baron and Kenny (1986) | p. 116 |
Baron and Kenny's (1986) Four-Step Procedure Is the Optimal Test of Mediation Hypotheses | p. 120 |
Fulfilling the Conditions Articulated in the Baron and Kenny (1986) Four-Step Test Is Sufficient for Drawing Conclusions About Mediated Relationships | p. 122 |
Suggestions for Testing Mediation Hypotheses | p. 124 |
Structural Equation Modeling (SEM) as an Analytic Framework | p. 124 |
Summary of Tests of Mediation | p. 127 |
A Heuristic Framework for Classifying Mediation Models | p. 129 |
Summary | p. 135 |
Conclusion | p. 136 |
Author Note | p. 136 |
References | p. 137 |
Seven Deadly Myths of Testing Moderation in Organizational Research | p. 143 |
The Seven Myths | p. 144 |
Product Terms Create Multicollinearity Problems | p. 144 |
Coefficients on First-Order Terms Are Meaningless | p. 146 |
Measurement Error Poses Little Concern When First-Order Terms Are Reliable | p. 148 |
Product Terms Should Be Tested Hierarchically | p. 150 |
Curvilinearity Can Be Disregarded When Testing Moderation | p. 151 |
Product Terms Can Be Treated as Causal Variables | p. 156 |
Testing Moderation in Structural Equation Modeling Is Impractical | p. 158 |
Myths Beyond Moderation | p. 159 |
Conclusion | p. 160 |
References | p. 160 |
Alternative Model Specifications in Structural Equation Modeling: Facts, Fictions, and Truth | p. 165 |
The Core of the Issue | p. 167 |
AMS Strategies | p. 170 |
Equivalent Models | p. 170 |
Nested Models | p. 174 |
Nonnested Alternative Models | p. 177 |
Summary | p. 179 |
AMS in Practice | p. 181 |
Summary | p. 186 |
References | p. 187 |
On the Practice of Allowing Correlated Residuals Among Indicators in Structural Equation Models | p. 193 |
Unraveling the Urban Legend | p. 195 |
Extent of the Problem | p. 195 |
Origins | p. 196 |
A Brief Review of Structural Equation Modeling | p. 197 |
Indicator Residuals | p. 199 |
Model Fit | p. 200 |
An Example | p. 202 |
Why Correlated IRs Improve Fit | p. 204 |
Problems With Correlated Residuals | p. 207 |
Recommendations | p. 209 |
Summary and Conclusions | p. 211 |
References | p. 212 |
Methodological Issues | |
Qualitative Research: The Redheaded Stepchild in Organizational and Social Science Research? | p. 219 |
Definitional Issues | p. 221 |
Philosophical Differences in Qualitative and Quantiative Research | p. 222 |
Quantitative and Qualitative Conceptualizations of Validity | p. 223 |
Caveats and Assumptions | p. 225 |
Beliefs Associated With Qualitative Research | p. 225 |
Qualitative Research Does Not Utilize the Scientific Method | p. 225 |
Qualitative Research Lacks Methodological Rigor | p. 226 |
Qualitative Research Contributes Little to the Advancement of Knowledge | p. 228 |
Evaluating the Beliefs Associated With Qualitative Research | p. 229 |
Qualitative Research Does Not Utilize the Scientific Method | p. 234 |
Qualitative Research Is Methodologically Weak | p. 236 |
Qualitative Research Has Weak Internal Validity | p. 236 |
Qualitative Research Has Weak Construct Validity | p. 237 |
Qualitative Research Has Weak External Validity | p. 238 |
Qualitative Research Contributes Little to the Advancement of Knowledge | p. 239 |
The Future of Qualitative Research in the Social and Organizational Sciences | p. 240 |
Concluding Thoughts | p. 241 |
Author Note | p. 242 |
References | p. 242 |
Do Samples Really Matter That Much? | p. 247 |
Kernel of Truth | p. 248 |
Background | p. 251 |
History of the Concern | p. 251 |
The Research Base | p. 253 |
Why Do Samples Seem to Matter So Much? | p. 255 |
People Confuse Random Sampling With Random Assignment | p. 255 |
People Focus on the Wrong Things | p. 257 |
People Rely on Superficial Similarities | p. 259 |
Concluding Thoughts | p. 260 |
Author Note | p. 262 |
References | p. 262 |
Sample Size Rules of Thumb: Evaluating Three Common Practices | p. 267 |
Determine Whether Sample Size Is Appropriate by Conducting a Power Analysis Using Cohen's Definitions of Small, Medium, and Large Effect Size | p. 269 |
Discussion | p. 271 |
Increase the A Priori Type I Error Rate to .10 Because of Your Small Sample Size | p. 273 |
Discussion | p. 275 |
Sample Size Should Include at Least 5 Observations per Estimated Parameter in Covariance Structure Analyses | p. 277 |
Discussion | p. 279 |
Discussion | p. 280 |
Author Note | p. 283 |
References | p. 284 |
When Small Effect Sizes Tell a Big Story, and When Large Effect Sizes Don't | p. 287 |
Effect Size Defined | p. 289 |
The Urban Legend | p. 290 |
The Kernel of Truth | p. 291 |
Quine and Ontological Relativism | p. 292 |
Contextualization | p. 295 |
Inauspicious Designs | p. 296 |
Phenomena With Obscured Consequences | p. 299 |
Phenomena That Challenge Fundamental Assumptions | p. 300 |
The Flip Side: Trivial "Large" Effects | p. 302 |
Conclusion | p. 305 |
References | p. 306 |
So Why Ask Me? Are Self-Report Data Really That Bad? | p. 309 |
The Urban Legend of Self-Report Data and Its Historical Roots | p. 310 |
Construct Validity of Self-Report Data | p. 313 |
Interpreting the Correlations in Self-Report Data | p. 316 |
Social Desirability Responding in Self-Report Data | p. 319 |
Value of Data Collected From Non-Self-Report Measures | p. 325 |
Conclusion and Moving Forward | p. 330 |
References | p. 332 |
If It Ain't Trait It Must Be Method: (Mis)application of the Multitrait-Multimethod Design in Organizational Research | p. 337 |
Background | p. 338 |
Literature Review | p. 342 |
Range of Traits Studied | p. 342 |
Range of Methods Studied | p. 343 |
Not All "Measurement Methods" Are Created Equal | p. 344 |
The Case of Multisource Performance Appraisal | p. 345 |
The Case of AC Construct Validity | p. 347 |
Other Cases | p. 349 |
So, Are Any "Method" Facets Really Method Facets? | p. 350 |
Discriminating Method From Substance, or "If It Looks Like a Method and Quacks Like a Method..." | p. 351 |
References | p. 353 |
Chopped Liver? OK. Chopped Data? Not OK | p. 361 |
Urban Legends Regarding Chopped Data | p. 362 |
Urban Legends Associated With the Occurrence of Chopped Data | p. 363 |
Urban Legends Associated With Chopped Data Techniques | p. 364 |
Urban Legends Associated With Chopped Data Justifications | p. 365 |
Literature Review | p. 366 |
Chopped Data Through the Years | p. 367 |
Prevalence of Chopped Data | p. 370 |
The Occurrence of Chopped Data Over Time | p. 371 |
Chopped Data Across Disciplines | p. 372 |
Types of Chopped Data Approaches | p. 372 |
Evaluating Justifications for Using Chopped Data | p. 374 |
Insufficient or Faulty Justifications (Myths) | p. 374 |
Legitimate Justifications (Truths) | p. 376 |
Advantages of, Disadvantages of, and Recommendations for Using Chopped Data | p. 377 |
(Perceived) Advantages of Chopping Data | p. 378 |
Disadvantages of Chopping Data | p. 378 |
Recommendations When Faced With Chopping Data | p. 382 |
Conclusion | p. 383 |
References | p. 383 |
Subject Index | p. 387 |
Author Index | p. 401 |
Table of Contents provided by Ingram. All Rights Reserved. |
What is included with this book?
The New copy of this book will include any supplemental materials advertised. Please check the title of the book to determine if it should include any access cards, study guides, lab manuals, CDs, etc.
The Used, Rental and eBook copies of this book are not guaranteed to include any supplemental materials. Typically, only the book itself is included. This is true even if the title states it includes any access cards, study guides, lab manuals, CDs, etc.