Uses of artificial and composite treatments in experimental methods: Reconsidering the problem of validity and its implications for stratification research
Publication in refereed journal

Times Cited
Altmetrics Information

Other information
AbstractThis paper promotes the incorporation of the expanding uses of experimental methods in stratification research and beyond. Social science experiments are usually criticized for their lack of external validity, especially their limited generalizability to real-life settings. We offer a critique of this concern by reaffirming the classical insight of experimental methodology that external validity should not be a key concern and focusing on some under-appreciated scientific payoffs to experimental control. We illustrate the opportunities for productive use of artificial and composite treatments in tandem to obtain three measures of inequality preference: generalized egalitarianism (derived from an incentivized behavioral game), acceptable merit-based economic differential (from a vignette experiment), and motivational value of income differentials (from a standard attitudinal item). This approach enables us to examine their roles in determining the perceived fairness of living standard given one’s effort—an outcome of much interest in subjective stratification research. The empirical results are based on a mobile phone survey of over 62,000 urban respondents, covering more than 330 of the largest cities in China. Using fixed-effects to control for locational unobservables, we show that perceived fairness is positively associated with tolerance of merit-based pay differential and, surprisingly, strict generalized egalitarian preference, but not with belief in income differentials as motivator for effort. Since the seemingly equivalent measures have very different predictive relationships with an individual’s perceived fairness, the results offer a cautionary tale of invalid interpretations that overlook the subtle nonequivalence among valid measurement strategies with differing degrees of realism.
Acceptance Date21/11/2019
All Author(s) ListJacqueline Chen Chen, Tony Tam
Journal nameResearch in Social Stratification and Mobility
Volume Number65
Article number100443
LanguagesEnglish-United States
KeywordsExperimental validity, Artificial treatment, Composite treatment, Inequality preference, China

Last updated on 2021-18-09 at 00:01