I've been looking at some empirical work that's been done gauging people's attitudes about race. I've been taking these studies to be empirically sound and up to social science standards. As I'm going through the comments on my chapter dealing with this, I'm seeing some worry that my arguments rely on surveys that aren't up to social science standards because they're not representative enough. I have no idea how to evaluate such a charge.
What exactly does a study need to do to meet social science standards? One study I'm looking at was an internet survey of 449 people. The initial contacts responded to ads on a university campus and were told to invite other people they knew to participate, and the authors of the study say that the invitations allowed it to mushroom into a more diverse group. The ages ranged from 18 to 82, with a mean age of 35 (SD=13.38). They asked for self-identification racially, and they got 64% identifying as European American, 14% African American, 9% Latino/a, 5% Asian American, 3% Biracial/Multiracial, .2% Native American/Alaskan, and 4% Other or None of the Above. For educational background, 3% had just high school, 23% some college, 23% college grads, 15% some graduate school, and 36% completed graduate school. 29% were from the Midwest, 24% from the West, 17% Midatlantic, 18% South, 7% New England, and 5% Southwest.
The full text of the study is here.
Can anyone with a background in social science give me a sense of whether it's fair to charge studies like this one with not being representative enough and therefore not up to social science standards? The only thing I can detect is some geographical skewing and a heavy emphasis on people with higher education backgrounds, but the authors acknowledge the latter and checked to see if responses differed significantly from one education level to another and detected no problems there. Is the sample size large enough? I don't have a sense of how these things are supposed to be done.