Quota sampling using Facebook advertisements

Quota sampling using Facebook advertisements

Our new article “Quota sampling using Facebook advertisements” explores the use of the popular social media and advertising platform as a tool for obtaining survey samples.  It was published in December 2018 in the journal Political Science Research and Methods.

Corporations have long used Facebook to conduct market research, but academic use has been more limited because Facebook samples are not representative of the broader population. The enormity of Facebook’s active monthly users (241 million in the U.S. and Canada and still growing), however, makes it a potentially valuable research tool, if there were a way to ensure sample representativeness.

Some survey researchers already recruit respondents on Facebook using ads. As individuals click on an ad, however, Facebook’s algorithms start to serve the ad to other people with similar characteristics to the people who were first to click. The resulting sample of survey respondents is then likely to be biased in a number of ways. So, as a simple example, if men are more likely to click on an ad, Facebook’s algorithm starts to serve the ad to a higher proportion of men. As more men respond, the algorithm further serves the ad to an even higher proportion of men, and so on. Yet, representativeness of the diversity of a population is vital for accurate research results. In our simple example, a survey sample comprised almost entirely of men cannot accurately represent the views of a larger population that is half women.

Our study addressed this sampling problem by developing a method that allows researchers to define diverse characteristics of a desired population (e.g., men and women, accurate proportions of different age groups, etc.), and then serve survey recruitment ads to Facebook users in a way that produces a diverse sample of respondents that better represents the diversity of the entire population.

In the paper, we compare measures of public opinion about global warming using this new method against an independent, concurrent wave of our nationally representative Climate Change in the American Mind survey (GfK) and data from the U.S. Census. We find that many of the results from the Facebook-sampled survey are very similar to the independent survey and Census results. These findings suggest that using Facebook to recruit respondents can be a viable and less expensive option for survey researchers wishing to approximate population-level public opinion.

We have also produced a statistical tool called fbsample using the R programming language to facilitate sampling with the new method. The R package semi-automates the quota sampling process using the Facebook Marketing API; it can be downloaded here: https://github.com/13bzhang/fbsample.

We hope the article can support your own research efforts. Stay tuned by joining our email list for additional research. For a copy of the paper, please email us at climatechange@yale.edu with the subject line: FACEBOOK paper.