By Young Mi Kwon ’15, THURJ Staff
The rapid development and expansion of the Internet in the last few decades has led to its ubiquitous role in our everyday lives. Currently, over 80 percent of people in the United States alone use the Internet for a wide range of purposes (1). Now, the web has broadened its role to include the field of research. Within the past few years, there has been a significant trend towards an increased use of web-based research. Particularly in the area of psychology, researchers view the Internet as a vast, critical resource. Due to its ability to access and connect populations around the world, the Internet can be perceived as a gateway to a wealth of opportunities. However, the usage of web-based research has also raised a number of concerns. How reliable is the data collected? How is it comparable to laboratory-based research? Does it meet the standards that have already been established for laboratory research? Although the advantages and disadvantages of web-based research have been debated intensely, researchers are clearly relying on the Internet more than ever before.
Yet, the Web’s greatest contribution to research may also be considered its greatest danger. Certainly, through the Web, one can reach millions of people from across the globe with relatively little effort, ultimately saving costs. Thus, through one computer, one can acquire a sample size unimaginable in a laboratory setting. At the same time, though, biases in the sample arise from the characteristics of people who decide to participate in online research gathering.
Those who often participate in the web-based research actively seek out these experiments through a web search or website, creating a “self-selecting” process. Most often these participants already have some prior knowledge of psychology. Professor Laura Germine of Wellesley College, the creator of Testmybrain.org—an online study focusing on facial perception—explains that a bias could exist since a majority of the participants are those with access to computers. Thus, these participants tend to be more technologically savvy and better educated than others. Such factors limit the ability of web-based research to generalize results. However, as Professor Germine notes, similar factors also bias laboratory research. Often, the sample is collected within the vicinity of the research center, creating a homogenous sample. This problem is particularly clear within college campuses. Many psychology research studies mostly utilize the available college students, narrowing the generalizability of the sample. While web research cannot be completely random, one cannot simply claim that it is “worse” than the laboratory’s sample. In fact, web sampling may be more diverse in certain aspects like age.
Alternatively, a concern may arise for the opposite situation: the ability to select a specific group with certain characteristics instead of a group representing a broad population. While web sampling can produce a greater sample, how reliable is it in selecting a particular group? Certainly, people may falsify information in order to complete an experiment and see the outcome. However, alternatives exist that minimize these occurrences. For example, if left-handed people are the specific group being studied, the study could also allow right-handed people to also take the test but to note that they are not left-handed. The assumption behind this measure is that the people who would like to participate but do not fulfill the characteristics can also be involved. In other words, by eliminating the motive to falsify information, the likelihood of such occurrences will be decreased.
While it is impossible to obtain a completely controlled sample from web sampling, the impact of extraneous variables can be minimized in such a manner. Other extraneous variables also include factors like visual stimuli (for instance, the size of the monitor), the environment, degree of concentration, and the computer system. According to Professor Germine, an assumption can be made under the theory of large numbers. In this theory, if the sampling size is sufficiently large, the statistical noise generated by such variables can be diluted to have a smaller effect. That is, if the sample is large and the noise is small, the noise will be lost within the large sample. Given the web’s ability to access large numbers of people, the theory of large numbers is particularly applicable to web-based research.
Though web sampling seems to produce a reasonable data sample, concerns about the quality of the data remain. Several studies have compared laboratory and web sampling to find few differences between the results of the two data sets (Gosling and Vazire 2004). In order to measure the data quality, means of the variance, measurement reliability, and performance were also examined in these data sets (Germine et al. 2012). Just as laboratory research is regulated through established standards, web research must also have comparable standards to produce reliable and valid data.
Web sampling and research initially may seem to be an area of uncertainty, but the advantages of utilizing web sampling popularize it among researchers. Comparisons to laboratory research cannot simply claim that either form is superior to the other. However, web-based research certainly opens a new avenue for potential research that was previously unfeasible. As the technological revolution persists, we will continue to see the expansion and development of research, especially the broadening of web-based research from psychological to clinical research.
- Gosling, S., Vazire, S., Srivastava, S., & John, O. (2004). Should we trust Web-based studies? A comparative analysis of six preconceptions about Internet questionnaires. American Psychologist, 59, 93–104.
- Germine, L., Nakayama, K., Duchaine, B., Chabris, C., Chatterjee, G., & Wilmer, J. (2012). Is the Web as good as the lab? Comparable performance from Web and lab in cognitive and perceptual experiments. Springer. Online.