For citation purposes: van Gelder MMHJ, Pijpe A. E-epidemiology: a comprehensive update. OA Epidemiology 2013 Jun 04;1(1):5.

Critical review

 
Education

E-epidemiology: a comprehensive update

MMH van Gelder1*†, A Pijpe2†
 

Authors affiliations

(1) Department for Health Evidence (HP 133), Radboud University Nijmegen Medical Centre, P.O. Box 9101, 6500 HB Nijmegen, The Netherlands

(2) Department of Epidemiology, Netherlands Cancer Institute, Plesmanlaan 121, 1066 CX, Amsterdam, The Netherlands

* Corresponding author Email: M.vanGelder@ebh.umcn.nl

†These authors contributed equally to this study.

Abstract

Introduction

Interests in using information technology and specifically the Internet for epidemiological research (e-epidemiology) are increasing since it is expected to add several advantages to the traditional modes of data collection. These may include increased response rates, improved data quality and a more rapid and cost-efficient method for data collection and study population recruitment. In this study, we provide an overview of the latest developments in this relatively new field within epidemiologic research, focusing on methods of data collection and recruitment of study participants.

Conclusion

Information technology for data collection within epidemiological research offers many possibilities for large-scale studies: the current developments in the field of e-epidemiology show promising results, especially with regard to Web-based questionnaires, which showed that depending on the study population, the response rates for Web-based questionnaires may be higher than for paper-based questionnaires. However, future research should focus on the internal and external validity of the data collected, as well as on other aspects of e-epidemiologic research, including participant recruitment through the Internet, use of Web 2.0 content, data collection through smartphone applications, short message service and videoconferencing, and inherent privacy and ethical concerns.

Introduction

Due to the declining participation rates and increased costs of epidemiologic studies, interest in alternatives to the traditional methods of data collection, such as interviews and paper-based questionnaires, is increasing. In 2007, Ekman and Litton[1] introduced the term e-epidemiology, defining it as ‘the science underlying the acquisition, maintenance and application of epidemiological knowledge and information using digital media such as the Internet, mobile phones, digital paper, and digital TV. E-epidemiology also refers to the large-scale epidemiological studies that are increasingly conducted through distributed global collaborations enabled by the Internet’. They expected that e-epidemiology would have several advantages over the traditional modes of data collection and participant recruitment, leading to increased response rates, improvements in data quality, possibilities to reach specifically targeted populations that might otherwise be hard or impossible to contact and a more rapid and cost-efficient means for data collection and processing. However, many issues needed to be explored and tested before e-epidemiological tools could fulfil their expectations. In this study, we provide an overview of the current state of knowledge and latest developments in e-epidemiology, focusing on data collection methods and means for participant recruitment used in this relatively new field within epidemiological research.

Discussion

The authors have referenced some of their own studies in this review. These referenced studies have been conducted in accordance with the Declaration of Helsinki (1964) and the protocols of these studies have been approved by the relevant ethics committees related to the institution in which they were performed. All human subjects, in these referenced studies, gave informed consent to participate in these studies.

Web-based questionnaires

In populations with high Internet access rates, Web-based questionnaires may be considered as an alternative for or complementary method of data collection to the traditional modes[2]. The advantages of Web-based questionnaires are straightforward: (1) data quality is improved through validation checks during questionnaire completion, automated skipping of non-applicable items and not having to make the error-prone steps of data entry and coding, (2) automatic data management systems may be used to send invitations for follow-up questionnaires and reminders by e-mail, (3) information is returned more rapidly compared with paper-based questionnaires and (4) the costs of data collection may be reduced. Indeed, some large-scale cohort studies, such as the Snart Gravid study[3], the PRIDE Study[4] and the Nightingale Study[5], use Web-based questionnaires as their primary method of data collection.

However, one of the major concerns of Web-based questionnaires is bias resulting from survey non-response, which leads to selection bias if the association between exposure and outcome differs between participants and all targeted subjects. A 2008 meta-analysis showed that mail surveys had higher response rates than Web surveys[6], but studies conducted in the last five years that directly compared response rates for Web-based and paper-based questionnaires showed that the gap in response rates is closing (Table 1)[7,8,9,10,11,12,13,14], especially in higher educated populations and when e-mail invitations were used. According to the same meta-analysis, postal invitations to complete a Web-based questionnaire may negatively influence response rates due to the additional step of switching from the postal invitation to the Internet by typing in the web address, login and password for participation, which could explain up to 15% lower response rates for Web-based questionnaires[6], and thereby also the lower response rates for Web-based questionnaires compared to paper-based questionnaires in the studies by Boschman et al.[7] and Sinclair et al.[10].

Table 1

Summary of studies conducted since 2008 that directly compared response rates for Web-based and paper-based questionnaires

Another concern regarding Web-based questionnaires is the validity of the data collected. Compared with the traditional modes of data collection, Web-based questionnaires were expected to yield larger measurement errors due to simple errors, including faster reading by Internet users, the respondents not scrolling down to all questions or answering options and suboptimal questionnaire design[15,16]. Although evidence on the validity of data collected through Web-based questionnaires is still limited (Table 2), the validity of data on anthropometric, dietary and socio-demographic factors is generally reported to be high[17,18,19,20,21,22,23,24,25,26]. More studies to validate data collected through Web-based questionnaires are needed to establish whether this approach is suitable for epidemiological research in various study populations. However, it should be noticed that none of the traditional methods is perfect either.

Table 2

Results of studies validating data on anthropometric, dietary and socio-demographic factors collected through Web-based questionnaires

Recruitment of study participants through the Internet

Besides approaching potential study participants through a postal letter or e-mail, the Internet is increasingly being used for study recruitment purposes. The first experiences with the social networking site Facebook as a method to recruit a specific target group for health-related research showed modest success. Facebook allows advertisement targeting based on age, gender, location and listed interests, which makes it possible to show the advertisement only to users that potentially fulfil the study eligibility criteria. To evaluate the advertisement’s performance, Facebook provides a number of metrics, including the number of times the advertisement is shown to a user (‘impressions’) and the number of clicks the advertisement received. The costs are determined either as cost per 1000 impressions or cost per click. For health-related studies, the click-through rates varied between 0.02% and 0.05%[27,28], and the final costs of the Facebook campaign per completed survey ranged between ≈ €3.25 for a study recruiting young US cigarette users (n = 1548)[28] and €25 for a study recruiting Italian pregnant women (n = 8)[29]. Similarly, but less often reported in the scientific literature, Google advertising (AdWords) can be used. Using this service, a short advertisement is displayed when a user searches for certain keywords, of which the settings can be adjusted based on the budget, targeted locations, time of the day and day of the week. For a study recruiting participants for an online preventive depression intervention, the click-through rate was 6.01%, resulting in an average cost of $10.75 per signed-up person (n = 602)[30]. Other methods for online recruitment could include posts in forums, Internet support groups and noticeboards, respondent-driven sampling (snowballing), link exchanges with relevant web sites and sending e-mails to specific groups or lists.

In 2009, Edwards et al.[31] published a systematic review in which they identified multiple strategies to increase response to Web-based questionnaires. In addition to multiple design issues such as shorter questionnaires, using a white background and simple header and personalised invitations, non-monetary incentives can be used to increase response rates. In Web-based studies, lotteries are popular incentives to increase response and completion rates. However, a few recent studies on their effectiveness show that the effects are small. In an online panel, lotteries may slightly increase response among those panellists with a low motivation to respond (e.g. if the subject did not participate in a previous study)[32]. For community-based surveys, a lottery with a small number of large prizes was the most cost-effective incentive compared with no incentive, prepaid cash, which yielded the highest response rate, and a lottery with a relatively large number of small prizes[33].

User-driven Internet content: Web 2.0

Data resulting from participation in Web 2.0, which refers to web sites that allow users to interact and communicate with each other to produce user-generated content, may yield new possibilities for epidemiologic data collection[34]. Participation in Web 2.0 takes place on three levels. Firstly, it can occur unnoticed to the user, such as by using a search engine that keeps track of the searches performed (even stratified by geographic location and time, for example Google Trends) or by clicking on web feeds. Secondly, it may include active searching of user-generated content, for example reading hotel or restaurant reviews. Finally, it comprises the production of content to be read by others, for example content in blogs, microblogs (Twitter), social networking sites (Facebook, LinkedIn) and photo-sharing sites (Instagram, Flickr, Pinterest).

The best-known example of using Web 2.0 content for epidemiologic research is the study by Ginsberg et al.[35], who used Google search query data to track influenza-like illness in the United States. The relative frequency of selected queries was highly correlated with the data from the traditional surveillance networks and the resulting influenza-like illness estimates were 1-2 weeks ahead of the surveillance network reports. Recently, Twitter data were used for the same purpose, in which sophisticated methods to distinguish between actual flu cases and people merely talking about it were applied[36]. Furthermore, Twitter may produce other data valuable for epidemiological research, including measuring behavioural risk factors, symptoms and medication use, but it is as yet unknown what new empirical knowledge can be gained by using this method of data collection[37]. User profiles of social networking sites could also contain a wealth of data on risk behaviours, relationships and attitudes, whereas blog posts can automatically be screened for more detailed information on personality, health status and psychological profiles. Moreover, user-driven Internet content often holds detailed geographical information, either through automatically publishing the user’s location concurrently with a status update or through use of specialised applications, such as Foursquare, Google Latitude and Find My Friends, which could be of interest for epidemiologists quantifying mobility patterns in relation to environmental exposures, activities and health outcomes.

For clinical epidemiologists, social networking sites that focus on health-related problems may be of special interest. For example, on PatientsLikeMe, almost 190 000 people with various conditions share data on many factors, such as their disease status, symptoms, treatments, quality of life and weight. With information from these social networking sites, a number of research studies have already been conducted. The PatientsLikeMe web site has been integrated with ClinicalTrials.gov, so that patients can search for trials they might be eligible for. DailyStrength has over 500 support groups and health blogs; in which, users share their medical conditions, mental health issues and life challenges. On MedHelp, patients can document their health by using condition-specific applications and personal health records, which can be tracked by other members (and researchers). Furthermore, companies that offer direct-to-consumer genetic testing, such as 23andMe, create databases with genetic profiles, which can be used for research purposes. For example, two novel loci for Parkinson’ disease were identified using data from customers of 23andMe[38].

Other methods for data collection using information technology

In addition to Web-based questionnaires and Web 2.0, some other methods for data collection using information technology have been explored on a small scale for application in epidemiological studies. If mobile phone numbers of potential study participants are known, short message service (SMS) may be used to administer very short, easy questionnaires with one question for each SMS and set answering options. In two Swedish studies using SMS to assess influenza vaccination and physical activity in subjects randomly sampled from the population registry, 45% and 91% of subjects who received the questions completed the questionnaires, respectively[9,39]. In these studies, SMS data on the variables of interest were comparable with data originating from more traditional methods of data collection. Another application of SMS is that it may be used to send reminders for completing Web-based questionnaires.

Smartphones may be more feasible to collect epidemiological data than low-end mobile phones since they combine the functions of a personal digital assistant, portable media players, cameras and global positioning system navigation and allow the user to install and run dedicated applications (‘apps’). The first studies showed promising results for collecting data on physical activity (using the device’s accelerometer and gyroscope or an app)[40,41], infant feeding practices[42] and health-related diaries[43]. Future research should develop and make these tools available to the full extent of their capabilities for health-related research, in which new possibilities, such as real-time exposure measurement and instant individual feedback, could be explored.

Videoconferencing has been proposed as a low-cost alternative for face-to-face interviews, especially in studies in which study participants are geographically dispersed over a large area, provided that both a high-bandwidth connection (i.e. ≥ 1024 kbps) and the hardware and software are both present[44]. However, a German randomised pilot study among young adults showed that interviewing using Skype was not feasible: the response rate was 10% (95% CI: 5%-15%) in the Skype group and 22% (95% CI: 15%-28%) in the telephone interview group[45].

Concerns

New developments in epidemiological research, such as using information technology for medical research, generate new discussions, even though some of them refer to the basic epidemiological principles of internal and external validity. Although Internet access rates are high in developed countries, still a selective part of the population uses all the possibilities the Internet has to offer, which may be a threat to the generalisability of the results of Internet-based studies. Comparable with the traditional paper-based questionnaires and personal and telephone interviews, response rates for Web-based questionnaires are less than 100%. Mixed-mode or crossover designs may be used to increase survey participation rates. Data on response rates for studies using SMS, smartphone apps and videoconferencing are insufficient and need further investigation before they can be added to the epidemiological toolbox. Reassuringly, the traditional methods of data collection have shown little bias in exposure-outcome associations resulting from selective participation[46,47], and this is expected not to be different for e-epidemiological tools. Regarding user-driven Internet content, there is not only self-selection in participants in Web 2.0, but there is also selection in the information available, as the user decides which information he shares on the Internet. Furthermore, even for Web-based questionnaires, information on data quality is often lacking and specifically for Web 2.0 content, data on potential confounding factors are often unavailable, possibly leading to residual confounding.

In addition to the discussions about the internal and external validity of the results of e-epidemiological studies, privacy and ethical issues are being raised, especially concerning studies using Web 2.0 content[48]. Users of Web 2.0 are likely unaware that the information they share may be used for research purposes, even though this is stated in the web site’s User Agreement. Questions may be raised whether clicking on the ‘I agree’ button for the web site’s terms of service represents an acceptable informed consent for health-related research, especially for web sites that offer direct-to-consumer genetic testing. However, the use of Web-based questionnaires also raises questions since there are no general guidelines regarding participant protection: how should informed consent be obtained? Is an online-administered form sufficient or should it be a paper-based, signed form? Which security measures should be taken to protect the (personal) data collected? How can we verify the identity of the participants? Institutional Review Boards and other regulatory bodies may need to develop guidelines to solve these issues.

Conclusion

Using information technology and specifically the Internet for data collection and participant recruitment within epidemiological research offers many possibilities for large-scale studies. Web-based questionnaires are increasingly being considered as a fully-fledged alternative to the traditional modes of data collection with, in many populations, response rates comparable to those for paper-based questionnaires. However, many issues need to be further investigated and tested before e-epidemiology can be completely implemented in health-related research. These include determining the internal and external validity of the data of e-epidemiological studies, addressing privacy and ethical concerns regarding participant recruitment through the Internet and use of Web 2.0 content, and the development of data collection through SMS, smartphone apps or videoconferencing. Network meetings and collaborations between epidemiologists pioneering in this relatively new research area, such as the 2012 ∊-epi symposium in Cardiff and the initiation of focus group e-Epidemiology of The Netherlands Epidemiology Society, certainly contribute to further improvements of e-epidemiological tools.

Authors Contribution

All authors contributed to the conception, design, and preparation of the manuscript, as well as read and approved the final manuscript.

Competing interests

None declared.

Conflict of interests

None declared.

A.M.E

All authors abide by the Association for Medical Ethics (AME) ethical rules of disclosure.

References

  • 1. Ekman A, Litton JE. New times, new needs; e-epidemiology. Eur J Epidemiol 2007 May;22(5):285-92.
  • 2. van Gelder MM, Bretveld RW, Roeleveld N. Web-based questionnaires: the future in epidemiology. Am J Epidemiol 2010 Dec;172(11):1292-8.
  • 3. Huybrechts KF, Mikkelsen EM, Christensen T, Riis AH, Hatch EE, Wise LA. A successful implementation of e-epidemiology: the Danish pregnancy planning study ‘Snart-Gravid’. Eur J Epidemiol 2010 May;25(5):297-304.
  • 4. van Gelder MM, Bretveld RW, Roukema J, Steenhoek M, van Drongelen J, Spaanderman ME. Rationale and design of the PRegnancy and Infant DEvelopment (PRIDE) Study. Paediatr Perinat Epidemiol 2013 Jan;27(1):34-43.
  • 5. Pijpe A, Slottje P, van Pelt C, Stehmann F, Kromhout H, van Leeuwen FE. The Nightingale Study: rationale, study design and baseline characteristics of a prospective cohort study on shift work and breast cancer risk among nurses. Submitted .
  • 6. Manfreda KL, Bosniak M, Berzelak J, Haas I, Vehovar V. Web surveys versus other survey modes: a meta-analysis comparing response rates. Int J Market Res 2008;50(1):79-104.
  • 7. Boschman JS, van der Molen HF, Frings-Dresen MHW, Sluiter JK. Response rate of bricklayers and supervisors on an internet or a paper-and-pencil questionnaire. Int J Ind Ergon 2012 Jan;42(1):178-82.
  • 8. Kamo N, Dandapani SV, Miksad RA, Houlihan MJ, Kaplan I, Regan M. Evaluation of the SCA instrument for measuring patient satisfaction with cancer care administered via paper or via the Internet. Ann Oncol 2011 Mar;22(3):723-9.
  • 9. Lagerros YT, Sandin S, Bexelius C, Litton JE, Löf M. Estimating physical activity using a cell phone questionnaire sent by means of short message service (SMS): a randomized population-based study. Eur J Epidemiol 2012 Jul;27(7):561-6.
  • 10. Sinclair M, O’Toole J, Malawaraarachchi M, Leder K. Comparison of response rates and cost-effectiveness for a community-based survey: postal, internet and telephone modes with generic or personalised recruitment approaches. BMC Med Res Methodol 2012 Aug;12132.
  • 11. Greenlaw C, Brown-Welty S. A comparison of web-based and paper-based survey methods: testing assumptions of survey mode and response cost. Eval Rev 2009 Oct;33(5):464-80.
  • 12. Derby DC, Haan A, Wood K. Data quality assurance: an analysis of patient non-response. Int J Health Care Qual Assur 2011 Mar;24(3):198-210.
  • 13. Barrios M, Villarroya A, Borrego A, Ollé C. Response rates and data quality in web and mail surveys administered to PhD holders. Soc Sci Comput Rev 2011 May;29(2):208-20.
  • 14. Whitehead L . Methodological issues in Internet-mediated research: a randomized comparison of internet versus mailed questionnaires. J Med Internet Res 2011 Dec;13(4):e109.
  • 15. Wyatt JC . When to use Web-based surveys. J Am Med Inform Assoc 2000 Jul-Aug;7(4):426-9.
  • 16. Manfreda KL, Batagelj Z, Vehovar V. Design of web survey questionnaires: three basic experiments. J Comput Mediated Comm 2002 Apr;7(3):0.
  • 17. Namba H, Yamaguchi Y, Yamada Y, Tokushima S, Hatamoto Y, Sagayama H. Validation of Web-based physical activity measurement systems using doubly labeled water. J Med Internet Res 2012 Sep;14(5):e123.
  • 18. Graham AL, Papandonatos GD, Bock BC, Cobb NK, Baskin-Sommers A, Niaura R. Internet- vs. telephone-administered questionnaires in a randomized trial of smoking cessation. Nicotine Tob Res 2006 Dec;8(Suppl 1):S49-57.
  • 19. Touvier M, Méjean C, Kesse-Guyot E, Pollet C, Malon A, Castetbon K. Comparison between web-based and paper versions of a self-administered anthropometric questionnaire. Eur J Epidemiol 2010 May;25(5):287-96.
  • 20. Bae J, Joung H, Kim JY, Kwon KN, Kim Y, Park SW. Validity of self-reported height, weight, and body mass index of the Korea Youth Risk Behavior Web-based Survey questionnaire. J Prev Med Public Health 2010 Sep;43(5):396-402.
  • 21. West R, Gilsenan A, Coste F, Zhou X, Brouard R, Nonnemaker J. The ATTEMPT cohort: a multi-national longitudinal study of predictors, patterns and consequences of smoking cessation; introduction and evaluation of internet recruitment and data collection methods. Addiction 2006 Sep;101(9):1352-61.
  • 22. Beasley JM, Davis A, Riley WT. Evaluation of a web-based, pictorial diet history questionnaire. Public Health Nutr 2009 May;12(5):651-9.
  • 23. Bliven BD, Kaufman SE, Spertus JA. Electronic collection of health-related quality of life data: validity, time benefits, and patient preference. Qual Life Res 2001 Feb;10(1):15-22.
  • 24. Vergnaud AC, Touvier M, Méjean C, Kesse-Guyot E, Pollet C, Malon A. Agreement between web-based and paper versions of a socio-demographic questionnaire in the NutriNet-Santé study. Int J Public Health 2011 Aug;56(4):407-17.
  • 25. Arab L, Tseng CH, Ang A, Jardack P. Validity of a multipass, web-based, 24-hour self-administered recall for assessment of total energy intake in blacks and whites. Am J Epidemiol 2011 Dec;174(11):1256-65.
  • 26. Bonn SE, Trolle Lagerros Y, Christensen SE, Möller E, Wright A, Sjölander A. Active-Q: validation of the web-based physical activity questionnaire using doubly labeled water. J Med Internet Res 2012 Feb;14(1):e29.
  • 27. Chu JL, Snider CE. Use of a social networking web site for recruiting Canadian youth for medical research. J Adolesc Health 2013 Jun;52(6):792-4.
  • 28. Ramo DE, Prochaska JJ. Broad reach and targeted recruitment using Facebook for an online survey of young adult substance use. J Med Internet Res 2012 Feb;14(1):e28.
  • 29. Richiardi L, Pivetta E, Merletti F. Recruiting study participants through Facebook. Epidemiology 2012 Jan;23(1):175.
  • 30. Morgan AJ, Jorm AF, Mackinnon AJ. Internet-based recruitment to a depression prevention intervention: lessons from the Mood Memos study. J Med Internet Res 2013 Feb;15(2):e31.
  • 31. Edwards PJ, Roberts I, Clarke MJ, Diguiseppi C, Wentz R, Kwan I. Methods to increase response to postal and electronic questionnaires. Cochrane Database Syst Rev 2009 Jul;3MR000008.
  • 32. Göritz AS, Luthe SC. Effects of lotteries on response behavior in online panels. Field Methods 2013 Feb.
  • 33. Gajic A, Cameron D, Hurley J. The cost-effectiveness of cash versus lottery incentives for a web-based, stated-preference community survey. Eur J Health Econ 2012 Dec;13(6):789-99.
  • 34. Lee BK . Epidemiologic research and Web 2.0: the user-driven Web. Epidemiology 2010 Nov;21(6):760-3.
  • 35. Ginsberg J, Array Mohebbi MH, Patel RS, Brammer L, Smolinski MS, Brilliant L. Detecting influenza epidemics using search engine query data. Nature 2009 Feb;457(7232):1012-4.
  • 36. . Using Twitter to track the flu: researchers find a better way to screen the tweets. http://releases.jhu.edu/2013/01/24/using-twitter-to-track-the-flu/. Baltimore: Johns Hopkins University 2013 [updated 24 Jan 2013; cited 04 Apr 2013].
  • 37. Paul M, Dredze M. You are what you tweet: analyzing Twitter for public health. Proceedings of the 5th International AAAI Conference on Weblogs and Social Media; July 17-21; Barcelona, Spain 2011265-72.
  • 38. Do CB, Tung JY, Dorfman E, Kiefer AK, Drabant EM, Francke U. Web-based genome-wide association study identifies two novel loci and a substantial genetic component for Parkinson’s disease. PLoS Genet 2011 Jun;7(6):e1002141.
  • 39. Bexelius C, Merk H, Sandin S, Ekman A, Nyrén O, Kühlmann-Berenzon S. SMS versus telephone interviews for epidemiological data collection: feasibility study estimating influenza vaccination coverage in the Swedish population. Eur J Epidemiol 2009 Feb;24(2):73-81.
  • 40. Wu W, Dasgupta S, Ramirez EE, Peterson C, Norman GJ. Classification accuracies of physical activities using smartphone motion sensors. J Med Internet Res 2012 Oct;14(5):e130.
  • 41. Kirwan M, Duncan MJ, Vandelanotte C, Mummery WK. Using smartphone technology to monitor physical activity in the 10,000 Steps program: a matched case-control trial. J Med Internet Res 2012 Apr;14(2):e55.
  • 42. Zhang S, Wu Q, van Velthoven MH, Chen L, Car J, Rudan I. Smartphone versus pen-and-paper data collection of infant feeding practices in rural China. J Med Internet Res 2012 Sep;14(5):e119.
  • 43. Nes AA, van Dulmen S, Eide E, Finset A, Kristjánsdóttir ÓB, Steen IS. The development and feasibility of a web-based intervention with diaries and situational feedback via smartphone to support self-management in patients with diabetes type 2. Diabetes Res Clin Pract 2012 Sep;97(3):385-93.
  • 44. Sedgwick M, Spiers J. The use of videoconferencing as a medium for the qualitative interview. Int J Qual Meth 2009;8(1):1-11.
  • 45. Weinmann T, Thomas S, Brilmayer S, Heinrich S, Radon K. Testing Skype as an interview method in epidemiologic research: response and feasibility. Int J Public Health 2012 Dec;57(6):959-61.
  • 46. Nohr EA, Frydenberg M, Henriksen TB, Olsen J. Does low participation in cohort studies induce bias. Epidemiology 2006 Jul;17(4):413-8.
  • 47. Nilsen RM, Vollset SE, Gjessing HK, Skjærven R, Melve KK, Schreuder P. Self-selection and bias in a large prospective pregnancy cohort in Norway. Paediatr Perinat Epidemiol 2009 Nov;23(6):597-608.
  • 48. Vayena E, Mastroianni A, Kahn J. Ethical issues in health research with novel online sources. Am J Public Health 2012 Dec;102(12):2225-30.
Licensee to OAPL (UK) 2013. Creative Commons Attribution License (CC-BY)

Summary of studies conducted since 2008 that directly compared response rates for Web-based and paper-based questionnaires

Study population Age (years) Study year Recruitment details Response rate Differencea P-value References
WBQ PBQ
Bricklayers 46 (17–64)b 2009 Postal invitation 28.4% (42/148) 43.5% (64/147) −15.1% 0.007 Boschman et al.7
Cancer patients 59 (30–80)c 2007–2008 NR 63.9% (39/61) 63.9% (39/61) 0.0% 1.00 Kamo et al.8
Construction supervisors 49 (19–65)b 2009 Postal invitation 43.0% (64/149) 45.9% (67/146) −2.9% 0.61 Boschman et al.7
General populationd 18–80e 2010 E-mail/postal invitation 93.4% (170/182) 89.1% (188/211) +4.3% 0.14 Lagerros et al.9
Households NR 2011 Personalised postal invitation 4.7% (289/6210) 10.5% (575/5500) −5.8% < 0.001 Sinclair et al.10
Households NR 2011 Generic postal invitation 2.2% (101/4515) 7.5% (299/4000) −5.3% < 0.001 Sinclair et al.10
Members AEA NR NR E-mail/postal invitation 52.5% (672/1281) 42.0% (538/1280) +10.5% < 0.001 Greenlaw and Brown-Welty11
Patients chiropractor clinic 18+e 2008 In-clinic administration 74.6% (47/63) 62.8% (135/215) +11.8% 0.08 Derby et al.12
PhD holders 43f 2008 Personalised e-mail/postal invitation 64.8% (250/386) 48.8% (62/127) +16.0% 0.001 Barrios et al.13
University students 24f NR E-mail/postal invitation 54.4% (536/986) 50.7% (498/983) +3.7% 0.10 Whitehead14
Total 15.8% (2210/13981) 19.5% (2465/12670) −3.7% < 0.001
Total excl. Sinclair et al.10 55.9% (1820/3256) 50.2% (1591/3170) +5.7% < 0.001

AEA, American Evaluation Association; NR, not reported; PBQ, paper-based questionnaire; WBQ, web-based questionnaire.

aresponse rate WBQ minus response rate PBQ; bmedian (range); cmean (range); damong those subjects that agreed to participate; erange; fmean.

Results of studies validating data on anthropometric, dietary and socio-demographic factors collected through Web-based questionnaires

Variable/instrument Reference standard Results References
Activity energy expenditure (seven-day recall) Doubly labelled water r = 0.14 (p = 0.54) Namba et al.17
Activity energy expenditure (diary) Doubly labelled water r = 0.68 (p < 0.001) Namba et al.17
Alcohol consumption Telephone interview kappa = 0.88 Graham et al.18
Anthropometry Paper-based questionnaire ICC = 0.86–1.00; kappa = 0.7–0.9 Touvier et al.19
BMI (weight, height) Anthropometric measurements Se = 0.69; Sp = 1.00 Bae et al.20
Body weight In-home assessment r = 0.98a West et al.21
Dietary assessment measures Paper-based questionnaire r = 0.28–0.73 Beasley et al.22
Dietary assessment measures Four-day food record r = 0.21–0.64 Beasley et al.22
Dietary assessment measures Twenty-four-hour recall r = 0.10–0.57 Beasley et al.22
General health status (RAND-36) Paper-based questionnaire ICC = 0.54–0.75 Bliven et al.23
Perceived health status Telephone interview kappa = 0.73 Graham et al.18
Socio-demographic factors Paper-based questionnaire ICC = 0.81–1.00; kappa = 0.76–1.00 Vergnaud et al.24
Total energy expenditure Doubly labelled water r = 0.45 Arab et al.25
Total energy expenditure (seven-day recall) Doubly labelled water r = 0.59 (p = 0.006) Namba et al.17
Total energy expenditure (diary) Doubly labelled water r = 0.88 (p < 0.001) Namba et al.17
Total energy expenditure (Active-Q) Doubly labelled water r = 0.52 (p < 0.001) Bonn et al.26
Total household income Telephone interview kappa = 0.93 Graham et al.18

BMI, body mass index; ICC, intraclass correlation coefficient; r, correlation coefficient; Se, sensitivity; Sp, specificity.

aalthough highly correlated, self-reported weight was on an average approximately 1 kg lower compared with the weight measured by a trained professional.

Keywords