American Journal of Clinical Dermatology

, Volume 13, Issue 6, pp 405–416 | Cite as

Harnessing the Power of Crowds

Crowdsourcing as a Novel Research Method for Evaluation of Acne Treatments
  • April W. Armstrong
  • Safia Cheeney
  • Julie Wu
  • Caitlin T. Harskamp
  • Clayton W. Schupp
Original Research Article


Background: Crowdsourcing is a novel process of data collection that can provide insight into the effectiveness of acne treatments in real-world settings. Little is known regarding the feasibility of crowdsourcing as a means of collecting dermatology research data, the quality of collected data, and how the data compare to the published literature.

Objective: The objective of this analysis is to compare acne data collected from a medical crowdsourcing site with high-quality controlled studies from peer-reviewed medical literature.

Methods: Crowdsourced data was collected from 662 online acne patients. Online patients reported data in a Likert-type format to characterize their symptom severity (740 total responses) and their treatment outcomes (958 total responses). The crowdsourced data were compared with meta-analyses and reviews on acne treatment from August 20, 2010 to August 20, 2011.

Results: We compared topical, oral systemic, alternative, phototherapy, and physical acne treatments of crowdsourced data to published literature. We focused on topical tretinoin due to the large number of online patient responses. While approximately 80% of tretinoin users observed clinical improvement after a 12-week treatment period in clinical trials, 46% of online users reported improvement in an unspecified time period. For most topical treatments, medication with high efficacy in clinical trials did not produce high effectiveness ratings based on the crowdsourced online data.

Conclusion: While limitations exist with the current methods of crowdsourced data collection, with standardization of data collection and use of validated instruments, crowdsourcing will provide an important and valuable platform for collecting high-volume patient data in real-world settings.


Acne Isotretinoin Benzoyl Peroxide Tretinoin Tazarotene 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



The authors would like to thank Alexandra Carmichael and Daniel Reda at CureTogether for the crowdsourced data. We would also like to thank Will Dampier, PhD, for his helpful comments in reviewing the manuscript draft and Lynda Ledo for her help with the initial research. No funding sources supported this work. The authors have no conflicts of interest that are directly relevant to the content of this article.


  1. 1.
    Horowitz CR, Brenner BL, Lachapelle S, et al. Effective recruitment of minority populations through community-led strategies. Am J Prev Med 2009 Dec; 37 (6 Suppl. 1): S195–200PubMedCrossRefGoogle Scholar
  2. 2.
    Ejiogu N, Norbeck JH, Mason MA, et al. Recruitment and retention strategies for minority or poor clinical research participants: lessons from the Healthy Aging in Neighborhoods of Diversity across the Life Span study. Gerontologist 2011 Jun; 51 Suppl. 1: S33–45PubMedCrossRefGoogle Scholar
  3. 3.
    Adams SA. Sourcing the crowd for health services improvement: The reflexive patient and ‘share-your-experience’ websites. Soc Sci Med 2011 Apr; 72(7): 1069–76PubMedCrossRefGoogle Scholar
  4. 4.
    Scoble R, Israel S. Naked conversations: how blogs are changing the way businesses talk with customers. Hoboken (NJ): John Wiley & Sons, Inc., 2006Google Scholar
  5. 5.
    Behrend TS, Sharek DJ, Meade AW, et al. The viability of crowdsourcing for survey research. Behav Res Methods 2011; 43(3): 800–13PubMedCrossRefGoogle Scholar
  6. 6.
    Bradley JC, Lancashire RJ, Lang AS, et al. The spectral game: leveraging open data and crowdsourcing for education. J Cheminform 2009; 1: 9PubMedCrossRefGoogle Scholar
  7. 7.
    Ekins S, Williams AJ. Reaching out to collaborators: crowdsourcing for pharmaceutical research. Pharm Res 2010 Mar; 27(3): 393–5PubMedCrossRefGoogle Scholar
  8. 8.
    Johnston SC, Hauser SL. Crowdsourcing scientific innovation. Ann Neurol 2009 Jun; 65(6): A7–8Google Scholar
  9. 9.
    Norman TC, Bountra C, Edwards AM, et al. Leveraging crowdsourcing to facilitate the discovery of new medicines. Sci Transl Med 2011 Jun 22; 3(88): 88mr1CrossRefGoogle Scholar
  10. 10.
    Prainsack B, Wolinsky H. Direct-to-consumer genome testing: opportunities for pharmacogenomics research? Pharmacogenomics 2010 May; 11(5): 651–5PubMedCrossRefGoogle Scholar
  11. 11.
    Thurzo A, Stanko P, Urbanova W, et al. The WEB 2.0 induced paradigm shift in the e-learning and the role of crowdsourcing in dental education. Bratisl Lek Listy 2010; 111(3): 168–75PubMedGoogle Scholar
  12. 12.
    Whitla P. Crowdsourcing and its application in marketing activities. Contemp Manage Res 2009; 5(1): 15–28Google Scholar
  13. 13.
    Schmidt L. Crowdsourcing for human subjects research. San Francisco (CA): CrowdConf, 2010Google Scholar
  14. 14.
    Stern RS. Acne therapy: medication use and sources of care in office-based practice. Arch Dermatol 1996 Jul; 132(7): 776–80PubMedCrossRefGoogle Scholar
  15. 15.
    Collier CN, Harper JC, Cafardi JA, et al. The prevalence of acne in adults 20 years and older. J Am Acad Dermatol 2008 Jan; 58(1): 56–9PubMedCrossRefGoogle Scholar
  16. 16.
    Kim RH, Armstrong AW. Current state of acne treatment: highlighting lasers, photodynamic therapy, and chemical peels. Dermatol Online J 2011; 17(3): 2PubMedGoogle Scholar
  17. 17.
    Carmichael A, Reda D. CureTogether 2011 [online]. Available from URL: [Accessed 2011 Sep 1]
  18. 18.
    Chu A, Huber FJ, Plott RT. The comparative efficacy of benzoyl peroxide 5%/erythromycin 3% gel and erythromycin 4%/zinc 1.2% solution in the treatment of acne vulgaris. Br J Dermatol 1997 Feb; 136(2): 235–8PubMedCrossRefGoogle Scholar
  19. 19.
    Kraft J, Freiman A. Management of acne. CMAJ 2011 Apr 19; 183(7): E430–5PubMedGoogle Scholar
  20. 20.
    Imahiyerobo-Ip JI, Dinulos JG. Changing the topography of acne with topical medications. Curr Opin Pediatr 2011 Feb; 23(1): 121–5PubMedCrossRefGoogle Scholar
  21. 21.
    Hsu P, Litman GI, Brodell RT. Overview of the treatment of acne vulgaris with topical retinoids. Postgrad Med 2011 May; 123(3): 153–61PubMedGoogle Scholar
  22. 22.
    Cunliffe WJ, Poncet M, Loesche C, et al. A comparison of the efficacy and tolerability of adapalene 0.1% gel versus tretinoin 0.025% gel in patients with acne vulgaris: a meta-analysis of five randomized trials. Br J Dermatol 1998 Oct; 139 Suppl. 52: 48–56PubMedCrossRefGoogle Scholar
  23. 23.
    Leyden JJ, Tanghetti EA, Miller B, et al. Once-daily tazarotene 0.1 % gel versus once-daily tretinoin 0.1 % microsponge gel for the treatment of facial acne vulgaris: a double-blind randomized trial. Cutis 2002 Feb; 69(2 Suppl.): 12–9PubMedGoogle Scholar
  24. 24.
    Lucky AW, Cullen SI, Funicella T, et al. Double-blind, vehicle-controlled, multicenter comparison of two 0.025% tretinoin creams in patients with acne vulgaris. J Am Acad Dermatol 1998 Apr; 38(4): S24–30PubMedCrossRefGoogle Scholar
  25. 25.
    Mills Jr OH, Kligman AM, Pochi P, et al. Comparing 2.5%, 5%, and 10% benzoyl peroxide on inflammatory acne vulgaris. Int J Dermatol 1986 Dec; 25(10): 664–7PubMedCrossRefGoogle Scholar
  26. 26.
    Haider A, Shaw JC. Treatment of acne vulgaris. JAMA 2004 Aug 11; 292(6): 726–35PubMedCrossRefGoogle Scholar
  27. 27.
    Lesher Jr JL, Chalker DK, Smith Jr JG, et al. An evaluation of a 2% erythromycin ointment in the topical therapy of acne vulgaris. J Am Acad Dermatol 1985 Mar; 12(3): 526–31PubMedCrossRefGoogle Scholar
  28. 28.
    Becker LE, Bergstresser PR, Whiting DA, et al. Topical clindamycin therapy for acne vulgaris: a cooperative clinical study. Arch Dermatol 1981 Aug; 117(8): 482–5PubMedCrossRefGoogle Scholar
  29. 29.
    Lookingbill DP, Chalker DK, Lindholm JS, et al. Treatment of acne with a combination clindamycin/benzoyl peroxide gel compared with clindamycin gel, benzoyl peroxide gel and vehicle gel: combined results of two double-blind investigations. J Am Acad Dermatol 1997 Oct; 37(4): 590–5PubMedCrossRefGoogle Scholar
  30. 30.
    Reuter J, Merfort I, Schempp CM. Botanicals in dermatology: an evidence-based review. Am J Clin Dermatol 2010; 11(4): 247–67PubMedGoogle Scholar
  31. 31.
    Enshaieh S, Jooya A, Siadat AH, et al. The efficacy of 5% topical tea tree oil gel in mild to moderate acne vulgaris: a randomized, double-blind placebo-controlled study. Indian J Dermatol Venereol Leprol 2007 Jan–Feb; 73(1): 22–5PubMedCrossRefGoogle Scholar
  32. 32.
    Strauss JS, Leyden JJ, Lucky AW, et al. A randomized trial of the efficacy of a new micronized formulation versus a standard formulation of isotretinoin in patients with severe recalcitrant nodular acne. J Am Acad Dermatol 2001 Aug; 45(2): 187–95PubMedCrossRefGoogle Scholar
  33. 33.
    Braathen LR. Topical clindamycin versus oral tetracycline and placebo in acne vulgaris. Scand J Infect Dis Suppl 1984; 43: 71–5PubMedGoogle Scholar
  34. 34.
    Skidmore R, Kovach R, Walker C, et al. Effects of subantimicrobial-dose doxycycline in the treatment of moderate acne. Arch Dermatol 2003 Apr; 139(4): 459–64PubMedCrossRefGoogle Scholar
  35. 35.
    Leyden J, Shalita A, Hordinsky M, et al. Efficacy of a low-dose oral contraceptive containing 20 microg of ethinyl estradiol and 100 microg of levo-norgestrel for the treatment of moderate acne: A randomized, placebo-controlled trial. J Am Acad Dermatol 2002 Sep; 47(3): 399–409PubMedCrossRefGoogle Scholar
  36. 36.
    Hongcharu W, Taylor CR, Chang Y, et al. Topical ALA-photodynamic therapy for the treatment of acne vulgaris. J Invest Dermatol 2000 Aug; 115(2): 183–92PubMedCrossRefGoogle Scholar
  37. 37.
    Bowe WP, Joshi SS, Shalita AR. Diet and acne. J Am Acad Dermatol 2010 Jul; 63(1): 124–41PubMedCrossRefGoogle Scholar
  38. 38.
    Seidler EM, Kimball AB. Meta-analysis comparing efficacy of benzoyl peroxide, clindamycin, benzoyl peroxide with salicylic acid, and combination benzoyl peroxide/clindamycin in acne. J Am Acad Dermatol 2010 Jul; 63(1): 52–62PubMedCrossRefGoogle Scholar
  39. 39.
    Touitou E, Godin B, Shumilov M, et al. Efficacy and tolerability of clindamycin phosphate and salicylic acid gel in the treatment of mild to moderate acne vulgaris. J Eur Acad Dermatol Venereol 2008 May; 22(5): 629–31PubMedCrossRefGoogle Scholar
  40. 40.
    Plewig G, Holland KT, Nenoff P. Clinical and bacteriological evaluation of nadifloxacin 1% cream in patients with acne vulgaris: a double-blind, phase III comparison study versus erythromycin 2% cream. Eur J Dermatol 2006 Jan–Feb; 16(1): 48–55PubMedGoogle Scholar
  41. 41.
    NilFroushzadeh MA, Siadat AH, Baradaran EH, et al. Clindamycin lotion alone versus combination lotion of clindamycin phosphate plus tretinoin versus combination lotion of clindamycin phosphate plus salicylic acid in the topical treatment of mild to moderate acne vulgaris: a randomized control trial. Indian J Dermatol Venereol Leprol 2009 May–Jun; 75(3): 279–82PubMedCrossRefGoogle Scholar
  42. 42.
    Peck GL, Olsen TG, Butkus D, et al. Isotretinoin versus placebo in the treatment of cystic acne: a randomized double-blind study. J Am Acad Dermatol 1982 Apr; 6(4 Pt 2 Suppl.): 735–45PubMedCrossRefGoogle Scholar
  43. 43.
    Blaney DJ, Cook CH. Topical use of tetracycline in the treatment of acne: a double-blind study comparing topical and oral tetracycline therapy and placebo. Arch Dermatol 1976 Jul; 112(7): 971–3PubMedCrossRefGoogle Scholar
  44. 44.
    Rapaport M, Puhvel SM, Reisner RM. Evaluation of topical erythromycin and oral tetracycline in acne vulgaris. Cutis 1982 Jul; 30(1): 122–6, 30, 32–5PubMedGoogle Scholar
  45. 45.
    Arowojolu Gallo MF, Lopez LM, et al. Combined oral contraceptive pills for treatment of acne. Cochrane Database Syst Rev 2009; (3): CD004425Google Scholar
  46. 46.
    Zahradnik HP, Goldberg J, Andreas JO. Efficacy and safety of the new anti-androgenic oral contraceptive Belara. Contraception 1998 Feb; 57(2): 103–9PubMedCrossRefGoogle Scholar
  47. 47.
    Plewig G, Cunliffe WJ, Binder N, et al. Efficacy of an oral contraceptive containing EE 0.03 mg and CMA 2mg (Belara) in moderate acne resolution: a randomized, double-blind, placebo-controlled phase III trial. Contraception 2009 Jul; 80(1): 25–33PubMedCrossRefGoogle Scholar
  48. 48.
    Lucky AW, Henderson Olson WH, et al. Effectiveness of norgestimate and ethinyl estradiol in treating moderate acne vulgaris. J Am Acad Dermatol 1997 Nov; 37(5 Pt 1): 746–54PubMedCrossRefGoogle Scholar
  49. 49.
    Kokandi A. Evaluation of acne quality of life and clinical severity in acne female adults. Dermatol Res Pract 2010; 2010: 410809PubMedGoogle Scholar
  50. 50.
    Doshi A, Zaheer A, Stiller MJ. A comparison of current acne grading systems and proposal of a novel system. Int J Dermatol 1997 Jun; 36(6): 416–8PubMedCrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2012

Authors and Affiliations

  • April W. Armstrong
    • 1
  • Safia Cheeney
    • 2
  • Julie Wu
    • 1
  • Caitlin T. Harskamp
    • 1
  • Clayton W. Schupp
    • 1
  1. 1.Department of DermatologyUniversity of California Davis Health SystemSacramentoUSA
  2. 2.University of ArizonaTucsonUSA

Personalised recommendations