Behavior Research Methods

, Volume 39, Issue 2, pp 205–211 | Cite as

A multimethod approach to examining usability of Web privacy polices and user agents for specifying privacy preferences

  • Robert W. ProctorEmail author
  • Kim-Phuong L. Vu
Articles From the SCiP Conference


Because all research methods have strengths and weaknesses, a multimethod approach often provides the best way to understand human behavior in applied settings. We describe how a multimethod approach was employed in a series of studies designed to examine usability issues associated with two aspects of online privacy: comprehension of privacy policies and configuration of privacy preferences for an online user agent. Archival research, user surveys, data mining, quantitative observations, and controlled experiments each yielded unique findings that, together, contributed to increased understanding of online-privacy issues for users. These findings were used to evaluate the accessibility of Web privacy policies to computer-literate users, determine whether people can configure user agents to achieve specific privacy goals, and discover ways in which the usability of those agents can be improved.


Privacy Policy Privacy Concern User Agent Positive Word Identifiable Information 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. Aaronson, D. (1994). Computer-based driving systems for research, assessment, and advisement: An introduction.Behavior Research Methods, Instruments, & Computers,26, 181–182.CrossRefGoogle Scholar
  2. Adkinson, W. F., Eisenach, J. A., &Lenard, T. M. (2002).Privacy online: A report on the information practices and policies of commercial Web sites [Special Report]. Washington, DC. Progress and Freedom Foundation.Google Scholar
  3. Antón, A. I., Earp, J. B., Vail, M. W., Jain, N., Gheen, C., &Frink, J. M (2004).An analysis of Web site privacy policy evolution in the presence of HIPAA (Tech. Rep. No. 2004-21). Raleigh: North Carolina State University. Computer Science Department.Google Scholar
  4. Bamberg, M. (2003). Foreword. In P. M. Camic, J. E. Rhodes, & L. Yardley (Eds.),Qualitative research in psychology: Expanding perspectives in methodology and design (pp. ix-xi). Washington, DC: American Psychological Association.Google Scholar
  5. Cranor, L. F., &Garfinkel, F. (2005).Security and usability: Designing secure systems that people can use. Sebastopol, CA: O’Reilly.Google Scholar
  6. Cranor, L. F., Guduru, P., &Arjula, M. (2006). User interfaces for privacy agents.ACM Transactions on Computer-Human Interaction,13, 135–178.CrossRefGoogle Scholar
  7. Eid, M., &Diener, E. (Eds.), (2006).Handbook of multimethod measurement in psychology. Washington, DC: American Psychological Association.CrossRefGoogle Scholar
  8. Flesch, R. (1949).The art of readable writing. New York: Macmillan.Google Scholar
  9. Han, J., &Kamber, M. (2001).Data mining: Concepts and techniques. San Francisco: Morgan Kaufmann.Google Scholar
  10. Hoc, J.-M. (2001). Toward ecological validity of research in cognitive ergonomics.Theoretical Issues in Ergonomics Science,2, 278–288.CrossRefGoogle Scholar
  11. Hwang, W. (2006). Data mining in ergonomics. In W. Karwowski (Ed.),Encyclopedia of ergonomics and human factors (pp. 3077–3081). Boca Raton, FL: CRC Press.Google Scholar
  12. Jensen, C., &Potts, J. (2004). Privacy policies as decision-making tools: An evaluation of online privacy notices.Proceedings of the Special Interest Group on Human-Computer Interaction Conference on Human Factors in Computing Systems,6, 471–478.Google Scholar
  13. Moores, T. T., &Dhillon, G. (2003). Do privacy seals in e-commerce really work?Communications of the ACM,46, 265–271.CrossRefGoogle Scholar
  14. National Science Foundation (2005). Cyber Trust 2005. Retrieved October 21, 2006, from htm.Google Scholar
  15. Oates, J. (2005, February 7). AOL man pleads guilty to selling 92m email addies. The Register. Retrieved October 29, 2006, from www Scholar
  16. Proctor, R. W. (2005). Methodology is more than research design and technology.Behavior Research Methods,37, 197–201.CrossRefPubMedGoogle Scholar
  17. Proctor, R. W., Ali, M. A., & Vu, K.-P. L. (in press). Examining usability of Web privacy policies.International Journal of Human-Computer Interaction.Google Scholar
  18. Proctor, R. W., &Capaldi, E. J. (2006).Why science matters: Understanding the methods of psychological research. Malden, MA: Blackwell.Google Scholar
  19. Proctor, R. W., Schultz, E. E., & Vu, K.-P. L. (in press). Human factors in information security and privacy. In J. Gupta & S. Sharma (Eds.),Handbook of research on information security and assurance. Hershey, PA: Idea Group Reference.Google Scholar
  20. Proctor, R. W., Vu, K.-P. L., &Ali, M. A. (2007). Usability of user agents for privacy-preference specification. In M. J. Smith & G. Salvendy (Eds.),Human Interface: Part II. HCII 2007 (Lecture Notes in Computer Science No. 4558, pp. 766–776). Berlin: Springer.Google Scholar
  21. Schultz, E. E., Proctor, R. W., Lien, M.-C., &Salvendy, G. (2001). Usability and security: An appraisal of usability issues in information security methods.Computers & Security,20, 620–634.CrossRefGoogle Scholar
  22. Simonton, D. K. (2000). Archival research. In A. E. Kazdin (Ed.),Encyclopedia of psychology (pp. 234–235). Washington, DC: American Psychological Association.Google Scholar
  23. Spiekermann, S., Grossklags, J., &Berendt, B. (2001, October). E-privacy in 2nd generation E-commerce: Privacy preferences versus actual behavior. InProceedings of the 3rd ACM Conference on Electronic Commerce (pp. 38–47). New York: ACM.CrossRefGoogle Scholar
  24. Stone, P. J., Dunphy, D. C., Smith, M. S., &Ogilvie, D. M. (1966).The General Inquirer: A computer approach to content analysis. Cambridge, MA: MIT Press.Google Scholar
  25. Vu, K.-P. L., Proctor, R. W., Bhargav-Spanzel, A., Tai, B.-L., Cook, J., &Schultz, E. E. (2007). Improving password security and memorability to protect personal and organizational information.International Journal of Human-Computer Studies,65, 744–757.CrossRefGoogle Scholar
  26. Zhu, W., Vu, K.-P. L, &Proctor, R. W. (2005). Evaluating Web usability. In R. W. Proctor & K.-P. L. Vu(Eds.),Handbook of human factors in Web design (pp. 321–337). Mahwah, NJ: Erlbaum.Google Scholar

Copyright information

© Psychonomic Society, Inc. 2007

Authors and Affiliations

  1. 1.Department of Psychological SciencesPurdue UniversityW. Lafayette
  2. 2.California State UniversityLong Beach

Personalised recommendations