Behavior Research Methods

, Volume 50, Issue 1, pp 57–83 | Cite as

How honest are the signals? A protocol for validating wearable sensors

  • Varol Onur Kayhan
  • Zheng (Chris) Chen
  • Kimberly A. French
  • Tammy D. Allen
  • Kristen Salomon
  • Alison Watkins
Article

Abstract

There is growing interest among organizational researchers in tapping into alternative sources of data beyond self-reports to provide a new avenue for measuring behavioral constructs. Use of alternative data sources such as wearable sensors is necessary for developing theory and enhancing organizational practice. Although wearable sensors are now commercially available, the veracity of the data they capture is largely unknown and mostly based on manufacturers’ claims. The goal of this research is to test the validity and reliability of data captured by one such wearable badge (by Humanyze) in the context of structured meetings where all individuals wear a badge for the duration of the encounter. We developed a series of studies, each targeting a specific sensor of this badge that is relevant for structured meetings, and we make specific recommendations for badge data usage based on our validation results. We have incorporated the insights from our studies on a website that researchers can use to conduct validation tests for their badges, upload their data, and assess the validity of the data. We discuss this website in the corresponding studies.

Keywords

Wearable sensors Unobtrusive measures Machine learning 

References

  1. Chaffin, D., Heidl, R., Hollenbeck, J. R., Howe, M., Yu, A., Voorhees, C., & Calantone, R. (2017). The promise and perils of wearable sensors in organizational research. Organizational Research Methods, 20, 3-31. doi: https://doi.org/10.1177/1094428115617004 CrossRefGoogle Scholar
  2. Curhan, J. R., & Pentland, A. (2007). Thin slices of negotiation: Predicting outcomes from conversational dynamics within the first 5 minutes. Journal of Applied Psychology, 92(3), 802-811. doi: https://doi.org/10.1037/0021-9010.92.3.802 CrossRefPubMedGoogle Scholar
  3. Dockweiler, S. (2014). How much time do we spend in meetings? (Hint: It's Scary). Retrieved from https://www.themuse.com/advice/how-much-time-do-we-spend-in-meetings-hint-its-scary
  4. Hall, J. A., Coats, E. J., & LeBeau, L. S. (2005). Nonverbal behavior and the vertical dimension of social relations: A meta-analysis. Psychological Bulletin, 131(6), 898-924. doi: https://doi.org/10.1037/0033-2909.131.6.898 CrossRefPubMedGoogle Scholar
  5. Kim, T., Chang, A., Holland, L., & Pentland, A. S. (2008). Meeting mediator: Enhancing group collaborationusing sociometric feedback. In Proceedings of the 2008 ACM conference on Computer supported cooperative work, San Diego, CA, pp.457-466.Google Scholar
  6. Linoff, G. S., & Berry, M. J. (2010). Data mining techniques: For marketing, sales, and customer relationship management (3rd ed.). Indianapolis, IN: John Wiley & Sons.Google Scholar
  7. Olguin-Olguin, D., Gloor, P. A., & Pentland, A. S. (2009). Capturing individual and group behavior with wearable sensors. In Proceedings of the 2009 AAAI Spring Symposium on Human Behavior Modeling, Palo Alto, CA, pp.68–74.Google Scholar
  8. Olguin-Olguin, D., & Pentland, A. (2010a). Assessing group performance from collective behavior. Paper presented at the ACM Conference on Computer Supported Cooperative Work, Workshop on Collective Intelligence In Organizations, Savannah, GA.Google Scholar
  9. Olguin-Olguin, D., & Pentland, A. (2010b). Sensor-based organisational design and engineering. International Journal of Organisational Design and Engineering, 1(1-2), 69-97. doi: https://doi.org/10.1504/IJODE.2010.035187 CrossRefGoogle Scholar
  10. Olguin-Olguin, D., Waber, B. N., Kim, T., Mohan, A., Ara, K., & Pentland, A. (2009). Sensible organizations: Technology and methodology for automatically measuring organizational behavior. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 39(1), 43-55. doi: https://doi.org/10.1109/TSMCB.2008.2006638 CrossRefGoogle Scholar
  11. Orbach, M., Demko, M., Doyle, J., Waber, B. N., & Pentland, A. (2015). Sensing informal networks in organizations. American Behavioral Scientist, 59(4), 508-524. doi: https://doi.org/10.1177/0002764214556810 CrossRefGoogle Scholar
  12. Shmueli, G., Patel, N., & Bruce, P. (2010). Data mining for business intelligence: Concepts, techniques, and applications in Microsoft Office Excel with XLMiner (2nd ed.). Hoboken, NJ: John Wiley & Sons.Google Scholar
  13. Sociometric Solutions. (2014). Sociometric badge 03-02. Preliminary user guide. Revision 1.21.Google Scholar
  14. Sociometric Solutions. (2015). Data collection and export recommendations for researchers.Google Scholar
  15. Titze, I. R. (1994). Principles of voice production (1st ed.). Englewood Cliffs, N.J: Prentice Hall.Google Scholar
  16. Tripathi, P., & Burleson, W. (2012). Predicting creativity in the wild: Experience sample and sociometric modeling of teams. In Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work, Seattle, WA, pp.1203-1212.Google Scholar
  17. Waber, B. N., Olguin-Olguin, D., Kim, T., & Pentland, A. S. (2008). Understanding Organizational Behavior with Wearable Sensing Technology. Available at SSRN: https://ssrn.com/abstract=1263992
  18. Wu, L., Waber, B. N., Aral, S., Brynjolfsson, E., & Pentland, A. S. (2008). Mining face-to-face interaction networks using sociometric badges: Predicting productivity in an IT configuration task. In Proceedings of the 29th International Conference on Information Systems, Paris, France, pp.1–19.Google Scholar

Copyright information

© Psychonomic Society, Inc. 2018

Authors and Affiliations

  • Varol Onur Kayhan
    • 1
    • 2
  • Zheng (Chris) Chen
    • 1
  • Kimberly A. French
    • 3
  • Tammy D. Allen
    • 4
  • Kristen Salomon
    • 4
  • Alison Watkins
    • 1
  1. 1.Kate Tiedemann College of BusinessUniversity of South Florida St. PetersburgSt. PetersburgUSA
  2. 2.St. PetersburgUSA
  3. 3.School of PsychologyGeorgia Institute of TechnologyAtlantaUSA
  4. 4.College of Arts and SciencesUniversity of South FloridaTampaUSA

Personalised recommendations