Skip to main content

Ranking vs. Preference: A Comparative Study of Self-reporting

  • Conference paper

Part of the Lecture Notes in Computer Science book series (LNIP,volume 6974)

Abstract

This paper introduces a comparative analysis between rating and pairwise self-reporting via questionnaires in user survey experiments. Two dissimilar game user survey experiments are employed in which the two questionnaire schemes are tested and compared for reliable affect annotation. The statistical analysis followed to test our hypotheses shows that even though the two self-reporting schemes are consistent there are significant order of reporting effects when subjects report via a rating questionnaire. The paper concludes with a discussion of the appropriateness of each self-reporting scheme under conditions drawn from the experimental results obtained.

An Erratum for this chapter can be found at http://dx.doi.org/10.1007/978-3-642-24600-5_65 .

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-642-24600-5_47
  • Chapter length: 10 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   84.99
Price excludes VAT (USA)
  • ISBN: 978-3-642-24600-5
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   109.99
Price excludes VAT (USA)

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Agresti, A.: Analysis of ordinal paired comparison data. Journal of the Royal Statistical Society. Series C (Applied Statistics) 41(2), 287–297 (1992)

    MATH  Google Scholar 

  2. Chan, J.C.: Response-order effects in Likert-type scales. Educational and Psychological Measurement 51(3), 531–540 (1991)

    CrossRef  Google Scholar 

  3. Cohen, J.: A coefficient of agreement for nominal scales. Educational and Psychological Measurement 20, 37–46 (1960)

    CrossRef  Google Scholar 

  4. Cronbach, J.L.: Coefficient alpha and the internal structure of tests. Psychometrika 16(3), 297–334 (1951)

    CrossRef  Google Scholar 

  5. Likert, R.: A technique for the measurement of attitudes. Archives of Psychology 140, 1–55 (1932)

    Google Scholar 

  6. Linn, R., Gronlund, N.: Measurement and assessment in teaching. Prentice-Hall, Englewood Cliffs (2000)

    Google Scholar 

  7. Lund, H.H., Klitbo, T., Jessen, C.: Playware technology for physically activating play. Artifical Life and Robotics Journal 9(4), 165–174 (2005)

    CrossRef  Google Scholar 

  8. Poels, K., IJsselsteijn, W.: Development and validation of the game experience questionnaire. In: FUGA Workshop Mini-Symposium, Helsinki, Finland (2008)

    Google Scholar 

  9. Read, J., MacFarlane, S., Cassey, C.: Endurability, engagement and expectations. In: Proceedings of International Conference for Interaction Design and Children (2002)

    Google Scholar 

  10. Scheffe, H.: An analysis of variance for paired comparisons. Journal of the American Statistical Association 47(259), 381–400 (1952)

    MathSciNet  MATH  Google Scholar 

  11. Stevens, S.S.: On the Theory of Scales of Measurement. Science 103(2684), 677–680 (1946)

    CrossRef  MATH  Google Scholar 

  12. Tognetti, S., Garbarino, M., Bonarini, A., Matteucci, M.: Modeling enjoyment preference from physiological responses in a car racing game. In: Proceedings of the IEEE Conference on Computational Intelligence and Games, Copenhagen, Denmark, August 18-21, pp. 321–328 (2010)

    Google Scholar 

  13. Viswanathan, M.: Measurement of individual differences in preference for numerical information. Journal of Applied Psychology 78(5), 741–752

    Google Scholar 

  14. Yannakakis, G.N.: Preference Learning for Affective Modeling. In: Proceedings of the Int. Conf. on Affective Computing and Intelligent Interaction, pp. 126–131. IEEE, Amsterdam (2009)

    Google Scholar 

  15. Yannakakis, G.N., Hallam, J.: Towards Optimizing Entertainment in Computer Games. Applied Artificial Intelligence 21, 933–971 (2007)

    CrossRef  Google Scholar 

  16. Yannakakis, G.N., Hallam, J., Lund, H.H.: Entertainment Capture through Heart Rate Activity in Physical Interactive Playgrounds. User Modeling and User-Adapted Interaction, Special Issue: Affective Modeling and Adaptation 18(1-2), 207–243 (2008)

    CrossRef  Google Scholar 

  17. Yannakakis, G.N., Maragoudakis, M., Hallam, J.: Preference Learning for Cognitive Modeling: A Case Study on Entertainment Preferences. IEEE Systems, Man and Cybernetics; Part A: Systems and Humans 39(6), 1165–1175 (2009)

    CrossRef  Google Scholar 

  18. Yannakakis, G.N., Martínez, H.P., Jhala, A.: Towards Affective Camera Control in Games. User Modeling and User-Adapted Interaction 20(4), 313–340 (2010)

    CrossRef  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2011 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Yannakakis, G.N., Hallam, J. (2011). Ranking vs. Preference: A Comparative Study of Self-reporting. In: D’Mello, S., Graesser, A., Schuller, B., Martin, JC. (eds) Affective Computing and Intelligent Interaction. ACII 2011. Lecture Notes in Computer Science, vol 6974. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-24600-5_47

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-24600-5_47

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-24599-2

  • Online ISBN: 978-3-642-24600-5

  • eBook Packages: Computer ScienceComputer Science (R0)