Quality & Quantity

, Volume 53, Issue 1, pp 221–246 | Cite as

Data generation methods across the empirical sciences: differences in the study phenomena’s accessibility and the processes of data encoding

  • Jana UherEmail author


Data generation methods differ across the empirical sciences. Today’s physicists and engineers primarily generate data with automated technologies. Behavioural, psychological and social scientists explore phenomena that are not technically accessible (e.g., attitudes, social beliefs) or only in limited ways (e.g., behaviours) and therefore generate data primarily with persons. But human abilities are involved in any data generation, even when technologies are used and developed. This article explores concepts and methods of data generation of different sciences from transdisciplinary and philosophy-of-science perspectives. It highlights that empirical data can reveal information about the phenomena under study only if relevant properties of these phenomena have been encoded systematically in the data. Metatheoretical concepts and methodological principles are elaborated that open up new perspectives on methods of data generation across the empirical sciences, highlighting commonalities and differences in two pivotal points: (1) in the accessibility that various kinds of phenomena have for the persons generating the data and for the researchers, and (2), as a consequence thereof, in the processes involved in the encoding of information from these phenomena in the signs (symbols) used as data. These concepts and principles cut across establish method categorisations (e.g., human-generated versus instrument-generated data; quantitative versus qualitative methods), highlighting fundamental issues equally important in all sciences as well as essential differences. They also provide novel lines of argumentation that substantiate psychologists’ and social scientists’ increasing criticism of their own disciplines’ focus on standardised assessment methods and establish connections to concepts of data generation developed in metrology.


Assessment method Data generation Experience sampling Human-based measurement Introspection Measurement with persons Metrology Observation Qualitative–quantitative methods 



The author gratefully acknowledges funding from a Marie Curie Fellowship (EC Grant Agreement Number 629430). She thanks the editor and the reviewer for thoughtful comments on the manuscript.


  1. Althusser, L., Balibar, E.: Reading Capital. New Left Books, London (1970)Google Scholar
  2. Arro, G.: Peeking into personality test answers: inter- and intraindividual variety in item interpretations. Integr. Psychol. Behav. Sci. 47(1), 56–76 (2013)Google Scholar
  3. Berger, J.: Ways of Seeing. Penguin, London (1972)Google Scholar
  4. Berglund, B., Rossi, G.B., Townsend, J.T., Pendrill, L.R.: Measurement with Persons: Theory, Methods, and Implementation Areas. Taylor Francis, New York (2012)Google Scholar
  5. BIPM, Bureau International des Poids et Mesures. The international system of units (SI), 8th edn. Organisation Intergouvernementale de la Convention du Mètre. (2006).
  6. Bohr, N.: Causality and complementarity. Philos. Sci. 4(3), 289–298 (1937)Google Scholar
  7. Bruschi, A.: Measurement in social research: some misunderstandings. Qual. Quant. 51(5), 2219–2243 (2017)Google Scholar
  8. Bühler, K.: Sprachtheorie: Die Darstellungsfunktion der Sprache. Lucius und Lucius (1999)Google Scholar
  9. Buntins, M., Buntins, K., Eggert, F.: Psychological tests from a (fuzzy-)logical point of view. Qual. Quant. 50(6), 2395–2416 (2016)Google Scholar
  10. Butler, J.: Rethinking Introspection. Palgrave Macmillan, London (2013)Google Scholar
  11. Capra, F.: The Web of Life: A New Synthesis of Mind and Matter. Anchor Books, New York (1997)Google Scholar
  12. Chalmers, A.F.: What is this Thing Called Science? Hackett Publishing, Indianapolis (2013)Google Scholar
  13. Chang, H.: Inventing Temperature: Measurement and Scientific Progress. Oxford University Press, Oxford (2004)Google Scholar
  14. Clark, A., Chalmers, D.: The extended mind. Analysis 58(1), 7–19 (1998)Google Scholar
  15. Cripps, B.: Psychometric Testing: Critical Perspectives. Wiley, Hoboken (2017)Google Scholar
  16. Daston, L., Galison, P.: Objectivity. Zone Books (2007)Google Scholar
  17. Dehaene, S., Izard, V., Spelke, E., Pica, P.: Log or linear? Distinct intuitions of the number scale in Western and Amazonian indigene cultures. Science 320(5880), 1217–1220 (2008)Google Scholar
  18. Eagle, N., Pentland, A.: Reality mining: sensing complex social systems. Pers. Ubiquitous Comput. 10(4), 255–268 (2006)Google Scholar
  19. Ellis, A., Beattie, G.: The Psychology of Language and Communication. Guilford Press, New York (1986)Google Scholar
  20. Fahrenberg, J.: The complementarity principle in psychophysiological research and somatic medicine. Z. Klin. Psychol. Psychother. 27(2), 151–167 (1979)Google Scholar
  21. Fahrenberg, J.: Psychologische Interpretation. Biographien-Texte-Tests. Hans Huber. ebook, Bern (2002)
  22. Fahrenberg, J.: Zur Kategorienlehre der Psychologie: Komplementaritätsprinzip. Pabst Science Publishers, Perspektiven und Perspektiven-Wechsel (2013)Google Scholar
  23. Fahrenberg, J., Myrtek, M., Pawlik, K., Perrez, M.: Ambulatory assessment—monitoring behavior in daily life settings. Eur. J. Psychol. Assess. 23(4), 206–213 (2007)Google Scholar
  24. Faßnacht, G.: Theory and Practice of Observing Behaviour, 2nd edn. Academic Press, London (1982)Google Scholar
  25. Fechner, G.T.: Elemente der Psychophysik. Breitkopf und Härtel, Leipzig (1860)Google Scholar
  26. Fechner, G.T.: Ueber die psychischen Maßprincipien und das Webersche Gesetz. Philos. Stud. 4, 161–230 (1888)Google Scholar
  27. Fisher, W.P.: Invariance and traceability for measures of human, social, and natural capital: theory and application. Measurement 13(42), 1278–1287 (2009)Google Scholar
  28. Flick, U.: Managing Quality in Qualitative Research. Sage Publications, London (2008)Google Scholar
  29. Flick, U.: An introduction to qualitative research. Sage Publications, London (2014)Google Scholar
  30. Freud, S.: The unconscious. Stand. Ed. 14, 159–214 (1915)Google Scholar
  31. Furr, R.M.: Personality psychology as a truly behavioural science. Eur. J. Pers. 23(5), 369–401 (2009)Google Scholar
  32. Gibson, J.J.: The Ecological Approach to Visual Perception. Houghton Mifflin, Boston (1979)Google Scholar
  33. Gigerenzer, G.: Probabilistic thinking and the fight against subjectivity. In: Krüger, L., Gigerenzer, G., Morgan, M.S. (eds.) Probabilistic Revolution, Vol. 2: Ideas in the Science, pp. 11–33. MIT Press, Cambridge (1987)Google Scholar
  34. Hammersley, M.: The Myth of Research-Based Policy and Practice. SAGE Publications, London (2013)Google Scholar
  35. Hassabis, D., Maguire, E.A.: Deconstructing episodic memory with construction. Trends Cogn. Sci. 11(7), 299–306 (2007)Google Scholar
  36. JCGM, Joint Committee for Guides in Metrology. International vocabulary of metrology – Basic and general concepts and associated terms (VIM 3rd edition). Working Group 2 (Eds.), Joint Committee for Guides in Metrology (2012)Google Scholar
  37. Kant, I.: Kritik der reinen Vernunft. In: Timmermann, J. (Ed.) Felix Meiner Verlag, Hamburg (1998)Google Scholar
  38. Kant, I.: Anthropologie in Pragmatischer Hinsicht. In: Brandt, R. (Ed.) Felix Meiner (2000)Google Scholar
  39. Kant I. Metaphysische Anfangsgründe der Naturwissenschaft. 4. Auflage. In: Holzinger, M. (Ed.) CreateSpace Independent Publishing Platform (2016)Google Scholar
  40. Kelly, G.: A Theory of Personality: The Psychology of Personal Constructs. W.W. Norton, New York (1963)Google Scholar
  41. Lahlou, S.: How can we capture the subject’s perspective? An evidence-based approach for the social scientist. Soc. Sci. Inf. 50(3–4), 607–655 (2011)Google Scholar
  42. Larson, R., Csikszentmihalyi, M.: The experience sampling method. In: Flow and the Foundations of Positive Psychology, pp. 21–34. Springer, Dordrecht (2014)Google Scholar
  43. Little, B.R.: Well-doing: personal projects and the quality of lives. Sch. Field 12(3), 329–346 (2014)Google Scholar
  44. Lundmann, L., Villadsen, J.W.: Qualitative variations in personality inventories: subjective understandings of items in a personality inventory. Qual. Res. Psychol. 13(2), 166–187 (2016)Google Scholar
  45. Mari, L.: Epistemology of measurement. Measurement 34, 17–30 (2003)
  46. Mari, L., Maul, A., Irribarra, D.T., Wilson, M.: A meta-structural understanding of measurement. J. Phys. Conf. Ser. 772(1), 12009 (2016)Google Scholar
  47. McAdams, D.P.: The psychology of life stories. Rev. Gen. Psychol. 5(2), 100–122 (2001)Google Scholar
  48. Mehl, M.R., Pennebaker, J.W., Crow, D.M., Dabbs, J., Price, J.H.: The Electronically Activated Recorder (EAR): a device for sampling naturalistic daily activities and conversations. Behav. Res. Methods Instrum. Comput. 33(4), 517–523 (2001)Google Scholar
  49. Mehl, M.R., Conner, T.S.: Handbook of Research Methods for Studying Daily Life. Guilford Press, New York (2012)Google Scholar
  50. Michell, J.: The quantity/quality interchange: a blind spot on the highway of science. In: Toomela, A., Valsiner, J. (eds.) Methodological Thinking in Psychology: 60 Years Gone Astray? Information Age Publishing, Charlotte (2010)Google Scholar
  51. Morin, E.: On Complexity. Hampton Press, Cresskill (2008)Google Scholar
  52. Morris, S.D., Grice, J.W., Cox, R.A.: Scale imposition as quantitative alchemy: studies on the transitivity of neuroticism ratings. Basic Appl. Soc. Psychol. 39(1), 1–18 (2017)Google Scholar
  53. Pendrill, L.: Man as a measurement instrument. NCSLI Meas. 9(4), 24–35 (2014)Google Scholar
  54. Pendrill, L., Fisher, W.P.: Counting and quantification: comparing psychometric and metrological perspectives on visual perceptions of number. Measurement 10(71), 46–55 (2015)Google Scholar
  55. Pendrill, L., Petersson, N.: Metrology of human-based and other qualitative measurements. Meas. Sci. Technol. 27(9), 94003 (2016)Google Scholar
  56. Rosenbaum, P.J., Valsiner, J.: The un-making of a method: From rating scales to the study of psychological processes. Theory Psychol. 21(1), 47–65 (2011)Google Scholar
  57. Rowlands, M.: Externalism: Putting mind and world back together again. Acumen (2003)Google Scholar
  58. Russell, B.: A priori justification and knowledge. In: Zalta, E.N. (Ed.) Stanford Encyclopedia of Philosophy (2014).
  59. Sale, J.E.M., Lohfeld, L.H., Brazil, K.: Revisiting the quantitative-qualitative debate: implications for mixed-methods research. Qual. Quant. 36(1), 43–53 (2002)Google Scholar
  60. Schacter, D.L.: The seven sins of memory. Insights from psychology and cognitive neuroscience. Am. Psychol. 54(3), 182–203 (1999)Google Scholar
  61. Schacter, D.L., Addis, D.R.: Constructive memory: the ghosts of past and future. Nature 445(7123), 27 (2007)Google Scholar
  62. Schwitzgebel, E.: Introspection. Stanford Encycl. Philos. [Internet]. Winter 2016. Metaphysics Research Lab, Stanford University; 2016 [cited 2017 Sep 30].
  63. Steinke, I.: Quality criteria in qualitative research. In: Flick, U., Kardorff, E.V., Steinke, I. (eds.) A companion to qualitative research, pp. 184–190. Sage, Thousand Oaks (2004)Google Scholar
  64. Tafreshi, D., Slaney, K.L., Neufeld, S.D.: Quantification in psychology: critical analysis of an unreflective practice. J. Theor. Philos. Psychol. 36(4), 233–249 (2016)Google Scholar
  65. Thorndike, E.L.: Notes on child study, 2nd edn. Macmillan, New York (1903)Google Scholar
  66. Thurstone, L.L.: The method of paired comparisons for social values. J. Abnorm. Soc. Psychol. 21, 384–400 (1927)Google Scholar
  67. Thurstone, L.L.: Attitudes can be measured. Am. J. Sociol. 33, 529–554 (1928)Google Scholar
  68. Toomela, A.: Travel into a fairy land: a critique of modern qualitative and mixed methods psychologies. Integr. Psychol. Behav. Sci. 45(1), 21–47 (2011)Google Scholar
  69. Uher, J.: Comparative personality research: methodological approaches. Eur. J. Pers. 22(5), 427–455 (2008a)Google Scholar
  70. Uher, J.: Three methodological core issues of comparative personality research. Eur. J. Pers. 22(5), 475–496 (2008b)Google Scholar
  71. Uher, J.: Personality in nonhuman primates: What can we learn from human personality psychology? Personal. Temperament Nonhum. Primates, pp. 41–76. Springer, New York (2011)Google Scholar
  72. Uher, J.: Personality psychology: Lexical approaches, assessment methods, and trait concepts reveal only half of the story-Why it is time for a paradigm shift. Integr. Psychol. Behav. Sci. 47(1), 1–55 (2013)Google Scholar
  73. Uher, J.: Agency enabled by the psyche: Explorations using the transdisciplinary philosophy-of-science paradigm for research on individuals. In: Gruber, C.W., Clark, M.G., Klempe, S.H., Valsiner, J. (eds.) Constraints of agency: Explorations of theory in everyday life. Annals of Theoretical Psychology, vol. 12, pp. 177–228. Springer International Publishing, New York (2015a)Google Scholar
  74. Uher, J.: Interpreting, “personality” taxonomies: Why previous models cannot capture individual-specific experiencing, behaviour, functioning and development. Major taxonomic tasks still lay ahead. Integr. Psychol. Behav. Sci. 49(4), 600–655 (2015b)Google Scholar
  75. Uher, J.: Comparing individuals within and across situations, groups and species: Metatheoretical and methodological foundations demonstrated in primate behaviour. In: Emmans, D., Laihinen, A. (eds.) Comp. Neuropsychol. Brain Imaging, pp. 223–284. Lit Verlag, Berlin (2015c)Google Scholar
  76. Uher, J.: Conceiving, “personality”: Psychologist’s challenges and basic fundamentals of the Transdisciplinary Philosophy-of-Science Paradigm for Research on Individuals. Integr. Psychol. Behav. Sci. 49(3), 398–458 (2015d)Google Scholar
  77. Uher, J.: Developing, “personality” taxonomies: Metatheoretical and methodological rationales underlying selection approaches, methods of data generation and reduction principles. Integr. Psychol. Behav. Sci. 49, 531–589 (2015e)Google Scholar
  78. Uher, J.: What is behaviour? And (when) is language behaviour? A metatheoretical definition. J. Theory Soc. Behav. 46(4), 475–501 (2016a)Google Scholar
  79. Uher, J.: Exploring the workings of the Psyche: Metatheoretical and methodological foundations. In: Valsiner, J., Marsico, G., Chaudhary, N., Sato, T., Dazzani, V. (eds.) Psychology as the science of human being: The Yokohama Manifesto, pp. 299–324. Springer International Publishing, New York (2016b)Google Scholar
  80. Uher, J.: Taxonomic models of individual differences: a guide to transdisciplinary approaches. Philos. Trans. R. Soc. B Biol. Sci. 373(1744), 20170171 (2018). Google Scholar
  81. Uher, J., Addessi, E., Visalberghi, E.: Contextualised behavioural measurements of personality differences obtained in behavioural tests and social observations in adult capuchin monkeys (Cebus apella). J. Res. Pers. 47(4), 427–444 (2013a)Google Scholar
  82. Uher, J., Werner, C.S., Gosselt, K.: From observations of individual behaviour to social representations of personality: Developmental pathways, attribution biases, and limitations of questionnaire methods. J. Res. Pers. 47(5), 647–667 (2013b)Google Scholar
  83. Uher, J., Visalberghi, E.: Observations versus assessments of personality: A five-method multi-species study reveals numerous biases in ratings and methodological limitations of standardised assessments. J. Res. Pers. 61, 61–79 (2016)Google Scholar
  84. Valsiner, J.: The Guided Mind. A Sociogenetic Approach to Personality. Harvard University Press, Cambridge (1998)Google Scholar
  85. Valsiner, J.: From Methodology to Methods in Human Psychology. Springer, Cham (2017)Google Scholar
  86. Valsiner, J., Diriwächter, R., Sauck, C.: Diversity in unity: standard questions and nonstandard interpretations. In: Bibace, R., Laird, J., Noller, K., Valsiner, J. (eds.) Science Mediterranean Dialogue, pp. 289–307. Westport, Praeger-Greenwood (2005)Google Scholar
  87. Weber, M.: On the methodology of the social sciences. In: Shils, E., Finch, H. (Ed.) Free Press, New York (1949)Google Scholar
  88. Wong, W.: Retracing the footsteps of Wilhelm Wundt: explorations in the disciplinary frontiers of psychology and in Völkerpsychologie. Hist. Psychol. 12(4), 229–265 (2009)Google Scholar
  89. Wundt, W.: Über psychische Kausalität und das Prinzip des psycho-physischen Parallelismus. Philos. Stud. 10, 1–124 (1894)Google Scholar
  90. Wundt, W.: Grundriss der Psychologie. Stuttgart: Körner (1896).
  91. Wundt, W.: Principles of Physiological Psychology. Allen, London (1904)Google Scholar

Copyright information

© Springer Science+Business Media B.V., part of Springer Nature 2018

Authors and Affiliations

  1. 1.University of GreenwichLondonUK
  2. 2.London School of EconomicsLondonUK

Personalised recommendations