Data generation methods across the empirical sciences: differences in the study phenomena’s accessibility and the processes of data encoding

Article
  • 20 Downloads

Abstract

Data generation methods differ across the empirical sciences. Today’s physicists and engineers primarily generate data with automated technologies. Behavioural, psychological and social scientists explore phenomena that are not technically accessible (e.g., attitudes, social beliefs) or only in limited ways (e.g., behaviours) and therefore generate data primarily with persons. But human abilities are involved in any data generation, even when technologies are used and developed. This article explores concepts and methods of data generation of different sciences from transdisciplinary and philosophy-of-science perspectives. It highlights that empirical data can reveal information about the phenomena under study only if relevant properties of these phenomena have been encoded systematically in the data. Metatheoretical concepts and methodological principles are elaborated that open up new perspectives on methods of data generation across the empirical sciences, highlighting commonalities and differences in two pivotal points: (1) in the accessibility that various kinds of phenomena have for the persons generating the data and for the researchers, and (2), as a consequence thereof, in the processes involved in the encoding of information from these phenomena in the signs (symbols) used as data. These concepts and principles cut across establish method categorisations (e.g., human-generated versus instrument-generated data; quantitative versus qualitative methods), highlighting fundamental issues equally important in all sciences as well as essential differences. They also provide novel lines of argumentation that substantiate psychologists’ and social scientists’ increasing criticism of their own disciplines’ focus on standardised assessment methods and establish connections to concepts of data generation developed in metrology.

Keywords

Assessment method Data generation Experience sampling Human-based measurement Introspection Measurement with persons Metrology Observation Qualitative–quantitative methods 

Notes

Acknowledgements

The author gratefully acknowledges funding from a Marie Curie Fellowship (EC Grant Agreement Number 629430). She thanks the editor and the reviewer for thoughtful comments on the manuscript.

References

  1. Althusser, L., Balibar, E.: Reading Capital. New Left Books, London (1970)Google Scholar
  2. Arro, G.: Peeking into personality test answers: inter- and intraindividual variety in item interpretations. Integr. Psychol. Behav. Sci. 47(1), 56–76 (2013)CrossRefGoogle Scholar
  3. Berger, J.: Ways of Seeing. Penguin, London (1972)Google Scholar
  4. Berglund, B., Rossi, G.B., Townsend, J.T., Pendrill, L.R.: Measurement with Persons: Theory, Methods, and Implementation Areas. Taylor Francis, New York (2012)Google Scholar
  5. BIPM, Bureau International des Poids et Mesures. The international system of units (SI), 8th edn. Organisation Intergouvernementale de la Convention du Mètre. (2006). https://www.bipm.org/
  6. Bohr, N.: Causality and complementarity. Philos. Sci. 4(3), 289–298 (1937)CrossRefGoogle Scholar
  7. Bruschi, A.: Measurement in social research: some misunderstandings. Qual. Quant. 51(5), 2219–2243 (2017)CrossRefGoogle Scholar
  8. Bühler, K.: Sprachtheorie: Die Darstellungsfunktion der Sprache. Lucius und Lucius (1999)Google Scholar
  9. Buntins, M., Buntins, K., Eggert, F.: Psychological tests from a (fuzzy-)logical point of view. Qual. Quant. 50(6), 2395–2416 (2016)CrossRefGoogle Scholar
  10. Butler, J.: Rethinking Introspection. Palgrave Macmillan, London (2013)CrossRefGoogle Scholar
  11. Capra, F.: The Web of Life: A New Synthesis of Mind and Matter. Anchor Books, New York (1997)Google Scholar
  12. Chalmers, A.F.: What is this Thing Called Science? Hackett Publishing, Indianapolis (2013)Google Scholar
  13. Chang, H.: Inventing Temperature: Measurement and Scientific Progress. Oxford University Press, Oxford (2004)CrossRefGoogle Scholar
  14. Clark, A., Chalmers, D.: The extended mind. Analysis 58(1), 7–19 (1998)CrossRefGoogle Scholar
  15. Cripps, B.: Psychometric Testing: Critical Perspectives. Wiley, Hoboken (2017)CrossRefGoogle Scholar
  16. Daston, L., Galison, P.: Objectivity. Zone Books (2007)Google Scholar
  17. Dehaene, S., Izard, V., Spelke, E., Pica, P.: Log or linear? Distinct intuitions of the number scale in Western and Amazonian indigene cultures. Science 320(5880), 1217–1220 (2008)CrossRefGoogle Scholar
  18. Eagle, N., Pentland, A.: Reality mining: sensing complex social systems. Pers. Ubiquitous Comput. 10(4), 255–268 (2006)CrossRefGoogle Scholar
  19. Ellis, A., Beattie, G.: The Psychology of Language and Communication. Guilford Press, New York (1986)Google Scholar
  20. Fahrenberg, J.: The complementarity principle in psychophysiological research and somatic medicine. Z. Klin. Psychol. Psychother. 27(2), 151–167 (1979)Google Scholar
  21. Fahrenberg, J.: Psychologische Interpretation. Biographien-Texte-Tests. Hans Huber. ebook, Bern (2002) http://www.jochen-fahrenberg.de
  22. Fahrenberg, J.: Zur Kategorienlehre der Psychologie: Komplementaritätsprinzip. Pabst Science Publishers, Perspektiven und Perspektiven-Wechsel (2013)Google Scholar
  23. Fahrenberg, J., Myrtek, M., Pawlik, K., Perrez, M.: Ambulatory assessment—monitoring behavior in daily life settings. Eur. J. Psychol. Assess. 23(4), 206–213 (2007)CrossRefGoogle Scholar
  24. Faßnacht, G.: Theory and Practice of Observing Behaviour, 2nd edn. Academic Press, London (1982)Google Scholar
  25. Fechner, G.T.: Elemente der Psychophysik. Breitkopf und Härtel, Leipzig (1860)Google Scholar
  26. Fechner, G.T.: Ueber die psychischen Maßprincipien und das Webersche Gesetz. Philos. Stud. 4, 161–230 (1888)Google Scholar
  27. Fisher, W.P.: Invariance and traceability for measures of human, social, and natural capital: theory and application. Measurement 13(42), 1278–1287 (2009)CrossRefGoogle Scholar
  28. Flick, U.: Managing Quality in Qualitative Research. Sage Publications, London (2008)Google Scholar
  29. Flick, U.: An introduction to qualitative research. Sage Publications, London (2014)Google Scholar
  30. Freud, S.: The unconscious. Stand. Ed. 14, 159–214 (1915)Google Scholar
  31. Furr, R.M.: Personality psychology as a truly behavioural science. Eur. J. Pers. 23(5), 369–401 (2009)CrossRefGoogle Scholar
  32. Gibson, J.J.: The Ecological Approach to Visual Perception. Houghton Mifflin, Boston (1979)Google Scholar
  33. Gigerenzer, G.: Probabilistic thinking and the fight against subjectivity. In: Krüger, L., Gigerenzer, G., Morgan, M.S. (eds.) Probabilistic Revolution, Vol. 2: Ideas in the Science, pp. 11–33. MIT Press, Cambridge (1987)Google Scholar
  34. Hammersley, M.: The Myth of Research-Based Policy and Practice. SAGE Publications, London (2013)CrossRefGoogle Scholar
  35. Hassabis, D., Maguire, E.A.: Deconstructing episodic memory with construction. Trends Cogn. Sci. 11(7), 299–306 (2007)CrossRefGoogle Scholar
  36. JCGM, Joint Committee for Guides in Metrology. International vocabulary of metrology – Basic and general concepts and associated terms (VIM 3rd edition). Working Group 2 (Eds.), Joint Committee for Guides in Metrology (2012)Google Scholar
  37. Kant, I.: Kritik der reinen Vernunft. In: Timmermann, J. (Ed.) Felix Meiner Verlag, Hamburg (1998)Google Scholar
  38. Kant, I.: Anthropologie in Pragmatischer Hinsicht. In: Brandt, R. (Ed.) Felix Meiner (2000)Google Scholar
  39. Kant I. Metaphysische Anfangsgründe der Naturwissenschaft. 4. Auflage. In: Holzinger, M. (Ed.) CreateSpace Independent Publishing Platform (2016)Google Scholar
  40. Kelly, G.: A Theory of Personality: The Psychology of Personal Constructs. W.W. Norton, New York (1963)Google Scholar
  41. Lahlou, S.: How can we capture the subject’s perspective? An evidence-based approach for the social scientist. Soc. Sci. Inf. 50(3–4), 607–655 (2011)CrossRefGoogle Scholar
  42. Larson, R., Csikszentmihalyi, M.: The experience sampling method. In: Flow and the Foundations of Positive Psychology, pp. 21–34. Springer, Dordrecht (2014)Google Scholar
  43. Little, B.R.: Well-doing: personal projects and the quality of lives. Sch. Field 12(3), 329–346 (2014)CrossRefGoogle Scholar
  44. Lundmann, L., Villadsen, J.W.: Qualitative variations in personality inventories: subjective understandings of items in a personality inventory. Qual. Res. Psychol. 13(2), 166–187 (2016)CrossRefGoogle Scholar
  45. Mari, L.: Epistemology of measurement. Measurement 34, 17–30 (2003) http://philpapers.org/rec/MAREOM-2
  46. Mari, L., Maul, A., Irribarra, D.T., Wilson, M.: A meta-structural understanding of measurement. J. Phys. Conf. Ser. 772(1), 12009 (2016)CrossRefGoogle Scholar
  47. McAdams, D.P.: The psychology of life stories. Rev. Gen. Psychol. 5(2), 100–122 (2001)CrossRefGoogle Scholar
  48. Mehl, M.R., Pennebaker, J.W., Crow, D.M., Dabbs, J., Price, J.H.: The Electronically Activated Recorder (EAR): a device for sampling naturalistic daily activities and conversations. Behav. Res. Methods Instrum. Comput. 33(4), 517–523 (2001)CrossRefGoogle Scholar
  49. Mehl, M.R., Conner, T.S.: Handbook of Research Methods for Studying Daily Life. Guilford Press, New York (2012)Google Scholar
  50. Michell, J.: The quantity/quality interchange: a blind spot on the highway of science. In: Toomela, A., Valsiner, J. (eds.) Methodological Thinking in Psychology: 60 Years Gone Astray? Information Age Publishing, Charlotte (2010)Google Scholar
  51. Morin, E.: On Complexity. Hampton Press, Cresskill (2008)Google Scholar
  52. Morris, S.D., Grice, J.W., Cox, R.A.: Scale imposition as quantitative alchemy: studies on the transitivity of neuroticism ratings. Basic Appl. Soc. Psychol. 39(1), 1–18 (2017)CrossRefGoogle Scholar
  53. Pendrill, L.: Man as a measurement instrument. NCSLI Meas. 9(4), 24–35 (2014)CrossRefGoogle Scholar
  54. Pendrill, L., Fisher, W.P.: Counting and quantification: comparing psychometric and metrological perspectives on visual perceptions of number. Measurement 10(71), 46–55 (2015)CrossRefGoogle Scholar
  55. Pendrill, L., Petersson, N.: Metrology of human-based and other qualitative measurements. Meas. Sci. Technol. 27(9), 94003 (2016)CrossRefGoogle Scholar
  56. Rosenbaum, P.J., Valsiner, J.: The un-making of a method: From rating scales to the study of psychological processes. Theory Psychol. 21(1), 47–65 (2011)CrossRefGoogle Scholar
  57. Rowlands, M.: Externalism: Putting mind and world back together again. Acumen (2003)Google Scholar
  58. Russell, B.: A priori justification and knowledge. In: Zalta, E.N. (Ed.) Stanford Encyclopedia of Philosophy (2014). http://plato.stanford.edu/archives/sum2014/entries/apriori
  59. Sale, J.E.M., Lohfeld, L.H., Brazil, K.: Revisiting the quantitative-qualitative debate: implications for mixed-methods research. Qual. Quant. 36(1), 43–53 (2002)CrossRefGoogle Scholar
  60. Schacter, D.L.: The seven sins of memory. Insights from psychology and cognitive neuroscience. Am. Psychol. 54(3), 182–203 (1999)CrossRefGoogle Scholar
  61. Schacter, D.L., Addis, D.R.: Constructive memory: the ghosts of past and future. Nature 445(7123), 27 (2007)CrossRefGoogle Scholar
  62. Schwitzgebel, E.: Introspection. Stanford Encycl. Philos. [Internet]. Winter 2016. Metaphysics Research Lab, Stanford University; 2016 [cited 2017 Sep 30]. http://plato.stanford.edu/archives/win2016/entries/introspection/
  63. Steinke, I.: Quality criteria in qualitative research. In: Flick, U., Kardorff, E.V., Steinke, I. (eds.) A companion to qualitative research, pp. 184–190. Sage, Thousand Oaks (2004)Google Scholar
  64. Tafreshi, D., Slaney, K.L., Neufeld, S.D.: Quantification in psychology: critical analysis of an unreflective practice. J. Theor. Philos. Psychol. 36(4), 233–249 (2016)CrossRefGoogle Scholar
  65. Thorndike, E.L.: Notes on child study, 2nd edn. Macmillan, New York (1903)Google Scholar
  66. Thurstone, L.L.: The method of paired comparisons for social values. J. Abnorm. Soc. Psychol. 21, 384–400 (1927)CrossRefGoogle Scholar
  67. Thurstone, L.L.: Attitudes can be measured. Am. J. Sociol. 33, 529–554 (1928)CrossRefGoogle Scholar
  68. Toomela, A.: Travel into a fairy land: a critique of modern qualitative and mixed methods psychologies. Integr. Psychol. Behav. Sci. 45(1), 21–47 (2011)CrossRefGoogle Scholar
  69. Uher, J.: Comparative personality research: methodological approaches. Eur. J. Pers. 22(5), 427–455 (2008a)CrossRefGoogle Scholar
  70. Uher, J.: Three methodological core issues of comparative personality research. Eur. J. Pers. 22(5), 475–496 (2008b)CrossRefGoogle Scholar
  71. Uher, J.: Personality in nonhuman primates: What can we learn from human personality psychology? Personal. Temperament Nonhum. Primates, pp. 41–76. Springer, New York (2011)CrossRefGoogle Scholar
  72. Uher, J.: Personality psychology: Lexical approaches, assessment methods, and trait concepts reveal only half of the story-Why it is time for a paradigm shift. Integr. Psychol. Behav. Sci. 47(1), 1–55 (2013)CrossRefGoogle Scholar
  73. Uher, J.: Agency enabled by the psyche: Explorations using the transdisciplinary philosophy-of-science paradigm for research on individuals. In: Gruber, C.W., Clark, M.G., Klempe, S.H., Valsiner, J. (eds.) Constraints of agency: Explorations of theory in everyday life. Annals of Theoretical Psychology, vol. 12, pp. 177–228. Springer International Publishing, New York (2015a)Google Scholar
  74. Uher, J.: Interpreting, “personality” taxonomies: Why previous models cannot capture individual-specific experiencing, behaviour, functioning and development. Major taxonomic tasks still lay ahead. Integr. Psychol. Behav. Sci. 49(4), 600–655 (2015b)CrossRefGoogle Scholar
  75. Uher, J.: Comparing individuals within and across situations, groups and species: Metatheoretical and methodological foundations demonstrated in primate behaviour. In: Emmans, D., Laihinen, A. (eds.) Comp. Neuropsychol. Brain Imaging, pp. 223–284. Lit Verlag, Berlin (2015c)Google Scholar
  76. Uher, J.: Conceiving, “personality”: Psychologist’s challenges and basic fundamentals of the Transdisciplinary Philosophy-of-Science Paradigm for Research on Individuals. Integr. Psychol. Behav. Sci. 49(3), 398–458 (2015d)CrossRefGoogle Scholar
  77. Uher, J.: Developing, “personality” taxonomies: Metatheoretical and methodological rationales underlying selection approaches, methods of data generation and reduction principles. Integr. Psychol. Behav. Sci. 49, 531–589 (2015e)CrossRefGoogle Scholar
  78. Uher, J.: What is behaviour? And (when) is language behaviour? A metatheoretical definition. J. Theory Soc. Behav. 46(4), 475–501 (2016a)CrossRefGoogle Scholar
  79. Uher, J.: Exploring the workings of the Psyche: Metatheoretical and methodological foundations. In: Valsiner, J., Marsico, G., Chaudhary, N., Sato, T., Dazzani, V. (eds.) Psychology as the science of human being: The Yokohama Manifesto, pp. 299–324. Springer International Publishing, New York (2016b)CrossRefGoogle Scholar
  80. Uher, J.: Taxonomic models of individual differences: a guide to transdisciplinary approaches. Philos. Trans. R. Soc. B Biol. Sci. 373(1744), 20170171 (2018).  https://doi.org/10.1098/rstb.2017-0171 CrossRefGoogle Scholar
  81. Uher, J., Addessi, E., Visalberghi, E.: Contextualised behavioural measurements of personality differences obtained in behavioural tests and social observations in adult capuchin monkeys (Cebus apella). J. Res. Pers. 47(4), 427–444 (2013a)CrossRefGoogle Scholar
  82. Uher, J., Werner, C.S., Gosselt, K.: From observations of individual behaviour to social representations of personality: Developmental pathways, attribution biases, and limitations of questionnaire methods. J. Res. Pers. 47(5), 647–667 (2013b)CrossRefGoogle Scholar
  83. Uher, J., Visalberghi, E.: Observations versus assessments of personality: A five-method multi-species study reveals numerous biases in ratings and methodological limitations of standardised assessments. J. Res. Pers. 61, 61–79 (2016)CrossRefGoogle Scholar
  84. Valsiner, J.: The Guided Mind. A Sociogenetic Approach to Personality. Harvard University Press, Cambridge (1998)Google Scholar
  85. Valsiner, J.: From Methodology to Methods in Human Psychology. Springer, Cham (2017)CrossRefGoogle Scholar
  86. Valsiner, J., Diriwächter, R., Sauck, C.: Diversity in unity: standard questions and nonstandard interpretations. In: Bibace, R., Laird, J., Noller, K., Valsiner, J. (eds.) Science Mediterranean Dialogue, pp. 289–307. Westport, Praeger-Greenwood (2005)Google Scholar
  87. Weber, M.: On the methodology of the social sciences. In: Shils, E., Finch, H. (Ed.) Free Press, New York (1949)Google Scholar
  88. Wong, W.: Retracing the footsteps of Wilhelm Wundt: explorations in the disciplinary frontiers of psychology and in Völkerpsychologie. Hist. Psychol. 12(4), 229–265 (2009)CrossRefGoogle Scholar
  89. Wundt, W.: Über psychische Kausalität und das Prinzip des psycho-physischen Parallelismus. Philos. Stud. 10, 1–124 (1894)Google Scholar
  90. Wundt, W.: Grundriss der Psychologie. Stuttgart: Körner (1896). http://archive.org/
  91. Wundt, W.: Principles of Physiological Psychology. Allen, London (1904)Google Scholar

Copyright information

© Springer Science+Business Media B.V., part of Springer Nature 2018

Authors and Affiliations

  1. 1.University of GreenwichLondonUK
  2. 2.London School of EconomicsLondonUK

Personalised recommendations