Abstract
Measuring and analyzing spatial, multimodal biosensor data may effectively model how the built environment influences neurophysiological processes. This article presents the Multimer Data Collection and Analysis System (MDCAS), which records data from several kinds of commonly available, wearable sensors (wearables) including electroencephalogram (EEG), electrocardiogram (ECG), pedometer, accelerometer, and gyroscope modules. Data from these wearables is sent to a custom smartphone application, which also records surveys and associates these with global positioning system (GPS) readings. MDCAS then collects and analyzes data from its smartphone app. MDCAS aims to help space professionals like architects, workplace strategists, and urban planners make better design interventions. As a case study of the MDCAS, this article discusses the analysis results of biometric data (EEG, ECG in addition to survey reports) collected from a 2017 study focused on pedestrians, cyclists, and drivers (N = 101) in New York City. Signal and spatial validation of the data indicated usability—data that is not randomly distributed—for biometric data types. Exploratory regressions of the biometric data (regressors) with exogenous data (predictors including environmental and municipal data sets) revealed spatiotemporal relationships that warrant further investigation. Notable relationships include 1) EEG beta and gamma frequencies were more strongly predicted by street features like service capacity (e.g. delivery levels) and speed limit, while EEG delta and theta frequencies were more strongly predicted by amenities like cultural institutions and trees; 2) pedestrians and cyclists were more impacted by street features during weekdays, and 3) a non-oppositional relationship between EEG beta/gamma and delta/theta frequencies.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availability
Available upon request. Raw data requires permission from participants who contributed data.
References
Amodio, D. M., & Frith, C. D. (2006). Meeting of minds: The medial frontal cortex and social cognition. Nature Reviews Neuroscience, 7(4), 268–277. https://doi.org/10.1038/nrn1884.
Anselin, L. (1995). Local indicators of spatial association—LISA. Geographical Analysis, 27(2), 93–115. https://doi.org/10.1111/j.1538-4632.1995.tb00338.x.
Anselin, L. (2013). Spatial econometrics: Methods and models. Vol. 4. Springer Science & Business Media. Heidelberg, Germany: Springer.
ArcGIS. (n.d.). How exploratory regression works. https://desktop.arcgis.com/en/arcmap/10.3/tools/spatial-statistics-toolbox/how-exploratory-regression-works.htm. Accessed Nov 2018.
Arribas, D., & Rey, S. (n.d.). Geographic Data Science with PySAL and the pydata stack. http://darribas.org/gds_scipy16/gds_scipy16.pdf. Accessed Feb 2018.
Arribas-Bel, D. (n.d.). Spatial Regression, https://darribas.org/gds_scipy16/ipynb_md/08_spatial_regression.html. Accessed Feb 2019.
Aspinall, P., Mavros, P., Coyne, R., & Roe, J. (2015). The urban brain: Analysing outdoor physical activity with mobile EEG. British Journal of Sports Medicine, 49(4), 272–276. https://doi.org/10.1136/bjsports-2012-091877.
Berger, R. D., Akselrod, S., Gordon, D., & Cohen, R. J. (1986). An efficient algorithm for spectral analysis of heart rate variability. IEEE Transactions on Biomedical Engineering, (9), 900–904. 10.1109/TBME.1986.325789
Bird, S., Klein, E., & Loper, E. (2009). Natural language processing with python: Analyzing text with the natural language toolkit. s.
Boeing, G. (2017). OSMnx: New methods for acquiring, constructing, analyzing, and visualizing complex street networks. Computers, Environment and Urban Systems, 65, 126–139. https://doi.org/10.1016/j.compenvurbsys.2017.05.004.
Bostrom, N. (2014). Superintelligence: Paths, dangers, strategies. Oxford University Press.
Brayne, C., Ince, P. G., Keage, H. A., Mckeith, I. G., Matthews, F. E., Polvikoski, T., & Sulkava, R. (2010). Education, the brain and dementia: Neuroprotection or compensation? Brain, 133(8), 2210–2216. https://doi.org/10.1093/brain/awq185.
Burton, E. J., Mitchell, L., & Stride, C. B. (2011). Good places for ageing in place: Development of objective built environment measures for investigating links with older people’s wellbeing. BMC Public Health, 11(1), 839.
Cerin, E., Nathan, A., Cauwenberg, J. V., Barnett, D. W., & Barnett, A. (2017). The neighbourhood physical environment and active travel in older adults: A systematic review and meta- analysis. International Journal of Behavioral Nutrition and Physical Activity, 14(1), 1–23.
Dahlberg, B. (2018). Cornell food researcher's downfall raises larger questions for science. NPR. http://www.npr.org/sections/thesalt/2018/09/26/651849441/cornell-food-researchers-downfall-raises-larger-questions-for-science. Accessed Feb 2019.
Ducao, A. (2019). Neurophysiological experience of cyclists in kuala lumpur and nairobi. International Journal of Traffic Safety Innovation: Vision Zero Cities, https://www.transalt.org/sites/default/files/2019-1/VZC_Journal_2019_Full_Update-Small.pdf. Accessed Feb 2020.
Ducao A., Koen, I., & Guo, Z. (2018). Multimer: validating multimodal, cognitive data in the city: Towards a model of how the urban environment influences streetscape users. In Proceedings of the Workshop on Modeling Cognitive Processes from Multimodal Data (MCPMD ‘18). Association for Computing Machinery, New York, NY, USA, Article 11, 1–8. https://doi.org/10.1145/3279810.3279853
Ducao A., Koen, I., van Bergen, T., Berry, Y., Sheu, S., & Mitchell, T. (2020). Multimer: User experience exercises from a bio-spatial study in the urban context. Proceedings of the Distributed, Ambient and Pervasive Interactions 8th International Conference, DAPI 2020, Held as Part of the 22st HCI International Conference, HCII 2020. (forthcoming).
EU GDPR portal. (n.d.). https://eugdpr.org. Accessed Oct 2019.
Ferguson, C. J. (2016). An effect size primer: A guide for clinicians and researchers. In A. E. Kazdin (Ed.), Methodological issues and strategies in clinical research (pp. 301–310). Professional Psychology: Research and Practice, American Psychological Association. https://doi.org/10.1037/14805-020
Fortune, S. (1992). Voronoi diagrams and delaunay triangulations. In Computing in euclidean geometry, 193–233. https://doi.org/10.1142/9789812831699_0007
GeoDa, an introduction to Spatial Data Analysis. (n.d.). https://geodacenter.github.io. Accessed Feb 2019.
Head, M. L., Holman, L., Lanfear, R., Kahn, A. T., & Jennions, M. D. (2015). The extent and consequences of p-hacking in science. PLoS Biology, 13(3). https://doi.org/10.1371/journal.pbio.1002106.
Hutto, C. J. (n.d). Valence aware dictionary and sentiment reasoner, https://github.com/cjhutto/vaderSentiment. Accessed Nov 2018.
Jones, F. M. (2012). Accumulating neighborhood stress exposure: Effects on hypertension, obesity, and depression. UCLA. ProQuest ID: Jones_ucla_0031D_10293. Merritt ID: ark:/13030/m57p90cs. Retrieved from https://escholarship.org/uc/item/2n9500gf. Accessed Jan 2020.
Jones, M., & Pebley, A. R. (2014). Redefining neighborhoods using common destinations: Social characteristics of activity spaces and home census tracts compared. Demography, 51(3), 727–752. https://doi.org/10.1007/s13524-014-0283-z.
Karandinou, A., & Turner, L. (2017). Architecture and neuroscience; what can the EEG recording of brain activity reveal about a walk through everyday spaces? International Journal of Parallel, Emergent and Distributed Systems, 32(sup1), S54–S65. https://doi.org/10.1080/17445760.2017.1390089.
Kurzweil, R. (2012). How to create a mind: The secret of human thought revealed. New York: Viking Penguin.
Leys, C., Ley, C., Klein, O., Bernard, P., & Licata, L. (2013). Detecting outliers: Do not use standard deviation around the mean, use absolute deviation around the median. Journal of Experimental Social Psychology, 49(4), 764–766. https://doi.org/10.1016/j.jesp.2013.03.013.
Luu, P., Flaisch, T., & Tucker, D. M.. (2000). Medial frontal cortex in action monitoring. Journal of Neuroscience, 464–469. https://doi.org/10.1523/JNEUROSCI.20-01-00464.2000
McCarthy, Minsky, Rochester, & Shannon, (1955). A proposal for the dartmouth summer research project on artificial intelligence. AI Magazine Volume 27 Number 4 (2006) (© AAAI). 31 August 1955. https://doi.org/10.1609/aimag.v27i4.1904.
Mouratidis, K. (2018). Built environment and social well-being: How does urban form affect social life and personal relationships? Cities, 74, 7–20. https://doi.org/10.1016/j.cities.2017.10.020.
Muggeo, V. M. R. (2003). Estimating regression models with unknown break-points. Statistics in Medicine, 22.19, 3055–3071. https://doi.org/10.1002/sim.1545.
Musikanski, L., Rakova, B., Bradbury, J., Phillips, R., & Manson, M. (2020). Artificial Intelligence and community well-being: A proposal for an emerging area of research. International Journal of Community Well-Being, 1–17. https://doi.org/10.1007/s42413-019-00054-6.
National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, Department of Health, Education and Welfare. (1978). The Belmont Report. Washington, DC: United States Government Printing Office.
NeuroSky: Mindset Communications Protocol. (2015). http://developer.neurosky.com/docs/lib/exe/fetch.php?media=mindset_communications_protocol.pdf. Accessed Feb 2019.
NYC Open Data. (n.d.). https://opendata.cityofnewyork.us. Accessed Feb 2019.
OpenStreetMap contributors. (2015). Planet dump [Data file from 1 May 2018]. Retrieved from https://planet.openstreetmap.org.
Ord, J. K., & Getis, A. (1995). Local spatial autocorrelation statistics: distributional issues and an application. Geographical Analysis, 27(4), 286–306. https://doi.org/10.1111/j.1538-4632.1995.tb00912.x.
Palva, S., & Palva, J. M. (2007). New vistas for α-frequency band oscillations. Trends in Neurosciences, 30(4), 150–158. https://doi.org/10.1016/j.tins.2007.02.001.
Python object serialization. (n.d.). https://docs.python.org/3/library/pickle.html. Accessed Feb 2019.
Renalds, A., Smith, T. H., & Hale, P. J. (2010). A systematic review of built environment and health. Family & Community Health, 33(1), 68–78. https://doi.org/10.1097/fch.0b013e3181c4e2e5.
Rey, S. J., & Anselin, L. (2010). PySAL: A python library of spatial analytical methods. In Handbook of applied spatial analysis (pp. 175–193). Heidelberg: Springer.
Russell, S. J., & Norvig, P. (2016). Artificial intelligence: A modern approach. London: Pearson.
Sarmiento, O. L., Schmid, T. L., Parra, D. C., Díaz-Del-Castillo, A., Gómez, L. F., Pratt, M., et al. (2010). Quality of life, physical activity, and built environment characteristics among Colombian adults. Journal of Physical Activity and Health, 7(S2). https://doi.org/10.1123/jpah.7.s2.s181.
Tatum, W. O. (2014). Ellen R. Grass lecture: extraordinary EEG. The Neurodiagnostic Journal, 54(1), 3–21.
Tilley, S., Neale, C., Patuano, A., & Cinderby, S. (2017). Older people’s experiences of mobility and mood in an urban environment: A mixed methods approach using electroencephalography (EEG) and interviews. International Journal of Environmental Research and Public Health, 14(2), 151. https://doi.org/10.3390/ijerph14020151.
Turing, A. M. (1950). I-computing machinery and intelligence. Mind, LIX(236), 433–460. https://doi.org/10.1093/mind/LIX.236.433.
UN SDG. United Nations: Sustainable development goals. (n.d.). https://unstats.un.org/sdgs/indicators/indicators-list. Accessed Dec 2019.
Funding
National Science Foundation, Award #1721679.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of Interest/Competing Interests
Matter Ventures and NYU Leslie E-Lab contributed meeting space, office space, and office supplies to this study. A preliminary phase of this study, conducted in 2016, was funded in part by SOSV and BMW.
Research Involving Human Participants and/or Animals
This study involved human participants. Its protocol was approved by the Biomedical Research Alliance of New York (BRANY), Protocol “NYC-003,” BRANY file number 17–08–184-455.
Informed Consent
All participants in this study signed informed consent forms, which were approved by the Biomedical Research Alliance of New York (BRANY), Protocol “NYC-003,” BRANY file number 17–08–184-455.
Code Availability
Available upon request, pending permission from code authors.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
ESM 1
(DOCX 20 kb)
Rights and permissions
About this article
Cite this article
Ducao, A., Koen, I., Guo, Z. et al. Multimer: Modeling Neurophysiological Experience in Public Urban Space. Int. Journal of Com. WB 3, 465–490 (2020). https://doi.org/10.1007/s42413-020-00082-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s42413-020-00082-7