Skip to main content
SpringerLink
Log in

Sensate Media — Multimodal Electronic Skins as Dense Sensor Networks

  • Published:
BT Technology Journal

Abstract

In this paper, we introduce the concept of building electronic sensate skins as extremely dense, multimodal, systolic sensor networks. In this fashion, the copious signals produced by the skin's receptors are reduced by the network itself, and only high-level features are routed out peer-to-peer, avoiding complex wiring requirements while promising to enable scalability across large areas. Our architectures and algorithms have been inspired by biological skin, where signals from receptors are enhanced or suppressed by processing in the receptor cells and nervous system before arriving at the brain. We illustrate this concept with work in progress on two test beds, the Pushpin Computer, an easily configurable, planar array of over 100 nodes set up as 'smart wallpaper', and the Tribble, a sphere coated with over 500 diverse tactile and noncontact sensor channels processed in 32 interconnected sensor 'cells'. We also introduce the Z-Tiles, a collaborative project that has resulted in a pressure-imaging floor realised through a collection of pixilated floor tiles configured as a mesh sensor network. We conclude with a technology roadmap for scaling to higher densities.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Johansson R S and Vallbo A B: 'Tactile sensory coding in the glabrous skin of the human hand', Trends in NeuroSciences, 6, No 1, pp 27—32 (1983).

    Google Scholar 

  2. Aizenberg J, Tkachenko A, Weiner S, Addadi L and Hendler G: 'Calcitic microlenses as part of the photoreceptor system in brittlestars', Nature, 412, pp 819—822, (2001).

    Google Scholar 

  3. Bastian J: 'Electrosensory Organisms', Physics Today, 47, No 2, pp 30—37 (February 1994).

    Google Scholar 

  4. Lumelsky V J, Shur M S and Wagner S: 'Sensitive Skin', IEEE Sensors J, 1, No 1, pp 41—51 (June 2001).

    Google Scholar 

  5. Lumelsky V, Shur M S and Wagner S (Eds): 'Sensitive skin' Singapore: World Scientific (2000).

  6. Lee M H and Nicholls H R: 'Tactile sensing for mechatronics — a state of the art survey', Mechatronics, 9, No 1, pp 1—31 (February 1999).

    Google Scholar 

  7. Papakostas T, Lima J and Lowe M: 'Large area force sensor for smart skin applications', in Proceedings of the IEEE Sensors 2002 Conference, 2, pp 1614—1619 (2002).

    Google Scholar 

  8. Someya T and Sakurai T: 'Integration of organic field-effect transistors and rubbery pressure sensors for artificial skin applications', in the Proc of the 2003 IEEE International Electron Devices Meeting (IEDM), Washington DC, pp 8.4.1—8.4.4 (December 2003).

  9. Xu Y, Tai Y, Huang A and Ho C: 'IC-integrated flexible shear-stress sensor skin',Technical Digest, Solid State Sensor and Actuator Workshop (SSAW), pp 247—250 (2002).

  10. Richardson D: 'Smart skins and structures', Armada International, 22, No 2, pp 50—56 (April/May 1998).

    Google Scholar 

  11. Dietz P and Leigh D: 'DiamondTouch: a multi-user touch technology', in Proc of ACM UIST 2001, Orlando, Florida, USA, pp 219—226 (November 2001).

  12. Rekimoto J: 'Smartskin: An infrastructure for freehand manipulation on interactive surfaces', in Proceedings of CHI 2002 Conference on Human Factors in Computing Systems, ACM, Minneapolis, pp 113—120 (2002).

  13. Lee S K, Buxton W and Smith K C: 'A multi-touch three dimensional touch-sensitive tablet', Proceedings of the SIGCHI conference on Human Factors in Computing Systems (CHI 85), San Francisco, CA, ACM Press, pp 21—25 (April 1985).

    Google Scholar 

  14. Westerman W and Elias J G: 'Method and apparatus for integrating manual input', US Patent No 6,323,846 (November 2001).

  15. Omojola O, Post E R, Hancher M D, Maguire Y, Pappu R, Schoner B, Russo P R, Fletcher R and Gershenfeld N: 'An installation of interactive furniture', IBM Systems J, 39, No 3 and 4, pp 861—879 (2000).

    Google Scholar 

  16. De Rossi D, Carpi F, Lorussi F, Mazzoldi A, Scilingo E P and Tognetti A: 'Electroactive fabrics for distributed, conformable and interactive systems', in Proc of the 2002 IEEE Sensors Conference, IEEE Press, 2, pp 1608—1613, (2002).

    Google Scholar 

  17. Çelik-Butler Z, Butler D P, Yardanakul A and Yildiz A: 'Infrared sensors on flexible substrates for smart skin', SPIE Annual International Symposium on Aerospace/Defense Sensing, Simulation, and Controls (AEROSENSE): Infrared Detectors and Focal Plane Arrays VII, Dereniak E L and Sampson R E (Eds),Orlando, FL, SPIE 4721, pp 260—268 (April 2002).

    Google Scholar 

  18. Engel J, Chen J and Liu C: 'Development of a multi-modal, flexible, tactile sensing skin using polymer micromachining', in the Proceedings of Transducers' 03, 2, Boston MA, IEEE Press, pp 1027—1030 (June 2003).

    Google Scholar 

  19. NASA Robonaut project — http://robonaut.jsc.nasa.gov/

  20. Stiehl W D: 'Tactile perception in robots: from the somatic alphabet to the realisation of a fully sensitive skin', BS Thesis, MIT Mechanical Engineering Dept (June 2003).

  21. Hristu D, Ferrier N and Brockett R: 'The performance of a deformable membrane tactile sensor: basic results on geometrically defined tasks', IEEE Conference on Robotics and Automation, San Francisco, California, pp 508—513 (April 2000).

  22. Son J S, Monteverde E A and Howe R D: 'A tactile sensor for localising transient events in manipulation', in Proceedings of the IEEE International Conference on Robotics and Automation, pp 471—476 (May 1994).

  23. Yamada Y, Maeno T, Fujimoto I, Morizono T and Umetani Y: 'Identification of incipient slip phenomena based on the circuit output signals of PVDF film strips embedded in artificial finger ridges', in Proceedings of the SICE Annual Conference, pp 3272—3277 (2002).

  24. Kamiyama K, Vlack K, Mizota T, Kajimoto H, Kawakami N and Tachi S: 'GelForce: a vision-based force sensor for real-time measurement of force vector distributions', to appear in IEEE Computer Graphics and Applications (January 2005).

  25. Hakozaki M, Hatori A and Shinoda H A: 'Sensitive skin using wireless tactile sensing elements', in Technical Digest of the 18th Sensor Symposium, Kawasaki, Japan, pp 147—150 (May 2001).

  26. Yamada K, Goto K, Nakajima Y, Koshida N and Shinoda H: 'A sensor skin using wire-free tactile sensing elements based on optical connection', in Proceedings of the SICE Annual Conference, Osaka, Japan, pp 319—322 (August 2002).

  27. Um D, Stankovic B, Giles K, Hammond T and Lumelsky V: 'A modularised sensitive skin for motion planning in uncertain environments', Proc of the IEEE International Conference on Robotics and Automation, 1, pp 7—12 (May 1998).

    Google Scholar 

  28. Wang P, Valencia P, Prokopenko M, Price D and Poulton G: 'Self-reconfigurable sensor networks in ageless aerospace vehicles', in Proceedings of the 11th International Conference on Advanced Robotics (ICAR-03), Portugal, Paper P100 (June—July 2003).

  29. Cholewiak R W and Collins A: 'Sensory and physiological bases of touch', in Heller M A and Schiff W (Eds): 'The Psychology of Touch', Lawrence Erlbaum (Hillsdale, NJ), pp 23—60 (1991).

    Google Scholar 

  30. Pohja S: 'Survey of studies on tactile senses', Report R96:02, RWCP Neuro SICS Laboratory, Kista, Sweden (March 1996).

    Google Scholar 

  31. Howe R D: 'Tactile sensing and control of robotic manipulation', Advanced Robotics, 8, No 3, pp 245—261 (1994).

    Google Scholar 

  32. Brooks M: 'Highly redundant sensing in robotics — analogies from biology: distributed sensing and learning', in Tou J T and Balchen J G (Eds): 'Highly Redundant Sensing in Robotic Systems', NATO ASI Series, F58, Springer, Berlin and Heidelberg, pp 35—42 (1990).

    Google Scholar 

  33. Deutsch S and Deutsch A: 'Understanding the nervous system: an engineering perspective', IEEE Press, Piscataway, New Jersey (1993).

    Google Scholar 

  34. Gupta P and Kumar P R: 'The capacity of wireless networks', IEEE Transactions on Information Theory, IT-46, No 2, pp 388—404 (March 2000).

    Google Scholar 

  35. Scaglione A and Sergio S S D: 'On the interdependence of routing and data compression in multi-hop sensor networks', in the Proceedings of the Eighth Annual International Conference on Mobile Computing and Networks (MobiCOM 2002), Atlanta, GA, ACM Press, New York, pp 140—147 (September 2002).

    Google Scholar 

  36. Pradhan S S, Kusuma J and Ramchandran K: 'Distributed compression in a dense microsensor network', IEEE Signal Processing Magazine, 19, No 2, pp 51—60 (March 2002).

    Google Scholar 

  37. Gharavi H and Kumar S P (Eds): 'Special issue on sensor networks and applications', IEEE Proceedings, 91, No 8 (August 2003).

  38. Hogg T and Huberman B: 'Controlling smart matter', Smart Materials and Structures, 7, pp R1—R14 (1998).

    Google Scholar 

  39. Abelson H, Knight T and Sussman J: 'Amorphous computing', MIT AI Laboratory Internal Report (1996) — http://www-swiss.ai.mit.edu/projects/amorphous/white-paper/amorph-new/amorph-new.htm

  40. Butera W: 'Programming a paintable computer', MIT PhD Thesis, Media Laboratory, Massachusetts Institute of Technology, Cambridge MA (February 2002).

    Google Scholar 

  41. Sim H L: 'Scheme-based High-Level Simulator for Gunk (on the Wall)', — http://www.swiss.ai.mit.edu/projects/amorphous/hlsim/

  42. NS2: 'C++ based discrete time network event simulator with TCP support', — http://www.isi.edu/nsnam/ns/

  43. Swarm: 'Objective C based agent-based modeling toolkit', — http://wiki.swarm.org

  44. Fang Q, Zhao F and Guibas L: 'Lightweight sensing and communication protocols for target enumeration and aggregation', in Proc of ACM Symp on Mobile Ad Hoc Networking and Computing (MobiHoc), Annapolis, MD, pp 165—176 (June 2003).

  45. Lifton J, Seetharam D, Broxton M and Paradiso J: 'Pushpin computing system overview: a platform for distributed, embedded, ubiquitous sensor networks', in Mattern F and Naghshineh M (Eds): 'Pervasive 2002', Proceedings of the First Pervasive Computing Conference, Zurich Switzerland, Springer Verlag, Berlin Heidelberg, pp 139—151 (August 2002).

    Google Scholar 

  46. Lifton J: 'Pushpin computing: a platform for distributed sensor networks', MIT MSc Thesis, Media Laboratory, Massachusetts Institute of Technology, Cambridge MA (September 2002).

    Google Scholar 

  47. Bletsas A and Lippman A: 'Natural spontaneous order in wireless sensor networks: time synchronisation based on entrainment', Technical Report, MIT Media Lab (Viral Networking Group) (December 2003) — http://web.media.mit.edu/~aggelos/papers/spontime.pdf

  48. Dipline — http://www.saitec.fr/english/cadre.htm

  49. Silicon Laboratories — http://www.silabs.com

  50. Heaton K B, Poor R D and Wheeler A J: 'Nami', in SIGGRAPH 99 Conference Abstracts and Applications, ACM Press, New York, NY pp 215 (1999).

    Google Scholar 

  51. Intanagonwiwat C, Govindan R and Estrin D: 'Directed diffusion: a scalable and robust communication paradigm for sensor networks', in Proceedings of the Sixth Annual International Conference on Mobile Computing and Networks (MobiCOM 2000), Boston, Massachusetts, pp 56—67 (August 2002).

  52. Bachrach J: 'Position determination in sensor networks', to appear in Stojmenovic I (Ed), 'Mobile ad hoc networking', John Wiley & Sons (2004).

  53. Nagpal R, Shrobe H and Bachrach J: 'Organising a global coordinate system from local information on an ad hoc sensor network', in the 2nd International Workshop on Information Processing in Sensor Networks (IPSN' 03), Palo Alto, CA, Springer-Verlag, Berlin, LNCS 2634, pp 333—348 (April 2003).

    Google Scholar 

  54. Broxton M, Lifton J and Paradiso J A: 'Localising a sensor network via collaborative processing of global stimuli', to be published (2004).

  55. Hu A and Servetto S D: 'Algorithmic aspects of the time synchronisation problem in large-scale sensor networks', to appear in the ACM Journal on Mobile Networks and Applications (2004).

  56. Fitzpatrick S and Meertens L: 'Experiments on dense graphs with a stochastic, peer-to-peer colourer', in Gomes C and Walsh T (Eds): 'Probabilistic approaches in search', Workshop at Eighteenth National Conf on Artificial Intelligence (AAAI 2002), pp 24—28, AAAI Press (2002).

  57. Allen D, Mitros P and Sussman G J: 'Acoustic aperture synthesis on an amorphous computer', MIT AI Laboratory Internal Report (Spring 2000).

  58. Chen J C, Yip L, Hanbiao J E, Maniezzo W, Hudson D A, Yao R E K and Estrin D: 'Coherent acoustic array processing and localisation on wireless sensor networks', Proceedings of the IEEE, 91, No 8, pp 1154—1162 (August 2003).

    Google Scholar 

  59. Scott J and Dragovic B: 'Audio location: accurate low-cost location sensing', to appear in the UbiComp 2004 Adjunct Proceedings (2004).

  60. Lifton J, Broxton M and Paradiso J A: 'Distributed sensor networks as sensate skin', in Proc of the 2003 IEEE International Conference on Sensors, Toronto, Ontario, pp 743—747 (October 2003).

  61. Bush K: 'A true sense of touch', Catnip, 12, No 8, pp 3—5 (2004).

    Google Scholar 

  62. Dehnhardt G et al: 'Hydrodynamic trail-following in harbor seals (Phoca vitulina)', Science, 293, pp 102—104 (2001).

    Google Scholar 

  63. Moore C I: 'Frequency-Dependent Processing in the Vibrissa Sensory System', Journal of Neurophysiology, 91, No 6 (June 2004).

  64. Roads C: 'The computer music tutorial', MIT Press, Boston, MA (1996).

    Google Scholar 

  65. Trueman D, Bahn C and Cook P: 'Alternative voices for electronic sound: spherical speakers and sensor-speaker arrays (SenSAs)', in Proc of the International Computer Music Conference, Berlin, San Francisco, International Computer Music Association, pp 248—251 (August 2000).

  66. Artbots: Robot Talent Show (2003) — http://www.artbots.org/

  67. McElligott L, Dillon M, Leydon K, Richardson B, Fernstrom M and Paradiso J A: 'ForSe FIElds — Force Sensors for Interactive Environments', in Borriello G and Holmquist L E (Eds): 'UbiComp 2002', LNCS 2498, Springer-Verlag Berlin Heidelberg, pp 168—175 (October 2002).

    Google Scholar 

  68. Richardson B, Leydon K, Fernstrom M and Paradiso J A: 'Z-Tiles: building blocks for modular, pressure-sensing floorspaces', in Proc of CHI 2004, Vienna Austria, ACM Press, New York, pp 1529—1532 (April 2004).

    Google Scholar 

  69. Paradiso J, Hsiao K, Strickon J, Lifton J and Adler A: 'Sensor systems for interactive surfaces', IBM Systems Journal, 39, Nos 3 and 4, pp 892—914 (October 2000).

    Google Scholar 

  70. Griffith N and Fernström M: 'LiteFoot: A floor space for recording dance and controlling media', in Proc of the 1998 Int Comp Music Conf (ICMC98), Ann Arbor, Michigan, International Computer Music Ass, San Francisco, CA, pp 475—481 (October 1998) — http://www.ul.ie/~pal/litefoot/index.htm

    Google Scholar 

  71. Pinkston R, Kerkhoff J and McQuilken M: 'A touch sensitive dance floor MIDI controller', Proceedings of the 1995 International Computer Music Conference (ICMC95), Banff, Canada, San Francisco, International Computer Music Association, pp 224—225 (September 1995).

  72. Suutala J, Pirttikangas S, Riekki J and Roning J: 'Reject-optional LVQ-based two-level classifier to improve reliability in footstep identification', in Fresch A and Mattern F (Eds): 'Pervasive computing', Second International Conference, Pervasive 2004, Linz/Vienna, Austria, Springer-Verlag, Berlin, pp 182—187 (2004).

    Google Scholar 

  73. Eng K: 'Designing neuromorphic interactive spaces', PhD thesis, Swiss Federal Institute of Technology, (ETH) Zurich, Switzerland, Diss ETH No 15451 (2004).

  74. Richardson B: 'Implementation of efficient data extraction from a scalable sensor system', MS Thesis, University of Limerick (August 2004).

  75. Huang Q, Qin M, Zhang Z, Zhou M, Gu L, Zhu H, Hu D, Hu Z, Xu G and Liu Z: 'Weather Station on a Chip', Proc of IEEE Sensors 2003,Toronto, Canada, pp 1106—1113 (October 2003).

  76. van Baar J J, Dijkstra M, Wiegerink R J, Lammerink T S J and Krijnen G M: 'Fabrication of arrays of artifical hairs for complex flow pattern recognition', Proc of IEEE Sensors 2003, Toronto, Canada, pp 332—336 (October 2003).

  77. Chen J, Engel J and Liu C: 'Development of Polymer-Based Artificial Haircell Using Surface Micromachining and 3D Assembly', in the Proceedings of Transducers' 03, 2, Boston MA, IEEE Press, pp 1035—1038 (June 2003).

    Google Scholar 

  78. Collins G P: 'Next stretch for plastic electronics',Scientific American, 291, No 2, pp 74—81 (August 2004).

    Google Scholar 

  79. Ridley B A, Nivi B, Jacobson J M: 'All-inorganic field effect transistors fabricated by printing', Science, 286 (5440), pp 746—749 (1999).

    Google Scholar 

  80. Mihm S: 'Print your next PC', Technology Review, 103, No 6, pp 66—70 (November—December 2000).

    Google Scholar 

  81. Butera W and Knaian A: 'V1.0 Dense Mixel Array', Personal communication (Summer 2004).

  82. Benbasat A Y, Paradiso J A: 'Design of real-time adaptive power optimal sensor systems', to appear in the Proc of IEEE Sensors 2004, Vienna, Austria (October 2004).

  83. Min R, Bhardwaj M, Cho S W, Ickes N, Shih E, Sinha A, Wang A and Chandrakasan A: 'Energy-centric enabling technologies for wireless sensor networks', IEEE Wireless Communications, 9, No 4, pp 28—39 (August 2002).

    Google Scholar 

  84. Sarpeshkar R: 'Analogue versus digital: extrapolating from electronics to neurobiology', Neural Computation, 10, No 7, pp 1601—38 (October 1998).

    Google Scholar 

  85. Kottapalli V A, Kiremidjian A S, Lynch J P, Carryer E, Kenny T W, Law K H and Lei Y: 'Two-tiered wireless sensor network architecture for structural health monitoring', in the SPIE 10th International Symposium on Smart Structures and Materials, San Diego, CA, Proc of the SPIE, 5057, pp 8—19 (March 2003).

    Google Scholar 

Download references

Authors
  1. J A Paradiso
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. J Lifton
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M Broxton
    View author publications

    You can also search for this author in PubMed Google Scholar

About this article

Cite this article

Paradiso, J.A., Lifton, J. & Broxton, M. Sensate Media — Multimodal Electronic Skins as Dense Sensor Networks. BT Technology Journal 22, 32–44 (2004). https://doi.org/10.1023/B:BTTJ.0000047581.37994.c2

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/B:BTTJ.0000047581.37994.c2

Keywords

Search

Navigation