E-Textile Couch: Towards Smart Garments Integrated Furniture

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10217)

Abstract

Application areas like health-care and smart environments have greatly benefited from embedding sensors into every-day-objects, enabling for example sleep apnea detection. We propose to further integrate parts of sensors into the very own materials of the objects. Thus, in this work we explore integrating smart garments into furniture using a couch as our use-case. Equipped with textile capacitive sensing electrodes, we show that our prototype outperforms existing systems achieving an F-measure of 94.1%. Furthermore, we discuss implications and limitation of the integration process.

Keywords

Capacitive sensing Conductive materials E-textiles Posture detection 

1 Introduction

The usage of smart textiles has expanded from an initial state of single prototypes [1] and fashionable technology [2] into a great number of applications, especially in the area of wearables. On-body controls [3, 4], easy prototyping of user interfaces for disabled people [5] as well as posture and motion recognition of body parts [6, 7] are few of these examples.

The concept of self-aware materials [8] and producing digital textiles at scale [9] enables to view our surrounding materials and surfaces differently, leveraging unexpected invisible ubiquitous interactivity [10, 11]. To address these advances, we investigate how smart textiles can be seamlessly integrated within furniture and demonstrate this on the couch as our use-case. The interactions between human and couch, create implications about the surrounding context, creating a self-aware object of every-day use. Measurements conducted by Rus et al. have confirmed the suitability of conductive textile as capacitive sensing electrode [12].

Our contribution with this work, is to extend the usage of smart textiles from the on-body wearables to the seamlessly integrated ambient objects, like furniture. In this paper we cover an ordinary living room couch as our use-case. We extend state of the art by analyzing a set of several fine-grained postures which will contribute to adjusting the environment to the users needs.

2 Related Work

Posture recognition has been subject of many works [13, 14]. Especially in the area of Ambient Intelligence it is of utmost interest to know as much as possible about the human as interacting counterpart in the surrounding intelligent landscape, to which knowing the posture is an important contribution.

2.1 Posture Recognition Using Smart Textiles

In many works posture recognition has been attempted using different variations of smart textiles [6, 7]. Zhou et al. [7] have built a sensing band which monitors gym exercises. They use textile pressure sensors in order to track leg activity during exercising. Focusing on posture monitoring Wang has interconnected smart garments with wearable electronics on a vest for rehabilitation purposes [6]. Few works have already partly integrated smart textiles into furniture. Braun et al. have created a chair to recognize poses and activities creating awareness of correct posture [15]. The chair is endowed with capacitive sensors where one electrode integrated in the backrest woven through the mesh of the chair using conductive thread.

2.2 Posture Recognizing Furniture

Examples of furniture able to recognize the posture of the occupying human are bed, chair and couch. In the bed the sleeping posture is investigated by several works, where different types of unobtrusively placed sensors are used. For example Chang et al., Braun et al. and Rus et al. use capacitive sensors placed underneath the mattress, attached to the frame of the bed, respectively underneath the bed-sheet in order to detect sleeping postures, lying postures and prevent decubitus ulcers as a consequence [16, 17, 18]. Liu et al. use capacitive pressure sensors in a high density sensor bed sheet for monitoring the patients rehabilitation exercises [19].

First approaches of detecting seating postures have been made by Tan et al. using a pressure sensor mat [20]. They classify 14 postures achieving more than 90% accuracy per posture. Eight postures were identified by using pressure sensors endowed in an office chair created by Nazari et al. [21]. Braun et al. have created several prototypes of sensing chairs by using capacitive sensing [22, 23]. One prototype is meant to support training micro-breaks in the office while another is a sensing system for car seats. The second one is based on 16 electrodes connected to capacitive sensors with the goal of identifying different properties of the driver, like e.g. drivers head posture.

Couches have been endowed with sensors several times, mostly using capacitive sensors. Kivikunnas et al. present a sofa equipped with six metal foil capacitive sensors analyzing basic sensor data [24]. Grosse-Puppendahl et al. evaluate nine different postures with a couch equipped with 8 capacitive proximity sensors achieving 97% precision and recall [25]. By creating a network of furniture composed of bed, couch and chair Heikkil et al. envisage posture and activity tracking throughout the day [26]. Even though, the couch has been also equipped with sensors only long time evaluations with chair and bed have been reported. More recently, the couch has been used as a sensing device by Pohl et al. for context sensing in a livingroom, controlling ambient lightning, music and tv [27]. The couch is equipped with six capacitive proximity sensors, evaluating eight postures with an achieved accuracy of 92.9%.

3 Prototype

The production of conductive textiles at large scale envisages that sensing electrodes will one day be fully integrated into the covering materials and thus into the production process of furniture. Following this chain of thought our interactive couch prototype is enhanced by placing eight textile electrodes on the surface of the couch, see Fig. 1. We created the electrodes by using pieces of 15 x 16 cm\(^{2}\) conductive fabric, sewing a loop of wire to the fabric using conductive thread and gluing it to pieces of ordinary couch cover, as shown in Fig. 2. This process ensures that the electrodes are isolated and utilize materials used for the production of an actual couch.

The electrodes are connected to sensors which are connected to a capacitive sensing prototyping board, the OpenCapSense board [28]. The raw sensor data is collected and processed in the following steps in order to extract the posture of a person on the couch.
Fig. 1.

Couch endowed with eight textile electrodes.

Fig. 2.

a) Sensor and connected electrode made of conductive textile taped to regular couch cover sample. b) Sewn connection with conductive thread between textile and wire.

4 Evaluation Setup

We evaluated the couch by asking 15 test persons (2 female) to execute 14 different postures: 12 sitting poses, of which 3 using the armrest of the couch (see Fig. 3), and 2 lying postures. At all times there was only one person on the couch. Including the empty couch we have evaluated 15 distinguished classes:
  • Class 1 Empty couch

  • Class 2 Sitting upright, on right side

  • Class 3 Sitting on edge, on right side

  • Class 4 Sitting leaned back, on right side

  • Class 5 Sitting upright, on right side, using armrest in front

  • Class 6 Sitting leaned back, on right side, using armrest in front

  • Class 7 Sitting leaned back, on right side, using armrest at back

  • Class 8 Sitting upright, in the middle

  • Class 9 Sitting on edge, in the middle

  • Class 10 Sitting leaned back, in the middle

  • Class 11 Sitting upright, on left side

  • Class 12 Sitting on edge, on left side

  • Class 13 Sitting leaned back, on left side

  • Class 14 Lying down, head on right side

  • Class 15 Lying down, head on left side

Fig. 3.

a) Sitting upright; b) Sitting upright using armrest in front; c) Sitting leaned back using armrest in front; d) Sitting leaned back using armrest at back;

For each class we collected 30 data samples per sensor, which correspond to spending about 10 s in a given posture. The test persons were verbally instructed on how the posture should be executed. Only the desired position of the arm using the armrest has been marked at the position in front and back due to the more specific and smaller change in posture, harder to convey verbally.

We evaluated the data with leave-one-subject-out cross-validation using four different classifiers form the WEKA [29] framework. All classifiers were applied with their standard settings. The four classifiers are k nearest neighbors (kNN), naive Bayes, C4.5 decision tree (Weka J.48) and Support Vector Machine (SVM). At first we applied them on the raw sensor data and subsequently on the normalized sensor data. In order to be able to compare the performance of conductive fabric electrodes with the performance of proximity capacitance measurements we selected the classes equivalent to the ones which were evaluated within the work of Pohl et al. [27]. These correspond to our classes 1–4 and 11–15.

5 Evaluation Results

As described in Sect. 4 we have collected the raw data of 15 subjects and evaluated it with different classifiers. As input we used the raw data, the per sensor normalized data and a subset of classes of the raw respectively the normalized data. The subset was chosen in order to compare the results of the fabric electrodes to the proximity sensing electrodes. The detailed results of the leave-one-subject-out cross-validation F-measure are shown in Fig. 5. For each classifier we have calculated the overall accuracy and F-measure by compiling the mean of all leave-one-subject-out cross-validation results for the particular classifier. Table 1 shows an overview of the results.
Table 1.

Overview of classification results for C4.5, kNN, naive Bayes and SVM on different data sets.

C4.5 decision tree

kNN

Naive Bayes

SVM

Data

Acc.[%]

F-m.[%]

Acc.[%]

F-m.[%]

Acc.[%]

F-m.[%]

Acc.[%]

F-m.[%]

Raw

82.3

77.0

80.7

75.6

84.0

79.8

85.7

81.9

Normalized

89.1

86.0

88.9

86.2

87.2

84.1

91.2

88.8

Subset raw

83.3

78.9

87.5

83.9

89.7

87.0

90.45

88.1

Subset normalized

89.9

87.2

91.6

88.9

95.3

94.1

95.5

94.1

Comparing the overall results of the different classifiers, SVM produces the highest accuracy and F-measure. SVM performs on the normalized data an accuracy of 91.3% and an F-measure of 88.8%. On the subset of classes 1–4 and 11–15 SVM reaches even higher values of 95.5% accuracy and 94.1% F-measure.

These results outperform the results achieved by Pohl et al. [27]. Table 2 compares the accuracy achieved with the two classifiers kNN and naive Bayes which we used in common. For kNN our results were significantly better 91.6% compared to 79.4%. Pohl et al. achieved their best results with the naive Bayes classifier, reaching 92.9% accuracy, whereas our prototype has achieved slightly more 95.3% accuracy, only 0.2% less than our overall best result of 95.5% accuracy using SVM.

Grosse-Puppendahl et al. [25] have evaluated their prototype with a total of 9 classes. Six of these classes correspond to the classes evaluated using the current prototype. These classes are sitting upright on left, middle and right side and lying down with the head on the right and the left side which correspond to classes 1, 2, 8, 11, 14, 15. The F-measure calculated from their precision and recall values of the individual classes is 97.5% achieved using the RBF network. Selecting the same classes, using the current prototype, we achieve an F-measure of 99.8%.

These results indicate, that using conductive textile electrodes reaches equally good results, slightly outperforming a system with electrodes placed under the couch cushions.
Table 2.

Performance comparison to related work.

kNN

Naive Bayes

SVM

Pohl et al. [27]

79.4 %

92.9 %

-

Our work

91.6 %

95.3 %

95.5 %

The difference between the results of SVM on the normalized data and on the subset of normalized data is of 6%. In order to find out, which of the classes cause the miss-classification, we inspected the confusion matrices of particular subjects. We chose to look at the subject with the lowest success rate, subject 4 (see Fig. 4), and a middle success rate, subject 3. The confusion matrices indicate that classes sitting on the right, upright and on the edge were not differentiated at all for both test persons. Looking at the performance over all classes, in the case of subject 4 sitting upright and on edge were correctly identified, however differentiating between leaned back with arm in front and arm at the back were miss-classified as can be observed in Fig. 4.
Fig. 4.

Confusion matrices of subject 4 for the subset and for all classes.

Fig. 5.

F-measure of leave one subject out cross-valuation using different classifiers and on different data sets.

6 Discussion

Taking only the miss-classification of sitting upright and on the edge on the right side, we could consider improving this by placing two electrodes on the sitting area, as has been done by Pohl et al. However, the fact that the two classes were correctly identified in the case of sitting on the left side and in the middle shows that it is possible to differentiate these poses in most of the cases. This means that one needs to consider a trade-off between the cost of using one ore more additional sensors and accuracy.

Regarding the placement of the arm, one single electrode does not seem to be enough to detect the position of the arm in a robust way. We are considering to improve recognition rates with placing two electrodes on the armrest, one towards the front and one towards the back.

The evaluation results show, that using conductive textile electrodes is equally suitable in order to detect postures. However, while attaching the textile electrodes to the couch cover, it became clear that integrating the electrodes with the couch cover has to be done by taking the design and shape of the couch into account. On a couch where three persons can sit down, but the sitting surface is made up of only two couch cushions one needs to consider the placement of the electrodes. Placing the electrodes underneath the couch cushion needs only one electrode to sense the user. Integrating the electrode into the cushion cover material would mean in the case of our couch creating two different electrodes, which could be connected to two sensors or connecting the two electrodes to one single sensor. Connecting multiple electrodes to one sensor could be used in order to increase the sensing area, and would still send their signal to one single sensing unit. This approach could reduce conductive fabric material costs.

Combining proximity sensing electrodes and multiple smaller cover electrodes all connected to one sensing unit could be used to create sensing electrode pairs which act as stand-alone sensor and are connected to a single sensing unit. This approach has been recently proposed by Tsuruta et al. [30].

7 Conclusion and Outlook

In this paper we have contributed to extending the usage of smart textiles from the on-body wearables to seamless integration within ambient objects, like a couch. We have shown that conductive textile used as capacitive electrode yields as good results as capacitive proximity electrodes, slightly outperforming previous works. Next steps to be considered are: exploring conductive thread as electrodes, refining the placement of the electrodes and the number of electrodes needed. These efforts will result in being able to track a human skeleton motion model, measuring fine-grained postures.

We envision sensing couches with higher sensing resolution, in order to detect fine-grained postures. Combined with other physiological signals, like breathing detection, furniture would extend the possibilities of implicitly adjusting the ambient surrounding to the users needs.

References

  1. 1.
    Holleis, P., Schmidt, A., Paasovaara, S., Puikkonen, A., Häkkilä, J.: Evaluating capacitive touch input on clothes. In: Proceedings of the 10th International Conference on Human Computer Interaction with Mobile Devices and Services, MobileHCI 2008, pp. 81–90. ACM, New York (2008)Google Scholar
  2. 2.
    Seymour, S.: Fashionable Technology: The Intersection of Design, Fashion, Science, and Technology, 1st edn. Springer Publishing Company, Heidelberg (2008)CrossRefGoogle Scholar
  3. 3.
    Schneegass, S., Voit, A,: GestureSleeve: using touch sensitive fabrics for gestural input on the forearm for controlling smartwatches. In: Proceedings of the 2016 ACM International Symposium on Wearable Computers, ISWC 2016, pp. 108–115. ACM, New York (2016)Google Scholar
  4. 4.
    Hamdan, N.A.-H., Blum, J.R., Heller, F., Kosuru, R.K., Borchers, J.: Grabbing at an angle: menu selection for fabric interfaces. In: Proceedings of the 2016 ACM International Symposium on Wearable Computers, ISWC 2016, pp. 1–7. ACM, New York (2016)Google Scholar
  5. 5.
    Singh, G., Nelson, A., Robucci, R., Patel, C., Banerjee, N.: Inviz: low-power personalized gesture recognition using wearable textile capacitive sensor arrays. In: 2015 IEEE International Conference on Pervasive Computing and Communications (PerCom), pp. 198–206, March 2015Google Scholar
  6. 6.
    Wang, Q., Toeters, M., Chen, W., Timmermans, A., Markopoulos, P.: Zishi: A smart garment for posture monitoring. In: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, CHI EA 2016, pp. 3792–3795. ACM, New York (2016)Google Scholar
  7. 7.
    Zhou, B., Sundholm, M., Cheng, J., Cruz, H., Lukowicz, P.: Never skip leg day: A novel wearable approach to monitoring gym leg exercises. In: 2016 IEEE International Conference on Pervasive Computing and Communications (PerCom), pp. 1–9, March 2016Google Scholar
  8. 8.
    Dementyev, A.: Towards self-aware materials. In: Proceedings of the TEI 2016: Tenth International Conference on Tangible, Embedded, and Embodied Interaction, TEI 2016, pp. 685–688. ACM, New York (2016)Google Scholar
  9. 9.
    Poupyrev, I., Gong, N.-W., Fukuhara, S., Emre Karagozler, M., Schwesig, C., Robinson, K. E.: Project Jacquard: interactive digital textiles at scale. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, CHI 2016, pp. 4216–4227. ACM, New York (2016)Google Scholar
  10. 10.
    Mennicken, S., Bernheim Brush, A.J., Roseway, A., Scott, J.: Finding roles for interactive furniture in homes with Emotocouch. In: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication, UbiComp 2014 Adjunct, pp. 923–930. ACM, New York (2014)Google Scholar
  11. 11.
    Mennicken, S., Bernheim Brush, A.J., Roseway, A., Scott, J.: Exploring interactive furniture with EmotoCouch. In: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication, UbiComp 2014 Adjunct, pp. 307–310. ACM, New York (2014)Google Scholar
  12. 12.
    Rus, S., Sahbaz, M., Braun, A., Kuijper, A.: Design factors for flexible capacitive sensors in ambient intelligence. In: Ruyter, B., Kameas, A., Chatzimisios, P., Mavrommati, I. (eds.) AmI 2015. LNCS, vol. 9425, pp. 77–92. Springer, Heidelberg (2015). doi:10.1007/978-3-319-26005-1_6 CrossRefGoogle Scholar
  13. 13.
    Xu, X., Lin, F., Wang, A., Hu, Y., Huang, M.C., Xu, W.: Body-earth mover’s distance: a matching-based approach for sleep posture recognition. IEEE Trans. Biomed. Circ. Syst. 10(5), 1023–1035 (2016)CrossRefGoogle Scholar
  14. 14.
    Enokibori, Y., Ito, Y., Suzuki, A., Mizuno, H., Shimakami, Y., Kawabe, T., Mase,K.: Spirovest: An E-textile-based wearable spirometer with posture change adaptability. In: Proceedings of the 2013 ACM Conference on Pervasive and Ubiquitous Computing Adjunct Publication, UbiComp 2013 Adjunct, pp. 203–206. ACM, New York (2013)Google Scholar
  15. 15.
    Braun, A., Frank, S., Wichert, R.: The capacitive chair. In: Streitz, N., Markopoulos, P. (eds.) DAPI 2015. LNCS, vol. 9189, pp. 397–407. Springer, Cham (2015). doi:10.1007/978-3-319-20804-6_36 CrossRefGoogle Scholar
  16. 16.
    Chang, W.-Y., Chen, C.-C., Chang, C.-C., Yang, C.-L.: An enhanced sensing application based on a flexible projected capacitive-sensing mattress. Sensors 14(4), 6922–6937 (2014)CrossRefGoogle Scholar
  17. 17.
    Djakow, M., Braun, A., Marinc, A.: MoviBed - sleep analysis using capacitive sensors. In: Stephanidis, C., Antona, M. (eds.) UAHCI 2014. LNCS, vol. 8516, pp. 171–181. Springer, Heidelberg (2014). doi:10.1007/978-3-319-07509-9_17 CrossRefGoogle Scholar
  18. 18.
    Rus, S., Grosse-Puppendahl, T., Kuijper, A.: Recognition of bed postures using mutual capacitance sensing. In: Aarts, E., et al. (eds.) AmI 2014. LNCS, vol. 8850, pp. 51–66. Springer, Cham (2014)Google Scholar
  19. 19.
    Liu, J.J., Xu, W., Huang, M.-C., Alshurafa, N., Sarrafzadeh, M., Raut, N., Yadegar, B.: A dense pressure sensitive bedsheet design for unobtrusive sleep posture monitoring. In: IEEE International Conference on Pervasive Computing and Communications (PerCom), p. 22 (2013)Google Scholar
  20. 20.
    Tan, H.Z., Slivovsky, L.A., Pentland, A.: A sensing chair using pressure distribution sensors. IEEE/ASME Trans. Mechatron. 6(3), 261–268 (2001)CrossRefGoogle Scholar
  21. 21.
    Shirehjini, A.A.N., Yassine, A., Shirmohammadi, S.: Design and implementation of a system for body posture recognition. Multimedia Tools Appl. 70(3), 1637–1650 (2014)CrossRefGoogle Scholar
  22. 22.
    Braun, A., Schembri, I., Frank, S.: ExerSeat - sensor-supported exercise system for Ergonomic microbreaks. In: Ruyter, B., Kameas, A., Chatzimisios, P., Mavrommati, I. (eds.) AmI 2015. LNCS, vol. 9425, pp. 236–251. Springer, Heidelberg (2015). doi:10.1007/978-3-319-26005-1_16 CrossRefGoogle Scholar
  23. 23.
    Braun, A., Frank, S., Majewski, M., Wang, X.: Capseat: capacitive proximity sensing for automotive activity recognition. In: Proceedings of the 7th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI 2015, pp. 225–232. ACM, New York (2015)Google Scholar
  24. 24.
    Kivikunnas, S., Strmmer, E., Korkalainen, M., Heikkil, T., Haverinen, M.: Sensing sofa and its ubiquitous use. In: 2010 International Conference on Information and Communication Technology Convergence (ICTC), pp. 559–562, November 2010Google Scholar
  25. 25.
    Große-Puppendahl, T.A., Marinc, A., Braun, A.: Classification of user postures with capacitive proximity sensors in AAL-environments. In: Keyson, D.V., Maher, M.L., Streitz, N., Cheok, A., Augusto, J.C., Wichert, R., Englebienne, G., Aghajan, H., Kröse, B.J.A. (eds.) AmI 2011. LNCS, vol. 7040, pp. 314–323. Springer, Heidelberg (2011). doi:10.1007/978-3-642-25167-2_43 CrossRefGoogle Scholar
  26. 26.
    Heikkil, T., Strmmer, E., Kivikunnas, S., Jrviluoma, M., Korkalainen, M., Kyllnen, V., Sarjanoja, E.M., Peltomaa, I.: Low intrusive Ehealth monitoring: human posture and activity level detection with an intelligent furniture network. IEEE Wirel. Commun. 20(4), 57–63 (2013)CrossRefGoogle Scholar
  27. 27.
    Pohl, H., Hettig, M., Karras, O., Ötztürk, H., Rohs, M.: CapCouch: home control with a posture-sensing couch. In: Adjunct Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2015 ACM International Symposium on Wearable Computers, UbiComp/ISWC2015 Adjunct, pp. 229–232. ACM, New York (2015)Google Scholar
  28. 28.
    Grosse-Puppendahl, T., Berghoefer, Y., Braun, A., Wimmer, R., Kuijper, A.: OpenCapSense: a rapid prototyping toolkit for pervasive interaction using capacitive sensing. In: IEEE International Conference on Pervasive Computing and Communications (PerCom), pp. 152–159, March 2013Google Scholar
  29. 29.
    Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The WEKA data mining software: an update. SIGKDD Explor. Newsl. 11(1), 10–18 (2009)CrossRefGoogle Scholar
  30. 30.
    Tsuruta, M., Nakamae, S., Shizuki, B.: RootCap: touch detection on multi-electrodes using single-line connected capacitive sensing. In: Proceedings of the 2016 ACM on Interactive Surfaces and Spaces, ISS 2016, pp. 23–32. ACM, New York (2016)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Fraunhofer IGDDarmstadtGermany
  2. 2.Technische Universität DarmstadtDarmstadtGermany

Personalised recommendations