Type independent hierarchical analysis for the recognition of folded garments’ configuration

Abstract

This paper proposes a hierarchical visual architecture for perceiving garments’ configuration independently from their type for the robotic unfolding task. Special focus is given on the decomposition of folded configurations into low- and high-level features. The low-level features comprise junctions of edges, which act as localized indicators of the clothing article’s state, while the high-level components refer to its layers and the axis that unites them. The proposed methodology extracts and classifies the low-level components into indicators of folds, overlaps, garment’s edges and corners and through their combination reconstructs the axis and the layers of the garment. The methodology is independent from the garment’s shape while it uses depth sensors so that it can deal with garments of various colours, patterns and decorative features. Experiments showed the effectiveness of the method in scenarios with onefold or twofold and in different datasets, proving the extensibility of the approach.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27

References

  1. 1.

    Triantafyllou D, Mariolis I, Kargakos A, Malassiotis S, Aspragathos N (2016) A geometric approach to robotic unfolding of garments. J Robotics Auton Syst 75:233–243

    Article  Google Scholar 

  2. 2.

    Triantafyllou D, Aspragathos N (2019) Upper layer extraction of a folded garment towards unfolding by a robot. In: RAAD pp 597–604

  3. 3.

    Tu Z, Chen X (2014) Junction detection based on line segments. In: 9th IEEE conference on industrial electronics and applications, Hangzhou, pp 1231–1234

  4. 4.

    Doumanoglou A, Kim T K, Zhao X, Malassiotis S (2014) Active random forests: an application to autonomous clothes unfolding. In: ECCV 2014: computer vision—ECCV, pp 644–658

  5. 5.

    Doumanoglou A, Kargakos A, Kim TK, Malassiotis S (2014) Autonomous active recognition and unfolding of clothes using random decision forests and probabilistic planning. In: IEEE international conference on robotics and automation pp 987–993

  6. 6.

    Corona E, Alenya G, Gabas A, Torras C (2018) Active recognition and target grasping point detection using deep learning. Pattern Recogn 74:629–641

    Article  Google Scholar 

  7. 7.

    Saxenaand K, Shibata T (2019) Garment recognition and grasping point detection for clothing assistance task using deep learning. In: IEEE/SICE international symposium on system integration, pp 14–16

  8. 8.

    Mariolis I, Peleka G, Kargakos A, Malassiotis S (2015) Pose and category recognition of highly deformable objects using deep learning. In: International conference on advanced robotics, pp 655–662

  9. 9.

    Maitin-Shepard J, Cusumano-Towner M, Lei J, Abbeel P (2010) Cloth grasp point detection based on multiple-view geometric cues with application to robotic towel folding. In: IEEE international conference on robotics and automation, pp 2308–2315

  10. 10.

    Yamazaki K (2015) A method of grasp point selection from an item of clothing using hem element relations. Adv Robot 29

  11. 11.

    Kita Y, Sian Neo E, Ueshiba T, Kita N (2010) Clothes handling using visual recognition in cooperation with actions. In: IEEE/RSJ international conference on intelligent robots and systems, pp 2710–2715

  12. 12.

    Li Y, Wang Y, Case M, Chang SF, Allen KP (2014) Real-time estimation of deformable objects using a volumetric approach. In: IROS, pp 987–993

  13. 13.

    Osawa F, Seki H, Kamiya Y (2007) Unfolding of massive laundry and classification types by dual manipulator. J Adv Comput Intell Intell Inform 11:5

    Google Scholar 

  14. 14.

    Cusumano-Towner M, Singh A, Miller S, O’Brien JF, Abbeel P (2011) Bringing clothing into desired configurations with limited perception. In: 2011 IEEE international conference on robotics and automation, pp 3893–3900

  15. 15.

    Hamajima K, Kakikura M (2000) Planning strategy for task of unfolding clothes (classification of cloths). J Robot Mechatron 12:577–584

    Article  Google Scholar 

  16. 16.

    Manabu K, Massayoshi K (2003) Study on handling clothes-task planning of deformation for unfolding laundry. J Robots Mechatron 15:429–434

    Google Scholar 

  17. 17.

    Stria J, Petrik V, Hlavac V (2017) Model free approach to garments unfolding based on detection of folded layers. In: 2017 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 3274–3280

  18. 18.

    Estevez D, Victores JD, Fernandez-Fernandez R et al (2020) Enabling garment-agnostic laundry tasks for a robot household companion. Robotics Auton Syst 123:134–139

    Article  Google Scholar 

  19. 19.

    Triantafyllou D, Aspragathos N (2014) Definition and classification of primitives for the robotic unfolding of a piece of clothing. In: KEOD, vol 417, p 422

  20. 20.

    Zhou P, Ye W, Wang Q (2011) An improved Canny algorithm for edge detection. J Comput Inform Syst 7(5):1516–1523

    Google Scholar 

  21. 21.

    Hershberger J, Snoeyink J (1994) An o (nlogn) implementation of the Douglas–Peucker algorithm for line simplification. In: Proceedings of the 10th annual symposium on computational geometry, pp 383–384

  22. 22.

    Khoshelham K (2012) Accuracy analysis of Kinect depth data. In: ISPRS—international archives of the photogrammetry, remote sensing and spatial information sciences

  23. 23.

    Breiman L (2001) Random forests. Mach Learn 45(1):5–32

    Article  Google Scholar 

  24. 24.

    Geron A (2019) Hands on machine learning. O’Reilly Media, Inc., Newton

    Google Scholar 

  25. 25.

    Platt J (1999) Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Adv Large Margin Classif 10(3):61–74

    Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Dimitra Triantafyllou.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Triantafyllou, D., Koustoumpardis, P. & Aspragathos, N. Type independent hierarchical analysis for the recognition of folded garments’ configuration. Intel Serv Robotics (2021). https://doi.org/10.1007/s11370-021-00365-8

Download citation

Keywords

  • Garments
  • Unfolding
  • Layer detection
  • Axis detection
  • Depth images