Advertisement

Comparative Quantitative Evaluation of Distributed Methods for Explanation Generation and Validation of Floor Plan Recommendations

  • Christian Espinoza-Stapelfeld
  • Viktor EisenstadtEmail author
  • Klaus-Dieter Althoff
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11352)

Abstract

In this work, we compare different explanation generation and validation methods for semantic search pattern-based retrieval results returned by a case-based framework for support of early conceptual design phases in architecture. Compared methods include two case- and rule-based explanation engines, the third one is the discriminant analysis-based method for explanation and validation prediction and estimation. All of the explanation methods use the same data set for retrieval and subsequent explainability operations for results. We describe the main structure of each method and evaluate their quantitative validation performance against each other. The goal of this work is to examine which method performs better under which circumstances, at which point in time, and how good the potential explanation ant its validation can be predicted in general. To evaluate these issues, we compare not only the general performance, i.e., the average rate of valid explanations but also how the validation rate changes over time using a number of time steps for this comparison. We also show for which search pattern type which methods perform better.

Keywords

Case-based design Ditributed artificial intelligence Explainable agents Comparative evaluation Discriminant analysis Validation Semantic search 

References

  1. 1.
    Ayzenshtadt, V., Espinoza-Stapelfeld, C.A., Langenhahn, C., Althoff, K.D.: Multi-agent-based generation of explanations for retrieval results within a case-based support framework for architectural design. In: Proceedings of the 10th International Conference on Agents and Artificial Intelligence. International Conference on Agents and Artificial Intelligence (ICAART-18), Scitepress (2018)Google Scholar
  2. 2.
    Aamodt, A.: Explanation-driven case-based reasoning. In: Wess, S., Althoff, K.-D., Richter, M.M. (eds.) EWCBR 1993. LNCS, vol. 837, pp. 274–288. Springer, Heidelberg (1994).  https://doi.org/10.1007/3-540-58330-0_93CrossRefGoogle Scholar
  3. 3.
    Roth-Berghofer, T.R.: Explanations and case-based reasoning: foundational issues. In: Funk, P., González Calero, P.A. (eds.) ECCBR 2004. LNCS (LNAI), vol. 3155, pp. 195–209. Springer, Heidelberg (2004).  https://doi.org/10.1007/978-3-540-28631-8_29CrossRefGoogle Scholar
  4. 4.
    Cassens, J., Kofod-Petersen, A.: Designing explanation aware systems: the quest for explanation patterns. In: ExaCt, pp. 20–27 (2007)Google Scholar
  5. 5.
    Broekens, J., Harbers, M., Hindriks, K., van den Bosch, K., Jonker, C., Meyer, J.-J.: Do you get it? user-evaluated explainable BDI agents. In: Dix, J., Witteveen, C. (eds.) MATES 2010. LNCS (LNAI), vol. 6251, pp. 28–39. Springer, Heidelberg (2010).  https://doi.org/10.1007/978-3-642-16178-0_5CrossRefGoogle Scholar
  6. 6.
    Harbers, M., van den Bosch, K., Meyer, J.J.: Design and evaluation of explainable BDI agents. In: 2010 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT), vol. 2, pp. 125–132. IEEE (2010)Google Scholar
  7. 7.
    Voss, A.: Case design specialists in FABEL. In: Issues and Applications of Case-Based Reasoning in Design, pp. 301–335 (1997)Google Scholar
  8. 8.
    Oxman, R., Oxman, R.: Precedents: memory structure in design case libraries. In: CAAD Futures. vol. 93, pp. 273–287 (1993)Google Scholar
  9. 9.
    Flemming, U.: Case-based design in the SEED system. Autom. Constr. 3, 123–133 (1994)CrossRefGoogle Scholar
  10. 10.
    Lai, I.C.: Dynamic idea maps: a framework for linking ideas with cases during brainstorming. Int. J. Architectural Comput. 3, 429–447 (2005)CrossRefGoogle Scholar
  11. 11.
    Lin, C.J.: Visual architectural topology. In: Open Systems: Proceedings of the 18th International Conference on Computer-Aided Architectural Design Research in Asia, pp. 3–12 (2013)Google Scholar
  12. 12.
    Inanc, B.S.: Casebook. an information retrieval system for housing floor plans. In: The Proceedings of 5th Conference on Computer Aided Architectural Design Research (CAADRIA), pp. 389–398 (2000)Google Scholar
  13. 13.
    Richter, K., Heylighen, A., Donath, D.: Looking back to the future-an updated case base of case-based design tools for architecture. In: Knowledge Modelling-eCAADe (2007)Google Scholar
  14. 14.
    Richter, K.: Augmenting Designers’ Memory: Case-based Reasoning in Architecture. Logos-Verlag, Berlin (2011)Google Scholar
  15. 15.
    Richter, K.: What a shame-why good ideas can’t make it in architecture: a contemporary approach towards the case-based reasoning paradigm in architecture. In: FLAIRS Conference (2013)Google Scholar
  16. 16.
    Sabri, Q.U., Bayer, J., Ayzenshtadt, V., Bukhari, S.S., Althoff, K.D., Dengel, A.: Semantic pattern-based retrieval of architectural floor plans with case-based and graph-based searching techniques and their evaluation and visualization. In: ICPRAM, pp. 50–60 (2017)Google Scholar
  17. 17.
    Ayzenshtadt, V., et al.: Comparative evaluation of rule-based and case-based retrieval coordination for search of architectural building designs. In: Goel, A., Díaz-Agudo, M.B., Roth-Berghofer, T. (eds.) ICCBR 2016. LNCS (LNAI), vol. 9969, pp. 16–31. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-47096-2_2CrossRefGoogle Scholar
  18. 18.
    Jackson, M.: Problem analysis using small problem frames. S. Afr. Comput. J. 47–60 (1999)Google Scholar
  19. 19.
    Espinoza, C.: Case-based classification of explanation expressions in search results of a retrieval system for Support of the early conceptual design phase in architecture. Bachelor Thesis. University of Hildesheim (2018)Google Scholar
  20. 20.
    Bayer, J., et al.: Migrating the classical pen-and-paper based conceptual sketching of architecture plans towards computer tools - prototype design and evaluation. In: Lamiroy, B., Dueire Lins, R. (eds.) GREC 2015. LNCS, vol. 9657, pp. 47–59. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-52159-6_4CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Christian Espinoza-Stapelfeld
    • 1
  • Viktor Eisenstadt
    • 1
    Email author
  • Klaus-Dieter Althoff
    • 1
    • 2
  1. 1.Institute of Computer Science, Intelligent Information Systems Lab (IIS)University of HildesheimHildesheimGermany
  2. 2.German Research Center for Artificial IntelligenceKaiserslauternGermany

Personalised recommendations