Skip to main content
SpringerLink
Log in

Evaluating representational systems in artificial intelligence

  • Published:
Artificial Intelligence Review Aims and scope Submit manuscript

A Correction to this article was published on 08 January 2018

This article has been updated

Abstract

All artificial reasoners work within representational systems. These systems, which may have varying levels of formality or detail, determine the space of possible representations over which the artificial reasoner can operate, by defining the syntactic and semantic properties of the symbols, structures, and inferences that they manipulate. But we are now seeing an increasing need for the ability to reason over representational systems, rather than just working within them. A prerequisite of performing such reasoning is the ability to evaluate and compare representational objects (and to know the difference between them). We survey the criteria that are used for such evaluations in AI, machine learning, and other AI-related fields. To aid our survey, we introduce a formalism of representations, representational systems, and representational spaces that lends itself nicely to an analysis of the criteria typically used for evaluating them.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Change history

  • 08 January 2018

    In the original publication, the first paragraph of introduction section was published incorrectly. The corrected text was given in this correction.

Notes

  1. By “AI-related research” we mean the areas of research that have criteria for comparing or evaluating representational objects which might be of use to AI researchers.

  2. It is sometimes claimed (e.g. Griffiths et al. 2012) that frameworks should be thought of as tools for describing models, and cannot be falsified. If the analogy from frameworks and models to RSes and representations holds, then, we might infer that RSes cannot be falsified. That is, however, not a claim we support in this paper; we instead argue that there exist criteria, in regular use, to evaluate RSes. Whether a negative evaluation of an RS counts as a falsification is to be determined.

  3. E.g., a cat can purr, or meow; and a legislator can legislate. A legislator can also conceivably purr or meow, but these are not actions we typically associate with them, and would thus likely not be members of the legislator RS.

  4. Those familiar with object-oriented programming may notice this is a weakened version of the inheritance relationship; e.g., no multiple inheritances or polymorphisms are defined. This is because the work in this paper does not require specifying relationships any more complex than what we present here.

  5. But it is interesting to note that this RS seems to be falling out of favor in the machine learning community. Popular implementations of neural networks, such as Google’s TensorFlow, view networks not as collections of neurons, but rather as tensors in a “computational graph” (Abadi et al. 2015). Our OO-inspired formalism would describe this as a shift in RSes, to one that views the propagations of activations and gradients to be operations defined over tensors. Such tensor operations can be easier to optimize mathematically and computationally, a clear benefit of the tensor-based RS.

  6. We leave the proof of this discovery to the reader.

  7. In the latter cases, it is often not clearly specified what the neural networks are supposed to be representations of.

  8. E.g., see Gentner and Forbus (2011) for a review of models of analogical reasoning.

  9. Though cognitive architectures are supposed to be more general than cognitive models of particular phenomena, in practice many cognitive architectures tend to focus on some subset of cognitive phenomena, rendering the authors’ claim of model generality difficult to verify. See Kotseruba et al. (2016) for a comprehensive review of cognitive architectures and their foci.

  10. We can refer to these RSes nested within other RSes as level RSes.

  11. It is outside of the scope of this paper to take a definitive position on the level of ontological commitment in our OO-inspired formalism, other than saying that it is typically the case that the inclusion of members or methods in an RS constitutes an ontological commitment to the realism of those elements. We have drawn some ideas from Floridi’s theory of levels of abstraction, which itself comes with a theory of ontological commitment built in. Interested readers can refer to Floridi (2011b) for more.

  12. ‘Contact’ here means something akin to theoretical or field utility, a criterion we will define in Sect. 6.

  13. Podsakoff et al. (2016) give an example in which candidate attributes for a definition of “submarine” are compared to objects that are or are not considered submarines: diving bells, viking longboats, fishing boats, etc.

References

  • Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, Corrado GS, Davis A, Dean J, Devin M, Ghemawat S, Goodfellow I, Harp A, Irving G, Isard M, Jia Y, Jozefowicz R, Kaiser L, Kudlur M, Levenberg J, Mané D, Monga R, Moore S, Murray D, Olah C, Schuster M, Shlens J, Steiner B, Sutskever I, Talwar K, Tucker P, Vanhoucke V, Vasudevan V, Viégas F, Vinyals O, Warden P, Wattenberg M, Wicke M, Yu Y, Zheng X (2015) TensorFlow: large-scale machine learning on heterogeneous systems. Software available from tensorflow.org. http://tensorflow.org/

  • Anderson B (2011) The myth of computational level theory and the vacuity of rational analysis. Behav Brain Sci 34:189–190

    Article  Google Scholar 

  • Aouchiche M, Hansen P (2010) A survey of automated conjectures in spectral graph theory. Linear Algebra Appl. 432:2293

    Article  MathSciNet  MATH  Google Scholar 

  • Barker-Plummer D, Barwise J, Etchemendy J (2011) Language proof and logic, 2nd edn. Center for the Study of Language and Information, Stanford

    MATH  Google Scholar 

  • Barwise J, Etchemendy J (1998) A computational architecture for heterogeneous reasoning. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.24.5942

  • Bauer AJ, Just MA (2015) Monitoring the growth of the neural representations of new animal concepts. Hum Brain Mapp 36:3213

    Article  Google Scholar 

  • Bengio Y et al (2009) Learning deep architectures for AI. Found Trends Mach Learn 2(1):1–127

    Article  MATH  MathSciNet  Google Scholar 

  • Bengio Y, Courville A, Vincent P (2013) Representation learning: a review and new perspectives. IEEE Trans Pattern Anal Mach Intell 35(8):1798–1828

    Article  Google Scholar 

  • Besnard P, Garcia A, Hunter A, Modgil S, Prakken H, Simari G, Toni F (2014) Introduction to structured argumentation. Argum Comput 5(1):1–4

    Article  Google Scholar 

  • Borsboom D, Wagenmakers EJ, Romejin JW (2011) Mechanistic curiosity will not kill the Bayesian cat. Behav Brain Sci 34:192–194

    Article  Google Scholar 

  • Bowers JS, Davis CJ (2012a) Bayesian just-so stories in psychology and neuroscience. Psychol Bull 138(3):389–414

    Article  Google Scholar 

  • Bowers JS, Davis CJ (2012) Is that what Bayesians believe? Reply to Griffiths, Chater, Norris, and Pouget (2012). Psychol Bull 138(3):423–426

    Article  Google Scholar 

  • Bricker P (2016) Ontological commitment. In: Zalta EN (ed) The Stanford encyclopedia of philosophy, winter 2016 edn. Metaphysics Research Lab, Stanford University, Stanford

    Google Scholar 

  • Bringsjord S (1991) Is the connectionist–logicist clash one of AI’s wonderful red herrings? J Exp Theor AI 3(4):319–349

    Article  Google Scholar 

  • Bringsjord S, Ferrucci D (1998) Logic and artificial intelligence: divorced, separated, still married...? Mind Mach 8:273–308

    Article  Google Scholar 

  • Bringsjord S, Licato J, Bringsjord A (2016) The contemporary craft of creating characters meets today’s cognitive architectures: a case study in expressivity. In: Turner J, Nixon M, Bernardet U, DiPaola S (eds) Integrating cognitive architectures into virtual character design, Advances in computational intelligence and robotics (ACIR), Information science reference. IGI Global, Hershey

    Google Scholar 

  • Brooks RA (1990) Elephants don’t play chess. Robot Auton Syst 6:3–15

    Article  Google Scholar 

  • Brun G (2015) Explication as a method of conceptual re-engineering. Erkenntnis 1:1–31

    Google Scholar 

  • Bundy A, Ireland A, Hutter D, Basin D (2005) Rippling: meta-level guidance for mathematical reasoning, vol 56. Cambridge University Press, Cambridge

    Book  MATH  Google Scholar 

  • Carnap R (1950) Logical foundations of probability. University of Chicago Press, Chicago

    MATH  Google Scholar 

  • Chater N, Goodman N, Griffiths TL, Kemp C, Oaksford M, Tenenbaum JB (2011) The imaginary fundamentalists: the unshocking truth about bayesian cognitive science. Behav Brain Sci 34:194–196

    Article  Google Scholar 

  • Cheng PCH (2002) Electrifying diagrams for learning: principles for complex representational systems. Cogn Sci 26:685–736

    Article  Google Scholar 

  • Cheng PCH (2016) What constitutes an effective representation? In: Jamnik M, Uesaka Y, Elzer Schwartz S (eds) Diagrammatic representation and inference: proceedings from the 9th international conference, Diagrams 2016, Lecture Notes in Computer Science vol 9781. Springer

  • Cunningham P (2009) A taxonomy of similarity mechanisms for case-based reasoning. IEEE Trans Knowl Data Eng 21(11):1532

    Article  Google Scholar 

  • Doumas LA, Hummel JE (2013) Comparison and mapping facilitate relation discovery and predication. PLoS ONE 8(6):e63889

    Article  Google Scholar 

  • Doumas LA, Hummel JE, Sandhofer C (2008) A theory of the discovery and predication of relational concepts. Psychol Rev 115(1):1–43

    Article  Google Scholar 

  • Eickenberg M, Gramfort A, Varoquaux G, Thirion B (2016) Seeing it all: convolutional network layers map the function of the human visual system. NeuroImage 152:184

    Article  Google Scholar 

  • Eliasmith C, Thagard P (2001) Integrating structure and meaning: a distributed model of analogical mapping. Cogn Sci 25(2):245–286

    Article  Google Scholar 

  • Emruli B, Sandin F (2013) Analogical mapping with sparse distributed memory: a simple model that learns to generalize from examples. Cogn Comput 6:74

    Article  Google Scholar 

  • Falkenhainer B, Forbus KD, Gentner D (1989) The structure-mapping engine: algorithm and examples. Artif Intell 41(1):1–63

    Article  MATH  Google Scholar 

  • Fernbach PM, Sloman SA (2011) Don’t throw out the Bayes with the bathwater. Behav Brain Sci 34:198–199

    Article  Google Scholar 

  • Floridi L (2011a) A defence of constructionism: philosophy as conceptual engineering. Metaphilosophy 42(3):282–304

    Article  Google Scholar 

  • Floridi L (2011b) The philosophy of information. Oxford University Press, Oxford

    Book  MATH  Google Scholar 

  • Fodor JA (1980) The language of thought, 2nd edn. Harvard University Press, Cambridge

    Google Scholar 

  • Fodor J (1998) Concepts: where cognitive science went wrong. Oxford University Press, New York

    Book  Google Scholar 

  • Fodor JA, Pylyshyn Z (1988) Connectionism and cogntive architecture: a critical analysis. Cognition 28(1–2):3–71

    Article  Google Scholar 

  • Forbus K, Mostek T, Ferguson R (2002) An analogy ontology for integrating analogical processing and first-principles reasoning. In: AAAI/IAAI, pp 878–885

  • Forbus KD, Ferguson RW, Lovett A, Gentner D (2017) Extending SME to handle large-scale cognitive modeling. Cogn Sci 41:1152

    Article  Google Scholar 

  • Gauthier T, Kaliszyk C, Urban J, Vyskočil J (2016) Conjecturing over large corpora. In: Proceedings from the first conference on artificial intelligence and theorem proving (AITP 2016)

  • Gentner D (1983) Structure-mapping: a theoretical framework for analogy. Cogn Sci 7(2):155–170

    Article  Google Scholar 

  • Gentner D, Forbus K (2011) Computational models of analogy. Wiley Interdiscip Rev Cogn Sci 2(3):266–276

    Article  Google Scholar 

  • Gerring J (1999) What makes a concept good? A criterial framework for understanding concept formation in the social sciences. Polity 31(3):357–393

    Article  Google Scholar 

  • Goodman N, Frank MC, Griffiths TL, Tenenbaum JB, Battaglia PW, Hamrick JB (2015) Relevant and robust: a response to Marcus and Davis (2013). Psychol Sci 26(4):539–541

    Article  Google Scholar 

  • Gopnik A (2011) Probabilistic models as theories of children’s minds. Behav Brain Sci 34:200–201

    Article  Google Scholar 

  • Griffiths TL, Chater N, Norris D, Pouget A (2012) How the Bayesians got their beliefs (and what those beliefs actually are): comment on Bowers and Davis (2012). Psychol Bull 138(3):415–422

    Article  Google Scholar 

  • Guo J, Wang C, Roman-Rangel E, Chao H, Rui Y (2016) Building hierarchical representations for oracle character and sketch recognition. IEEE Trans Image Process 25(1):104–118

    Article  MathSciNet  MATH  Google Scholar 

  • Gust H, Kühnberger Ku, Schmid U (2003) Anti-Unification of Axiomatic Systems. University of Osnabrück, Technical report

  • Gust H, Kühnberger Ku, Schmid U (2006) Metaphors and heuristic-driven theory projection (HDTP). Theor Comput Sci 354(1):98–117

    Article  MathSciNet  MATH  Google Scholar 

  • Han J, Zhang D, Wen S, Guo L, Liu T, Li X (2016) Two-stage learning to predict human eye fixations via SDAEs. IEEE Trans Cybern 46(2):487–498

    Article  Google Scholar 

  • Hansson SO (2000) Formalization in philosophy. Bull Symb Logic 6(2):162–175

    Article  Google Scholar 

  • Hinton GE, Zemel RS (1994) Autoencoders, minimum description length and Helmholtz free energy. In Advances in neural information processing systems, pp 3–10

  • Hinton GE, McClelland J, Rumelhart DE (1986) Distributed representations. In: Rumelhart DE, McClelland J (eds) Parallel distributed processing: explorations in the microstructure of cognition, vol 1. MIT Press, Cambridge

    Google Scholar 

  • Hofstadter DR (2001) Epilogue: analogy as the core of cognition. In: Gentner D, Holyoak KJ, Kokinov BN (eds) The analogical mind: perspectives from cognitive science, chap 15. The MIT Press, Cambridge

    Google Scholar 

  • Hofstadter DR, Sander E (2013) Surfaces and essences: analogy as the fuel and fire of thinking. Basic Books, London

    Google Scholar 

  • Holyoak KJ, Hummel JE (2000) The proper treatment of symbols in a connectionist architecture. In: Deitrich E, Markman A (eds) Cognitive dynamics: conceptual change in humans and machines. MIT Press, Cambridge

    Google Scholar 

  • Hummel JE (2001) Complementary solutions to the binding problem in vision: implications for shape perception and object recognition. Vis Cogn 8(3):489–517

    Article  Google Scholar 

  • Hummel JE (2010) Symbolic versus associative learning. Cogn Sci 34:958–965

    Article  Google Scholar 

  • Hummel JE (2016) Putting distributed representations into context. Lang Cogn Neurosci 32:359

    Article  Google Scholar 

  • Hummel JE, Biederman I (1992) Dynamic binding in a neural network for shape recognition. Psychol Rev 99(3):480–517

    Article  Google Scholar 

  • Hummel JE, Holyoak KJ (1997) Distributed representations of structure: a theory of analogical access and mapping. Psychol Rev 104(3):427–466

    Article  Google Scholar 

  • Hummel JE, Holyoak KJ (2003) Relational reasoning in a neurally-plausible cognitive architecture: an overview of the LISA Project. Cogn Stud Bull Jpn Cogn Sci Soc 10:58–75

    Google Scholar 

  • Ireland A, Bundy A (1996) Productive use of failure in inductive proof. J Autom Reason 16(1–2):79–111

    Article  MathSciNet  MATH  Google Scholar 

  • Jones M, Love BC (2011) Bayesian fundamentalism or enlightenment? On the explanatory status and theoretical contributions of Bayesian models of cognition. Behav Brain Sci 34:169–231

    Article  Google Scholar 

  • Kahneman D (2011) Thinking, fast and slow. Farrar, Straus and Girous, New York

    Google Scholar 

  • Kotseruba I, Gonzalez OJA, Tsotsos JK (2016) A review of 40 years of cognitive architecture research: focus on perception, attention, learning and applications. arXiv:1610.08602v1

  • Li Y, Yosinski J, Clune J, Lipson H, Hopcroft J (2016) Convergent learning: do different neural networks learn the same representations? In: Proceedings of international conference on learning representation (ICLR)

  • Licato J (2015) Analogical constructivism: the emergence of reasoning through analogy and action schemas. PhD thesis, Rensselaer Polytechnic Institute, Troy

  • Licato J (2017) Two paradoxes and their implications for AI-assisted analysis. In: Proceedings of the 2017 conference of the International Association for Computing and Philosophy (IACAP 2017)

  • Licato J, Bringsjord S (2016) A physically realistic, general-purpose simulation environment for developmental AI systems. In: Proceedings of the ECAI 2016 workshop on evaluating general-purpose AI (EGPAI 2016)

  • Licato J, Govindarajulu NS, Bringsjord S, Pomeranz M, Gittelson L (2013) Analogico-deductive generation of Gödel’s first incompleteness theorem from the liar paradox. In: Proceedings of the 23rd annual international joint conference on artificial intelligence (IJCAI-13)

  • Licato J, Bringsjord S, Govindarajulu NS (2014a) How models of creativity and analogy need to answer the tailorability concern. In: Besold TR, Kühnberger KU, Schorlemmer M, Smaill A (eds) Computational creativity research: towards creative machines, chap 5. Atlantis Press, Paris

    Google Scholar 

  • Licato J, Sun R, Bringsjord S (2014b) Using a hybrid cognitive architecture to model children’s errors in an analogy task. In: Proceedings of CogSci 2014

  • Licato J, Marton N, Dong B, Sun R, Bringsjord S (2015) Modeling the creation and development of cause-effect pairs for explanation generation in a cognitive architecture. In: Proceedings of the 2015 international workshop on artificial intelligence and cognition (AIC 2015)

  • Macagno F, Walton D (2009) Argument from analogy in law, the classical tradition, and recent theories. Philos Rhetor 42(2):154

    Article  Google Scholar 

  • Maher P (2007) Explication defended. Stud Log 86(2):331–341

    Article  MathSciNet  MATH  Google Scholar 

  • Marcus G, Davis E (2012) How robust are probabilistic models of higher-level cognition? Psychol Sci 24(12):2351–2360

    Article  Google Scholar 

  • Marcus G, Davis E (2015) Still searching for principles: a response to Goodman et al (2015). Psychol Sci 26(4):542–544

    Article  Google Scholar 

  • Markman AB, Otto AR (2011) Cognitive systems optimize energy rather than information. Behav Brain Sci 34:207

    Article  Google Scholar 

  • Marr D (1982) Vision: a computational approach. Freeman and Co., New York

    Google Scholar 

  • McClamrock R (1991) Marr’s three levels: a re-evaluation. Mind Mach 1(2):185–196

    Article  Google Scholar 

  • McCulloch WS, Pitts W (1943) A logical calculus of the ideas immanent in nervous activity. Bull Math Biophys 5(4):115–133

    Article  MathSciNet  MATH  Google Scholar 

  • Mikolov T, Chen K, Corrado G, Dean J (2013a) Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781

  • Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J (2013b) Distributed representations of words and phrases and their compositionality. In: Advances in neural information processing systems, pp 3111–3119

  • Newell A (1982) The knowledge level. Artif Intell 18(1):87–127

    Article  MathSciNet  Google Scholar 

  • Novaes CD, Reck E (2017) Carnapian explication, formalisms as cognitive tools, and the paradox of adequate formalization. Synthese 194(1):195–215

    Article  MathSciNet  MATH  Google Scholar 

  • Pearl J (1996) The art and science of cause and effect. Lecture given as part of the UCLA Faculty Research Lectureship Program

  • Peirce CS (1933) Existential graphs. In: Hartshorne C, Weiss P (eds) The simplest mathematics, The collected papers of Charles Sanders Peirce, vol 4. Harvard University Press, Cambridge, pp 347–584

    Google Scholar 

  • Podsakoff PM, McKenzie SB, Podsakoff NP (2016) Recommendations for creating better concept definitions in the organizational, behavioral, and social sciences. Organ Res Methods 19(2):159–203

    Article  Google Scholar 

  • Pylyshyn Z (1984) Computation and cognition. MIT Press, Cambridge

    Google Scholar 

  • Rachkovskij DA, Kussul EM, Baidyk TN (2013) Building a world model with structure-sensitive sparse binary distributed representations. Biologically inspired cognitive architectures 3:64–86

    Article  Google Scholar 

  • Reed C, Rowe G (2004) Araucaria: software for argument analysis, diagramming and representation. Int J AI Tools 14(3–4):961–980

    Article  Google Scholar 

  • Reed C, Budzynska K, Duthie R, Janier M, Konat B, Lawrence J, Pease A, Snaith M (2017) The argument web: an online ecosystem of tools, systems and services for argumentation. Philos Technol 30(2):137–160

    Article  Google Scholar 

  • Satel S, Lilenfeld SO (2013) Brainwashed: the seductive appeal of mindless neuroscience, kindle edition. Basic Books, London

    Google Scholar 

  • Schmidt M, Krumnack U, Gust H, Kühnberger KU (2014) Heuristic-driven theory projection: an overview. In: Computational approaches to analogical reasoning: current trends studies in computational intelligence, vol 548

  • Schwering A, Krumnack U, Kühnberger Ku, Gust H (2009) Syntactic principles of heuristic-driven theory projection. Cogn Syst Res 10(3):251–269

    Article  Google Scholar 

  • Searle J (2004) Mind: a brief introduction. Oxford University Press, New York

    Google Scholar 

  • Smith P (2007) An introduction to Gödel’s theorems. Cambridge University Press, Cambridge

    Book  MATH  Google Scholar 

  • Stapleton G, Jamnik M, Shimojima A (2015) Effective representation of information: generalizing free rides. In: Proceedings of the 9th international conference on diagrammatic representation and inference, pp 296–299

  • Stewart T, Eliasmith C (2012) Compositionality and biologically plausible models. In: Hinzen W, Werning M, Machery E (eds) Oxford handbook of compositionality. Oxford University Press, Oxford

    Google Scholar 

  • Sun R (1991) Connectionist models of rule-based reasoning. In: Proceedings of the 13th cognitive science society conference. Lawrence Erlbaum, pp 437–442

  • Sun R (2001a) Computation, reduction and teleology of consciousness. J Cogn Syst Res 1(4):241–249

    Article  Google Scholar 

  • Sun R (2001b) From implicit skills to explicit knowledge: a bottom-up model of skill learning. Cogn Sci 25(2):203–244

    Article  Google Scholar 

  • Sun R (2002) Duality of the mind: a bottom up approach toward cognition. Lawrence Erlbaum, Mahwah

    Google Scholar 

  • Sun R (2004) Desiderata for cognitive architectures. Philos Psychol 17(3):341–373

    Google Scholar 

  • Taddeo M, Floridi L (2007) A praxical solution of the symbol grounding problem. Mind Mach 17(4):369–389

    Article  Google Scholar 

  • Thibodeau PH, Sikos L, Durgin FH (2016) What do we learn from rating metaphors? In: Proceedings from the 2016 conference of the Cognitive Science Society (CogSci 2016)

  • Turner R, Angius N (2017) The philosophy of computer science. In: Zalta EN (ed) The Stanford encyclopedia of philosophy, spring 2017 edn. Metaphysics Research Lab, Stanford University, Stanford

    Google Scholar 

  • Uesaka Y, Igarashi M, Suetsugu R (2016) Promoting multi-perspective integration as a 21st century skill: the effects of instructional methods encouraging students’ spontaneous use of tables for organizing information. In: Jamnik M, Uesaka Y, Elzer Schwartz S (eds) Diagrammatic representation and inference: proceedings from the 9th international conference, Diagrams 2016, Lecture Notes in Computer Science, vol 9781. Springer

  • Urbas M, Jamnik M (2014) A framework for heterogeneous reasoning in formal and informal domains. In: Proceedings of the third international conference on diagrammatic representation and inference, pp 277–292

  • Vernon D, van Hofsten C, Fadiga L (2016) Desiderata for developmental cognitive architectures. Biol Inspir Cogn Archit 18:116–127

    Google Scholar 

  • Vincent P, Larochelle H, Lajoie I, Bengio Y, Manzagol PA (2010) Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J Mach Learn Res 11:3371–3408

    MathSciNet  MATH  Google Scholar 

  • Vorms M (2009) Formats of Representation in Scientific Theorising. In: Models and simulations 3: emergence, computation, and reality, Charlottesville, VA, USA

  • Walton D (1985) Arguer’s position: a pragmatic study of ad hominem attack, criticism, refutation, and fallacy. Greenwood Press, Westport

    Google Scholar 

  • Walton D (1999) One-sided arguments: a dialectical analysis of bias. State University of New York Press, Albany

    Google Scholar 

  • Walton D, Reed C, Macagno F (2008) Argumentation schemes. Cambridge University Press, Cambridge

    Book  MATH  Google Scholar 

  • Willingham DB, Nissen MJ, Bullemer P (1989) On the development of procedural knowledge. J Exp Psychol Learn Mem Cogn 15:1047–1060

    Article  Google Scholar 

  • Winograd T (1975) Frame representations and the declarative-procedural controversy. In: Bobrow D, Collins A (eds) Representation and understanding: studies in cognitive science. Academic Press, New York

    Google Scholar 

  • Wyss M, Thieme A, Licato J (2017) Can AI reason over representational systems? In: Licato J, Hayes A (eds) Proceedings of the 28th modern artificial intelligence and cognitive science (MAICS) conference

  • Xing C, Corter JE, Zahner D (2016) Diagrams affect choice of strategy in probability problem solving. In: Jamnik M, Uesaka Y, Elzer Schwartz S (eds) Diagrammatic representation and inference: proceedings from the 9th international conference, Diagrams 2016. Lecture Notes in Computer Science, vol 9781. Springer

  • Zoph B, Le QV (2017) Neural architecture search with reinforcement learning. CoRR arXiv:1611.01578

Download references

Author information

Authors and Affiliations

  1. University of South Florida, Tampa, FL, USA

    John Licato

  2. Purdue University, Fort Wayne, Fort Wayne, IN, USA

    Zhitian Zhang

Authors
  1. John Licato
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhitian Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to John Licato.

Additional information

The original version of this article was revised: In the original publication, the first paragraph of introduction section was published incorrectly. Full information regarding corrections made can be found in the correction article for this article.

This material is based upon work supported by the Air Force Office of Scientific Research under Award Number FA9550-16-1-0308. Any opinions, finding, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the United States Air Force.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Licato, J., Zhang, Z. Evaluating representational systems in artificial intelligence. Artif Intell Rev 52, 1463–1493 (2019). https://doi.org/10.1007/s10462-017-9598-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10462-017-9598-7

Keywords

Search

Navigation