Logical Vision: One-Shot Meta-Interpretive Learning from Real Images

  • Wang-Zhou DaiEmail author
  • Stephen Muggleton
  • Jing Wen
  • Alireza Tamaddoni-Nezhad
  • Zhi-Hua Zhou
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10759)


Statistical machine learning is widely used in image classification. However, most techniques (1) require many images to achieve high accuracy and (2) do not provide support for reasoning below the level of classification, and so are unable to support secondary reasoning, such as the existence and position of light sources and other objects outside the image. In recent work an Inductive Logic Programming approach called Logical Vision (LV) was shown to overcome some of these limitations. LV uses Meta-Interpretive Learning combined with low-level extraction of high-contrast points sampled from the image to learn recursive logic programs describing the image. This paper extends LV by using (a) richer background knowledge enabling secondary reasoning from raw images, such as light reflection that can itself be learned and used for resolving visual ambiguities, which cannot be easily modelled using statistical approaches, (b) a wider class of background models representing classical 2D shapes such as circles and ellipses, (c) primitive-level statistical estimators to handle noise in real images. Our results indicate that the new noise-robust version of LV is able to handle secondary reasoning task in real images with few data, which is very similar to scientific discovery process of humans. Specifically, it uses a single example (i.e. one-shot LV) converges to an accuracy at least comparable to thirty-shot statistical machine learner on the prediction of hidden light sources. Moreover, we demonstrate that the learned theory can be used to identify ambiguities in the convexity/concavity of objects such as craters.



This research was supported by the National Science Foundation of China (61751306). The second author acknowledges support from his Royal Academy of Engineering/Syngenta Research Chair at the Department of Computing at Imperial College London. Authors want to thank reviewers and ILP’17 attendees for helpful comments.


  1. 1.
    Antanas, L., van Otterlo, M., Oramas Mogrovejo, J., Tuytelaars, T., De Raedt, L.: There are plenty of places like home: using relational representations in hierarchies for distance-based image understanding. Neurocomputing 123, 75–85 (2014)CrossRefGoogle Scholar
  2. 2.
    Cecchini, R., Del Bimbo, A.: A programming environment for imaging applications. Pattern Recogn. Lett. 14(10), 817–824 (1993)CrossRefzbMATHGoogle Scholar
  3. 3.
    Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2, 27:1–27:27 (2011)CrossRefGoogle Scholar
  4. 4.
    Cohn, A.G., Hogg, D.C., Bennett, B., Devin, V., Galata, A., Magee, D.R., Needham, C., Santos, P.: Cognitive vision: integrating symbolic qualitative representations with computer vision. In: Christensen, H.I., Nagel, H.-H. (eds.) Cognitive Vision Systems. LNCS, vol. 3948, pp. 221–246. Springer, Heidelberg (2006). CrossRefGoogle Scholar
  5. 5.
    Cox, D.: Do we understand high-level vision? Curr. Opin. Neurobiol. 25, 187–193 (2014)CrossRefGoogle Scholar
  6. 6.
    Cropper, A., Muggleton, S.H.: Logical minimisation of meta-rules within meta-interpretive learning. In: Davis, J., Ramon, J. (eds.) ILP 2014. LNCS (LNAI), vol. 9046, pp. 62–75. Springer, Cham (2015). CrossRefGoogle Scholar
  7. 7.
    Cropper, A., Muggleton, S.: Learning higher-order logic programs through abstraction and invention. In: Proceedings of the 25th International Joint Conference Artificial Intelligence, pp. 1418–1424 (2016)Google Scholar
  8. 8.
    Cucchiara, R., Piccardi, M., Mello, P.: Image analysis and rule-based reasoning for a traffic monitoring system. IEEE Trans. Intell. Transp. Syst. 1(2), 119–130 (2000)CrossRefGoogle Scholar
  9. 9.
    Dai, W.-Z., Muggleton, S.H., Zhou, Z.-H.: Logical vision: meta-interpretive learning for simple geometrical concepts. In: Late Breaking Paper Proceedings of the 25th International Conference on Inductive Logic Programming, pp. 1–16. CEUR (2015)Google Scholar
  10. 10.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Proceedings of the 13rd IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Diego, CA, pp. 886–893. IEEE Computer Society (2005)Google Scholar
  11. 11.
    Del Bimbo, A., Vicario, E., Zingoni, D.: A spatial logic for symbolic description of image contents. J. Vis. Lang. Comput. 5(3), 267–286 (1994)CrossRefGoogle Scholar
  12. 12.
    Duan, K., Parikh, D., Crandall, D.J., Grauman, K.: Discovering localized attributes for fine-grained recognition. In: Proceedings of the 25th IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, pp. 3474–3481. IEEE Computer Society (2012)Google Scholar
  13. 13.
    Esposito, F., Ferilli, S., Basile, T.M.A., Di Mauro, N.: Machine learning for digital document processing: from layout analysis to metadata extraction. Mach. Learn. Doc. Anal. Recogn. 90, 105–138 (2008)Google Scholar
  14. 14.
    Farid, R., Sammut, C.: Plane-based object categorisation using relational learning. Mach. Learn. 94(1), 3–23 (2014)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Ferilli, S., Basile, T.M., Esposito, F., Biba, M.: A contour-based progressive technique for shape recognition. In: Proceedings of 2011 International Conference on Document Analysis and Recognition, pp. 723–727 (2011)Google Scholar
  16. 16.
    Galilei, G.: The Herald of the Stars (1610). English translation by Edward Stafford Carlos, Rivingtons, London, 1880; edited by Peter Barker, Byzantium Press, 2004Google Scholar
  17. 17.
    Gregory, R.: Concepts and Mechanics of Perception. Duckworth, London (1974)Google Scholar
  18. 18.
    Gregory, R.: Eye and Brain: The Psychology of Seeing. Oxford University Press, Oxford (1998)Google Scholar
  19. 19.
    Heath, D., Ventura, D.: Before a computer can draw, it must first learn to see. In: Proceedings of the 7th International Conference on Computational Creativity, pp. 172–179 (2016)Google Scholar
  20. 20.
    von Helmholtz, H.: Treatise on Physiological Optics, vol. 3. Dover Publications, New York (1962). Originally published in German in 1825Google Scholar
  21. 21.
    Horn, B.: Obtaining Shape from Shading Information. MIT Press, Cambridge (1989)Google Scholar
  22. 22.
    Hu, R., Xu, H., Rohrbach, M., Feng, J., Saenko, K., Darrell, T.: Natural language object retrieval. In: Proceedings of the 29th IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, pp. 4555–4564. IEEE Computer Society (2016)Google Scholar
  23. 23.
    Lake, B., Salakhutdinov, R., Gross, J., Tenenbaum, J.: One shot learning of simple visual concepts. In: Proceedings of the 33rd Annual Conference of the Cognitive Science Society, pp. 2568–2573 (2011)Google Scholar
  24. 24.
    Lake, B., Salakhutdinov, R., Tenenbaum, J.: Human-level concept learning through probabilistic program induction. Science 350, 1332–1338 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Lampert, C.H., Nickisch, H., Harmeling, S.: Attribute-based classification for zero-shot visual object categorization. IEEE Trans. Pattern Anal. Mach. Intell. 36(3), 453–465 (2014)CrossRefGoogle Scholar
  26. 26.
    Li, Z., Gavves, E., Mensink, T., Snoek, C.G.M.: Attributes make sense on segmented objects. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part VI. LNCS, vol. 8694, pp. 350–365. Springer, Cham (2014). CrossRefGoogle Scholar
  27. 27.
    Lin, D., Dechter, E., Ellis, K., Tenenbaum, J., Muggleton, S.: Bias reformulation for one-shot function induction. In: Proceedings of the 23rd European Conference on Artificial Intelligence (ECAI 2014), pp. 525–530. IOS Press, Amsterdam (2014)Google Scholar
  28. 28.
    Mensink, T., Verbeek, J.J., Csurka, G.: Learning structured prediction models for interactive image labeling. In: The 24th IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs, CO, pp. 833–840. IEEE Computer Society (2011)Google Scholar
  29. 29.
    Muggleton, S.H., Lin, D., Chen, J., Tamaddoni-Nezhad, A.: MetaBayes: Bayesian meta-interpretative learning using higher-order stochastic refinement. In: Zaverucha, G., Santos Costa, V., Paes, A. (eds.) ILP 2013. LNCS (LNAI), vol. 8812, pp. 1–17. Springer, Heidelberg (2014). CrossRefGoogle Scholar
  30. 30.
    Muggleton, S., Lin, D., Pahlavi, N., Tamaddoni-Nezhad, A.: Meta-interpretive learning: application to grammatical inference. Mach. Learn. 94, 25–49 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  31. 31.
    Muggleton, S., Lin, D., Tamaddoni-Nezhad, A.: Meta-interpretive learning of higher-order dyadic datalog: predicate invention revisited. Mach. Learn. 100(1), 49–73 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  32. 32.
    Muggleton, S., Raedt, L.D., Poole, D., Bratko, I., Flach, P., Inoue, K.: ILP turns 20: biography and future challenges. Mach. Learn. 86(1), 3–23 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  33. 33.
    Ojala, T., Pietikainen, M., Mäenpää, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002)CrossRefzbMATHGoogle Scholar
  34. 34.
    Palatucci, M., Pomerleau, D., Hinton, G., Mitchell, T.M.: Zero-shot learning with semantic output codes. In: Advances in Neural Information Processing Systems, vol. 22, pp. 1410–1418. Curran Associates Inc. (2009)Google Scholar
  35. 35.
    Poppe, R.: A survey on vision-based human action recognition. Image Vis. Comput. 28(6), 976–990 (2010)CrossRefGoogle Scholar
  36. 36.
    Shanahan, M.: Perception as abduction: turning sensor data into meaningful representation. Cogn. Sci. 29(1), 103–134 (2005)CrossRefGoogle Scholar
  37. 37.
    Vinyals, O., Blundell, C., Lillicrap, T.P., Kavukcuoglu, K., Wierstra, D.: Matching networks for one shot learning. CoRR abs/1606.04080 (2016)Google Scholar
  38. 38.
    Wielemaker, J., Schrijvers, T., Triska, M., Lager, T.: SWI-prolog. Theor. Pract. Logic Program. 12(1–2), 67–96 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  39. 39.
    Zhang, H., Fritts, J.E., Goldman, S.A.: Image segmentation evaluation: a survey of unsupervised methods. Comput. Vis. Image Underst. 110(2), 260–280 (2008)CrossRefGoogle Scholar
  40. 40.
    Zhang, R., Tai, P., Cryer, J., Shah, M.: Shape-from-shading: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 21(8), 670–706 (1999)zbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.National Key Laboratory for Novel Software TechnologyNanjing UniversityNanjingChina
  2. 2.Department of ComputingImperial College LondonLondonUK
  3. 3.School of Computer and Information TechnologyShanxi UniversityTaiyuanChina
  4. 4.Department of Computer ScienceUniversity of SurreyGuildfordUK

Personalised recommendations