Skip to main content
SpringerLink
Log in

On the empirical justification of theoretical heuristic transference and learning

  • Published:
Information Systems Frontiers Aims and scope Submit manuscript

Abstract

The solution of intractable problems implies the use of heuristics. Quantum computers may find use for optimization problems, but have yet to solve any NP-hard problems. This paper demonstrates results in game theory for domain transference and the reuse of problem-solving knowledge through the application of learned heuristics. It goes on to explore the possibilities for the acquisition of heuristics for the solution of the NP-hard TSP problem. Here, it is found that simple heuristics (e.g., pairwise exchange) often work best in the context of more or less sophisticated experimental designs. Often, these problems are not amenable to exclusive logic solutions; but rather, require the application of hybrid approaches predicated on search. In general, such approaches are based on randomization and supported by parallel processing. This means that heuristic solutions emerge from attempts to randomize the search space. The paper goes on to present a constructive proof of the unbounded density of knowledge in support of the Semantic Randomization Theorem (SRT). It highlights this result and its potential impact upon the community of machine learning researchers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  • Behbood, V., Lu, J., & Zhang, G. (2011). Long term bank failure prediction using fuzzy refinement-based transductive transfer learning. In IEEE International Conference on Fuzzy Systems (pp. 2676–2683). Taiwan: IEEE.

    Google Scholar 

  • Behbood, V., Lu, J., & Zhang, G. (2013a). Fuzzy bridged refinement domain adaptation: Long-term bank failure prediction. International Journal of Computational Intelligence and Applications, 12(01).

  • Behbood, V., Lu, J., & Zhang, G. (2013b). Text categorization by fuzzy domain adaptation. In IEEE International Conference on Fuzzy Systems. Hyderabad: IEEE.

    Google Scholar 

  • Behbood, V., Lu, J., & Zhang, G. (2014). Fuzzy refinement domain adaptation for long term prediction in banking ecosystem. IEEE Transactions on Industrial Informatics, 10(2), 1637–1646.

    Article  Google Scholar 

  • Caruana, R. (1993). Multitask learning: a knowledge-based source of inductive bias. In Proceedings of the tenth international conference on machine learning (pp. 41–48). MA, USA.

  • Caruana, R. (1997). Multitask learning. Machine learning, 28(1), 41–75.

    Article  Google Scholar 

  • Celiberto, L.A., Matsuura, J.P., Lopez de Mantaras, R., & Bianchi, R.A.C. (2011). Using cases as heuristics in reinforcement learning: a transfer learning application. In International Joint Conference on Artificial Intelligence. Barcelona.

  • Chaitin, G.J. (1975). Randomness and mathematical proof. Scientific American, 232(5), 47–52.

    Article  Google Scholar 

  • Chopra, S., Balakrishnan, S., & Gopalan, R. (2013). DLID: Deep Learning for domain adaptation by interpolating between domains. In ICML Workshop on challenges in representation learning, Vol. 2. Atlanta.

  • Cireşan, D.C., Meier, U., & Schmidhuber, J. (2012). Transfer learning for latin and chinese characters with deep neural networks. In International Joint Conference on Neural Networks (IJCNN). Australia: IEEE.

    Google Scholar 

  • Cousins, N., & Eccles, J.C. (1985). Nobel Prize Conversations with Sir John Eccles, Roger Sperry, Ilya Prigogine, Brian Josephson. CA: Saybrook Publishers.

    Google Scholar 

  • Eccles, J.C. (1976). The understanding of the brain. New York: McGraw-Hill Co. 2d ed.

    Google Scholar 

  • Feigenbaum, E.A., & McCorduck, P. (1983). The fifth generation: Artificial intelligence and Japan’s computer challenge to the world reading. MA: Addison-Wesley Pub. Co.

    Google Scholar 

  • Fogel, D.B. (2001). Blondie24: Playing at the edge of AI. Mountain View: Morgan Kaufmann Publishers, Inc.

    Google Scholar 

  • Huang, J.-T., Li, J., Yu, D., Deng, L., & Gong, Y. (2013). Cross-language knowledge transfer using multilingual deep neural network with shared hidden layers. Vancouver: IEEE.

    Book  Google Scholar 

  • Kandaswamy, C., Silva, L.M., Alexandre, L.A., Santos, J.M., & de Sá, J.M. (2014). Improving deep neural network performance by reusing features trained with transductive transference. In 24Th international conference on artificial neural networks and machine learning–ICANN (pp. 265–272). Hampurg: Springer.

    Google Scholar 

  • Kfoury, A.J., Moll, R.N., & Arbib, M.A. (1982). A programming approach to computability. New York: Springer Verlag Inc.

    Book  Google Scholar 

  • Koçer, B., & Arslan, A. (2010). Genetic transfer learning. Expert Systems with Applications, 37(10), 6997–7002.

    Article  Google Scholar 

  • Lin, J.-H., & Vitter, J.S. (1991). Complexity results on learning by neural nets. Machine Learning, 6(3), 211–230.

    Article  Google Scholar 

  • Lu, J., Behbood, V., Hao, P., Zuo, H., Xue, S., & Zhang, G. (2015). Transfer learning using computational intelligence: a survey. Knowledge-Based Systems, 80, 14–23.

    Article  Google Scholar 

  • Luis, R., Sucar, L.E., & Morales, E.F. (2010). Inductive transfer for learning bayesian networks. Machine learning, 79(1-2), 227– 255.

    Article  Google Scholar 

  • Ma, Y., Luo, G., Zeng, X., & Chen, A. (2012). Transfer learning for cross-company software defect prediction. Information and Software Technology, 54(3), 248–256.

    Article  Google Scholar 

  • Michalski, R.S., Carbonell, J.G., & Mitchell, T.M. (1983). Machine learning: an artificial intelligence approach (volume 1). Palo Alto: Tioga Publishing Co.

    Book  Google Scholar 

  • Mitchell, T.M., Carbonell, J.G., & Michalski, R.S. (Eds.) (1986). Machine Learning: A guide to current research. New York: Springer-Verlag Inc.

    Google Scholar 

  • Niculescu-Mizil, A., & Caruana, R. (2007). Inductive Transfer for bayesian network structure learning. In 11Th international Conference on Artificial Intelligence and Statistics, Puerto Rico (pp. 339–346).

    Google Scholar 

  • Nilsson, N.J. (1980). Principles of artificial intelligence. Mountain View: Morgan Kaufmann Publishers Inc.

    Google Scholar 

  • Oyen, D., & Lane, T. (2013). Bayesian discovery of multiple bayesian networks via transfer learning. In 13Th international conference on data mining (ICDM) (pp. 577–586). Dallas: IEEE.

    Google Scholar 

  • Pan, S.J., & Yang, Q. (2010). A survey on transfer learning. IEEE Transactions Knowledge and Data Engineering, 22(10), 1345–1359.

    Article  Google Scholar 

  • Rubin, S.H. (2004). On the auto-randomization of knowledge. In: Information Reuse and Integration. In Proceedings of the 2004 IEEE International Conference On Information Reuse and Integration (pp. 308–313): IEEE.

  • Rubin, S.H. (2007). On randomization and discovery. Information Sciences, 177(1), 170–191.

    Article  Google Scholar 

  • Rubin, S.H. (2012). On creativity and intelligence in computational systems. In Advances in Reasoning-Based Image Processing Intelligent Systems (pp. 383–421). Berlin: Springer.

    Chapter  Google Scholar 

  • Rubin, S.H., & Bouabana-Tebibel, T. (2015a). Naval intelligent authentication and support through randomization and transformative search. In To appear New approaches in intelligent control and image analysis - techniques, methodologies and applications. Intelligent Systems Reference Library: Springer.

  • Rubin, S.H., & Bouabana-Tebibel, T. (2015b). NNCS: Randomization and informed search for novel naval cyber strategies. In Recent Advances in Computational Intelligence in Defense and Security. Studies in Computational Intelligence, (Vol. 621 pp. 193–223): Springer.

  • Rubin, S.H., Bouabana-Tebibel, T., Hoadjli, Y., Habib, K., & Belamiri, B.Y. (2015). On heuristic randomization and reuse as an enabler of domain transference. In The 16th IEEE international conference on information reuse and integration IEEE IRI 2015. San Francisco, USA, August 13–15 (pp. 411–418).

    Google Scholar 

  • Rubin, S.H., Chen, S.-C., Law, J.B., & Lee, G.K. (2005). On the inherent necessity of heuristic proofs. In 2005 IEEE International Conference on Systems, Man and Cybernetics, (Vol. 4 pp. 3890–3896): IEEE.

  • Rubin, S.H., Murthy, S.N.J., Smith, M.H., & Trajković, L. (2004). Kaser: knowledge amplification by structured expert randomization. IEEE Transactions on Systems, Man, and Cybernetics Part B: Cybernetics, 34(6), 2317–2329.

    Article  Google Scholar 

  • Shell, J. (2013). Fuzzy transfer learning. PhD thesis: De Montfort University.

  • Shell, J., & Coupland, S. (2012). Towards fuzzy transfer learning for intelligent environments, (Vol. 7683 pp. 145–160): Springer.

  • Shell, J., & Coupland, S. (2015). Fuzzy transfer learning: methodology and application. Information Sciences, 293, 59–79.

    Article  Google Scholar 

  • Silver, D.L., & Poirier, R. (2007). Context-sensitive MTL networks for machine lifelong learning. In 20th florida artificial intelligence research society conference (pp. 628–633). Key West.

  • Swietojanski, P., Ghoshal, A., & Renals, S. (2012). Unsupervised cross-lingual knowledge transfer in DNN-based LVCSR. In IEEE Workshop on spoken language technology (SLT) (pp. 246–251). Miami: IEEE.

    Chapter  Google Scholar 

  • Tesar, B. (2000). Learnability in optimality theory: Mit Press.

  • Uspenskii, V.A. (1987). Gödel’s incompleteness theorem. Translated from Russian. Ves Mir Publishers: Moscow.

    Google Scholar 

Download references

Acknowledgments

Stuart would like to extend special thanks to Michael Leyton, Rutgers Department of Mathematics, for reviewing his tremendously exciting Theory of Randomization (Rubin 2007) and to the kind genius, Sukarno Mertoguno for applicative discussions on cybersecurity. Stuart would also like to extend his thanks to the Office of Naval Research (ONR) for providing financial backing for this research with the support of his Branch Head, Ken Simonsen. This work was produced by a U.S. government employee as part of his official duties and is not subject to copyright. It is approved for public release with an unlimited distribution.

Author information

Authors and Affiliations

  1. Space and Naval Warfare Systems Center Pacific, San Diego, CA, 92152-5001, USA

    Stuart H. Rubin

  2. Ecole nationale Supérieure d’Informatique, LCSI Laboratory, Alger, Algeria

    Thouraya Bouabana-Tebibel & Yasmine Hoadjli

Authors
  1. Stuart H. Rubin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thouraya Bouabana-Tebibel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yasmine Hoadjli
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yasmine Hoadjli.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rubin, S.H., Bouabana-Tebibel, T. & Hoadjli, Y. On the empirical justification of theoretical heuristic transference and learning. Inf Syst Front 18, 981–994 (2016). https://doi.org/10.1007/s10796-016-9661-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10796-016-9661-y

Keywords

Search

Navigation