Advertisement

Dual Sum-Product Networks Autoencoding

  • Shengsheng Wang
  • Hang Zhang
  • Jiayun Liu
  • Qiang-yuan Yu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11061)

Abstract

Sum-Product Networks (SPNs) are a new class of deep probabilistic model allowing tractable and exact inference. Recently SPNs have been successfully employed as autoencoder framework in Representation Learning. However, SPNs autoencoding mechanism ignores the model structural duality and train the models separately and independently. In this paper, we propose the Dual-SPNs autoencoding mechanism which design model structure as a dual close loop. This approach training the models simultaneously, and explicitly exploiting their structural duality correlation to guide the training process. As shown in extensive multilabel classification experiments, Dual-SPNs autoencoding mechanism prove highly competitive against the ones employing SPNs autoencoding mechanism and other stacked autoencoder architectures.

Keywords

Sum-Product Networks Dual learning Representation Learning Multi-Label Classification 

Notes

Acknowledgments

This work is supported by the National Natural Science Foundation of China (61472161), Science & Technology Development Project of Jilin Province (20180101334JC).

References

  1. 1.
    Choi, A., Darwiche, A.: On relaxing determinism in arithmetic circuits. In: Thirty-Fourth International Conference on Machine Learning, pp. 825–833. ACM, Sydney (2017)Google Scholar
  2. 2.
    Poon, H., Domingos, P.: Sum-product networks: a new deep architecture. In: The 27th Conference on Uncertainty in Artificial Intelligence, pp. 337–346. AUAI, Barcelona (2011)Google Scholar
  3. 3.
    Cheng, W., Kok, S., Pham, H.: Language modeling with sum-product networks. In: Fifteenth Annual Conference of the International Speech Communication Association, pp. 2098–2102. ISSN, Singapore (2014)Google Scholar
  4. 4.
    Amer, M.R., Todorovic, S.: Sum product networks for activity recognition. IEEE Trans. Pattern Anal. Mach. Intell. 38(4), 800–813 (2016)CrossRefGoogle Scholar
  5. 5.
    Vergari, A., Peharz, R.: Sum-product autoencoding: encoding and decoding representations using sum-product networks. In: The Thirty-Second Conference on Artificial Intelligence. AAAI, New Orleans (2018)Google Scholar
  6. 6.
    He, D., Xia, Y., Qin, T., et al.: Dual learning for machine translation. In: The Thirtieth Annual Conference on Neural Information Processing Systems, pp. 820–828. MIT Press, Barcelona (2016)Google Scholar
  7. 7.
    Yi, Z., Zhang, H., Tan, P., et al.: DualGAN: unsupervised dual learning for image-to-image translation. In: International Conference on Computer Vision, pp. 2868–2876. IEEE, Venice (2017)Google Scholar
  8. 8.
    Gens, R., Pedro, D.: Learning the structure of sum-product networks. In: 30th International Conference on Machine Learning, pp. 873–880. ACM, Atlanta (2013)Google Scholar
  9. 9.
    Gens, R., Domingos, P.: Discriminative learning of sum-product networks, In: 26th Advances in Neural Information Processing Systems, pp. 3239–3247. MIT Press, Lake Tahoe (2012)Google Scholar
  10. 10.
    Vergari, A., Di Mauro, N., Esposito, F.: Simplifying, regularizing and strengthening sum-product network structure learning. In: Appice, A., Rodrigues, P.P., Santos Costa, V., Gama, J., Jorge, A., Soares, C. (eds.) ECML PKDD 2015. LNCS, vol. 9285, pp. 343–358. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-23525-7_21CrossRefGoogle Scholar
  11. 11.
    Kingma, D., Adam, J.B.: A method for stochastic optimization. Comput. Sci. (2014)Google Scholar
  12. 12.
    Finley, T., Joachims, T.: Training structural SVMs when exact inference is intractable. In: International Conference on Machine Learning, pp. 304–311. ACM, Helsinki (2008)Google Scholar
  13. 13.
    Germain, M., Gregor, K., Murray, I., et al.: MADE: masked autoencoder for distribution estimation. In: The 32nd International Conference on Machine Learning, pp. 881–889. ACM, Lille (2015)Google Scholar
  14. 14.
    Wicker, J., Tyukin, A., Kramer, S.: A nonlinear label compression and transformation method for multi-label classification using autoencoders. In: Bailey, J., Khan, L., Washio, T., Dobbie, G., Huang, J.Z., Wang, R. (eds.) PAKDD 2016. LNCS, vol. 9651, pp. 328–340. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-31753-3_27CrossRefGoogle Scholar
  15. 15.
    Ueda, N., Saito, K.: Single-shot detection of multiple categories of text using parametric mixture models. In: Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 626–631. ACM, Alberta (2001)Google Scholar
  16. 16.
    Cano, A., Luna, J.M., Gibaja, E.L.: LAIM discretization for multi-label data. Inf. Sci. 330, 370–384 (2016)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Shengsheng Wang
    • 1
  • Hang Zhang
    • 2
  • Jiayun Liu
    • 1
  • Qiang-yuan Yu
    • 1
  1. 1.College of Computer Science and TechnologyJilin UniversityChangchunChina
  2. 2.College of SoftwareJilin UniversityChangchunChina

Personalised recommendations