Abstract
Properties of randomized binary vector representations with adjustable sparseness are investigated. Such representations are formed from input vectors by projecting them using a random matrix with ternary elements {-1, 0, +1}. The accuracy of estimating measures of similarity-difference between initial vectors composed of floating-point numbers and output binary vectors is analyzed. The vector representations obtained can be used to efficiently process large arrays of input multidimensional vectors in applications related to searching, classification, associative memory, etc.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
G. Salton, Automatic Text Processing: The Transformation, Analysis, and Retrieval of Information by Computer, Addison-Wesley, Reading, MA (1989).
I. S. Misuno, D. A. Rachkovskij, S. V. Slipchenko, and A. M. Sokolov, “Searching for text information with the help of vector representations,” Problemy Programmuvannya, No. 4, 50–59 (2005).
E. M. Kussul, Associative Neuron-Like Structures [in Russian], Naukova Dumka, Kyiv (1992).
D. A. Rachkovskij, “Modeling of thinking as a way of increasing intellectuality of information technologies and systems,” System Technologies, No. 4 (57), 154–162 (2008).
D. A. Rachkovskij, “Linear classifiers based on binary distributed representations,” Intern. J. Inform. Theories & Appl., 14, No. 3, 270–274 (2007).
D. A. Rachkovskij, I. S. Misuno, E. G. Revunova, S. V. Slipchenko, and A. M. Sokolov, “Conception and methods of distributed neural net information representation in AI problems,” in: 14th Intern. Conf. “Problems of neurocybernetics,” Vol. 2, Rostov-on-Don, Russia (2005), pp. 30–33.
D. A. Rachkovsky, S. V. Slipchenko, A. A. Frolov, and D. Gusek, “Resolution of binary coding of real-valued vectors by hyperrectangular receptive fields,” Cybernetics and Systems Analysis, Vol. 41, No. 5, 635–646 (2005).
P. Kanerva, G. Sjodin, J. Kristoferson, R. Karlsson, B. Levin, A. Holst, J. Karlgren, and M. Sahlgren, “Computing with large random patterns,” in: Foundations of Real-World Intelligence, CSLI Publications, Stanford, California (2001), pp. 251–311.
W. B. Johnson and J. Lindenstrauss, “Extensions of Lipshitz mapping into Hilbert space,” Contemporary Mathemat., 26, 189–206 (1984).
S. S. Vempala, The Random Projection Method, American Mathemat. Soc., Providence, RI (2004).
D. Achlioptas, “Database-friendly random projections: Johnson–Lindenstrauss with binary coins,” J. of Comput. and System Sci., 66, No. 4, 671–687 (2003).
M. Charikar, “Similarity estimation techniques from rounding algorithms,” in: ACM Symp. on Theory of Comput., 1, ACM, Montreal, Canada (2002), pp. 380–388.
K. Liu, H. Kargupta, and J. Ryan, “Random projection-based multiplicative data perturbation for privacy preserving distributed data mining,” IEEE Transact. on Knowledge and Data Engineer., 18, No. 1, 92–106 (2006).
P. Li, T. J. Hastie, and K. W. Church, “Very sparse random projections,” in: 12th ACM SIGKDD Intern. Conf. on Knowledge Discovery and Data Mining, ACM Press, Philadelphia, PA, USA (2006), pp. 287–296.
X. G. Michel and P. W. David, “Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming,” J. of the Associat. for Comput. Machinery, 42, No. 6, 1115–1145 (1995).
D. A. Rachkovskij, “Representation and processing of structures with binary sparse distributed codes,” IEEE Trans. on Knowledge and Data Engineer. 13, No. 2, 261–276 (2001).
D. A. Rachkovskij, “Some approaches to analogical mapping with structure sensitive distributed representations,” J. of Experiment. and Theoret. Artificial Intell., 16, No. 3, 125–145 (2004).
D. A. Rachkovskij and E. M. Kussul, “Binding and normalization of binary sparse distributed representations by context-dependent thinning,” Neural Comput., 13, No. 2, 411–452 (2001).
D. A. Rachkovskii, S. V. Slipchenko, E. M. Kussul’, and T. N. Baidyk, “A binding procedure for distributed binary data representations,” Cybernetics and Systems Analysis, Vol. 41, No. 3, 319–331 (2005).
A. A. Frolov, D. Husek, and D. A. Rachkovskii, “Time of searching for similar binary vectors in associative memory,” Cybernetics and Systems Analysis, Vol. 42, No. 5, 615–623 (2006).
E. S. Venttsel’, Probability Theory [in Russian], Nauka, Moscow (1969).
E. M. Kussul, T. N. Baidyk, L. M. Kasatkina, and V. V. Lukovich, “Rosenblatt perceptrons for handwritten digit recognition,” in: Intern. Joint Conf. on Neural Networks, IEEE, 2, Washington, DC (2001), pp. 1516–1521.
E. Kussul, T. Baidyk, D. Wunsch, O. Makeyev, and A. Martin, “Permutation coding technique for image recognition systems,” IEEE Transact. on Neural Networks, 17, No. 6, 1566–1579 (2006).
I. S. Misuno, D. A. Rachkovskij, and S. V. Slipchenko, “Experimental investigations of handwritten digit classification,” System Technologies, No. 4 (39), 110–133 (2005).
N. M. Amosov, T. N. Baidyk, A. D. Goltsev, A. M. Kasatkin, L. M. Kasatkina, E. M. Kussul, and D. A. Rachkovskii, Neurocomputers and Intelligent Robots [in Russian], Naukova Dumka, Kyiv (1991).
Author information
Authors and Affiliations
Corresponding author
Additional information
Translated from Kibernetika i Sistemnyi Analiz, No. 1, pp. 175–187, January–February 2012.
Rights and permissions
About this article
Cite this article
Rachkovskij, D.A., Misuno, I.S. & Slipchenko, S.V. Randomized projective methods for the construction of binary sparse vector representations. Cybern Syst Anal 48, 146–156 (2012). https://doi.org/10.1007/s10559-012-9384-0
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10559-012-9384-0