Abstract
The mutual coherence of a matrix, defined as the maximum absolute value of the normalized inner-products between different columns, is an important property that characterizes the similarity between different matrix columns. Redundant matrices with very low mutual coherence are referred to as incoherent redundant matrices which play an important role in mathematical signal processing tasks. The problem of minimizing the mutual coherence in a give matrix space where each matrix has normalized columns is called the coherence optimization problem. In this paper, we transform equivalently the coherence optimization problem as a rank constrained semidefinite \(\ell _{\infty }\)-minimization problem. It is critical to analyze the projection operator onto the nonconvex set in the new matrix optimization constraints, i.e., the nonconvex set of symmetric positive semidefinite matrices whose rank is not greater than a give positive integer. By exploiting the projection operator, we express the nonconvex set mentioned above as the set of zero roots of a Difference of two Convex (DC) functions. For the convex function related to the projection operator in the DC function, we characterize its properties and obtain the concise form of its subdifferential. With the help of the DC function, a new algorithm based on DCA (DC Algorithms) is proposed to solve the Coherence Optimization problem, and thus the proposed algorithm is called DCACO for short. We also study the convergence analysis of DCACO. An advantage of DCACO is that subproblems in each iteration have closed-form solutions. Experimental results demonstrate that DCACO leads to state-of-art performance on generating highly incoherent redundant matrices, and DCACO can also compete with several other algorithms on designing optimized projection matrices for improving the performance of compressed sensing.
Similar content being viewed by others
References
Strohmer, T., Heath, Jr.R.W.: Grassmannian frames with applications to coding and communication. Appl. Comput. Harmon. Anal. 14(3), 257–275 (2003)
Zörlein, H., Bossert, M.: Coherence optimization and best complex antipodal spherical codes. IEEE Trans. Signal Process. 63(24), 6606–6615 (2015)
Goyal, V. K., Kovačević, J., Kelner, J. A.: Quantized frame expansions with erasures. Appl. Comput. Harmon. Anal. 10(3), 203–233 (2001)
Tropp, J. A.: Greed is good: Algorithmic results for sparse approximation. IEEE Trans. Inf. Theory 50(10), 2231–2242 (2004)
Elad, M.: Sparse and Redundant Representations: From Theory to Applications in Signal and Image Processing. Springer, New York (2010)
Foucart, S., Rauhut, H.: A mathematical introduction to compressive sensing birkhäuser Basel (2013)
Eldar, Y. C., Kutyniok, G.: Compressed Sensing: Theory and Applications. Cambridge Univ. Press, Cambridge (2012)
Elad, M.: Optimized projections for compressed sensing. IEEE Trans. Signal Process. 55(12), 5695–5702 (2007)
Xu, J., Pi, Y., Cao, Z.: Optimized projection matrix for compressive sensing. EURASIP J. Adv. Signal Process. 2010(1), 560349 (2010)
Abolghasemi, V., Ferdowsi, S., Sanei, S.: A gradient-based alternating minimization approach for optimization of the measurement matrix in compressive sensing. Signal Process. 92, 999–1009 (2012)
Tsiligianni, E. V., Kondi, L. P., Katsaggelos, A. K.: Construction of incoherent unit norm tight frames with application to compressed sensing. IEEE Trans. Inf. Theory 60(4), 2319–2330 (2014)
Rusu, C., González-Prelcic, N.: Designing incoherent frames through convex techniques for optimized compressed sensing. IEEE Trans. Signal Process. 64(9), 2334–2344 (2016)
Lu, C., Li, H., Lin, Z.: Optimized projections for compressed sensing via direct mutual coherence minimization. Signal Process. 151, 45–55 (2018)
Casazza, P. G., Kutyniok, G.: Finite frames: theory and applications birkhäuser Basel (2013)
Bajwa, W. U., Calderbank, R., Mixon, D. G.: Two are better than one: fundamental parameters of frame coherence. Appl. Comput. Harmon. Anal. 33(1), 58–78 (2012)
Waldron, S. F. D.: An introduction to finite tight frames birkhäuser Basel (2018)
Chen, X., Hardin, D. P., Saff, E. B.: On the search for tight frames of low coherence. J. Fourier Anal. Appl. 27(1), 1–27 (2021)
Welch, L. R: Lower bounds on the maximum cross correlation of signals. IEEE Trans. Inf. Theory 20(3), 397–399 (1974)
Xia, P., Zhou, S., Giannakis, G. B.: Achieving the Welch bound with difference sets. IEEE Trans. Inf. Theory 51(5), 1900–1907 (2005)
Sustik, M. A., Tropp, J. A., Dhillon, I. S., Heath, JrR. W.: On the existence of equiangular tight frames. Linear Algebra Appl. 426, 619–635 (2007)
Renes, J. M.: Equiangular tight frames from paley tournaments. Linear Algebra Appl. 426, 497–501 (2007)
Waldron, S. F. D.: On the construction of equiangular frames from graphs. Linear Algebra Appl. 431(11), 2228–2242 (2009)
Fickus, M., Mixon, D. G., Tremain, J. C.: Steiner equiangular tight frames. Linear Algebra Appl. 436(5), 1014–1027 (2012)
Jasper, J., Mixon, D. G., Fickus, M.: Kirkman equiangular tight frames and codes. IEEE Trans. Inf. Theory 60(1), 170–181 (2014)
Rusu, C., González-Prelcic, N., Heath, JrR. W.: Properties of real and complex ETFs and their application to the design of low coherence frames. Linear Algebra Appl. 508, 81–90 (2016)
Fickus, M., Mixon, D. G.: Tables of the existence of equiangular tight frames. https://arxiv.org/abs/1504.00253 (2016)
Tropp, J. A., Dhillon, I. S., Heath, R. W., Strohmer, T.: Designing structured tight frames via an alternating projection method. IEEE Trans. Inf. Theory 51(1), 188–209 (2005)
Thill, M., Hassibi, B.: Low-coherence frames from group fourier matrices. https://arxiv.org/abs/1509.05739 (2015)
Rusu, C.: Design of incoherent frames via convex optimization. IEEE Signal Process. Lett. 20(7), 673–676 (2013)
Sadeghi, M., Babaie-Zadeh, M.: Incoherent unit-norm frame design via an alternating minimization penalty method. IEEE Signal Process. Lett. 24(1), 32–36 (2017)
Dumitrescu, B.: Designing incoherent frames with only matrix-vector multiplications. IEEE Signal Process. Lett. 24(9), 1265–1269 (2017)
Theobald, C. M.: An inequality for the trace of the product of two symmetric matrices. Math. Proc. Cambridge Philos. Soc. 77(2), 265–267 (1975)
Bauschke, H. H., Combettes, P. L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces, 2nd edn. Springer, New York (2017)
Thi, H. A. L., Dinh, T. P.: DC programming and DCA: thirty years of developments. Math. Program. 169(1), 5–68 (2018)
Duchi, J., Shalev-Shwartz, S., Singer, Y., Chandra, T.: Efficient projections onto the ℓ1-ball for learning in high dimensions. Proc. 25th ICML:272–279 (2008)
Grant, M., Boyd, S.: CVX: matlab software for disciplined convex programming, version 2.0 beta. http://cvxr.com/cvx (2013)
Grant, M., Boyd, S.: Graph implementations for nonsmooth convex programs. Recent advances in learning and control (a tribute to M. Vidyasagar). In: Blondel, V., Boyd, S., Kimura, H. (eds.) Lecture Notes in Control and Information Sciences, pp. 95–110. Springer. http://stanford.edu/boyd/graph_dcp.html (2008)
Chamboulle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imag. Vis. 40(1), 120–145 (2011)
Acknowledgements
The authors would like to thank the editor-in chief and the anonymous reviewers for their valuable comments and suggestions, which help us very much in improving the original version of this paper. The authors would like to thank Professor Bogdan Dumitrescu for providing the source code of ISPM in [31]. The authors also would like to thank Chongyang Wang for improving numerical experiments.
Funding
This research was supported by the National Natural Science Foundation of China under the Grant nos. 61901404, 12031003 and 11771347, and supported by the Nanhu Scholars Program for Young Scholars of XYNU.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no competing interests.
Additional information
Data availability
All data and code generated or used during the study are available at https://github.com/sparseelad/DCACO.
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Yu, Y., Peng, J. DCACO: an algorithm for designing incoherent redundant matrices. Numer Algor 93, 785–813 (2023). https://doi.org/10.1007/s11075-022-01441-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11075-022-01441-5