Abstract
Fitting a matrix of a given rank to data in a least squares sense can be done very effectively using 2nd order methods such as Levenberg-Marquardt by explicitly optimizing over a bilinear parameterization of the matrix. In contrast, when applying more general singular value penalties, such as weighted nuclear norm priors, direct optimization over the elements of the matrix is typically used. Due to non-differentiability of the resulting objective function, first order sub-gradient or splitting methods are predominantly used. While these offer rapid iterations it is well known that they become inefficient near the minimum due to zig-zagging and in practice one is therefore often forced to settle for an approximate solution.
In this paper we show that more accurate results can in many cases be achieved with 2nd order methods. Our main result shows how to construct bilinear formulations, for a general class of regularizers including weighted nuclear norm penalties, that are provably equivalent to the original problems. With these formulations the regularizing function becomes twice differentiable and 2nd order methods can be applied. We show experimentally, on a number of structure from motion problems, that our approach outperforms state-of-the-art methods.
This work was supported by the Swedish Research Council (grants no. 2015-05639, 2016-04445 and 2018-05375), the Swedish Foundation for Strategic Research (Semantic Mapping and Visual Navigation for Smart Robots) and the Wallenberg AI, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Akhter, I., Sheikh, Y., Khan, S., Kanade, T.: Nonrigid structure from motion in trajectory space. In: Proceedings of the 21st International Conference on Neural Information Processing Systems, NIPS 2008, pp. 41–48. Curran Associates Inc., Red Hook (2008)
Bach, F.: Convex relaxations of structured matrix factorizations, September 2013
Basri, R., Jacobs, D., Kemelmacher, I.: Photometric stereo with general, unknown lighting. Int. J. Comput. Vision 72(3), 239–257 (2007). https://doi.org/10.1007/s11263-006-8815-7
Bhatia, R., Jain, T.: On symplectic eigenvalues of positive definite matrices. J. Math. Phys. 56(11), 112201 (2015)
Birkhoff, G.: Tres observaciones sobre el algebra lineal. Universidad Nacional de Tucuman Revista. Serie A. 5, 137–151 (1946)
Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011)
Bregler, C., Hertzmann, A., Biermann, H.: Recovering non-rigid 3D shape from image streams. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2000)
Buchanan, A.M., Fitzgibbon, A.W.: Damped newton algorithms for matrix factorization with missing data. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2005)
Cabral, R., De la Torre, F., Costeira, J.P., Bernardino, A.: Unifying nuclear norm and bilinear factorization approaches for low-rank matrix decomposition. In: International Conference on Computer Vision (ICCV) (2013)
Candès, E.J., Li, X., Ma, Y., Wright, J.: Robust principal component analysis? J. ACM 58(3), 11:1–11:37 (2011)
Candès, E.J., Recht, B.: Exact matrix completion via convex optimization. Found. Comput. Math. 9(6), 717–772 (2009). https://doi.org/10.1007/s10208-009-9045-5
Canyi, L., Tang, J., Yan, S., Lin, Z.: Generalized nonconvex nonsmooth low-rank minimization. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014)
Canyi, L., Tang, J., Yan, S., Lin, Z.: Nonconvex nonsmooth low-rank minimization via iteratively reweighted nuclear norm. IEEE Trans. Image Process. 25(2), 829–839 (2015)
Dai, Y., Li, H., He, M.: A simple prior-free method for non-rigid structure-from-motion factorization. Int. J. Comput. Vision 107(2), 101–122 (2014). https://doi.org/10.1007/s11263-013-0684-2
Fazel, M., Hindi, H., Boyd, S.P.: A rank minimization heuristic with application to minimum order system approximation. In: American Control Conference (2001)
Garg, R., Roussos, A., Agapito, L.: A variational approach to video registration with subspace constraints. Int. J. Comput. Vision 104(3), 286–314 (2013). https://doi.org/10.1007/s11263-012-0607-7
Garg, R., Roussos, A., de Agapito, L.: Dense variational reconstruction of non-rigid surfaces from monocular video. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2013)
Gu, S., Xie, Q., Meng, D., Zuo, W., Feng, X., Zhang, L.: Weighted nuclear norm minimization and its applications to low level vision. Int. J. Comput. Vision 121(2), 183–208 (2016). https://doi.org/10.1007/s11263-016-0930-5
Haeffele, B.D., Vidal, R.: Structured low-rank matrix factorization: global optimality, algorithms, and applications. IEEE Trans. Pattern Anal. Mach. Intell. 42(6), 1468–1482 (2020)
Hyeong Hong, J., Fitzgibbon, A.: Secrets of matrix factorization: approximations, numerics, manifold optimization and random restarts. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015)
Hong, J.H., Zach, C., Fitzgibbon, A., Cipolla, R.: Projective bundle adjustment from arbitrary initialization using the variable projection method. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 477–493. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_29
Hong, J.H., Zach, C., Fitzgibbon, A., Cipolla, R.: Projective bundle adjustment from arbitrary initialization using the variable projection method. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)
Hu, Y., Zhang, D., Ye, J., Li, X., He, X.: Fast and accurate matrix completion via truncated nuclear norm regularization. IEEE Trans. Pattern Anal. Mach. Intell. 35(9), 2117–2130 (2013)
Hyeong Hong, J., Zach, C.: pOSE: pseudo object space error for initialization-free bundle adjustment. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018
Jensen, S.H.N., Del Bue, A., Doest, M.E.B., Aanæs, H.: A benchmark and evaluation of non-rigid structure from motion (2018)
Krechetov, M., Marecek, J., Maximov, Y., Takac, M.: Entropy-penalized semidefinite programming. In: Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, pp. 1123–1129. International Joint Conferences on Artificial Intelligence Organization, July 2019
Kumar, S.: Non-rigid structure from motion: prior-free factorization method revisited. In: The IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 51–60, March 2020
Larsson, V., Olsson, C.: Convex low rank approximation. Int. J. Comput. Vision 120(2), 194–214 (2016). https://doi.org/10.1007/s11263-016-0904-7
Lewis, A.S.: The convex analysis of unitarily invariant matrix functions. J. Convex Anal. 2(1), 173–183 (1995)
Li, H., Lin, Z.: Provable accelerated gradient method for nonconvex low rank optimization. Mach. Learn. 109(1), 103–134 (2019). https://doi.org/10.1007/s10994-019-05819-w
Oh, T.H., Tai, Y.W., Bazin, J.C., Kim, H., Kweon, I.S.: Partial sum minimization of singular values in robust PCA: algorithm and applications. IEEE Trans. Pattern Anal. Mach. Intell. 38(4), 744–758 (2016)
Olsson, C., Enqvist, O.: Stable structure from motion for unordered image collections. In: Heyden, A., Kahl, F. (eds.) SCIA 2011. LNCS, vol. 6688, pp. 524–535. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-21227-7_49
Oymak, S., Mohan, K., Fazel, M., Hassibi, B.: A simplified approach to recovery conditions for low rank matrices. In: IEEE International Symposium on Information Theory Proceedings (ISIT), pp. 2318–2322 (2011)
Paladini, M., Del Bue, A., Stosic, M., Dodig, M., Xavier, J., Agapito, L.: Factorization for non-rigid and articulated structure using metric projections. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2898–2905 (2009)
Recht, B., Fazel, M., Parrilo, P.A.: Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization. SIAM Rev. 52(3), 471–501 (2010)
Russell, C., Fayad, J., Agapito, L.: Energy based multiple model fitting for non-rigid structure from motion. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3009–3016, July 2011
Shang, F., Cheng, J., Liu, Y., Luo, Z., Lin, Z.: Bilinear factor matrix norm minimization for robust PCA: algorithms and applications. IEEE Trans. Pattern Anal. Mach. Intell. 40(9), 2066–2080 (2018)
Stoyanov, D., Mylonas, G.P., Deligianni, F., Darzi, A., Yang, G.Z.: Soft-tissue motion tracking and structure estimation for robotic assisted MIS procedures. In: Duncan, J.S., Gerig, G. (eds.) MICCAI 2005. LNCS, vol. 3750, pp. 139–146. Springer, Heidelberg (2005). https://doi.org/10.1007/11566489_18
Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: a factorization method. Int. J. Comput. Vision 9(2), 137–154 (1992). https://doi.org/10.1007/BF00129684
Valtonen Ornhag, M., Olsson, C., Heyden, A.: Bilinear parameterization for differentiable rank-regularization. In: The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2020
Varol, A., Salzmann, M., Tola, E., Fua, P.: Template-free monocular reconstruction of deformable surfaces. In: International Conference on Computer Vision (ICCV), pp. 1811–1818, November 2009
Wang, N., Yao, T., Wang, J., Yeung, D.-Y.: A probabilistic approach to robust matrix factorization. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7578, pp. 126–139. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33786-4_10
Xiao, J., Chai, J., Kanade, T.: A closed-form solution to non-rigid shape and motion recovery. In: Pajdla, T., Matas, J. (eds.) ECCV 2004. LNCS, vol. 3024, pp. 573–587. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-24673-2_46
Xu, C., Lin, Z., Zha, H.: A unified convex surrogate for the Schatten-p norm. In: Proceedings of the Conference on Artificial Intelligence (AAAI) (2017)
Yan, J., Pollefeys, M.: A factorization-based approach for articulated nonrigid shape, motion and kinematic chain recovery from video. IEEE Trans. Pattern Anal. Mach. Intell. 30(5), 865–877 (2008)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Iglesias, J.P., Olsson, C., Valtonen Örnhag, M. (2020). Accurate Optimization of Weighted Nuclear Norm for Non-Rigid Structure from Motion. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, JM. (eds) Computer Vision – ECCV 2020. ECCV 2020. Lecture Notes in Computer Science(), vol 12372. Springer, Cham. https://doi.org/10.1007/978-3-030-58583-9_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-58583-9_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-58582-2
Online ISBN: 978-3-030-58583-9
eBook Packages: Computer ScienceComputer Science (R0)