Skip to main content
SpringerLink
Log in

Progressive completion of a panoramic image

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This paper proposes an exemplar-based modified priority belief propagation (MP-BP) method to complete a stitched panorama. The result is an image with a rectangular boundary in which the missing area is filled by a visually plausible background that copies the appearance of the source region. Global optimization is usually preferable to greedy algorithms for image completion. Komodakis (IEEE Trans. Image Process. 16 (11): 2649–2661, 2007) proposed a priority BP method, which is expressed in the form of a discrete global optimization problem with an MRF energy function. However, this priority BP method cannot be directly applied to stitched panoramas completing problem because there are very few useful messages. Instead, a progressive way to expand the boundary is proposed. The main contributions of this study are: (1) it defines a progressive way in which to apply a Markov Random Field (MRF) model to complete panoramas; (2) it defines a priority term that integrates the concepts of “confidence” and “breadth first”, to guide the filling order; and (3) a restricted source region and clustered candidate patches are used, to alleviate computation complexity. A number of examples of real stitched panoramas demonstrate the effectiveness of this algorithm. The results compare favorably with those obtained using existing techniques.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Adobe at http://www.adobe.com/technology/projects/content-aware-fill.html. Access on Oct 2014

  2. Arias P, Facciolo G, Caselles V, Sapiro G (2011) A variational framework for exemplar-based image inpainting. Int J Comput Vis 93(3):319–347. doi:10.1007/s11263-010-0418-7

    Article  MathSciNet  MATH  Google Scholar 

  3. Bertalmio M, Sapiro G, Caselles V, Ballester C (2000) Image inpainting. SIGGRAPH’00 Proc. 27th Annu. Conf. Computer Graphics and Interactive Techniques: 417–424. 10.1145/344779.344972

  4. Choi JH, Hahm CH (2013) An exemplar-based image inpainting method with search region Prior. IEEE 2nd Global Conf. Consumer Electronics: 68–71. 10.1109/GCCE.2013.6664927

  5. Chung SC, Kuan TW, Lu CP, Lin HY (2013) A new approach of image inpainting based on PSO algorithm. IEEE Int. Conf. Orange Technologies: 205–209. 10.1109/ICOT.2013.6521193

  6. Criminisi A, Perez P, Toyama K (2004) Region filling and object removal by exemplar-based image inpainting. IEEE Trans Image Process 13(9):1200–1212. doi:10.1109/TIP.2004.833105

    Article  Google Scholar 

  7. Doretto G, Soatto S (2003) Editable dynamic textures. Comput. Vis. Pattern Recognit. 137–142. 10.1109/CVPR.2003.1211463

  8. Felzenszwalb PF, Huttenlocher DP (2006) Efficient belief propagation for early vision. Int J Comput Vis 70(1):41–54. doi:10.1007/s11263-006-7899-4

    Article  Google Scholar 

  9. Geman D, Reynolds G (1992) Constrained restoration and the recovery of discontinuities. IEEE Trans Pattern Anal Mach Intell 14(3):367–383. doi:10.1109/34.120331

    Article  Google Scholar 

  10. Hassan ET, Abbas HM, Mohamed HK (2013) Image inpainting based on image segmentation and segment slassification. IEEE Int. Conf. Control System, Computing and Engineering, pp 28–33. doi:10.1109/ICCSCE.2013.6719927

    Google Scholar 

  11. Hays J, Efros A (2007) Scene completion using millions of photographs. ACM Trans Graph 26(3):4. doi:10.1145/1276377.1276382

    Article  Google Scholar 

  12. He K, Chang H, Sun J (2013) Rectangling panoramic images via warping. ACM Trans Graph 32(4):79. doi:10.1145/2461912.2462004

    MATH  Google Scholar 

  13. He K, Sun J (2012) Statistics of patch offsets for image completion. Proc. 12th Eur Conf. Computer Vision II : 16–29. 10.1007/978-3-642-33709-3_2

  14. Komodakis N, Tziritas G (2007) Image completion using efficient belief propagation via priority scheduling and dynamic Pruning. IEEE Trans Image Process 16(11):2649–2661. doi:10.1109/TIP.2007.906269

    Article  MathSciNet  Google Scholar 

  15. Kopf J, Kienzle W, Drucker S, Kang SB (2012) Quality prediction for image completion. ACM Trans Graph 31(6):131. doi:10.1145/2366145.2366150

    Google Scholar 

  16. Kopf’s homepage at http://research.microsoft.com/en-us/um/people/kopf/completion/supplementary/ comparison_panos.html Access on Oct 2014

  17. Lee J, Lee DK, Park RH (2012) Robust exemplar-based inpainting algorithm using region segmentation. IEEE Trans Consum Electron 58f(2):553–561. doi:10.1109/TCE.2012.6227460

    Article  Google Scholar 

  18. Liu C, Guo Y, Pan L, Peng Q, Zhang F (2007) Image completion based on views of large displacement. Vis Comput 23:833–841. doi:10.1007/s00371-007-0137-4

    Article  Google Scholar 

  19. Liu Y, Caselles V (2013) Exemplar-based image inpainting using multiscale graph cuts. IEEE Trans Image Process 22(5):1699–1711. doi:10.1109/TIP.2012.2218828

    Article  MathSciNet  Google Scholar 

  20. Martinez-Noriega R, Roumy A (2013) Prior and macro-filling order for image completion. IEEE Int. Conf. Image Processing: 719–723. 10.1109/ICIP.2013.6738148

  21. Martinez-Noriega R, Roumy A, Blanchard G (2012) Exemplar-based image inpainting: fast priority and coherent nearest neighbor search. IEEE Int. Workshop Mach. Learn. Signal Process:1–6. 10.1109/MLSP.2012.6349810

  22. Pritch Y, Kav-Venaki E, Peleg S (2009) Shift-map image editing. IEEE Int. Conf. Computer Vision: 151–158. 10.1109/ICCV.2009.5459159

  23. Weiss Y, Freeman WT (2001) On the optimality of solutions of the max-product belief-propagation algorithm in arbitrary graphs. IEEE Trans Inf Theory 47(2):723–735. doi:10.1109/18.910585

    Article  MathSciNet  MATH  Google Scholar 

  24. Wexler Y, Shechtman E, Irani M (2007) Space-time completion of video. IEEE Trans Pattern Anal Mach Intell 29(3):463–476. doi:10.1109/TPAMI.2007.60

    Article  Google Scholar 

  25. Zhang M, Zhang L, Sun Y, Feng L, Ma W (2005) Auto cropping for digital photographs. IEEE Int Conf Multimed Expo. doi:10.1109/ICME.2005.1521454

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. PRIA. Lab., Department of Computer Science and Information Engineering, Tamkang University, 151, Yingzhuan Road, New Taipei, 25137, Taiwan, Republic of China

    Shwu-Huey Yen & Hao-Yu Yeh

  2. Department of Computer Science and Information Engineering, China University of Science and Technology, 245, Sec. 3, Academia Road, Taipei, 11581, Taiwan, Republic of China

    Hsiao-Wei Chang

Authors
  1. Shwu-Huey Yen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hao-Yu Yeh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hsiao-Wei Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shwu-Huey Yen.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yen, SH., Yeh, HY. & Chang, HW. Progressive completion of a panoramic image. Multimed Tools Appl 76, 11603–11620 (2017). https://doi.org/10.1007/s11042-015-3157-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-015-3157-5

Keywords

Search

Navigation