Image and Video Retargetting by Darting
This paper considers the problem of altering an image by imperceptibly adding or removing pixels, for example, to fit a differently shaped frame with minimal loss of interesting content. We show how to construct a family of convex programs that suitably rearrange pixels while minimizing image artifacts and distortions. We call this “darting” on analogy to a tailor’s darts—small edits are discreetly distributed throughout the fabric of the image. We develop a reduction to integer dynamic programming on edit trellises, yielding fast algorithms. One- and two-pass variants of the method have O(1) per-pixel complexity. Of the many edits that darting supports, five are demonstrated here: image retargetting to smaller aspect ratios; adding or moving or removing scene objects while preserving image dimensions; image expansion with gaps filled by a rudimentary form of texture synthesis; temporal video summarization by “packing” motion in time; and an extension to spatial video retargetting that avoids motion artifacts by preserving optical flow.
KeywordsTexture Synthesis Small Aspect Ratio Nonrigid Motion Rudimentary Form Replica Node
Unable to display preview. Download preview PDF.
- 1.Setlur, V., Takagi, S., Raskar, R., Gleicher, M., Gooch, B.: Automatic image retargeting. In: Proc. Mobile and Ubiquitous Multimedia (2005)Google Scholar
- 2.Simakov, D., Caspi, Y., Shechtman, E., Irani, M.: Summarizing visual data using bidirectional similarity. In: Proc. CVPR (2008)Google Scholar
- 3.Cho, T.S., Butman, M., Avidan, S., Freeman, W.T.: The patch transform and its applications to image editing. In: Proc. CVPR (2008)Google Scholar
- 4.Wolf, L., Guttmann, M., Cohen-Or, D.: Non-homogeneous content-driven video-retargeting. In: Proc. ICCV (2007)Google Scholar
- 6.Avidan, S., Shamir, A.: Seam carving for content-aware image retargeting. In: Proc. SIGGRAPH (2007)Google Scholar
- 7.Rubinstein, M., Shamir, A., Avidan, S.: Improved seam carving for video retargeting. In: Proc. SIGGRAPH (2008)Google Scholar
- 8.Brand, M.: Graph cut formulation for minimizing artifacts in seam carving—archived MERL seminar whiteboard photo and follow-on email correspondence with S. Avidan (May 2007)Google Scholar
- 9.Baker, S., Scharstein, D., Lewis, J., Roth, S., Black, M.J., Szeliski, R.: A database and evaluation methodology for optical flow. In: Proc. ICCV 2007 (2007)Google Scholar
- 10.Li, Z., Ishwar, P., Konrad, J.: Video condensation by ribbon carving. IEEE Transactions on Image Processing (to appear, 2009)Google Scholar