Image and Video Retargetting by Darting

  • Matthew Brand
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5627)


This paper considers the problem of altering an image by imperceptibly adding or removing pixels, for example, to fit a differently shaped frame with minimal loss of interesting content. We show how to construct a family of convex programs that suitably rearrange pixels while minimizing image artifacts and distortions. We call this “darting” on analogy to a tailor’s darts—small edits are discreetly distributed throughout the fabric of the image. We develop a reduction to integer dynamic programming on edit trellises, yielding fast algorithms. One- and two-pass variants of the method have O(1) per-pixel complexity. Of the many edits that darting supports, five are demonstrated here: image retargetting to smaller aspect ratios; adding or moving or removing scene objects while preserving image dimensions; image expansion with gaps filled by a rudimentary form of texture synthesis; temporal video summarization by “packing” motion in time; and an extension to spatial video retargetting that avoids motion artifacts by preserving optical flow.


Texture Synthesis Small Aspect Ratio Nonrigid Motion Rudimentary Form Replica Node 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Setlur, V., Takagi, S., Raskar, R., Gleicher, M., Gooch, B.: Automatic image retargeting. In: Proc. Mobile and Ubiquitous Multimedia (2005)Google Scholar
  2. 2.
    Simakov, D., Caspi, Y., Shechtman, E., Irani, M.: Summarizing visual data using bidirectional similarity. In: Proc. CVPR (2008)Google Scholar
  3. 3.
    Cho, T.S., Butman, M., Avidan, S., Freeman, W.T.: The patch transform and its applications to image editing. In: Proc. CVPR (2008)Google Scholar
  4. 4.
    Wolf, L., Guttmann, M., Cohen-Or, D.: Non-homogeneous content-driven video-retargeting. In: Proc. ICCV (2007)Google Scholar
  5. 5.
    Tutte, W.T.: How to draw a graph. Proc. London Mathematical Society 13(1), 743–767 (1963)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Avidan, S., Shamir, A.: Seam carving for content-aware image retargeting. In: Proc. SIGGRAPH (2007)Google Scholar
  7. 7.
    Rubinstein, M., Shamir, A., Avidan, S.: Improved seam carving for video retargeting. In: Proc. SIGGRAPH (2008)Google Scholar
  8. 8.
    Brand, M.: Graph cut formulation for minimizing artifacts in seam carving—archived MERL seminar whiteboard photo and follow-on email correspondence with S. Avidan (May 2007)Google Scholar
  9. 9.
    Baker, S., Scharstein, D., Lewis, J., Roth, S., Black, M.J., Szeliski, R.: A database and evaluation methodology for optical flow. In: Proc. ICCV 2007 (2007)Google Scholar
  10. 10.
    Li, Z., Ishwar, P., Konrad, J.: Video condensation by ribbon carving. IEEE Transactions on Image Processing (to appear, 2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Matthew Brand
    • 1
  1. 1.Mitsubishi Electric Research LabsUSA

Personalised recommendations