Abstract
There is an increasing trend towards panel displays in consumer electronics, and they are already replacing conventional Cathode Ray Tube (CRT) displays due to their various advantages. However, the main problem of the panel displays, namely motion blur, still remains unsolved. This shortcoming should be overcome efficiently to satisfy increasing demands of viewers such as artifact-free interpolation in dynamic videos. Among many frame-rate up conversion (FRUC) methods that address this problem, motion-compensated frame interpolation (MCFI) algorithms yield superior results with relatively less artifacts. Conventional MCFI techniques utilize block-based translational motion models and, in general, linear interpolation schemes. These methods, however, suffer from blocking artifacts especially at object boundaries despite several attempts to avoid them. Region-based methods tackle this problem by segmenting homogeneous, or smoothly varying, motion regions that are supposed to correspond real objects (or their parts) in the scene. In this chapter, two region-based MCFI methods that adopt 2D homography and 3D rigid body motion models are presented in the order of increasing complexity. As opposed to the conventional MCFI approaches where motion model interpolation is performed in the induced 2D motion parameter space, the common idea behind both methods is to perform the interpolation in the parameter space of the original 3D motion and structure elements of the scene. Experimental results suggest that the proposed algorithms achieve visually pleasing results without halo effects on dynamic scenes with complex motion.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Chen, H., Kim, S.S., Lee, S.H., Kwon, O.J., Sung, J.H.: Nonlinearity Compensated Smooth Frame Insertion for Motion-blur Reduction in LCD. In: Proc. IEEE 7th Multimedia Signal Processing Workshop, Shangai, China (2005)
Ha, T., Lee, S., Kim, J.: Motion Compensated Frame Interpolation by New Block-based Motion Estimation Algorithm. IEEE Transactions on Consumer Electronics 50(2), 752–759 (2004)
Choi, B.D., Han, J.W., Kim, C.S., Ko, S.J.: Motion-Compensated Frame Interpolation Using Bilateral Motion Estimation and Adaptive Overlapped Block Motion Compensation. IEEE Transactions on Circuits and Systems for Video Technology 17(4), 407–416 (2007)
Fujiwara, S., Taguchi, A.: Motion-compensated Frame Rate Up-conversion Based on Block Matching Algorithm with Multi-size Blocks. In: Proc. IEEE International Symposium on Intelligent Signal Processing and Communication Systems, Hong Kong, China (2005)
Lee, S.H., Yang, S., Jung, Y.Y., Park, R.H.: Adaptive Motion-compensated Interpolation for Frame Rate Up-conversion. In: Proc. IEEE International Conference on Computers in Education, Auckland, New Zeland (2002)
Jeon, B.W., Lee, G.I., Lee, S.H., Park, R.H.: Coarse-to-Fine Frame Interpolation for Frame Rate Up-Conversion Using Pyramid Structure. IEEE Transactions on Consumer Electronics 49(3), 499–508 (2003)
Biswas, M., Nguyen, T.: A Novel Motion Estimation Algorithm Using Phase Plane Correlation for Frame Rate Conversion. In: Proc. IEEE Conference Record of the Thirty-Sixth Asilomar Conference on Signals, Systems and Computers (2002)
Choi, B.T., Lee, S.H., Ko, S.J.: New Frame Rate Up-Conversion Using Bi-Directional Motion Estimation. IEEE Transactions on Consumer Electronics 46(3), 603–609 (2000)
Mishima, N., ltoh, G.: Novel Frame Interpolation Method For Hold-Type Displays. In: Proc. IEEE International Conference on Image Processing, Singapore (2004)
Sugiyama, K., Aoki, T., Hangai, S.: Motion Compensated Frame Rate Conversion Using Normalized Motion Estimation. In: Proc. IEEE Workshop on Signal Processing Systems Design and Implementation, Athens, Greece (2005)
Chen, T.: Adaptive Temporal Interpolation Using Bi-directional Motion Estimation and Compensation. In: Proc. IEEE International Conference on Image Processing, Rochester, NY, USA (2002)
Haan, G., Biezen, P.W.A.C.: An Efficient True-Motion Estimator Using Candidate Vectors from a Parametric Motion Model. IEEE Transactions on Circuits and Systems for Video Technology 8(1), 85–91 (1998)
Hilman, K., Park, H.W., Kim, Y.: Using Motion-Compensated Frame-Rate Conversion for the Correction of 3: 2 Pulldown Artifacts in Video Sequences. IEEE Transactions On Circuits and Systems for Video Technology 10(6), 869–877 (2000)
Koga, T., Iinuma, K., Hirano, A., Iijima, Y., Ishigora, T.: Motion Compensated Inter-frame Coding for Video Conferencing. In: Proc. IEEE National Telecommunications Conference, New Orleans, LA, USA (1981)
Zhu, C., Lin, X., Chau, L.P.: Hexagon-Based Search Pattern for Fast Block Motion Estimation. IEEE Transactions on Circuits and Systems for Video Technology 12(5), 349–355 (2002)
Zhu, S., Ma, K.K.: A New Diamond Search Algorithm for Fast Block-Matching Motion Estimation. IEEE Transactions on Image Processing 9(2), 287–290 (2000)
Cheng, Y., Wang, Z., Dai, K., Guo, J.: A Fast Motion Estimation Algorithm Based on Diamond and Triangle Search Patterns. In: Marques, J.S., Pérez de la Blanca, N., Pina, P. (eds.) IbPRIA 2005. LNCS, vol. 3522, pp. 419–426. Springer, Heidelberg (2005)
Beric, A., Haan, G., Meerbergen, J., Sethuraman, R.: Towards An Efficient High Quality Picture-Rate Up-Converter. In: Proc. IEEE International Conference on Image Processing, Barcelona, Spain (2003)
Lee, S., Kwon, O., Park, R.: Weighted-adaptive Motion-compensated Frame Rate Up-conversion. IEEE Transactions on Consumer Electronics 49(3), 485–491 (2003)
Al-Mualla, M.E.: Motion Field Interpolation for Frame Rate Conversion. In: Proc. IEEE International Symposium on Circuit and Systems, Bankok, Thailand (2003)
Kuo, T., Kim, J., Jay Kuo, C.C.: Motion-Compensated Frame Interpolation Scheme For H.263 Codec. In: Proc. IEEE International Symposium on Circuit and Systems, Orlando, Florida, USA (1999)
Tiehan, L.U.: Motion-Compensated Frame Rate Conversion with Protection Against Compensation Artifacts. WIPO, Patent No. 2007123759 (2007)
Ohwaki, K., Takeyama, Y., Itoh, G., Mishima, N.: Apparatus, Method, and Computer Program Product for Detecting Motion Vector and for Creating Interpolation Frame. US Patent Office (US PTO), Patent No. 2008069221 (2008)
Sato, K., Yamasaki, M., Hirayama, K., Yoshimura, H., Hamakawa, Y., Douniwa, K., Ogawa, Y.: Interpolation Frame Generating Method and Interpolation Frame Generating Apparatus. US PTO, Patent No. 2008031338 (2008)
Chen, H.F., Kim, S.S., Sung, J.H.: Frame Interpolator, Frame Interpolation Method and Motion Reliability Evaluator. US PTO, Patent No. 2007140346 (2007)
Ohwaki, K., Itoh, G., Mishima, N.: Method, Apparatus and Computer Program Product for Generating Interpolation Frame. US PTO, Patent No. 2006222077 (2006)
Bugwadia, K., Petajan, E.D., Puri, N.N.: Motion Compensation Image Interpolation and Frame Rate Conversion for HDTV. US PTO, Patent No. 6229570 (2001)
Benois-Pineau, J., Nicolas, H.: A New Method for Region-based Depth Ordering in a Video Sequence: Application to Frame Interpolation. Journal of Visual Communication and Image Representation 13(3), 363–385 (2002)
Wang, J.: Video Temporal Reconstruction and Frame Rate Conversion. Wayne State University, Detroit (2006)
Turetken, E., Alatan, A.A.: Region-based Motion-compensated Frame Rate Up-conversion by Homography Parameter Interpolation. In: Proc. IEEE International Conference on Image Processing, Cairo, Egypt (2009)
Bleyer, M., Gelautz, M., Rhemann, C.: Region-based Optical Flow Estimation with Treatment of Occlusions. In: Proc. Joint Hungarian-Austrian Conference on Image Processing and Pattern Recognition, Veszprem, Hungary (2005)
Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2004)
Tsai, R.Y., Huang, T.S.: Estimating Three-dimensional Motion Parameters of a Rigid Planar Patch. IEEE Transactions on Acoustics, Speech and Signal Processing 29(6), 1147–1152 (1981)
Faugeras, O., Lustman, F.: Motion and Structure from Motion in a Piecewise Planar Environment. International Journal of Pattern Recognition and Artifcial Intelligence 2(3), 485–508 (1988)
Zhang, Z., Hanson, A.R.: 3D Reconstruction Based on Homography Mapping. In: Proc. ARPA Image Understanding Workshop, Palm Springs, CA, USA (1996)
Malis, E., Vargas, M.: Deeper Understanding of the Homography Decomposition for Vision-based Control. Research Report INRIA (2007)
Ozkalayci, B., Gedik, S., Alatan, A.: 3-D Structure Assisted Reference View Generation for H.264 Based Multi-View Video Coding. In: Proc. IEEE Picture Coding Symposium, Lisbon, Portugal (2007)
Cigla, C., Zabulis, X., Alatan, A.: Region-Based Dense Depth Extraction From Multi-View Video. In: Proc. IEEE International Conference on Image Processing, San Antonio Texas, USA (2007)
Cigla, C., Alatan, A.: Object Segmentation in Multi-view Video via Color, Depth and Motion Cues. In: Proc. IEEE International Conference on Image Processing, San Diego, California, USA (2008)
Lowe, D.G.: Distinctive Image Features from Scale-Invariant Keypoints. International Journal of Computer Vision 60(2), 91–110 (2004)
Fischler, M.A., Bolles, R.C.: Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography. Communications of the ACM 24(6), 381–395 (1981)
Paragios, N., Chen, Y., Faugeras, O.: Hand Book of Mathematical Models in Computer Vision. Springer, USA (2006)
Akko-Kayo: Multi-view Sequence by Department of Information Electronics at Nagoya University, http://www.tanimoto.nuee.nagoya-u.ac.jp/~fukushima/mpegftv/Akko.htm (accessed September 5, 2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Gedik, O.S., Türetken, E., Alatan, A.A. (2010). Next Generation Frame Rate Conversion Algorithms. In: Mrak, M., Grgic, M., Kunt, M. (eds) High-Quality Visual Experience. Signals and Communication Technology. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-12802-8_9
Download citation
DOI: https://doi.org/10.1007/978-3-642-12802-8_9
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-12801-1
Online ISBN: 978-3-642-12802-8
eBook Packages: EngineeringEngineering (R0)