Advertisement

MediaSync pp 451-473 | Cite as

Synchronization in MPEG-4 Systems

  • Jean Le Feuvre
  • Cyril Concolato
Chapter

Abstract

The MPEG-4 standard was defined in the early days of broadband Internet, after successful deployments of digital television networks, with the goal of unifying both broadcast and broadband media architectures and protocols in a single standard, tackling natural media (audio, video, images) as well as synthetic 2D or 3D graphics and audio. As such MPEG-4 can be seen as one of the first attempt at building the so-called convergence of Web and TV. Some parts of the standard have changed the media world forever (AAC audio and AVC|H264 video compression, MP4 file format), and while other parts have not always met their markets in successful way, they paved the way for more recent works, including HTML5 media. In this chapter, we explain how the MPEG-4 standard manages playback and synchronization of audio-visual streams and graphics animations and how multiple timelines can be used to provide rich interactive presentation over broadband and broadcast.

Keywords

Interactivity Synchronization Graphics Media Broadcast 

Notes

Definitions

Timestamp

time label assigned to a block of data in a media delivery system; timestamps are usually integer numbers, converted back to seconds using a timescale or timestamp resolution value expressing the number of integers in a second; for example, a timestamp of 180000 in a timescale of 90000 indicates a time of 2 s.

Decoding Time Stamp

time at which a media frame has to be available for the decoder.

Composition Time Stamp (or Presentation Time Stamp)

time at which a media frame must be presented to the user; composition and decoding times usually only differ for video stream using bi-directional motion estimation schemes, hence requiring a coding order different from the display order.

Intra-stream Synchronization

process and signaling required to reproduce a time-accurate, jitter-free playback of a given media stream; for example, signaling and maintaining the exact time lapse between successive video frames in a fixed or variable frame rate stream.

Inter-stream Synchronization

process and signaling required to reproduce a time-accurate playback of multiple media streams; for example, signaling and maintaining stream timestamps for achieve lip synchronization between an audio and a video stream.

Clock Reference

value against which decoding (resp. composition) timestamps are compared to trigger the decoding (resp. display) process; depending on the delivery systems, the clock reference may be delivered together with the content, as is the case in broadcast environments, derived from the wall-clock time another time source such as internet time, or derived from any other local system clock/counter (CPU cycle count, audio output sample rate, etc.).

References

  1. 1.
    Pereira, F.C., Ebrahimi, T.: The MPEG-4 Book. Prentice Hall PTR, Upper Saddle River, NJ, USA (2002)Google Scholar
  2. 2.
    Van der Meer, J.: Fundamentals and Evolution of Mpeg-2 Systems: Paving the MPEG Road, 1st edn. Wiley (2014)Google Scholar
  3. 3.
    Real time streaming protocol. https://www.ietf.org/rfc/rfc2326.txt
  4. 4.
    Synchronized multimedia integration language. https://www.w3.org/TR/SMIL3/
  5. 5.
    Dufourd, J.C., Avaro, O., Concolato, C.: LASeR: the MPEG standard for rich media services. IEEE Multimedia (2005)Google Scholar
  6. 6.
    Scalable vector graphics. https://www.w3.org/TR/SVG12/
  7. 7.
    Hyper-text transfer protocol. https://tools.ietf.org/html/rfc7230
  8. 8.
    RTP payload format for transport of MPEG-4 elementary streams. https://tools.ietf.org/html/rfc3640
  9. 9.
    Hyper text transfer protocol version 2 (HTTP/2). https://tools.ietf.org/html/rfc7540
  10. 10.
    File delivery over unidirectional transport. https://tools.ietf.org/html/rfc6726
  11. 11.
    FCAST: object delivery for the asynchronous layered coding (ALC) and NACK-oriented reliable multicast (NORM) protocols. https://tools.ietf.org/html/rfc6968
  12. 12.
    Meyer-Boudnik, T., Effelsberg, W.: MHEG explained. IEEE Multimedia 2(1), 26–38 (1995)CrossRefGoogle Scholar
  13. 13.
    Piesing, J.: The DVB multimedia home platform (MHP) and related specifications. Proc. IEEE 94(1), 237–247 (2006)CrossRefGoogle Scholar
  14. 14.
    ISO/IEC 14496-5, reference software, ISO/IEC publically available standards. http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=36086
  15. 15.
    Le Feuvre, J., Concolato, C., Moissinac, J. C.: GPAC: open source multimedia framework. In: Proceedings of the 15th ACM International Conference on Multimedia, pp. 1009–1012. ACM, Sept 2007. https://gpac.io

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Telecom ParisTech – LTCIParisFrance

Personalised recommendations