MPEG news: a report from the 103rd meeting, Geneva, Switzerland

MPEG plenary meeting at CICG in Geneva, CH

The 103rd MPEG meeting was held in Geneva, Switzerland, January 21-15, 2013. The official press release can be found here (doc only) and I'd like to introduce the new MPEG-H standard (ISO/IEC 23008) referred to as high efficiency coding and media delivery in heterogeneous environments:

• Part 1: MPEG Media Transport (MMT) - status: 2nd committee draft (CD)
• Part 2: High Efficiency Video Coding (HEVC) - status: final draft international standard (FDIS)
• Part 3: 3D Audio - status: call for proposals (CfP)

MPEG Media Transport (MMT)

The MMT project was started in order to address the needs of modern media transport applications going beyond the capabilities offered by existing means of transportation such as formats defined by MPEG-2 transport stream (M2TS) or ISO base media file format (ISOBMFF) group of standards. The committee draft was approved during the 101st MPEG meeting. As a response to the CD ballot, MPEG received more than 200 comments from national bodies and, thus, decided to issue the 2nd committee draft which will be publicly available by February 7, 2013.

High Efficiency Video Coding (HEVC) - ITU-T H.265 | MPEG HEVC

HEVC is the next generation video coding standard jointly developed by ISO/IEC JTC1/SC29/WG11 (MPEG) and the Video Coding Experts Group (VCEG) of ITU-T WP 3/16. Please note that both ITU-T and ISO/IEC MPEG use the term "high efficiency video coding" in the the title of the standard but one can expect - as with its predecessor - that the former will use ITU-T H.265 and the latter will use MPEG-H HEVC for promoting its standards. If you don't want to participate in this debate, simply use high efficiency video coding.

The MPEG press release says that the "HEVC standard reduces by half the bit rate needed to deliver high-quality video for a broad variety of applications" (note: compared to its predecessor AVC). The editing period for the FDIS goes until March 3, 2013 and then with the final preparations and a 2 month balloting period (yes|no vote only) once can expect the International Standard (IS) to be available early summer 2013. Please note that there are no technical differences between FDIS and IS.

The ITU-T press release describes HEVC as a standard that "will provide a flexible, reliable and robust solution, future-proofed to support the next decade of video. The new standard is designed to take account of advancing screen resolutions and is expected to be phased in as high-end products and services outgrow the limits of current network and display technology."

HEVC currently defines three profiles:

• Main Profile for the "Mass-market consumer video products that historically require only 8 bits of precision".
• Main 10 Profile "will support up to 10 bits of processing precision for applications with higher quality demands".
• Main Still Picture Profile to support still image applications, hence, "HEVC also advances the state-of-the-art for still picture coding"

3D Audio

The 3D audio standard shall complement MMT and HEVC assuming that in a "home theater" system a large number of loudspeakers will be deployed. Therefore, MPEG has issued a Call for Proposals (CfP) with the selection of the reference model v0 due in July 2013. The CfP says that MPEG-H 3D Audio "might be surrounding the user and be situated at high, mid and low vertical positions relative to the user’s ears. The desired sense of audio envelopment includes both immersive 3D audio, in the sense of being able to virtualize sound sources at any position in space, and accurate audio localization, in terms of both direction and distance."

"In addition to a “home theater” audio-visual system, there may be a “personal” system having a tablet-sized visual display with speakers built into the device, e.g. around the perimeter of the display. Alternatively, the personal device may be a hand-held smart phone. Headphones with appropriate spatialization would also be a means to deliver an immersive audio experience for all systems."

Complementary to the CfP, MPEG also provided the encoder input format for MPEG-H 3D audio and a draft MPEG audio core experiment methodology for 3D audio work.

Publicly available MPEG output documents

The following documents shall be come available at http://mpeg.chiariglione.org/ (note: some may have an editing period - YY/MM/DD). If you have difficulties to access one of these documents, please feel free to contact me.

• Study text of DIS of ISO/IEC 23000-13, Augmented Reality Application Format (13/01/25)
• Study text of DTR of ISO/IEC 23000-14, Augmented reality reference model (13/02/25)
• Text of ISO/IEC FDIS 23005-1 2nd edition Architecture (13/01/25)
• Text of ISO/IEC 2nd CD 23008-1 MPEG Media Transport (13/02/07)
• Text of ISO/IEC 23008-2:201x/PDAM1 Range Extensions (13/03/22)
• Text of ISO/IEC 23008-2:201x/PDAM2 Multiview Extensions (13/03/22)
• Call for Proposals on 3D Audio (13/01/25)
• Encoder Input Format for MPEG-H 3D Audio (13/02/08)
• Draft MPEG Audio CE methodology for 3D Audio work (13/01/25)
• Draft Requirements on MPEG User Descriptions (13/02/08)
• Draft Call for Proposals on MPEG User Descriptions (13/01/25)
• Draft Call for Proposals on Green MPEG (13/01/25)
• Context, Objectives, Use Cases and Requirements for Green MPEG (13/01/25)
• White Paper on State of the Art in compression and transmission of 3D Video (13/01/28)
• MPEG Awareness Event Flyer at 104th MPEG meeting in Incheon (13/02/28)

MPEG news: a report from the 101st meeting, Stockholm, Sweden

The 101st MPEG meeting was held in Stockholm, Sweden, July 16-20, 2012. The official press release can be found here and I would like to highlight the following topics:

• MPEG Media Transport (MMT) reaches Committee Draft (CD)
• High-Efficiency Video Coding (HEVC) reaches Draft International Standard (DIS)
• MPEG and ITU-T establish JCT-3V
• Call for Proposals: HEVC scalability extensions
• 3D audio workshop
• Green MPEG

MMT goes CD

The Committee Draft (CD) of MPEG-H part 1 referred to as MPEG Media Transport (MMT) has been approved and will be publicly available after an editing period which will end Sep 17th. MMT comprises the following features:

• Delivery of coded media by concurrently using more than one delivery medium (e.g., as it is the case of heterogeneous networks).
• Logical packaging structure and composition information to support multimedia mash-ups (e.g., multiscreen presentation).
• Seamless and easy conversion between storage and delivery formats.
• Cross layer interface to facilitate communication between the application layers and underlying delivery layers.
• Signaling of messages to manage the presentation and optimized delivery of media.

This list of 'features' may sound very high-level but as the CD usually comprises stable technology and is publicly available, the research community is more than welcome to evaluate MPEG's new way of media transport. Having said this, I would like to refer to the Call for Papers of  JSAC's special issue on adaptive media streaming which is mainly focusing on DASH but investigating its relationship to MMT is definitely within the scope.

HEVCs' next step towards completion: DIS

The approval of the Draft International Standard (DIS) brought the HEVC standard one step closer to completion. As reported previously, HEVC shows inferior performance gains compared to its predecessor and real-time software decoding on the iPad 3 (720p, 30Hz, 1.5 Mbps) has been demonstrated during the Friday plenary [1, 2]. It is expected that the Final Draft International Standard (FDIS) is going to be approved at the 103rd MPEG meeting in January 21-25, 2013. If the market need for HEVC is only similar as it was when AVC was finally approved, I am wondering if one can expect first products by mid/end 2013. From a research point of view we know - and history is our witness - that improvements are still possible even if the standard has been approved some time ago. For example, the AVC standard is now available in its 7th edition as a consolidation of various amendments and corrigenda.

JCT-3V

After the Joint Video Team (JVT) which successfully developed standards such as AVC, SVC, MVC and the Joint Collaborative Team on Video Coding (JCT-VC), MPEG and ITU-T establish the Joint Collaborative Team on 3D Video coding extension development (JCT-3V). That is, from now on MPEG and ITU-T also joins forces in developing 3D video coding extensions for existing codecs as well as the ones under development (i.e., AVC, HEVC). The current standardization plan includes the development of AVC multi-view extensions with depth to be completed this year and I assume HEVC will be extended with 3D capabilities once the 2D version is available.

In this context it is interesting that a call for proposals for MPEG Frame Compatible (MFC) has been issued to address current deployment issues of stereoscopic videos. The requirements are available here.

Call for Proposals: SVC for HEVC

In order to address the need for higher resolutions - Ultra HDTV - and subsets thereof, JCT-VC issued a call for proposals for HEVC scalability extensions. Similar to AVC/SVC, the requirements include that the base layer should be compatible with HEVC and enhancement layers may include temporal, spatial, and fidelity scalability. The actual call, the use cases, and the requirements shall become available on the MPEG Web site.

MPEG hosts 3D Audio Workshop

Part 3 of MPEG-H will be dedicated to audio, specifically 3D audio. The call for proposals will be issues at the 102nd MPEG meeting in October 2012 and submissions will be due at the 104th meeting in April 2013. At this meeting, MPEG has hosted a 2nd workshop on 3D audio with the following speakers.

• Frank Melchior, BBC R&D: “3D Audio? - Be inspired by the Audience!”
• Kaoru Watanabe, NHK and ITU: “Advanced multichannel audio activity and requirements”
• Bert Van Daele, Auro Technologies: “3D audio content production, post production and distribution and release”
• Michael Kelly, DTS: “3D audio, objects and interactivity in games”

The report of this workshop including the presentations will be publicly available by end of August at the MPEG Web site.

What's new: Green MPEG

Finally, MPEG is starting to explore a new area which is currently referred to as Green MPEG addressing technologies to enable energy-efficient use of MPEG standards. Therefore, an Ad-hoc Group (AhG) was established with the following mandates:

1. Study the requirements and use-cases for energy efficient use of MPEG technology.
2. Solicit further evidence for the energy savings.
3. Develop reference software for Green MPEG experimentation and upload any such software to the SVN.
4. Survey possible solutions for energy-efficient video processing and presentation.
5. Explore the relationship between metadata types and coding technologies.
6. Identify new metadata that will enable additional power savings.
7. Study system-wide interactions and implications of energy-efficient processing on mobile devices.

AhGs are usually open to the public and all discussions take place via email. To subscribe please feel free to join the email reflector.

MPEG news: a report from the 95th meeting, Daegu, Korea

For those who cannot wait until the official press release from the 95th ISO/IEC JTC 1/SC 29/WG 11 (MPEG) meeting, here comes a short review of what happened during this week. I must admit that I could not stay until the very end (the first time after a very long period) and, thus, only a few topics are presented here, i.e., those where I was heavily involved:

• MPEG-V Awareness Event
• Dynamic Adaptative Streaming over HTTP (DASH) reached Draft International Standard (DIS) stage
• MPEG Media Transport (MMT) has been successfully evaluated

MPEG-V Awareness Event

The purpose of this event is to present and demonstrate MPEG-V, a new standard for a variety of products and applications enabling multi-sensorial user experience and interchange between virtual worlds. The event started with a set of presentations coming from the editors of the individual parts and also from industry showing the productions and applications enabled by the standard. A demo session followed the presentations where individual companies, institutions, and universities demonstrated prototype implementations and promotional material. The presentations are publicly available (see below).

Mpeg v-awareness event

View more presentations from Marius Preda.

DASH reached DIS

ISO/IEC 23001-6 aka Dynamic Adaptive Streaming over HTTP (DASH) reached Draft International Standard (DIS) stage which means that most of the technical work is completed and now it's time to check and validate the specification as well as to start the implementation. It was a busy meeting where a lot of national body comments were addressed and related input contributions were reviewed. If you're interested to learn more about DASH, I would recommend the following events in the near future:

• Second W3C Web and TV Workshop, February 8-9, 2011 in Berlin, Germany
• Special Session related to DASH at ACM MMSys'11, February 23-25, 2011 in San Jose, CA, USA

Alternatively, you may just wait until early next week when the DIS of DASH will be made publicly available at the MPEG Web site (and also here on this blog). For technical discussions, I'd like to refer to the DASH email reflector.

MPEG Media Transport (MMT) has been successfully evaluated

The responses to the CfP on MMT has been evaluated and a first working draft has been created. As working drafts are (usually) not publicly available, that's more or less what I can say about this very interesting work item. Nevertheless, we received a reasonable number of submissions both comprehensive nature and specific to certain requirements. In general, they mainly aim to address limitations of existing transport formats in the context of the current and, hopefully, future Internet. If you would like to take part in the discussions, you may subscribe to the MMT email reflector. More to come on this topic during the next months, stay tuned ...

MPEG targets next generation teleconferencing and personalized audio in its newest technology

Kyoto, Japan – The 91st MPEG meeting was held in Kyoto, Japan from the 18th to the 22nd of January 2010.

Highlights of the 91st Meeting

• Advanced technology enables bitrate-efficient teleconferencing and personalized audio
• MPEG and ITU-T SG16 form Joint Collaborative Team (JCT) for High Performance Video Coding
• Creating solutions for Advanced IPTV Terminals
• MPEG Extensible Middleware (MXM) standard debuts to facilitate easy access to MPEG tools across a global market
• MPEG creates efficient and flexible standard for 3D graphics compression
• MPEG Media Transport (MMT) workshop targets requirements and technologies for streaming of MPEG content

Wanna know more/details? Please check out the official press release!

Overview of Selected Current MPEG Activities

--this covers a report from the 91st MPEG meeting in Kyoto, Japan

Previously, I've always provided a written report but this time it comes in form of a presentation (slideshow) - enjoy!

View more presentations from Klagenfurt University.

Kyoto MMT Workshop Program

The program for the Workshop on MMT is now available!

Date: Wednesday (13:00~ 17:00) January 20 during the 91th WG11 meeting in Japan

Session I, 13:00-14:50

• Consideration of a transport mechanism on broadcasting from the viewpoint of emerging hybrid content delivery system (by NHK)
• Industry Viewpoints of Internet TV over IP networks: Trends and Technologies (by Samsung)
• Future media adaptation and delivery (by Thomson)
• Experiences and Challenges of Emerging Multimedia Transport Environments from the Operator Point of View (by Avail-TVN)
• Panel Discussion

Session II, 15:10-17:00

• DVB Work on Multimedia Transport (by DVB)
• Cross Layer Optimization for reliable video delivery over mobile wireless networks (by ETRI)
• Extended NAL (Network Abstraction Layer) for MMT (by KHU)
• MPEG-21-based cross-layer optimization techniques for enabling Quality of Experience (by U. Klagenfurt)
• Panel Discussion

Workshop on MMT (MPEG Media Transport) – Call for Contributions

MPEG has been developed various technologies for multimedia transport such as MPEG-2 TS, MP4 file format, and so on. Both technologies have been widely accepted and heavily used by the various industries such as digital broadcasting, mobile phones and etc. On the other hands, the standardization organizations such as IETF, IEEE, and 3GPP have been providing various protocols to deliver multimedia contents packetized or packaged by such MPEG transport technologies. For example, several RTP payload formats have been developed to enable the streaming of media from a server to a client over IP networks. However, the development of the payload format was separate from the codec development since, traditionally, the coding and transport of media are associated with different layers of the OSI reference model. Such separation results in the current situation where the optimal streaming solution for MPEG media relies on proprietary information exchange over RTP and its associated protocols.

In order to develop standardized and efficient solutions for the transport of MPEG media, especially given the recent increase demand in the heterogeneous network environment, MPEG is gathering information on current limitations of available standards in the area of media streaming and challenges in emerging network environments.

To overcome existing limitations and face the challenges that emerging applications impose on the requirements of MMT standardization, ISO/IEC WG11 (MPEG) plans to organize a full-day workshop in Kyoto on Wednesday (11a.m~ 5p.m) January 20 during the 91th WG11 meeting in Japan.

The key intention of the workshop is to get overview state-of-art technologies as well as to require solid use cases and requirements for MMT. This will enable MPEG to draw conclusions for the needs and chances in new transport scheme standardization. For this purpose it is planned to invite speakers on key topics and in addition select a variety of proposed contributions. The following topics will be considered:

Industry experience of multimedia transport

• Delivery of media over IP networks in ptp/ptmp manner
• Progressive download and streaming
• MPEG TS Transport between heterogeneous network
• Delivery and sharing of User created contents

Challenges of emerging multimedia transport environments:

• Peer-to-Peer (P2P) traffic for IPTV services
• Cross-layer designs to improve the Quality of Service/Experience (QoS/QoE)
• Context- and Content-Aware Networks
• Conversion between a stream and a randomly accessible file

The workshop will be organized by a single track of oral presentations. When planning to propose a contribution, please send a summary by 12 December 2009, including title, author(s), area(s) as from the list above and an abstract of 500 words by email to the following persons (chairmen of MPEG systems and requirements subgroups):

• Youngkwon Lim, young@netntv.co.kr
• Jörn Ostermann, ostermann@tnt.uni-hannover.de

The final detail program will be made available by 10 January 2010. Information about acceptance/rejection of the contributions will be conveyed to proponents prior to that date. Note that contributions that cannot be considered for presentation at the workshop will be reviewed during the following week at the MPEG meeting.

MPEG Media Transport is taking shape

Achievements at the 90th MPEG meeting in Xi’an, China with respect to MPEG Media Transport (MMT):

• Context & Objectives;
• Ad-hoc Group; and
• Future workshop in January 2010 on the Wednesday during the Kyoto meeting focusing on “adaptive progressive transport” and “cross-layer design”.

The first document, context & objectives, describes the scope of the standard based – as usual – on a number of use cases. These are adaptivity, delivery, QoS/QoE-awareness, convergence, and transparency which are briefly highlighted in the following:

• Adaptivity is referred to adaptability of multimedia content towards the usage environment context in which the content is being used. Ultimately, this includes the end user and his environment but also the delivery and other areas.
• Delivery differentiates between (traditional) IPTV, progressive download, and peer-to-peer delivery.
• QoS/QoE-awareness refers to several optimization techniques which roughly can be clustered into application-level and cross-layer optimizations. The former is further divided into retransmission and buffer management as well as forward error correction (FEC). The latter can be categorized into bottom-up, top-down, and integrated approaches well-known from the research literature.
• Convergence includes topics related to network convergence, i.e., media-aware devices, and service convergence.
• Transparency is mainly referred to as transparency with respect to different content protection schemes and rights management systems.
  

The current requirements document, however, focuses on two main areas which are adaptive delivery and cross-layer optimization. A couple of general requirements (i.e., IMHO a wish list we all pretty much agree on) precede the requirements pertaining to these two selected topics which are listed below followed by my comments:

Anyway, all the technologies that will be submitted as a response to a call for proposals – most likely this will happen in April 2010 – will have to show evidence that they are better than their predecessors (i.e., MPEG-2 TS) but it is pretty much unclear how this evaluation is going to be performed.

Some people argue that the functionality of adaptive, progressive download could be achieved with (minor) extensions of the MP4 file format but is something the industry definitely needs right now. Therefore, solutions for this part of MMT shall be standardized without any delay, i.e., at maximum warp speed! In my view, such a requirement should be included in the list above although it is kind of obvious…

Cross-layer (XL) optimization is a research topic which is around for quite some time with pretty cool results. However, the results mainly stem from XL stuff being done at the lower layers, i.e., link layer and so on. So what MPEG can offer is to standardize an interface – parameter names & semantics – to XL optimization modules whereas these modules should be out-of-scope and left open for industry competition. A kind of interface that could be used for this purpose is Supplemental Enhancement Information (SEI) – originally standardized during the course of the AVC (MPEG-4 Part 10 | H.264) developments – which nowadays can be included in MPEG-2 TS, file formats based on the ISO Base Media File Format, and also RTP. Thus, what is needed is simply a registration authority for SEI messages in the context of XL optimization. However, in order to represent advanced XL models, further standardization might be needed but we have already demonstrated how this could be accomplished existing MPEG-21 Digital Item Adaptation tools [1][2][3]. As in the case of SEI message, the only missing link is some kind of registration authority for parameter names and semantics which have not been considered when the standard has been finally approved – that was already in 2003.

Finally, I’d like to mention the Ad-hoc Group (AhG) where discussions between meetings take place and the workshop in January 2010 on the Wednesday during the Kyoto meeting focusing on “adaptive progressive transport” and “cross-layer design”. Details about the workshop you can find here. Note that usually such a workshop is open to everybody, i.e., you don’t need to be a member of ISO/MPEG in order to attend this workshop.

References:
[1] I. Kofler, C. Timmerer, H. Hellwagner, and T. Ahmed: “Towards MPEG-21-based Cross-layer Multimedia Content Adaptation”, Proc. 2nd International Workshop on Semantic Media Adaptation and Personalization (SMAP 2007), London, UK, Dec. 2007.
[2] C. Timmerer, V. H. Ortega, J. M. González, and A. León: “Measuring Quality of Experience for MPEG-21-based Cross-Layer Multimedia Content Adaptation”, to appear in Proceedings of the 1st ACS/IEEE International Workshop on Wireless Internet Services (WISe’08), Doha, Qatar, April 1-4, 2008.
[3] I. Kofler, J. Seidl, C. Timmerer, H. Hellwagner, I. Djama and T. Ahmed, “Using MPEG-21 for cross-layer multimedia content adaptation”, Journal on Signal, Image and Video Processing, Springer, vol. 2, no. 4, Dec. 2008.