MPEG news: a report from the 110th meeting, Strasbourg, France

This blog post is also available at SIGMM records.

The 110th MPEG meeting was held at the Strasbourg Convention and Conference Centre featuring the following highlights:

• The future of video coding standardization
• Workshop on media synchronization
• Standards at FDIS: Green Metadata and CDVS
• What's happening in MPEG-DASH?

Additional details about MPEG's 110th meeting can be also found here including the official press release and all publicly available documents.

The Future of Video Coding Standardization

MPEG110 hosted a panel discussion about the future of video coding standardization. The panel was organized jointly by MPEG and ITU-T SG 16's VCEG featuring Roger Bolton (Ericsson), Harald Alvestrand (Google), Zhong Luo (Huawei), Anne Aaron (Netflix), Stéphane Pateux (Orange), Paul Torres (Qualcomm), and JeongHoon Park (Samsung).

As expected, "maximizing compression efficiency remains a fundamental need" and as usual, MPEG will study "future application requirements, and the availability of technology developments to fulfill these requirements". Therefore, two Ad-hoc Groups (AhGs) have been established which are open to the public:

• AHG on Future Video Coding Technology [email][subscription]
• AHG on industry needs for Future Video Coding [email][subscription]

The presentations of the brainstorming session on the future of video coding standardization can be found here.

Workshop on Media Synchronization

MPEG101 also hosted a workshop on media synchronization for hybrid delivery (broadband-broadcast) featuring six presentations "to better understand the current state-of-the-art for media synchronization and identify further needs of the industry".

• An overview of MPEG systems technologies providing advanced media synchronization, Youngkwon Lim, Samsung
• Hybrid Broadcast - Overview of DVB TM-Companion Screens and Streams specification, Oskar van Deventer, TNO
• Hybrid Broadcast-Broadband distribution for new video services :  a use cases perspective, Raoul Monnier, Thomson Video Networks
• HEVC and Layered HEVC for UHD deployments, Ye Kui Wang, Qualcomm
• A fingerprinting-based audio synchronization technology, Masayuki Nishiguchi, Sony Corporation
• Media Orchestration from Capture to Consumption, Rob Koenen, TNO

The presentation material is available here. Additionally, MPEG established an AhG on timeline alignment (that's how the project is internally called) to study use cases and solicit contributions on gap analysis and also technical contributions [email][subscription].

Standards at FDIS: Green Metadata and CDVS

My first report on MPEG Compact Descriptors for Visual Search (CDVS) dates back to July 2011 which provides details about the call for proposals. Now, finally, the FDIS has been approved during the 110th MPEG meeting. CDVS defines a compact image description that facilitates the comparison and search of pictures that include similar content, e.g. when showing the same objects in different scenes from different viewpoints. The compression of key point descriptors not only increases compactness, but also significantly speeds up, when compared to a raw representation of the same underlying features, the search and classification of images within large image databases. Application of CDVS for real-time object identification, e.g. in computer vision and other applications, is envisaged as well.

Another standard reached FDIS status entitled Green Metadata (first reported in August 2012). This standard specifies the format of metadata that can be used to reduce energy consumption from the encoding, decoding, and presentation of media content, while simultaneously controlling or avoiding degradation in the Quality of Experience (QoE). Moreover, the metadata specified in this standard can facilitate a trade-off between energy consumption and QoE. MPEG is also working on amendments to the ubiquitous MPEG-2 TS ISO/IEC 13818-1 and ISOBMFF ISO/IEC 14496-12 so that green metadata can be delivered by these formats.

What's happening in MPEG-DASH?

MPEG-DASH is in a kind of maintenance mode but still receiving new proposals in the area of SAND parameters and some core experiments are going on. Also, the DASH-IF is working towards new interoperability points and test vectors in preparation of actual deployments. When speaking about deployments, they are happening, e.g., a 40h live stream right before Christmas (by bitmovin, a top-100 company that matters most in online video). Additionally, VideoNext was co-located with CoNEXT'14 targeting scientific presentations about the design, quality and deployment of adaptive video streaming. Webex recordings of the talks are available here. In terms of standardization, MPEG-DASH is progressing towards the 2nd amendment including spatial relationship description (SRD), generalized URL parameters and other extensions. In particular, SRD will enable new use cases which can be only addressed using MPEG-DASH and the FDIS is scheduled for the next meeting which will be in Geneva, Feb 16-20, 2015. I'll report on this within my next blog post, stay tuned..

MPEG news: a report from the 107th meeting, San Jose, CA, USA

This blog post is also available at bitmovin tech blog and SIGMM records.

The MPEG-2 Transport Stream (M2TS; formally known as Rec. ITU-T H.222.0 | ISO/IEC 13818-1) has been awarded with the Technology & Engineering Emmy® Award by the National Academy of Television Arts & Sciences. It is the fourth time MPEG received an Emmy award. The M2TS is widely deployed across a broad range of application domain such as broadcast, cable TV, Internet TV (IPTV and OTT), and Blu-ray Disks. The Emmy was received during this year’s CES2014 in Las Vegas.

Plenary during the 107^th MPEG Meeting.

Other topics of the 107th MPEG meeting in San Jose include the following highlights:

• Requirements: Call for Proposals on Screen Content jointly with ITU-T’s Video Coding Experts Group (VCEG)
• Systems: Committee Draft for Green Metadata
• Video: Study Text Committee Draft for Compact Descriptors for Visual Search (CDVS)
• JCT-VC: Draft Amendment for HEVC Scalable Extensions (SHVC)
• JCT-3D: Proposed Draft Amendment for HEVC 3D Extensions (3D-HEVC)
• Audio: 3D audio plans to progress to CD at 108th meeting
• 3D Graphics: Working Draft 4.0 of Augmented Reality Application Format (ARAF) 2nd Edition

The official MPEG press release can be downloaded from the MPEG Web site. Some of the above highlighted topics will be detailed in the following and, of course, there’s an update on DASH-related matters at the end.

Call for Proposals on Screen Content

Screen content refers to content coming not from cameras but from screen/desktop sharing and collaboration, cloud computing and gaming, wirelessly connected displays, control rooms with high resolution display walls, virtual desktop infrastructures, tablets as secondary displays, PC over IP, ultra-thin client technology, etc. Also mixed-content is within the scope of this work item and may contain a mixture of camera-captured video and images with rendered computer-generated graphics, text, animation, etc.

Although this type of content was considered during the course of the HEVC standardization, recent studies in MPEG have led to the conclusion that significant further improvements in coding efficiency can be obtained by exploiting the characteristics of screen content and, thus, a Call for Proposals (CfP) is being issued for developing possible future extensions of the HEVC standard.

Companies and organizations are invited to submit proposals in response to this call –issued jointly by MPEG with ITU-T VCEG. Responses are expected to be submitted by early March, and will be evaluated during the 108th MPEG meeting. The timeline is as follows:

• 2014/01/17: Final Call for Proposals
• 2014/01/22: Availability of anchors and end of editing period for Final CfP
• 2014/02/10: Mandatory registration deadline
  One of the contact persons (see Section 10) must be notified, and an invoice for the testing fee will be sent after registration. Additional logistic information will also be sent to proponents by this date.
• 2014/03/05: Coded test material shall be available at the test site. By this date, the payment of the testing fee is expected to be finalized.
• 2014/03/17: Submission of all documents and requested data associated with the proposal.
• 2014/03/27-04/04: Evaluation of proposals at standardization meeting.
• 2015: Final draft standard expected.

It will be interesting to see the coding efficiency of the submitted proposals compared to a pure HEVC or even AVC approach.

DEC PDP-8 at Computer History
Museum during MPEG Social Event.

Committee Draft for Green Metadata

Green Metadata, formerly known as Green MPEG, shall enable energy-efficient media consumption and reached Committee Draft (CD) status at the 107th MPEG meeting. The representation formats defined within Green Metadata help reducing decoder power consumption and display power consumption. Clients may utilize such information for the adaptive selection of operating voltage or clock frequencies within their chipsets. Additional, it may be used to set the brightness of the backlights for the display to save power consumption.

Green Metadata also provides metadata for the signaling and selection of DASH representations to enable the reduction of power consumption for their encoding.

The main challenge in terms of adoption of this kind of technology is how to exploit these representation formats to actually achieve energy-efficient media consumption and how much!

What’s new on the DASH frontier?

The text of ISO/IEC 23009-1 2nd edition PDAM1 has been approved which may be referred to as MPEG-DASH v3 (once finalized and integrated into the second edition, possibly with further amendments and corrigenda, if applicable). This first amendment to MPEG-DASH v2 comprises accurate time synchronization between server and client for live services as well as a new profile, i.e., ISOBMFF High Profile which basically combines the ISOBMFF Live and ISOBMFF On-demand profiles and adds the Xlink feature.

Additionally, a second amendment to MPEG-DASH v2 has been started featuring Spatial Relationship Description (SRD) and DASH Client Authentication and Content Access Authorization (DAA).

Other DASH-related aspects include the following:

• The common encryption for ISOBMFF has been extended with a simple pattern-based encryption mode, i.e., a new method which should simply content encryption.
• The CD has been approved for the carriage of timed metadata metrics of media in ISOBMFF. This allows for the signaling of quality metrics within the segments enabling QoE-aware DASH clients.

What else? That is, some publicly available MPEG output documents… (Dates indicate availability and end of editing period, if applicable, using the following format YY/MM/DD):

• Report of 3D-AVC Subjective Quality Assessment (14/02/28)
• Working Draft 3 of Video Coding for Browsers (14/01/31)
• Common Test Conditions for Proposals on VCB Enhancements (14/01/17)
• Study Text of ISO/IEC CD 15938-13 Compact Descriptors for Visual Search (14/02/14)
• WD 4.0 of ARAF 2nd Edition (14/02/07)
• Text of ISO/IEC 23001-7 PDAM 1 Simple pattern-based encryption mode (14/01/31)
• Text of ISO/IEC CD 23001-10 Carriage of Timed Metadata Metrics of Media in the ISO Base Media File Format (14/01/31)
• Text of ISO/IEC CD 23001-11 Green Metadata (14/01/24)
• Preliminary Draft of ISO/IEC 23008-2:2013/FDAM1 HEVC Range Extensions (14/02/28)
• Text of ISO/IEC 23008-2:2013/DAM3 HEVC Scalable Extensions (14/01/31)
• Preliminary Draft of ISO/IEC 23008-2:2013/FDAM2 HEVC Multiview Extensions (14/02/28)
• Text of ISO/IEC 23008-2:2013/PDAM4 3D Extensions (14/03/14)
• Text of ISO/IEC CD 23008-12 Image File Format (14/01/17)
• Text of ISO/IEC 23009-1:201x DCOR 1 (14/01/24)
• Text of ISO/IEC 23009-1:201x PDAM 1 High Profile and Availability Time Synchronization (14/01/24)
• WD of ISO/IEC 23009-1 AMD 2 (14/01/31)
• Requirements for an extension of HEVC for coding of screen content (14/01/17)
• Joint Call for Proposals for coding of screen content (14/01/22)
• Draft requirements for Higher Dynamic Range (HDR) and Wide Color Gamut (WCG) video coding for Broadcasting, OTT, and Storage Media (14/01/17)
• Working Draft 1 of Internet Video Coding (IVC) (14/01/31)

Contact:

Dr. Christian Timmerer
Alpen-Adria-Universität Klagenfurt | christian.timmerer@aau.at
CIO bitmovin GmbH | christian.timmerer@bitmovin.net

MPEG news: a report from the 102nd meeting, Shanghai, China

The 102nd MPEG meeting was held in Shanghai, China, October 15-19, 2012. The official press release can be found here (not yet available) and I would like to highlight the following topics:

• Augmented Reality Application Format (ARAF) goes DIS
• MPEG-4 has now 30 parts: Let's welcome timed text and other visual overlays
• Draft call for proposals for 3D audio 
• Green MPEG is progressing
• MPEG starts a new publicity campaign by making more working documents publicly available for free

Augmented Reality Application Format (ARAF) goes DIS

MPEG's application format dealing with augmented reality reached DIS status and is only one step away from becoming in international standard. In a nutshell, the MPEG ARAF enables to augment 2D/3D regions of scene by combining multiple/existing standards within a specific application format addressing certain industry needs. In particular, ARAF comprises three components referred to as scene, sensor/actuator, and media. The scene component is represented using a subset of MPEG-4 Part 11 (BIFS), the sensor/actuator component is defined within MPEG-V, and the media component may comprise various type of compressed (multi)media assets using different sorts of modalities and codecs.

A tutorial from Marius Preda, MPEG 3DG chair, at the Web3D conference in August 2012 is provided below.

MPEG Augmented Reality Tutorial from Marius Preda

MPEG-4 has now 30 parts

Let's welcome timed text and other visual overlays in the family of MPEG-4 standards. Part 30 of MPEG-4 - in combination with an amendment to the ISO base media file format (ISOBMFF) -  addresses the carriage of W3C TTML including its derivative SMPTE Timed Text, as well as WebVTT. The types of overlays include subtitles, captions, and other timed text and graphics. The text-based overlays include basic text and XML-based text. Additionally, the standards provides support for bitmaps, fonts, and other graphics formats such as scalable vector graphics.

Draft call for proposals for 3D audio

MPEG 3D audio is concerned about various test items ranging from 9.1 over 12.1 up to 22.1 channel configurations. A public draft call for proposals has been issued at this meeting with the goal to finalize the call and the evaluation guidelines at the next meeting. The evaluation will be conducted in two phases. Phase one for higher bitrates (1.5 Mbps to 265 kbps) is foreseen to conclude in July 2013 with the evaluation of the answers to the call and the selection of the "Reference Model 0 (RM0)" technology which will serve as a basis for the development of an 3D audio standard. The second phase targets lower bitrates (96 kbps to 48 kbps) and builds on RM0 technology after this has been documented using text and code.

Green MPEG is progressing

The idea between green MPEG is to define signaling means that enable energy efficient encoding, delivery, decoding, and/or presentation of MPEG formats (and possibly others) without the loss of Quality of Experience. Green MPEG will address this issue from an end-to-end point of view with the focus - as usual - on the decoder. However, a codec-centric design is not desirable as the energy efficiency should not be affected at the expenses of the other components of the media ecosystem. At the moment, first requirements have been defined and everyone is free to join the discussions on the email reflector within the Ad-hoc Group.

MPEG starts a new publicity campaign by making more working documents publicly available for free

As a response to national bodies comments, MPEG is starting from now on to make more documents publicly available for free. Here's a selection of these documents which are publicly available here. Note that some may have an editing period and, thus, are not available at the of writing this blog post.

• Text of ISO/IEC 14496-15:2010/DAM 2 Carriage of HEVC (2012/11/02)
• Text of ISO/IEC CD 14496-30 Timed Text and Other Visual Overlays in ISO Base Media File Format (2012/11/02)
• DIS of ISO/IEC 23000-13, Augmented Reality Application Format (2012/11/07)
• DTR of ISO/IEC 23000-14, Augmented reality reference model (2012/11/21)
• Study of ISO/IEC CD 23008-1 MPEG Media Transport (2012/11/12)
• High Efficiency Video Coding (HEVC) Test Model 9 (HM 9) Encoder Description (2012/11/30)
• Study Text of ISO/IEC DIS 23008-2 High Efficiency Video Coding (2012/11/30)
• Working Draft of HEVC Full Range Extensions (2012/11/02)
• Working Draft of HEVC Conformance (2012/11/02)
• Report of Results of the Joint Call for Proposals on Scalable High Efficiency Video Coding (SHVC) (2012/11/09)
• Draft Call for Proposals on 3D Audio (2012/10/19)
• Text of ISO/IEC 23009-1:2012 DAM 1 Support for Event Messages and Extended Audio Channel Configuration (2012/10/31)
• Internet Video Coding Test Model (ITM) v 3.0 (2012/11/02)
• Draft Requirements on MPEG User Descriptions (2012/10/19)
• Draft Use Cases for MPEG User Description (Ver. 4.0) (2012/10/19)
• Requirements on Green MPEG (2012/10/19)
• White Paper on State of the Art in compression and transmission of 3D Video (Draft) (2012/10/19)
• White Paper on Compact Descriptors for Visual Search (2012/11/09)

MPEG news: a report from the 101st meeting, Stockholm, Sweden

The 101st MPEG meeting was held in Stockholm, Sweden, July 16-20, 2012. The official press release can be found here and I would like to highlight the following topics:

• MPEG Media Transport (MMT) reaches Committee Draft (CD)
• High-Efficiency Video Coding (HEVC) reaches Draft International Standard (DIS)
• MPEG and ITU-T establish JCT-3V
• Call for Proposals: HEVC scalability extensions
• 3D audio workshop
• Green MPEG

MMT goes CD

The Committee Draft (CD) of MPEG-H part 1 referred to as MPEG Media Transport (MMT) has been approved and will be publicly available after an editing period which will end Sep 17th. MMT comprises the following features:

• Delivery of coded media by concurrently using more than one delivery medium (e.g., as it is the case of heterogeneous networks).
• Logical packaging structure and composition information to support multimedia mash-ups (e.g., multiscreen presentation).
• Seamless and easy conversion between storage and delivery formats.
• Cross layer interface to facilitate communication between the application layers and underlying delivery layers.
• Signaling of messages to manage the presentation and optimized delivery of media.

This list of 'features' may sound very high-level but as the CD usually comprises stable technology and is publicly available, the research community is more than welcome to evaluate MPEG's new way of media transport. Having said this, I would like to refer to the Call for Papers of  JSAC's special issue on adaptive media streaming which is mainly focusing on DASH but investigating its relationship to MMT is definitely within the scope.

HEVCs' next step towards completion: DIS

The approval of the Draft International Standard (DIS) brought the HEVC standard one step closer to completion. As reported previously, HEVC shows inferior performance gains compared to its predecessor and real-time software decoding on the iPad 3 (720p, 30Hz, 1.5 Mbps) has been demonstrated during the Friday plenary [1, 2]. It is expected that the Final Draft International Standard (FDIS) is going to be approved at the 103rd MPEG meeting in January 21-25, 2013. If the market need for HEVC is only similar as it was when AVC was finally approved, I am wondering if one can expect first products by mid/end 2013. From a research point of view we know - and history is our witness - that improvements are still possible even if the standard has been approved some time ago. For example, the AVC standard is now available in its 7th edition as a consolidation of various amendments and corrigenda.

JCT-3V

After the Joint Video Team (JVT) which successfully developed standards such as AVC, SVC, MVC and the Joint Collaborative Team on Video Coding (JCT-VC), MPEG and ITU-T establish the Joint Collaborative Team on 3D Video coding extension development (JCT-3V). That is, from now on MPEG and ITU-T also joins forces in developing 3D video coding extensions for existing codecs as well as the ones under development (i.e., AVC, HEVC). The current standardization plan includes the development of AVC multi-view extensions with depth to be completed this year and I assume HEVC will be extended with 3D capabilities once the 2D version is available.

In this context it is interesting that a call for proposals for MPEG Frame Compatible (MFC) has been issued to address current deployment issues of stereoscopic videos. The requirements are available here.

Call for Proposals: SVC for HEVC

In order to address the need for higher resolutions - Ultra HDTV - and subsets thereof, JCT-VC issued a call for proposals for HEVC scalability extensions. Similar to AVC/SVC, the requirements include that the base layer should be compatible with HEVC and enhancement layers may include temporal, spatial, and fidelity scalability. The actual call, the use cases, and the requirements shall become available on the MPEG Web site.

MPEG hosts 3D Audio Workshop

Part 3 of MPEG-H will be dedicated to audio, specifically 3D audio. The call for proposals will be issues at the 102nd MPEG meeting in October 2012 and submissions will be due at the 104th meeting in April 2013. At this meeting, MPEG has hosted a 2nd workshop on 3D audio with the following speakers.

• Frank Melchior, BBC R&D: “3D Audio? - Be inspired by the Audience!”
• Kaoru Watanabe, NHK and ITU: “Advanced multichannel audio activity and requirements”
• Bert Van Daele, Auro Technologies: “3D audio content production, post production and distribution and release”
• Michael Kelly, DTS: “3D audio, objects and interactivity in games”

The report of this workshop including the presentations will be publicly available by end of August at the MPEG Web site.

What's new: Green MPEG

Finally, MPEG is starting to explore a new area which is currently referred to as Green MPEG addressing technologies to enable energy-efficient use of MPEG standards. Therefore, an Ad-hoc Group (AhG) was established with the following mandates:

1. Study the requirements and use-cases for energy efficient use of MPEG technology.
2. Solicit further evidence for the energy savings.
3. Develop reference software for Green MPEG experimentation and upload any such software to the SVN.
4. Survey possible solutions for energy-efficient video processing and presentation.
5. Explore the relationship between metadata types and coding technologies.
6. Identify new metadata that will enable additional power savings.
7. Study system-wide interactions and implications of energy-efficient processing on mobile devices.

AhGs are usually open to the public and all discussions take place via email. To subscribe please feel free to join the email reflector.