MPEG news: a report from the 105th meeting, Vienna, Austria

At the 105th MPEG meeting in Vienna, Austria, a lot of interesting things happened. First, this was not only the 105th MPEG meeting but also the 48th VCEG meeting, 14th JCT-VC meeting, 5th JCT-3V meeting, and 26th SC29 meeting bringing together more than 400 experts from more than 20 countries to discuss technical issues in the domain of coding of audio, [picture (SC29 only),] multimedia and hypermedia information. Second, it was the 3rd meeting hosted in Austria after the 62nd in July 2002 and 77th in July 2006. In 2002, “the new video coding standard being developed jointly with the ITU-T VCEG organization was promoted to Final Committee Draft (FCD)” and in 2006 "MPEG Surround completed its technical work and has been submitted for final FDIS balloting” as well as "MPEG has issued a Final Call for Proposals on MPEG-7 Query Format (MP7QF)”.

The official press release of the 105th meeting can be found here but I’d like to highlight a couple of interesting topics including research aspects covered or enabled by them. Although research efforts may lead to the standardization activities but also enables research as you may see below.

MPEG selects technology for the upcoming MPEG-H 3D audio standard
Based on the responses submitted to the Call for Proposals (CfP) on MPEG-H 3D audio, MPEG selected technology supporting content based on multiple formats, i.e., channels and objects (CO) and higher order ambisonics (HOA). All submissions have been evaluated by comprehensive and standardized subjective listening tests followed by statistical analysis of the results. Interestingly, when taking the highest bitrate of 1.2 Mb/s with a 22.2 channel configuration, both of the selected technologies have achieved excellent quality and are very close to true transparency. That is, listeners cannot differentiate between the encoded and uncompressed bitstream. A first version of the MPEG-H 3D audio standard with higher bitrates of around 1.2 Mb/s to 256 kb/s should be available by March 2014 (Committee Draft - CD), July 2014 (Draft International Standard - DIS), and January 2015 (Final Draft International Standards - FDIS), respectively.

Research topics: Although the technologies have been selected, it's still a long way until the standard gets ratified by MPEG and published by ISO/IEC. Thus, there's a lot of space for researching efficient encoding tools including the subjective quality evaluations thereof. Additionally, it may impact the way 3D Audio bitstreams are transferred from one entity to the another including file-based, streaming, on demand, and live services. Finally, within the application domain it may enable new use cases which are interesting to explore from a research point of view.

Augmented Reality Application Format reaches FDIS status

The MPEG Augmented Reality Application Format (ARAF, ISO/IEC 23000-13) enables the augmentation of the real world with synthetic media objects by combining multiple, existing standards within a single specific application format addressing certain industry needs. In particular, it combines standards providing representation formats for scene description (i.e., subset of BIFS), sensor/actuator descriptors (MPEG-V), and media formats such as audio/video coding formats. There are multiple target applications which may benefit from the MPEG ARAF standard, e.g., geolocation-based services, image-based object detection and tracking, mixed and augmented reality games and real-virtual interactive scenarios.

Research topics: Please note that MPEG ARAF only specifies the format to enable interoperability in order to support use cases enabled by this format. Hence, there are many research topics which could be associated to the application domains identified above.

What's new in Dynamic Adaptive Streaming over HTTP?

The DASH outcome of the 105th MPEG meeting comes with a couple of highlights. First, a public workshop was held on session management and control (#DASHsmc) which will be used to derive additional requirements for DASH. All position papers and presentations are publicly available here. Second, the first amendment (Amd.1) to part 1 of MPEG-DASH (ISO/IEC 23009-1:2012) has reached the final stage of standardization and together with the first corrigendum (Cor.1) and the existing part 1, the FDIS of the second edition of ISO/IEC 23009-1:201x has been approved. This includes support for event messages (e.g., to be used for live streaming and dynamic ad insertion) and a media presentation anchor which enables session mobility among others. Third and finally, the FDIS of conformance and reference software (ISO/IEC 23009-2) has been approved providing means for media presentation conformance, test vectors, a DASH access engine reference software, and various sample software tools.

Research topics: The MPEG-DASH conformance and reference software provides the ideal playground for researchers as it can be used both to generate and to consume bitstreams compliant to the standard. This playground could be used together with other open source tools from the DASH-IF, GPAC, and DASH@ITEC. An overview about DASH@ITEC's open source suite can be found here.

HEVC support in MPEG-2 Transport Stream and ISO Base Media File Format

After the completion of High Efficiency Video Coding (HEVC) - ITU-T H.265 | MPEG HEVC at the 103rd MPEG meeting in Geneva, HEVC bitstreams can be now delivered using the MPEG-2 Transport Stream (M2TS) and files based on the ISO Base Media File Format (ISOBMFF). For the latter, the scope of the Advanced Video Coding (AVC) file format has been extended to support also HEVC and this part of MPEG-4 has been renamed to Network Abstract Layer (NAL) file format. This file format now covers AVC and its family (Scalable Video Coding - SVC and Multiview Video Coding - MVC) but also HEVC.

Research topics: Research in the area of delivering audio-visual material is manifold and very well reflected in conference/workshops like ACM MMSys and Packet Video and associated journals and magazines. For these two particular standards, it would be interesting to see the efficiency of the carriage of HEVC with respect to the overhead.

Publicly available MPEG output documents

The following documents shall be come available at http://mpeg.chiariglione.org/ (availability in brackets - YY/MM/DD). If you have difficulties to access one of these documents, please feel free to contact me.

• Requirements for HEVC image sequences (13/08/02)
• Requirements for still image coding using HEVC (13/08/02)
• Text of ISO/IEC 14496-16/PDAM4 Pattern based 3D mesh compression (13/08/02)
• WD of ISO/IEC 14496-22 3rd edition (13/08/02)
• Study text of DTR of ISO/IEC 23000-14, Augmented reality reference model (13/08/02)
• Draft Test conditions for HEVC still picture coding performance evaluation (13/08/02)
• List of stereo and 3D sequences considered (13/08/02)
• Timeline and Requirements for MPEG-H Audio (13/08/02)
• Working Draft 1 of Video Coding for browsers (13/08/31)
• Test Model 1 of Video Coding for browsers (13/08/31)
• Draft Requirements for Full Gamut Content Distribution (13/08/02)
• Internet Video Coding Test Model (ITM) v 6.0 (13/08/23)
• WD 2.0 MAR Reference Model (13/08/13)
• Call for Proposals on MPEG User Description (MPEG-UD) (13/08/02)
• Use Cases for MPEG User Description (13/08/02)
• Requirements on MPEG User Description (13/08/02)
• Text of white paper on MPEG Query Format (13/07/02)
• Text of white paper on MPEG-7 AudioVisual Description Profile (AVDP) (13/07/02)

MPEG news: a report from the 102nd meeting, Shanghai, China

The 102nd MPEG meeting was held in Shanghai, China, October 15-19, 2012. The official press release can be found here (not yet available) and I would like to highlight the following topics:

• Augmented Reality Application Format (ARAF) goes DIS
• MPEG-4 has now 30 parts: Let's welcome timed text and other visual overlays
• Draft call for proposals for 3D audio 
• Green MPEG is progressing
• MPEG starts a new publicity campaign by making more working documents publicly available for free

Augmented Reality Application Format (ARAF) goes DIS

MPEG's application format dealing with augmented reality reached DIS status and is only one step away from becoming in international standard. In a nutshell, the MPEG ARAF enables to augment 2D/3D regions of scene by combining multiple/existing standards within a specific application format addressing certain industry needs. In particular, ARAF comprises three components referred to as scene, sensor/actuator, and media. The scene component is represented using a subset of MPEG-4 Part 11 (BIFS), the sensor/actuator component is defined within MPEG-V, and the media component may comprise various type of compressed (multi)media assets using different sorts of modalities and codecs.

A tutorial from Marius Preda, MPEG 3DG chair, at the Web3D conference in August 2012 is provided below.

MPEG Augmented Reality Tutorial from Marius Preda

MPEG-4 has now 30 parts

Let's welcome timed text and other visual overlays in the family of MPEG-4 standards. Part 30 of MPEG-4 - in combination with an amendment to the ISO base media file format (ISOBMFF) -  addresses the carriage of W3C TTML including its derivative SMPTE Timed Text, as well as WebVTT. The types of overlays include subtitles, captions, and other timed text and graphics. The text-based overlays include basic text and XML-based text. Additionally, the standards provides support for bitmaps, fonts, and other graphics formats such as scalable vector graphics.

Draft call for proposals for 3D audio

MPEG 3D audio is concerned about various test items ranging from 9.1 over 12.1 up to 22.1 channel configurations. A public draft call for proposals has been issued at this meeting with the goal to finalize the call and the evaluation guidelines at the next meeting. The evaluation will be conducted in two phases. Phase one for higher bitrates (1.5 Mbps to 265 kbps) is foreseen to conclude in July 2013 with the evaluation of the answers to the call and the selection of the "Reference Model 0 (RM0)" technology which will serve as a basis for the development of an 3D audio standard. The second phase targets lower bitrates (96 kbps to 48 kbps) and builds on RM0 technology after this has been documented using text and code.

Green MPEG is progressing

The idea between green MPEG is to define signaling means that enable energy efficient encoding, delivery, decoding, and/or presentation of MPEG formats (and possibly others) without the loss of Quality of Experience. Green MPEG will address this issue from an end-to-end point of view with the focus - as usual - on the decoder. However, a codec-centric design is not desirable as the energy efficiency should not be affected at the expenses of the other components of the media ecosystem. At the moment, first requirements have been defined and everyone is free to join the discussions on the email reflector within the Ad-hoc Group.

MPEG starts a new publicity campaign by making more working documents publicly available for free

As a response to national bodies comments, MPEG is starting from now on to make more documents publicly available for free. Here's a selection of these documents which are publicly available here. Note that some may have an editing period and, thus, are not available at the of writing this blog post.

• Text of ISO/IEC 14496-15:2010/DAM 2 Carriage of HEVC (2012/11/02)
• Text of ISO/IEC CD 14496-30 Timed Text and Other Visual Overlays in ISO Base Media File Format (2012/11/02)
• DIS of ISO/IEC 23000-13, Augmented Reality Application Format (2012/11/07)
• DTR of ISO/IEC 23000-14, Augmented reality reference model (2012/11/21)
• Study of ISO/IEC CD 23008-1 MPEG Media Transport (2012/11/12)
• High Efficiency Video Coding (HEVC) Test Model 9 (HM 9) Encoder Description (2012/11/30)
• Study Text of ISO/IEC DIS 23008-2 High Efficiency Video Coding (2012/11/30)
• Working Draft of HEVC Full Range Extensions (2012/11/02)
• Working Draft of HEVC Conformance (2012/11/02)
• Report of Results of the Joint Call for Proposals on Scalable High Efficiency Video Coding (SHVC) (2012/11/09)
• Draft Call for Proposals on 3D Audio (2012/10/19)
• Text of ISO/IEC 23009-1:2012 DAM 1 Support for Event Messages and Extended Audio Channel Configuration (2012/10/31)
• Internet Video Coding Test Model (ITM) v 3.0 (2012/11/02)
• Draft Requirements on MPEG User Descriptions (2012/10/19)
• Draft Use Cases for MPEG User Description (Ver. 4.0) (2012/10/19)
• Requirements on Green MPEG (2012/10/19)
• White Paper on State of the Art in compression and transmission of 3D Video (Draft) (2012/10/19)
• White Paper on Compact Descriptors for Visual Search (2012/11/09)

MPEG news: a report from the 100th meeting, Geneva, CH

The official press release is available here and I'd like to highlight two topics from MPEGs' 100th meeting in Geneva, Switzerland:

• "MP100E": MPEG celebrates its 100th meeting
• Systems news: ISOBMFF 4th edition, MDS social metadata, DASH conformance/refsw et al.
• WebVC and ARAF goes CD
• HEVC preliminary subjective test results publicly available

MPEG celebrates its 100th meeting

The 100th MPEG meeting was held in Geneva, Switzerland with a sponsored social event including honorific speeches by representatives of ISO, IEC, ITU-T, WIPO, JTC 1, SC 29, Sisvel, Samsung, and MERL. The ISO press release can be found here highlighting the win of three "Technology and Engineering Emmy Awards presented by the U.S. National Academy of Television Arts & Sciences (NATAS) for outstanding achievement in engineering and technical development. The awards covered:

• The MPEG-4 Advanced Video Coding (AVC) standard. The related ITU-T Video Coding Experts Group also received an Emmy Award for its role in the work on this standard
• MPEG-1 and MPEG-2 – compression coding associated with video CD and MP3, digital TV set top boxes and DVD."

Some additional pictures can be found here.

Systems news

ISOBMFF 4th edition: The "Text of ISO/IEC 14496-12 4th edition" has been approved which means the the ISO base media file format is available in its fourth edition (i.e., a consolidation of Amd.1, Amd.2, Amd.3, Cor.1-5). As previous editions, it shall become publicly available via ITTF Web site.

MDS social metadata: The fourth amendment of MPEG-7 MDS has been approved providing support  for social metadata such as ratings (like, 5-star) based on media, identity, and quality.

DASH conference/reference software: The committee draft for ISO/IEC 23009-2 has been approved adding conformance and reference software to DASH. In particular, MPD and segment conformance rules are defined and bitstreams are available. Furthermore, libdash has been selected as sample client. A public online MPD validator is available at http://dash.itec.aau.at.

WebVC and ARAF goes CD

WebVC: The committee draft of MPEG-4 Part 29 aka Web Video Coding (WebVC) has been approved which is compatible with the Constrained Baseline Profile of ISO/IEC 14996-10. In fact, WebVC is derived from the AVC specification with the purpose to define a coding format suitable for the Web. The CD is publicly available here.

ARAF: MPEG addresses the need for standards in the area of augmented realities by defining an application format. Hence, ARAF stands for Augmented Reality Application Format and will be defined as Part 13 of MPEG-A. The committee draft is publicly available here addressing use cases and requirements.

HEVC preliminary subjective test results publicly available

"The video subgroup recommends making the report on preliminary subjective testing N12475 publicly available."

N12475 is the Report on preliminary subjective testing of HEVC compression capability which can be found here. It shows impressive results as reported elsewhere, e.g., here. In particular, > 50% bitrate reduction, 67% in class B (HDTV), 49% in class C (WVGA) => mission accomplished! Currently, HEVC is between ballots and FDIS/IS is expected around Jan-Apr 2013.