Summary of results of the first JCT-VC meeting in Dresden

--text adopted from Gary Sullivan

JCT-VC stands for Joint Collaborative Team on Video Coding and is basically the successors of the Joint Video Team (JVT) which was responsible for standardizing the award-winning Advanced Video Coding (AVC) standard. Last week, the JCT-VC met the first time in Dresden for the evaluation of the joint call for proposals on video compression technology and Gary Sullivan provided a nice summary over the general JCT-VC reflector which is excerpted here.

The name for the new standardization project is now "High Efficiency Video Coding" (HEVC).

The Joint Call for Proposals (CfP) on Video Compression Technology, which was issued by ISO/IEC MPEG and ITU-T VCEG in January 2010, had a very successful outcome. Twenty-seven complete proposal submissions were received, and the associated video material was evaluated in extensive subjective tests that were conducted prior to the first meeting of the Joint Collaborative Team on Video Coding (JCT-VC). That meeting was held under MPEG auspices in Dresden during 15-23 April 2010. The contribution documents for those proposals are at the above link with contribution numbers JCTVC-A101 to JCTVC-A127.

The test results clearly indicated that some proposals exhibited a substantial improvement in compression performance as compared to the corresponding AVC anchors - and, in a number of cases, the performance of the best proposals can be roughly characterized as achieving similar quality when using only half of the bit rate.

All proposals basically used a coding architecture that was conceptually similar to AVC (and prior video coding standards), containing the following basic elements:

• Block-based
• Variable block sizes
• Block motion compensation
• Fractional-pel motion vectors
• Spatial intra prediction
• Spatial transform of residual difference
• Integer-based transform designs
• Arithmetic or VLC-based entropy coding
• In-loop filtering to form final decoded picture

However, there was a large variety of differences at the individual coding tool level.

For further details, please subscribe to the JCT-VC reflector and look for the corresponding email in the archives.

Heil The New xVC Heir: HVC

For the record and also as an update, here is my guest post at RADVISION's Video over Enterprise blog.

"The HVC efforts have already begun. Based on the current timeline, one can expect the new standard to be available around the end of 2012/beginning of 2013. This may seem far away, but as many video infrastructure products have a 2 year design process, this is very relevant to today’s design efforts.
And so we are looking forward to a new and very exciting xVC episode, and it will be very interesting to see how the new standard evolves and whether it will fit today’s expectations. Thus, stay tuned!"

The latest information about High-performance Video Coding (HVC) can be found at the MPEG Web site and here are links to the publicly available documents:

• Joint Call for Proposals on High Performance Video Coding
• Vision and Requirements for High-Performance Video Coding (HVC)
• Call for Evidence on High-Performance Video Coding
• Call for Test Materials for High-Performance Video Coding Standards Development

Most interestingly, the joint CfP on HVC - joint between MPEG and ITU-T - and the vision and requirements documents. If you're interested in the upcoming discussions around HVC you may join the mailing list or even the #HVC Twitter stream (alt) ;-)

MPEG targets next generation teleconferencing and personalized audio in its newest technology

Kyoto, Japan – The 91st MPEG meeting was held in Kyoto, Japan from the 18th to the 22nd of January 2010.

Highlights of the 91st Meeting

• Advanced technology enables bitrate-efficient teleconferencing and personalized audio
• MPEG and ITU-T SG16 form Joint Collaborative Team (JCT) for High Performance Video Coding
• Creating solutions for Advanced IPTV Terminals
• MPEG Extensible Middleware (MXM) standard debuts to facilitate easy access to MPEG tools across a global market
• MPEG creates efficient and flexible standard for 3D graphics compression
• MPEG Media Transport (MMT) workshop targets requirements and technologies for streaming of MPEG content

Wanna know more/details? Please check out the official press release!

Overview of Selected Current MPEG Activities

--this covers a report from the 91st MPEG meeting in Kyoto, Japan

Previously, I've always provided a written report but this time it comes in form of a presentation (slideshow) - enjoy!

View more presentations from Klagenfurt University.

Draft Call for Proposals on High-Performance Video Coding (HVC)

--excerpted from here!

Purpose: A new generation of video compression technology that has substantially higher compression capability than the existing AVC (ITU-T H.264 | ISO/IEC 14496-10) standard is targeted (see [1][2] for background information).

Timeline:

• 2009/11/06: Draft CfP
• 2009/11/06: Availability of all test materials defined in Draft CfP (download information can be requested from contact persons)
• 2009/11/30: Availability of anchors
• 2010/01/15: Deadline for Pre-registration (mandatory): one of the contact persons (see Section 9) must be notified An invoice for the testing fee will be sent after pre-registration. Additional logistic information will also be sent to proponents by this date.
• 2010/01/22: Final Call for Proposals (Note: The Final Call may contain minor updates)
• 2010/02/15: Formal registration and payment of the fee
  
• 2010/02/22: Coded test material available at the test site (Note: People who formally registered will receive instructions regarding how to submit the coded materials)
  
• 2010/03/15: Subjective assessment starts at this date or sooner
• 2010/04/12: Registration of documents describing the proposals with contact persons
• 2010/04/13: Submission of documents to contact persons (Note: Contact persons will compile all proposal documents into one or more input documents and upload)
  
• 2010/04/16: Cross-checking of bitstreams and binary decoders (participation mandatory for proponents)
• 2010/04/17: Subjective test results available within standardization body
• 2010/04/17-23: Evaluation of proposals at standardization meeting (Note: Proponents are requested to attend this standardization meeting)
  

Anticipated tentative timeline after CfP
(refers to first version of the new standard, which may be extended later):

• Core experiments begin 2010/04
• Test model 2010/10
• Final standard approval 2012/07
  

Test classes:

• Class A: Cropped areas of size 2560x1600 taken from the following sequences (frame rates unchanged): First 5 s of "Traffic" (4096x2048p 30 fps), "PeopleOnStreet" (3840x2160p 30 fps).
• Class B: 1920x1080p 24 fps: "ParkScene", "Kimono" and 1920x1080p 50-60 fps: "Cactus", "BasketballDrive", "BQTerrace"
• Class C: 832x480p 30-60 fps (WVGA): "BasketballDrill", "BQMall", "PartyScene", "RaceHorses"
• Class D: 416x240p 30-60 fps (WQVGA): "BasketballPass", "BQSquare", "BlowingBubbles", "RaceHorses"
• Class E: 1280x720p 60fps: "Vidyo1", "Vidyo3" and "Vidyo4"
  

For further information about constraint sets, target rate points, and anchors, please go to directly the Draft CfP.

Test sites and fees:
The proposals submission material for Classes B, C and E will be evaluated by means of a formal subjective assessment process. The tests will be conducted at FUB (Test Coordinator, Rome, Italy), EBU (Geneva, Switzerland) and EPFL (Lausanne, Switzerland).

Proponents will be charged a fee per submitted algorithm proposal. Such fee will be a flat charge for each proposal to cover the logistic cost (without any profit). The fee is non-refundable after the formal registration deadline has passed. The fee is estimated to be in the range of 4000 to 6000 Euros. The exact cost can only be determined after the number of submitted proposals is known, but over-payments will be reimbursed.

Additional requirements on submissions can be found here.

References:
[1] VCEG-AL96, Draft Requirements for Next Generation Video Coding, July 2009
[2] ISO/IEC JTC1/SC29/WG11, Vision and Requirements for High-Performance Video Coding, MPEG Document N10361, February 2009

MPEG news: a report from the 90th meeting in Xi'an, China

The 90th MPEG meeting in Xi’an, China is coming up with some very interesting news which are briefly highlighted here. First and, I think, most importantly, the timeline for the new MPEG/ITU-T video coding format has been discussed and it seems the final Call for Proposals (CfP) will be ready in January 2010. A draft CfP is available now and hopefully will be also publicly available if they solve all the editing issues until early November. This means that the proposals will be evaluated in April 2010 (note: this will be a busy meeting as a couple of other calls need to be evaluated too; see later). The CfP defines five classes of test sequences with the following characteristics (number of sequences available in brackets):

• Class A with 2560x1600 cropped from 4Kx2K (2);
• Class B with 1920x1080p at 24/50-60 fps (5);
• Class C with 832x480 WVGA (4);
• Class D with 416x240 WQVGA (4); and
• Class E with 1280x720p at 50-60fps (3).

For classes B, C, and E subjective tests will be performed whereas classes A and D will be only evaluated objectively using PSNR. The reason for evaluating A and D using objective measurements is due to its insignificant subjective differences with B and C respectively. Finally, they’re still discussing about the actual common nickname name of the standard as it seems some are not happy with high-performance video coding but that’s yet another story…

Second, 3D video coding is still a major topic in MPEG but you probably need to wait yet another year until a Call for Proposals will be issued. That is, a 3DV standard will be probably available around the beginning of 2013 at the earliest. The major issue right now is the availability of content – as usual – and different device manufacturer standards with respect to 3D video.

The third major topic at this meeting was around AIT and MMT, two acronyms you shall become more familiar in the future. The former is referred to as Advanced IPTV Terminal (AIT) and aims to develop an ecosystem for media value chains and networks. Therefore, basic (atomic) services will be defined including protocols (payload formats) to enable users to call these services, Application Programming Interfaces to access services, and bindings to specific programming languages. Currently, 30 of these basic services are foreseen which can be clustered in services pertaining to the identification, authentication, description, storage, adaptation, posting, packaging, delivery, presentation, interaction, aggregation, management, search, negotiation, and transaction. The timeline is similar as for HVC which means that proposals will be evaluated in April 2010. The latter is referred to as MPEG Media Transport (MMT) and basically aims to become a successor of the well-known MPEG-2 Transport Stream. Currently, two topics are explored for which also requirements have been formulated. The first topic covers adaptive, progressive transport and the second topic is in the area of cross-layer design. Further topics where this activity might look into are hybrid delivery and conversational services. As for HVC and AIT, the proposals are going to be evaluated in April 2010. However, in order to further refine this possible new work item, MPEG will held a workshop in January 2010 on the Wednesday during the Kyoto meeting focusing on “adaptive progressive transport” and “cross-layer design”.

However, MPEG is looking forward to a very busy meeting in April 2010 which by the way will be held in Dresden, Germany.

Another issue that has been discussed in Xi’an was (again) the development of a royalty free codec within MPEG. While some might say that within MPEG, trying to establish a royalty free codec is a first step towards failure, others argue that MPEG-1 is already royalty free, for MPEG-2 most patents expire in 2011, the Internet community is requesting this (e.g., IETF established coded group and Google has chosen On2, a royalty free codec), and, finally, MPEG-4 Part 10 royalty free baseline basically failed. Thus, maybe (or hopefully) this is the right time for a royalty free codec within MPEG and who can predict the future? Anyway, there’s some activity going on in this area and if you’re interested, stay tuned…

Finally, I’d like to note that MPEG-V (Media Context & Control) and MPEG-U (Rich Media User Interface) are progressing smoothly and both going hand in hand towards its finalization. This meeting, the FCDs have been approved which forms a major milestone as this was the last chance for substantial new contributions. One such input was related to advanced user interaction like the Wiimote, etc. which will become part of MPEG-V but used also by MPEG-U. Hence, one might argue merging these two standards into one single standard called MPEG-W (i.e., U+V=W) and a wedding ceremony could be performed at the next meeting in Kyoto with Geishas as witnesses … why not? Please raise your voice now or be silent forever!

MPEG Global Conference points the way to Ultra HD online services

Source: http://www.chiariglione.org/mpeg/meetings/london09/london_press.htm

London meeting sees significant improvement in compression for High Performance Video Coding

London, United Kingdom – The 89th MPEG meeting was held in London, United Kingdom from the 29th of June to the 3rd of July 2009.

Highlights of the 89th Meeting

Responses for Evidence Evaluated for HVC
During its 89th meeting, MPEG evaluated responses that were received on the Call for Evidence on High-Performance Video Coding (HVC), issued to obtain evidence of video coding technology providing compression capability clearly higher than that provided by the existing AVC standard (ITU-T H.264 | ISO/IEC 14496-10). Significant gains in compression were found when an assessment was made based on information brought by the contributors. A subjective comparison was performed in a blind test with a set of video test sequences encoded by the AVC High Profile at matching rate points. Gains were demonstrated for test cases ranging from resolutions as low as 412x240 pixels (Wide QVGA) up to resolutions for ultra-high definition. MPEG has therefore concluded that the development of the next generation of video compression technology is to be started with the issuing of a formal Call for Proposals by the next meeting.

AVC Extended with New Profiles for Baseline and MVC Technologies

At the 89th meeting, the AVC standard (ITU-T H.264 | ISO/IEC 14496-10) was further extended with the issuing of a Final Draft Amendment (FDAM) ballot containing the specification of two new profiles and new supplemental enhancement information. The first of the new profiles is the Constrained Baseline Profile, which forms the maximally-interoperable set of coding tools from the most widely deployed of existing profiles (the Baseline and High Profiles). The second new profile is a special case of multivew video coding (MVC) called the Stereo High Profile. The Stereo High profile enables all of the coding tools of the High Profile along with inter-view prediction capability for two-view (stereo) video applications such as 3D entertainment video.

Additionally, a new supplemental enhancement information (SEI) message has been defined for AVC. This new message – called the frame packing arrangement SEI message – enables the encoder to indicate to the decoder how to extract two distinct views of a video scene from a single decoded frame. The message also serves as a way to support stereo-view video in applications that require full compatibility with prior decoder designs that are not capable of supporting the new Stereo High Profile.

MPEG Promotes Technologies to link Real and Virtual Worlds

At its 88th meeting, MPEG had published a new call for proposals (N10526) with updated requirements (N10235) for an extension of the Media Context and Control project.

The technical contributions related to haptic and tactile devices, emotions, and virtual goods received at its 89th meeting have enabled MPEG to build a complete framework for defining haptic properties on top of virtual objects and to control haptic devices. This is now part of ISO/IEC 23005 or MPEG-V, a standard (formerly called Information Exchange with Virtual Worlds) providing a global framework and associated data representations to enable the interoperability between different virtual worlds (e.g. a digital content provider of a virtual world, a game with the exchange of real currency, or a simulator) and between virtual worlds and the real world (sensors, actuators, robotics, travel, real estate, or other physical systems).

MPEG Progresses Media Context and Control Project

MPEG has also advanced to the Committee Draft stage four parts of MPEG-V. The first part describes the architecture of the standard. The second part, “Control Information”, provides metadata representation of device capabilities and user preferences to be used for the information exchange between a controlling device and the real actuator or sensors. The third part, “Sensory Information”, provides metadata to represent sensory effects such as temperature, wind, vibration, fog, and more. The fourth part, “Avatar Characteristics”, provides metadata to commonly represent information about Avatars for the exchange of virtual characters between virtual worlds.

MPEG Hosts MXM Developer’s Day

The first MXM Developer’s Day workshop has been hosted by MPEG during its 89th meeting. The workshop featured demonstrations by companies and organisations that are developing MXM standards and applications. MXM, currently at its Final Committee Draft stage, provides specifications of APIs and an open source implementation (released under the BSD licence) to access various MPEG standards for easy deployment of applications. In this workshop detailed information about the APIs currently under standardization has been provided and several interesting demonstrations with the potential to create new business opportunities have also been presented. More information about this workshop can be found at http://mxm.wg11.sc29.org.

Rich Media User Interface Moves toward Completion

At its 89th meeting, MPEG has also advanced MPEG Rich Media UI (ISO/IEC 23007 or MPEG-U), to the Committee Draft stage. MPEG-U standardizes widget packaging, delivery, representation and communication formats. In its current draft, MPEG-U adopts and extends the W3C widget representation to provide a complete framework that can be used also in a non-Web based environment without a browser. Additionally, this standard enables communication among widgets on the same device or different devices, and other applications to better support connected environments.

Visual Signatures Enable New Applications

MPEG’s Visual Signatures define the world’s first standardized tools for content-based identification of any visual content even in very large databases, e.g. on the web. These tools enable a range of new applications including semantic linking, library management, metadata association (e.g. title, photographer, director, etc.) and content usage tracking. In the same way that a fingerprint or signature identifies a person, a Visual Signature is a compact descriptor uniquely representing either an image or video. The descriptor is derived directly from analysis of the visual content and is robust to heavy compression and editing.

The Image Signature and Video Signature are two separate amendments to MPEG-7. Collectively the two amendments are referred to as the MPEG-7 Visual Signatures. At the London meeting, the Video Signature advanced to the Proposed Draft Amendment (PDAM) stage with a target completion date of July 2010. The Image Signature was published as an ISO/IEC standard in April 2009.

Mobile Services to Be Enhanced by New BIFS Profile
At this meeting, MPEG advanced the new BInary Format for Scenes (BIFS) profile to the Committee Draft stage by incorporating additional nodes and technologies submitted as responses to the Call for Proposals for new BIFS technologies. The requirements for this profile (provided in N10567) originated from organizations of various industries and SDOs for digital radio and mobile television broadcasting. This profile will enable the development of more efficient and enhanced interactive services for mobile broadcasting services including digital radio or mobile television on small handheld devices. Moreover, it is backward compatible with Core2D@Level1 which is widely adopted by the industry.

Contact MPEG

Digging Deeper Once Again
Communicating the large and sometimes complex array of technology that the MPEG Committee has developed is not a simple task. The experts past and present have contributed a series of white-papers that explain each of these standards individually. The repository is growing with each meeting, so if something you are interested is not there yet, it may appear there shortly – but you should also not hesitate to request it. You can start your MPEG adventure at: http://www.chiariglione.org/mpeg/mpeg-tech.htm

Ends

Further Information
Future MPEG meetings are planned as follows:
No. 90, Xian, CN, 26-30 October, 2009
No. 91, Kyoto, JP, 18-22 January, 2010
For further information about MPEG, please contact:
Dr. Leonardo Chiariglione (Convener of MPEG, Italy)
Via Borgionera, 103
10040 Villar Dora (TO), Italy
Tel: +39 011 935 04 61
Email: mailto:leonardo@chiariglione.org
or
Dr. Arianne T. Hinds
Ricoh | IBM InfoPrint Solutions Company
6300 Diagonal Highway, MS 04N
Boulder, CO 80301, USA
Tel +1 720 663 3565
Email: arianne.hinds@infoprint.com

This press release and other MPEG-related information can be found on the MPEG homepage:
http://www.chiariglione.org/mpeg
The text and details related to the Calls mentioned above (together with other current Calls) are in the Hot News section, http://www.chiariglione.org/mpeg/hot_news.htm. These documents include information on how to respond to the Calls.
The MPEG homepage also has links to other MPEG pages which are maintained by the MPEG subgroups. It also contains links to public documents that are freely available for download by those who are not MPEG members. Journalists that wish to receive MPEG Press Releases by email should contact Dr. Arianne T. Hinds using the contact information provided above.

AhG on High-Performance Video Coding

A new interesting work item in the area of video coding is about to take off. The Ad-hoc Group (AhG) on High-performance Video Coding will discuss the following mandates:

1. To further discuss vision, applications and requirements of high-performance video coding.
2. To finalize the report on the Call for Evidence results N10721.
3. To discuss and refine the Draft Call for Proposals N10722, particularly with respect to communication coming from VCEG.
4. To organize the distribution of test sequences and anchors listed in the Draft CfP.
5. To actively seek for more test material according to N10362.

This activity will be chaired by Jens-Rainer Ohm, Vittorio Baroncini, and T.K. Tan and you may subscribe to its reflector via http://mailman.rwth-aachen.de/mailman/listinfo/mpeg-newvid.

Note that there will be an AhG meeting on Saturday and Sunday before the 90^th MPEG meeting.

MPEG news: a report from the 89th meeting in London, UK

A lot of interesting things happened at this meeting, notably the MXM Developer's Day, the Modern Media Transport workshop, MPEG-V and MPEG-U have been promoted to committee draft, and for MPEG High-performance Video Coding (HVC) enough evidence has been provided in order to start working towards a Call for Proposals (CfP).

The MXM Developer's Day was a great success with 45+ participants and all presentations are publicly available. Leonardo presented the MXM Vision while Filippo and Marius concentrated on the MXM Architecture and API respectively. This introductory session was followed by practical examples and demonstrations:

• Fully Interoperable Streaming of Media Resources in Heterogeneous Environments
• Including MPEG-4 3D graphics in your application
• An MXM-based application for sharing protected content

The workshop on Modern Media Transport (MMT) had even more participants (80+) and was clustered into two session. Session one was focusing on industry practice and presentations where given on how MPEG-2 TS and MP4 is being used. Furthermore, the DVB activity in the area of IPTV and InternetTV was presented. All the presentations will be publicly available through the MPEG Web site. The conclusion was that although MPEG-2 TS / MP4 is heavily used, it has some drawbacks due to their popularity. That is, MPEG-2 TS is running out of code points which is an issue. On the other hand, if MPEG is going to standardize something new, it has been recognized that it has to be to substantially better than what exists on the market with a high demand of backwards-compatibility to MPEG-2 TS. The issue will be further studied and stay tuned!

MPEG-V also known as Media Context and Control has promoted four parts to committee draft. The four parts are as follows:

• Part 1: Architecture
• Part 2: Control Information
• Part 3: Sensory Information
• Part 4: Avatar Characteristics

I've provided an overview during the final plenary and the presentation is accessible here.

MPEG-U is about Widgets and has been promoted to committee draft also. It seems to be an interesting activity which has a relationship to W3C's Widget activity. It will be interesting to see how these two standards co-exist.

Finally, the call for evidence for High-performance Video Coding (HVC) provided the following result: "Yes, we have enough evidence about improved compression technology (compared to AVC HP)". Thus, MPEG started working towards a call for proposals and a time schedule has been created. Furthermore, the future collaboration between MPEG and VCEG has been discussed.

That's it for now but I'll provide more details on the individual topics later. Please stay tuned!

Preparation of Call for Evidence in London

Just received this via the AhG on High-Performance Video Coding and so far they have
received 12 expressions of interest to participate in the Call for Evidence. They will have an AhG meeting on Sat-Sun prior to the next meeting where they will start reviewing the inputs, and in parallel start the real evaluation viewing with small groups of experts.

To subscribe or unsubscribe for this AhG, go to http://mailman.rwth-aachen.de/mailman/listinfo/mpeg-newvid.

MPEG Press Release: MPEG Explores New Technologies for High Performance Video Coding (HVC)

Maui, Hawaii, USA – The 88^th MPEG meeting was held in Maui, Hawaii, USA from the 20^th to the 24^th of April 2009.

Highlights of the 88^th Meeting

Call for Evidence of Technologies Issued for HVC

Technology evolution will soon make possible the capture and display of video material with a quantum leap in quality when compared to the quality of HDTV. However, networks are already finding it difficult to carry HDTV content to end users at data rates that are economical. Therefore, a further increase of data rates, such as soon will be possible, will put additional pressure on the networks. For example:

· High-definition (HD) displays and cameras are affordable for consumer usage today, while the currently available internet and broadcast network capacity is not sufficient to transfer large amount of HD content economically. While this situation may change slowly over time, the next generation of ultra-HD (UHD) contents and devices, such as 4Kx2K displays for home cinema applications and digital cameras, are already appearing on the horizon.

· For mobile terminals, lightweight HD resolutions such as 720p or beyond will be introduced to provide perceptual quality similar to that of home applications. Lack of sufficient data rates as well as the prices to be paid for transmission will remain a problem for the long term.

MPEG has concluded that video bitrate (when current compression technology is used) will go up faster than the network infrastructure will be able to carry economically, both for wireless and wired networks. Therefore a new generation of video compression technology with sufficiently higher compression capability than the existing AVC standard in its best configuration (the High Profile), is needed. Such High-Performance Video Coding (HVC) would be intended mainly for high quality applications, by providing performance improvements in terms of coding efficiency at higher resolutions, with applicability for entertainment-quality services such as HD mobile, home cinema and Ultra High Definition (UHD) TV.

To start a more rigorous assessment about the feasibility of HVC, a Call for Evidence has been issued, with the expectation that responses would report about the existence of technologies that would be able to fulfill the aforementioned goals. A set of appropriate test materials and rate points that would match the requirements of HVC application scenarios has been defined. Responses to this call will be evaluated at the 89^th MPEG meeting in July 2009. Depending on the outcome of this Call for Evidence, MPEG may issue a Draft Call for Proposals by the end of its 89^th meeting. The Call for Evidence can be found as document ISO/IEC JTC1/SC29/WG11 N10553 at http://www.chiariglione.org/mpeg/hot_news.htm

MPEG Seeks Technologies to link Real and Virtual Worlds

At its 88^th meeting, MPEG has published updated requirements (ISO/IEC JTC1/SC29/WG11 N10235) and issued an extended call for proposals (ISO/IEC JTC1/SC29/WG11 N10526) for an extension of the Media Context and Control project (ISO/IEC 23005 or MPEG-V) to standardize intermediate formats and protocols for the exchange of information between the (real) physical and virtual worlds. In particular, this extended call for proposals seeks technologies related to haptics and tactile, emotions, and virtual goods. Specifically, the goal of this project (formally called Information Exchange with Virtual Worlds) is to provide a standardized global framework and associated data representations to enable the interoperability between different virtual worlds (e.g. a digital content provider of a virtual world, a game with the exchange of real currency, or a simulator) and between virtual worlds and the real world (sensors, actuators, robotics, travel, real estate, or other physical systems). MPEG invites all parties with relevant technologies to submit these technologies for consideration. For more information, refer to the above documents, which are available at http://www.chiariglione.org/mpeg/hot_news.htm.

Digital Radio Service to be Extended with new BIFS

At its 88^th meeting, MPEG has been informed by the digital radio industry of the increasing need for a new interactive BInary Format for Scenes (BIFS) service for digital radio. This new service will enable the presentation of supplemental information like EPG or advertisements on radios with displays capable of supporting this service. In addition, such displays may be used for controlling the radio.

In order to fulfill the additional requirements for this new service, MPEG has issued a Call for Proposals for new BIFS technologies in N10568. The result of this call will be used to define a new amendment for BIFS and a profile, including the new technologies, backward compatible with Core2D@level1.

The requirements for Interactive Services for Digital Radio can be found in document ISO/IEC JTC1/SC29/WG11 N10567 available at http://www.chiariglione.org/mpeg/hot_news.htm.

New Presentation Element Added to Multimedia Framework

The MPEG-21 Multimedia Framework already provides flexible and efficient ways to package multimedia resources and associated metadata in a structured manner. At its 88^th meeting, MPEG advanced to the formal approval stage a new amendment to MPEG-21 (ISO/IEC JTC1/SC29/WG11 21000-2 PDAM 1 Presentation of Digital Item) to define a new element that can be used to provide information relevant to the presentation of multimedia resources. Specifically, the new element, called Presentation, will describe multimedia resources in terms of their spatio-temporal relationships and their interactions with users. In a related effort, MPEG also began the formal approval process for another amendment to MPEG-21 (ISO/IEC JTC1/SC29/WG11 21000-4 PDAM 2 Protection of Presentation Element) so that the new Presentation element can be associated with the Intellectual Property Management and Protection (IPMP) element for content protection and management.

Other Notable MPEG Events

MPEG Plans First MXM Developer’s Day

The first International MPEG Extensible Middleware (MXM) Developer’s Day will be held on 30 June at the Queen Mary University, London, U.K. The purpose of this event is to share with the software developer’s community the state of the art and the prospects of MPEG Extensible Middleware, a standard designed to promote the extended use of digital media content through increased interoperability and accelerated development of components, solutions, and applications. The event is free of charge. For more information, or to register, visit http://mxm.wg11.sc29.org.

MMT Workshop Targets Requirements for Streaming of MPEG Content

The Workshop for MPEG Media Transport (MMT) will be held on 1 July during the 89^th MPEG meeting at the Queen Mary University in London, U.K. The purpose of this event is to gather new requirements, use cases, and contributions related to the transport of multimedia content over heterogeneous networks. In particular, MPEG is gathering information on current limitations of available standards in the area of media streaming and associated challenges in emerging network environments. The MMT workshop is also free of charge. For more information, visit http://www.chiariglione.org/mpeg/hot_news.htm

Communicating the large and sometimes complex array of technology that the MPEG Committee has developed is not a simple task. The experts past and present have contributed a series of white-papers that explain each of these standards individually. The repository is growing with each meeting, so if something you are interested is not there yet, it may appear there shortly – but you should also not hesitate to request it. You can start your MPEG adventure at: http://www.chiariglione.org/mpeg/mpeg-tech.htm

Future MPEG meetings are planned as follows:

For further information about MPEG, please contact:

mailto:leonardo(at)chiariglione(dot)org

or

This press release and other MPEG-related information can be found on the MPEG homepage:

http://www.chiariglione.org/mpeg

The text and details related to the Calls mentioned above (together with other current Calls) are in the Hot News section, http://www.chiariglione.org/mpeg/hot_news.htm. These documents include information on how to respond to the Calls.

The MPEG homepage also has links to other MPEG pages which are maintained by the MPEG subgroups. It also contains links to public documents that are freely available for download by those who are not MPEG members. Journalists that wish to receive MPEG Press Releases by email should contact Dr. Arianne T. Hinds using the contact information provided above.

MPEG news: a report from the 88th meeting in Ka'anapali, HI, USA

MPEG-V has been renamed to Media Context & Control, a workshop on Modern Media Transport (MMT) will be held at the next meeting in London as well as an MXM Developer's Day, for High-Performance Video Coding (HVC) a Call for Evidence has been issues, and MPEG is about to start a new work item in the area of Rich Media User Interface. These are my headlines from the MPEG meeting last week at the beautiful Ka'anapali Beach.

MPEG-V, formerly known as Information Exchange with Virtual Worlds has been finally renamed to Media Context and Control. I think this is generic enough to cover everything in there now and what will come in the future. If you think this is too broad, then remember the name of MPEG-21 is just Multimedia Framework. Btw. an extended call for proposals has been issues for technolgies related to haptics & tactile, emotions, and virtual goods. For the rest, we have new working drafts for architecture, control information, sensory information, avatar information, reference software, and conformance.

At its next meeting, MPEG will host two sort of workshops which are also open to the public. The first one is related to the MPEG Extensible Middleware (MXM) and the second related to Modern Media Transport. I'm a co-organizer of the former but you'll also find me at the latter.

The Call for Evidence on High-Performance Video Coding has been issued with the following timeline

• Test sequences and AVC High Profile anchors available: 2009-04-30
• Expression of interest to participate: 2009-06-01
• Submission of contributions (descriptive document): 2009-06-22
• Decoded sequences, bitstreams and binary decoders available at London by 2009-06-27
• Evaluation of responses: July 2009 MPEG meeting (shall be attended by submitters)
• Depending on the outcome of the Call for Evidence, MPEG intents to issue a Draft Call for Proposals by the end of the July meeting

Finally, a first working draft for the MPEG Rich Media User Interface has been issued and a nice demo has been presented at the meeting in the area of widgets. More to come on this soon. However, in my view there's a strong relationship with W3C's Rich Web Client Activity which will be studied during the course of developing this standard.

If you ask yourself why I hardly ever post something about audio, the reason is very simply, I'm a video and systems guy but if you'd like me to dig into audio stuff, please let me know!

Call for Evidence on High-Performance Video Coding (HVC)

--excerpted from N10553 which can be found on http://www.chiariglione.org/MPEG/

1 Introduction

A large quantity of video material is already distributed in digital over broadcast channels, digital networks and packaged media. More and more of this material will be distributed with increased resolution and quality demand.

Technology evolution will soon make possible the capture and display of video material with a quantum leap in quality (temporal and spatial resolution, color fidelity, amplitude resolution). Networks are already finding it difficult to carry HDTV resolution and data rates economically to the end user. Therefore, further data rate increase will put additional pressure on the networks. For example:

• High-definition (HD) devices (displays and cameras) are affordable for consumer usage today, while the currently available internet and broadcast network capacity is not sufficient to transfer large amount of HD content economically. While this situation may change slowly over time, the next generation of ultra-HD (UHD) contents and devices, such as 4Kx2K displays for home cinema applications and digital cameras, are already appearing on the horizon.
• For mobile terminals, video quality using resolutions such as QCIF at low frame rates and low bit rates today is largely unacceptable. While the overall data rate will increase with the evolution of 3G/LTE and 4G networks, also the number of users increases simultaneously with their quality demand. Anticipating that lightweight HD resolutions such as 720p or even beyond will be introduced in the mobile sector to provide similar perceptual quality as for the home applications, lack of sufficient data rates as well as the prices to be paid for transmission will remain a problem for the long term.
  

MPEG has concluded that video bitrate (when current compression technology is used) will go up faster than the network infrastructure will be able to carry economically, both for wireless and wired networks. Therefore a new generation of video compression technology that has sufficiently higher compression capability than the existing AVC standard in its best configuration (the High Profile), is needed. A study has been started on the feasibility of HVC, which is mainly intended for high quality applications, in particular expecting

• Performance improvements in terms of coding efficiency at higher resolution,
• Applicability to entertainment-quality services such as HD mobile, home cinema and Ultra High Definition (UHD) TV.
  

First results that were reported to MPEG indicate that compression technology giving higher compression performance than AVC might exist for such application cases. It is therefore planned to enter a more rigorous evaluation phase, with a Call for Evidence on new high-performance video compression technology potentially followed by a formal Call for Proposals.

Companies and organizations who have developed compression technology that they believe to be better than AVC High Profile are kindly invited to bring such information to MPEG in the context of this Call for Evidence. The main purpose of a possible subsequent Call for Proposals and standards development will be to increase compression performance.

Timeline:

• Test sequences and AVC High Profile anchors available: 2009-04-30
• Expression of interest to participate: 2009-06-01
• Submission of contributions (descriptive document): 2009-06-22
• Decoded sequences, bitstreams and binary decoders available at London by 2009-06-27
• Evaluation of responses: July 2009 MPEG meeting (shall be attended by submitters)
• Depending on the outcome of the Call for Evidence, MPEG intents to issue a Draft Call for Proposals by the end of the July meeting
  

2 Test cases

Submitters are encouraged but not required to submit results for all test cases. However, submitters are required to provide results for all sequences in a given class.

2.1 Sequence formats and frame rates

Test material is progressively scanned in 4:2:0 color sampling and 8 bit per component.

The classes of video sequences are:
A) Cropped areas of size 2560x1600 taken from the following sequences (frame rates unchanged): Traffic (4096x2048/30), People on Street (3840x2160/30), Park Joy (3840x2160/50).

B) 1920x1080@24 frames per second: ParkScene, Tennis, Kimono1

C) 832x480@30 frames per second (WVGA): Mobisode 2, Keiba, Flowervase

D) Downsampled versions to half spatial resolution (416x240@30fps, WQVGA) of sequences in C)

2.2 Coding Conditions of AVC anchors
Anchors have been generated by encoding the above sequences using an AVC High Profile encoder (JM15.1) with

• QP settings: 25, 29, 33 and 37 for class A, 25, 28, 31 and 34 for class B, 25, 29, 33 and 37 for classes C, 25, 29, 33 and 37 for classes D
• GOP with an Intra picture every 24, 28 and 48 pictures for 24 fps, for 30 fps and for 50 fps sequences, respectively
• Hierarchical B pictures IbBbP coding structure
• 4 reference pictures
• RD Optimization enabled
• RDOQ enabled (fast mode)
• Weighted prediction enabled
• Fast motion estimation. (range 128x128)
  

For more precise information, refer to config files that come with the anchors.

Submissions to the call shall

• Be encoded to within +/-5 % of the target bit rates defined above
• Allow for random access at intervals not more than 1 second apart
• Not use preprocessing
• Only use postfilter if it is part of the decoding process.
  

3 Evaluation methodology

The test method most suitable when assessing video at high visual quality levels is the DSCQS test method as described in ITU-R Recommendation BT 500-11. This method is well known and properly described in the text of Recommendation BT-500, therefore this document does not includes further details on this method and its usage.
To speed up the evaluation of results of the Call for Evidence, we are considering a new effective approach to formal subjective testing of video, which is described in Annex C.

In addition, proponents are required to submit an input contribution with documentation of PSNR values (at least average of frame PSNR for each sequence and encoding point, separate for luma and chroma components, as well as Bjntegaard Delta-Rate and Delta-PSNR [1] [2] compared to the anchors) and, if possible, documentation of the compression technology. Furthermore, in addition to the decoded sequences, a binary decoder as well as bitstreams for all submitted test cases shall be brought to the MPEG July meeting, such that the results can be verified.

4 Logistics

Prospective contributors to the Call for Evidence should contact:

Prof. Dr. Jens-Rainer Ohm, chair of MPEG Video Subgroup
RWTH Aachen University, Institute of Communications Engineering
Melatener Str. 23, 52074 Aachen, Germany
Tel. +49-241-8027671, Fax. +49-241-8022196, email ohm@ient.rwth-aachen.de

Dr. Vittorio Baroncini,
Senior Researcher, Audio Video Signal Processing Area
Fondazione Ugo Bordoni
Via B. Castiglione, 59 00142 – Rome - Italy
Tel. +39-06-54802134, Fax. +39-06-54804405, email vittorio@fub.it

Expression of interest is requested by sending an email to Prof. Ohm by 2009-06-01 latest. Interested parties are kindly invited to express their intent of participation to the MPEG video chair as early as possible.

MPEG participants should upload their contributions as regular input documents. Prof. Ohm will collect contributions from outside MPEG and make them available via the MPEG document registry.

Test sequences and anchors (including configuration files used for encoding) will be made available by request by contacting one of the persons above.

[1] Gisle Bjontegaard, "Calculation of Average PSNR Differences between
RD curves", ITU-T SG16/Q6, 13th VCEG Meeting, Austin, Texas, USA, April 2001, Doc. VCEG-M33.

[2] Gisle Bjontegaard, "Improvements of the BD-PSNR model", ITU-T
SG16/Q6, 35th VCEG Meeting, Berlin, Germany, 16th - 18th July, 2008, Doc.VCEG-AI11.

MPEG news: a report from the 85th meeting in Busan, Korea

A lot of interesting things happened here and I'd like to report on three topics:

• MPEG RoSE
• Advanced IPTV Terminal
• High-performance Video Coding
  

MPEG RoSE: At this meeting we've issued the second version of the WD which will be publicly available and, thus, I can provide a more detailed overview here. I've also updated my slides from the last meeting which now can be found on SlideShare. The aim of RoSE to extend the traditional A/V content consumption to the dimension of sensory effects which are referred to as "an effect to augment perception by stimulating human senses in a particular scene of a multimedia application". With that definition in mind, sensory effects are composed by following the structure of the Sensory Effect Description Language (SEDL) and making use of terms (actually, effect types) of the Sensory Effect Vocabulary (SEV).

Sensory Effect Description Language (SEDL): Provides basic building blocks (declaration, effects, group of effects, reference to effects, parameters), common attributes (timing, priority, intensity, etc.), and data types (void at the moment) for constructing/authoring sensory effect metadata which is associated to A/V content.

Sensory Effect Vocabulary (SEV): Provides a clear set of extensible effect types which are currently comprising effects for color (illumination), temperature (°C), wind (Beaufort), and vibration (Richter).

It is foreseen that the sensory effect metadata which comes along with the A/V content is translated (or mapped or adapted) - by a module (hw/sw) called RoSE engine - to commands that are understood by so-called sensory devices with certain capabilities. Both commands and capabilities are also within the scope of standardization. Hence, it should be possible to consume the A/V content timely synchronized with its effects for an increased user experience. Furthermore, user preferences might also affect this translation/mapping/adaptation process.

Advanced IPTV Terminal: This activity is about to define a terminal suitable for IPTV scenarios jointly with ITU which may result in a similar construction as the JVT for video coding standardization. However, nothing has been fixed yet and interested parties are invited to join the discussions via the corresponding Ad-hoc Group (AhG) [subscribe]with the following mandates:

1. Look after the process of establishing the joint project with ITU-T so that SG16 may presented with a matured proposal for a new standard for Advanced IPTV Terminal
2. Gather use case scenarios and requirements for Advanced IPTV Terminal
3. Conduct collaboration with ITU-T Q.13/16 to prepare a joint meeting

High-performance Video Coding: I've already reported on that in previous blog posts, see part one and two for details.

Finally, MPEG got an Emmy! ;-)