MPEG news: a report from the 127th meeting, Gothenburg, Sweden

The original blog post can be found at the Bitmovin Techblog and has been modified/updated here to focus on and highlight research aspects. Additionally, this version of the blog post will be also posted at ACM SIGMM Records.

MPEG News Archive

Plenary of the 127th MPEG Meeting in Gothenburg, Sweden.

The 126th MPEG meeting concluded on March 29, 2019 in Geneva, Switzerland with the following topics:

• Versatile Video Coding (VVC) enters formal approval stage, experts predict 35-60% improvement over HEVC
• Essential Video Coding (EVC) promoted to Committee Draft
• Common Media Application Format (CMAF) 2nd edition promoted to Final Draft International Standard
• Dynamic Adaptive Streaming over HTTP (DASH) 4th edition promoted to Final Draft International Standard
• Carriage of Point Cloud Data Progresses to Committee Draft
• JPEG XS carriage in MPEG-2 TS promoted to Final Draft Amendment of ISO/IEC 13818-1 7th edition
• Genomic information representation – WG11 issues a joint call for proposals on genomic annotations in conjunction with ISO TC 276/WG 5
• ISO/IEC 23005 (MPEG-V) 4th Edition – WG11 promotes the Fourth edition of two parts of “Media Context and Control” to the Final Draft International Standard (FDIS) stage

The corresponding press release of the 127th MPEG meeting can be found here: https://mpeg.chiariglione.org/meetings/127

Versatile Video Coding (VVC)

The Moving Picture Experts Group (MPEG) is pleased to announce that Versatile Video Coding (VVC) progresses to Committee Draft, experts predict 35-60% improvement over HEVC.

The development of the next major generation of video coding standard has achieved excellent progress, such that MPEG has approved the Committee Draft (CD, i.e., the text for formal balloting in the ISO/IEC approval process).

The new VVC standard will be applicable to a very broad range of applications and it will also provide additional functionalities. VVC will provide a substantial improvement in coding efficiency relative to existing standards. The improvement in coding efficiency is expected to be quite substantial – e.g., in the range of 35–60% bit rate reduction relative to HEVC although it has not yet been formally measured. Relative to HEVC means for equivalent subjective video quality at picture resolutions such as 1080p HD or 4K or 8K UHD, either for standard dynamic range video or high dynamic range and wide color gamut content for levels of quality appropriate for use in consumer distribution services. The focus during the development of the standard has primarily been on 10-bit 4:2:0 content, and 4:4:4 chroma format will also be supported.

The VVC standard is being developed in the Joint Video Experts Team (JVET), a group established jointly by MPEG and the Video Coding Experts Group (VCEG) of ITU-T Study Group 16. In addition to a text specification, the project also includes the development of reference software, a conformance testing suite, and a new standard ISO/IEC 23002-7 specifying supplemental enhancement information messages for coded video bitstreams. The approval process for ISO/IEC 23002-7 has also begun, with the issuance of a CD consideration ballot.

Research aspects: VVC represents the next generation video codec to be deployed in 2020+ and basically the same research aspects apply as for previous generations, i.e., coding efficiency, performance/complexity, and objective/subjective evaluation. Luckily, JVET documents are freely available including the actual standard (committee draft), software (and its description), and common test conditions. Thus, researcher utilizing these resources are able to conduct reproducible research when contributing their findings and code improvements back to the community at large. 

Essential Video Coding (EVC)

MPEG-5 Essential Video Coding (EVC) promoted to Committee Draft

Interestingly, at the same meeting as VVC, MPEG promoted MPEG-5 Essential Video Coding (EVC) to Committee Draft (CD). The goal of MPEG-5 EVC is to provide a standardized video coding solution to address business needs in some use cases, such as video streaming, where existing ISO video coding standards have not been as widely adopted as might be expected from their purely technical characteristics.

The MPEG-5 EVC standards includes a baseline profile that contains only technologies that are over 20 years old or are otherwise expected to be royalty-free. Additionally, a main profile adds a small number of additional tools, each providing significant performance gain. All main profile tools are capable of being individually switched off or individually switched over to a corresponding baseline tool. Organizations making proposals for the main profile have agreed to publish applicable licensing terms within two years of FDIS stage, either individually or as part of a patent pool.

Research aspects: Similar research aspects can be described for EVC and from a software engineering perspective it could be also interesting to further investigate this switching mechanism of individual tools or/and fall back option to baseline tools. Naturally, a comparison with next generation codecs such as VVC is interesting per se. The licensing aspects itself are probably interesting for other disciplines but that is another story...

Common Media Application Format (CMAF)

MPEG ratified the 2nd edition of the Common Media Application Format (CMAF)

The Common Media Application Format (CMAF) enables efficient encoding, storage, and delivery of digital media content (incl. audio, video, subtitles among others), which is key to scaling operations to support the rapid growth of video streaming over the internet. The CMAF standard is the result of widespread industry adoption of an application of MPEG technologies for adaptive video streaming over the Internet, and widespread industry participation in the MPEG process to standardize best practices within CMAF.

The 2nd edition of CMAF adds support for a number of specifications that were a result of significant industry interest. Those include

• Advanced Audio Coding (AAC) multi-channel;
• MPEG-H 3D Audio;
• MPEG-D Unified Speech and Audio Coding (USAC);
• Scalable High Efficiency Video Coding (SHVC);
• IMSC 1.1 (Timed Text Markup Language Profiles for Internet Media Subtitles and Captions); and
• additional HEVC video CMAF profiles and brands.

This edition also introduces CMAF supplemental data handling as well as new structural brands for CMAF that reflects the common practice of the significant deployment of CMAF in industry. Companies adopting CMAF technology will find the specifications introduced in the 2nd Edition particularly useful for further adoption and proliferation of CMAF in the market.

Research aspects: see below (DASH).

Dynamic Adaptive Streaming over HTTP (DASH)

MPEG approves the 4th edition of Dynamic Adaptive Streaming over HTTP (DASH)

The 4th edition of MPEG-DASH comprises the following features:

service description that is intended by the service provider on how the service is expected to be consumed;

• a method to indicate the times corresponding to the production of associated media;
• a mechanism to signal DASH profiles and features, employed codec and format profiles; and
• supported protection schemes present in the Media Presentation Description (MPD).

It is expected that this edition will be published later this year. 

Research aspects: CMAF 2nd and DASH 4th edition come along with a rich feature set enabling a plethora of use cases. The underlying principles are still the same and research issues arise from updated application and service requirements with respect to content complexity, time aspects (mainly delay/latency), and quality of experience (QoE). The DASH-IF awards the excellence in DASH award at the ACM Multimedia Systems conference and an overview about its academic efforts can be found here. For example, see here our recent research on bandwidth prediction in low-latency chunked streaming. Additionally, our tutorial at ACM Multimedia 2019 about a journey towards fully immersive media access reviews state of the art in this area and how it could be extended enabling 6DoF HAS services through point cloud compression.

Carriage of Point Cloud Data

MPEG progresses the Carriage of Point Cloud Data to Committee Draft

At its 127th meeting, MPEG has promoted the carriage of point cloud data to the Committee Draft stage, the first milestone of ISO standard development process. This standard is the first one introducing the support of volumetric media in the industry-famous ISO base media file format family of standards.

This standard supports the carriage of point cloud data comprising individually encoded video bitstreams within multiple file format tracks in order to support the intrinsic nature of the video-based point cloud compression (V-PCC). Additionally, it also allows the carriage of point cloud data in one file format track for applications requiring multiplexed content (i.e., the video bitstream of multiple components is interleaved into one bitstream).

This standard is expected to support efficient access and delivery of some portions of a point cloud object considering that in many cases that entire point cloud object may not be visible by the user depending on the viewing direction or location of the point cloud object relative to other objects. It is currently expected that the standard will reach its final milestone by the end of 2020.

Research aspects: MPEG's Point Cloud Compression (PCC) comes in two flavors, video- and geometric-based but still requires to be packaged into file and delivery formats. MPEG's choice here is the ISO base media file format and the efficient carriage of point cloud data is characterized by both functionality (i.e., enabling the required used cases) and performance (such as low overhead).

MPEG 2 Systems/Transport Stream

JPEG XS carriage in MPEG-2 TS promoted to Final Draft Amendment of ISO/IEC 13818-1 7th edition

At its 127th meeting, WG11 (MPEG) has extended ISO/IEC 13818-1 (MPEG-2 Systems) – in collaboration with WG1 (JPEG) – to support ISO/IEC 21122 (JPEG XS) in order to support industries using still image compression technologies for broadcasting infrastructures. The specification defines a JPEG XS elementary stream header and specifies how the JPEG XS video access unit (specified in ISO/IEC 21122-1) is put into a Packetized Elementary Stream (PES). Additionally, the specification also defines how the System Target Decoder (STD) model can be extended to support JPEG XS video elementary streams.

Genomic information representation

WG11 issues a joint call for proposals on genomic annotations in conjunction with ISO TC 276/WG 5

The introduction of high-throughput DNA sequencing has led to the generation of large quantities of genomic sequencing data that have to be stored, transferred and analyzed. So far WG 11 (MPEG) and ISO TC 276/WG 5 have addressed the representation, compression and transport of genome sequencing data by developing the ISO/IEC 23092 standard series also known as MPEG-G. They provide a file and transport format, compression technology, metadata specifications, protection support, and standard APIs for the access of sequencing data in the native compressed format.

An important element in the effective usage of sequencing data is the association of the data with the results of the analysis and annotations that are generated by processing pipelines and analysts. At the moment such association happens as a separate step, standard and effective ways of linking data and meta information derived from sequencing data are not available.

At its 127th meeting, MPEG and ISO TC 276/WG 5 issued a joint Call for Proposals (CfP) addressing the solution of such problem. The call seeks submissions of technologies that can provide efficient representation and compression solutions for the processing of genomic annotation data.

Companies and organizations are invited to submit proposals in response to this call. Responses are expected to be submitted by the 8th January 2020 and will be evaluated during the 129th WG 11 (MPEG) meeting. Detailed information, including how to respond to the call for proposals, the requirements that have to be considered, and the test data to be used, is reported in the documents N18648, N18647, and N18649 available at the 127th meeting website (http://mpeg.chiariglione.org/meetings/127). For any further question about the call, test conditions, required software or test sequences please contact: Joern Ostermann, MPEG Requirements Group Chair (ostermann@tnt.uni-hannover.de) or Martin Golebiewski, Convenor ISO TC 276/WG 5 (martin.golebiewski@h-its.org).

ISO/IEC 23005 (MPEG-V) 4th Edition

WG11 promotes the Fourth edition of two parts of “Media Context and Control” to the Final Draft International Standard (FDIS) stage

At its 127th meeting, WG11 (MPEG) promoted the 4th edition of two parts of ISO/IEC 23005 (MPEG-V; Media Context and Control) standards to the Final Draft International Standard (FDIS). The new edition of ISO/IEC 23005-1 (architecture) enables ten new use cases, which can be grouped into four categories: 3D printing, olfactory information in virtual worlds, virtual panoramic vision in car, and adaptive sound handling. The new edition of ISO/IEC 23005-7 (conformance and reference software) is updated to reflect the changes made by the introduction of new tools defined in other parts of ISO/IEC 23005. More information on MPEG-V and its parts 1-7 can be found at https://mpeg.chiariglione.org/standards/mpeg-v.

Finally, the unofficial highlight of the 127th MPEG meeting we certainly found while scanning the scene in Gothenburg on Tuesday night...

One Year of MPEG

In my last MPEG report (index) I’ve mentioned that the 112th MPEG meeting in Warsaw was my 50th MPEG meeting which roughly accumulates to one year of MPEG meetings. That is, one year of my life I've spend in MPEG meetings - scary, isn't it? Thus, I thought it’s time to recap what I have done in MPEG so far featuring the following topics/standards where I had significant contributions:

• MPEG-21 - The Multimedia Framework 
• MPEG-M - MPEG extensible middleware (MXM), later renamed to multimedia service platform technologies 
• MPEG-V - Information exchange with Virtual Worlds, later renamed to media context and control
• MPEG-DASH - Dynamic Adaptive Streaming over HTTP

MPEG-21 - The Multimedia Framework

I started my work with standards, specifically MPEG, with Part 7 of MPEG-21 referred to as Digital Item Adaptation (DIA) and developed the generic Bitstream Syntax Description (gBSD) in collaboration with SIEMENS which allows for a coding-format independent (generic) adaptation of scalable multimedia content towards the actual usage environment (e.g., different devices, resolution, bitrate). The main goal of DIA was to enable the Universal Media Access (UMA) -- any content, anytime, anywhere on any device -- and also motivated me to start this blog. I also wrote a series of blog entries on this topic: O Universal Multimedia Access, Where Art Thou? which gives an overview about this topic and basically is also what I’ve done in my Ph.D. thesis. Later I helped a lot in various MPEG-21 parts including its dissemination and documented where it has been used. In the past, I saw many forms of Digital Items (e.g., iTunesLP was one of the first) but unfortunately the need for a standardised format is very low. Instead, proprietary formats are used and I realised that developers are more into APIs than formats. The format comes with the API but it’s the availability of an API that attracts developers and makes them to adopt a certain technology. 

MPEG-M

The lessons learned from MPEG-21 was one reason why I joined the MPEG-M project as it was exactly the purpose to create an API into various MPEG technologies, providing developers a tool that makes it easy for them to adopt new technologies and, thus, new formats/standards. We created an entire architecture, APIs, and reference software to make it easy for external people to adopt MPEG technologies. The goal was to hide the complexity of the technology through simple to use APIs which should enable the accelerated development of components, solutions, and applications utilising digital media content. A good overview about MPEG-M can found on this poster.

MPEG-V

When MPEG started working on MPEG-V (it was not called like that in the beginning), I saw it as an extension of UMA and MPEG-21 DIA to go beyond audio-visual experiences by stimulating potentially all human senses. We created and standardised an XML-based language that enables the annotation of multimedia content with sensory effects. Later the scope was extended to include virtual worlds which resulted in the acronym MPEG-V. It also brought me to start working on Quality of Experience (QoE) and we coined the term Quality of Sensory Experience (QuASE) as part of the (virtual) SELab at Alpen-Adria-Universität Klagenfurt which offers a rich set of open-source software tools and datasets around this topic on top of off-the-shelf hardware (still in use in my office).

MPEG-DASH

The latest project I’m working on is MPEG-DASH where I’ve also co-founded bitmovin, now a successful startup offering fastest transcoding in the cloud (bitcodin) and high quality MPEG-DASH players (bitdash). It all started when MPEG asked me to chair the evaluation of call for proposals on HTTP streaming of MPEG media. We then created dash.itec.aau.at that offers a huge set of open source tools and datasets used by both academia and industry worldwide (e.g., listed on DASH-IF). I think I can proudly state that this is the most successful MPEG activity I've been involved so far... (note: a live deployment can be found here which shows 24/7 music videos over the Internet using bitcodin and bitdash).

DASH and QuASE are also part of my habilitation which brought me into the current position at Alpen-Adria-Universität Klagenfurt as Associate Professor. Finally, one might ask the question, was it all worth spending so much time for MPEG and at MPEG meetings. I would say YES and there are many reasons which could easily results in another blog post (or more) but it’s better to discuss this face to face, I'm sure there will be plenty of possibilities in the (near) future or you come to Klagenfurt, e.g., for ACM MMSys 2016 ...

MPEG envisages royalty-free MPEG video coding standard

Daegu, KR – The 95th MPEG meeting was held in Daegu, Korea from the 24th to the 28th of January 2011.

--MPEG press release also available here.

Highlights of the 95th Meeting

MPEG anticipates March 2011 CfP for Type-1 Video Coding Standard
MPEG has been producing standards that provide industry with the best video compression technologies. In recognition of the growing importance that the Internet plays in the generation and consumption of video content, MPEG intends to develop a new video compression standard in line with the expected usage models of the Internet. The new standard is intended to achieve substantially better compression performance than that offered by MPEG-2 and possibly comparable to that offered by the AVC Baseline Profile. MPEG will issue a call for proposals on video compression technology at the end of its upcoming meeting in March 2011 that is expected to lead to a standard falling under ISO/IEC “Type-1 licensing”, i.e. intended to be “royalty free”.

MPEG moves toward a visual search standard by issuing Draft Call for Proposals
In its latest step toward creating a standard for efficient and interoperable designs of visual search applications, MPEG has issued a draft Call for Proposals at its 95th meeting. Like a barcode reader, but using regular images instead of barcodes, visual search enables the retrieval of related information from databases for tourists, simplified shopping, mobile augmented reality, and other applications.

Specifically, the call seeks technologies that deliver robust matching of images of objects, such as landmarks and text-based documents, that may be partially occluded or captured from various vantage points, and with different camera parameters, or lighting conditions. The underlying component technologies that are expected to be addressed by the standard include the format of the visual descriptors, and parts of the descriptor extraction process needed to ensure interoperability. Other component technologies, such as indexing and matching algorithms, may also be incorporated into the new standard.

Further details are outlined in the text of the call available at http://mpeg.chiariglione.org/hot_news.htm. The Final Call for Proposals will be issued at the 96th MPEG meeting in March 2011 with responses due in October 2011.

MPEG targets a new phase of 3D video coding standards
A Draft Call for Proposals on 3D Video Coding Technology has also been issued by MPEG at its 95th meeting. This call invites technology submissions providing efficient compression of 3D video and high quality view reconstruction that goes beyond the capabilities of existing standards. MPEG has already delivered 3D compression formats to the market, including MVC and frame-compatible stereoscopic formats, which are being deployed by industry for packaged media and broadcast services. However, the market needs are expected to evolve and new types of 3D displays and services will be offered. With this call, MPEG embarks on a new phase of 3D standardization that anticipates these future needs. The next-generation of 3D standards will define the 3D data format and associated compression technology to facilitate the generation of multiview output to enable both advanced stereoscopic display processing and improved support for auto-stereoscopic displays. Further details are outlined in MPEG's Vision on 3D Video (http://mpeg.chiariglione.org/visions/3dv/index.htm). The Final Call for Proposals will be issued at the 96th MPEG meeting in March 2011 with responses due in September 2011.

Amendment to MPEG-2 systems is finalized at 95th meeting
MPEG is continuously improving the popular MPEG-2 Transport Stream (TS) standard (ISO/IEC 13818-1), one of its most widely accepted standards for broadcast industries. At its 95th meeting, MPEG has finalized a new amendment to support recently developed video coding standards, Advanced Video Coding (AVC) and Multiview Video Coding (MVC), in MPEG-2 TS. This amendment extends the AVC video descriptor to signal the presence of a frame packing arrangement in an associated supplemental enhancement information message for the underlying AVC video stream component. The new amendment also adds signaling of an operating point descriptor of MVC which enables transmission systems to convey the relevant operating points that can be used by receiving devices.

In a related project, MPEG has also started a new amendment to signal stereoscopic video services carried in MPEG-2 TS. This amendment will support not only frame compatible video services but also service compatible video services which will allow implementation of backward compatible stereoscopic video services in HDTV systems.

MPEG hosts MPEG-V awareness event
At its 95th meeting, MPEG hosted the MPEG-V Awareness Event 2011, at which the full range of MPEG-V technologies, including several products and applications employing the standard, were showcased. These technologies cover applications for multi-sensorial user experience in the home environment, control of virtual worlds by real signals, motion capture systems and real-time avatar animation, multi-platform streaming for virtual worlds and mixed reality games. The workshop presentations are available at http://wg11.sc29.org/mpeg-v.


A hot standard moves fast
MPEG has approved the promotion of Dynamic Adaptive Streaming over HTTP (DASH) to Draft International Standard (DIS) status. The draft is available from the Hot News page of http://mpeg.chiariglione.org.

Responding to a Call – How to Contact MPEG
The text and details related to the Calls mentioned above (together with other current Calls) are in the Hot News section, http://www.chiariglione.org/mpeg/hot_news.htm. These documents include information on how to respond to the Calls.

Communicating the large and sometimes complex array of technologies that the MPEG Committee has developed is not a simple task. Experts, past and present, have contributed a series of tutorials and vision documents that explain each of these standards individually. The repository is growing with each meeting, but if something of interest cannot be found, do not hesitate to request it. You can start your MPEG adventure at: http://mpeg.chiariglione.org/technologies.htm.

Further Information

Future MPEG meetings are planned as follows:

• No. 96, Geneva, CH, 21-25 March, 2011
• No. 97, Torino, IT, 18-22 July, 2011
• No. 98, Geneva, CH 28 November – 2 December, 2011
• No. 99, San Jose, USA 6-10 February, 2012

For further information about MPEG, please contact:

Dr. Leonardo Chiariglione (Convener of MPEG, Italy)
Via Borgionera, 103
10040 Villar Dora (TO), Italy
Tel: +39 011 935 04 61
leonardo@chiariglione.org

This press release and other MPEG-related information can be found on the MPEG homepage:
http://mpeg.chiariglione.org/

The MPEG homepage also has links to other MPEG pages which are maintained by the MPEG subgroups. It also contains links to public documents that are freely available for download by those who are not MPEG members. Journalists that wish to receive MPEG Press Releases by email should contact Dr. Arianne T. Hinds at arianne.hinds@infoprint.com.

MPEG news: a report from the 95th meeting, Daegu, Korea

For those who cannot wait until the official press release from the 95th ISO/IEC JTC 1/SC 29/WG 11 (MPEG) meeting, here comes a short review of what happened during this week. I must admit that I could not stay until the very end (the first time after a very long period) and, thus, only a few topics are presented here, i.e., those where I was heavily involved:

• MPEG-V Awareness Event
• Dynamic Adaptative Streaming over HTTP (DASH) reached Draft International Standard (DIS) stage
• MPEG Media Transport (MMT) has been successfully evaluated

MPEG-V Awareness Event

The purpose of this event is to present and demonstrate MPEG-V, a new standard for a variety of products and applications enabling multi-sensorial user experience and interchange between virtual worlds. The event started with a set of presentations coming from the editors of the individual parts and also from industry showing the productions and applications enabled by the standard. A demo session followed the presentations where individual companies, institutions, and universities demonstrated prototype implementations and promotional material. The presentations are publicly available (see below).

Mpeg v-awareness event

View more presentations from Marius Preda.

DASH reached DIS

ISO/IEC 23001-6 aka Dynamic Adaptive Streaming over HTTP (DASH) reached Draft International Standard (DIS) stage which means that most of the technical work is completed and now it's time to check and validate the specification as well as to start the implementation. It was a busy meeting where a lot of national body comments were addressed and related input contributions were reviewed. If you're interested to learn more about DASH, I would recommend the following events in the near future:

• Second W3C Web and TV Workshop, February 8-9, 2011 in Berlin, Germany
• Special Session related to DASH at ACM MMSys'11, February 23-25, 2011 in San Jose, CA, USA

Alternatively, you may just wait until early next week when the DIS of DASH will be made publicly available at the MPEG Web site (and also here on this blog). For technical discussions, I'd like to refer to the DASH email reflector.

MPEG Media Transport (MMT) has been successfully evaluated

The responses to the CfP on MMT has been evaluated and a first working draft has been created. As working drafts are (usually) not publicly available, that's more or less what I can say about this very interesting work item. Nevertheless, we received a reasonable number of submissions both comprehensive nature and specific to certain requirements. In general, they mainly aim to address limitations of existing transport formats in the context of the current and, hopefully, future Internet. If you would like to take part in the discussions, you may subscribe to the MMT email reflector. More to come on this topic during the next months, stay tuned ...

MPEG advances Dynamic Adaptive Streaming over HTTP (DASH) toward completion

Guangzhou, CN – The 94th MPEG meeting was held in Guangzhou, People’s Republic of China from the 11th to the 15th of October 2010.

--MPEG press release also available here.

Attendees of the 94th MPEG meeting, 53rd JPEG meeting, and 3rd JCT-VC Meeting of MPEG and VCEG, Guangzhou, China. Photo courtesy of the GZ National Engineering Laboratory for Video Technology Research & Industrialization Center.

Highlights of the 94th Meeting

MPEG issues the Committee Draft for the carriage of multimedia data over HTTP
As HTTP became one of the most important protocols for the delivery of content over the Internet, MPEG launched an effort to use this standard for the delivery of multimedia data in an optimal way. At its 94th meeting, MPEG’s Dynamic Adaptive Streaming over HTTP (DASH) has reached the Committee Draft stage. The DASH committee draft is based on the 3GPP Adaptive HTTP Streaming specification and improves it by adding several new features and extensions such as, support of live streaming of content, additional annotation capabilities, flexibility in combining multiple contents, enhancement of trick modes and random access, and support of multiple content management and protection schemes, delivery of MPEG-2 Transport Stream, Scalable Video Coding (SVC) and Multi-view Video Coding (MVC). The new standard is expected to achieve Final Draft International Standard status in July 2011.

MPEG-M for multimedia services also progresses to Committee Draft
MPEG has advanced its joint activity with ITU-T SG16 (Question 13) on Advanced IPTV Terminal (AIT) at its 94th meeting, and progressed its work item called Multimedia Service Platform Technologies, or MPEG-M, to Committee Draft (CD). MPEG-M facilitates service aggregation so that service providers can offer users a plethora of innovative services by extending current IPTV technology toward the seamless integration of personal content creation and distribution, shopping-commerce, social networks and Internet distribution of digital media. The new draft MPEG-M standard (ISO/IEC 23006) comprises Part 2, specifying a revised and improved Middleware of MPEG technologies (MXM), Part 4 specifying a set of elementary services, and Part 5 specifying some mechanisms for aggregating services and illustrating how to use these mechanisms to aggregate services, e.g. for Advanced IPTV Terminal. The new standard is expected to achieve Final Draft International Standard status in July 2011.

MPEG-V Awareness Event is planned for January, 2011
MPEG announces the organization of the "MPEG-V Awareness Event 2011", that will be held on 27 January, 2011 in Daegu, Korea. The purpose of this event is to present and demonstrate MPEG-V, a new standard elaborated by MPEG and expected to be published by ISO in2011. MPEG-V based products and applications will enable enhanced and persistent multi-sensorial user experience and interchange between virtual worlds. The standard is composed of several parts including:

• architecture and use case scenarios,
• syntax and semantics of data formats for interaction devices, device commands, and sensed information,
• metadata to describe device capabilities and user preferences,
• metadata to represent sensory effects,
• metadata to represent the characteristics of virtual objects and avatars.

The MPEG-V awareness day is particularly targeted to developers of products and applications for multi-sensorial user experience, content creators, broadcasters, multimedia device manufacturers, sensor & actuator manufacturers, virtual worlds and online game developers, as well as persons interested in new opportunities in digital media. The event is free of charge. Registration is available at the MPEG-V Web Site at http://wg11.sc29.org/mpeg-v.

MPEG enriches scalable complexity 3DMC for 3DG compression model
At its 94th meeting, MPEG enriched the 3D Graphics Compression Model published in MPEG-4 Part 25 with a new amendment to add a new tool. This tool allows application and product developers to apply, in an interoperable manner, the state-of-the-art scalable complexity 3D mesh coding from MPEG-4 Part 25 to scene representations defined in third-party standards such as COLLADA (by Khronos) and X3D (by W3C).

MPEG conducts Spatial Audio Object Coding verification test
MPEG Audio has finalized the work on MPEG Spatial Audio Object Coding (SAOC) by conducting a verification test comprised of a music-remix and a teleconferencing oriented scenario. The test results show that, when operated at the same bitrate, MPEG SAOC technology offers better quality than legacy technology for discretely encoded audio objects. This holds true for both the Low Power and regular decoding modes of SAOC. Alternatively, for a comparable level of audio quality, MPEG SAOC requires a significantly lower bitrate than other technologies, while at the same time providing a backwards compatible downmix and user interactivity.

MPEG SAOC enables interactive (“re-mixable”) music by adding only about 10% of additional bitrate to the rate required for a stereo downmix signal. For telephone conferencing solutions, MPEG SAOC efficiently transmits a single mono audio signal while the embedded SAOC data offers receiver side control for a personalized rendering of the remote conference participants across the local sound stage.

MPEG SAOC is the preferred solution for interactive / personalized audio products and services at bitrates as low as those known today from non-interactive audio codecs.

MPEG moves toward a visual search standard
At its 94th meeting, MPEG progressed the definition of requirements and the drafting of a Call for Proposals on Compact Descriptors for Visual Search. Exploiting the colocation of the 94th MPEG meeting with JPEG's 53rd meeting, MPEG experts in the domain of visual search met with experts from JPEG's JPSearch group to exchange their views and ideas on MPEG’s visual search activity. Experts from both working groups expressed their intent to strengthen their collaboration and to exploit the legacies of both committees.

Digging Deeper – How to Contact MPEG
Communicating the large and sometimes complex array of technology that the MPEG Committee has developed is not a simple task. Experts, past and present, have contributed a series of white-papers and vision documents that explain each of these standards individually. The repository is growing with each meeting, so if something you are interested is not there yet, it may appear there shortly – but you should also not hesitate to request it. You can start your MPEG adventure at: http://mpeg.chiariglione.org/technologies.htm.


Further Information
Future MPEG meetings are planned as follows:

• No. 95, Daegu, KR, 24-28 January, 2011
• No. 96, Geneva, CH, 21-25 March, 2011
• No. 97, Torino, IT, 18-22 July, 2011
• No. 98, Geneva, CH 28 November – 2 December, 2011

For further information about MPEG, please contact:
Dr. Leonardo Chiariglione (Convener of MPEG, Italy)
Via Borgionera, 103
10040 Villar Dora (TO), Italy
Tel: +39 011 935 04 61
leonardo@chiariglione.org
This press release and other MPEG-related information can be found on the MPEG homepage:
http://mpeg.chiariglione.org/
The text and details related current Calls are in the Hot News section, http://mpeg.chiariglione.org/hot_news.htm. These documents include information on how to respond to Calls.
The MPEG homepage also has links to other MPEG pages which are maintained by the MPEG subgroups. It also contains links to public documents that are freely available for download by those who are not MPEG members. Journalists that wish to receive MPEG Press Releases by email should contact Dr. Arianne T. Hinds at arianne.hinds@infoprint.com.

Immersive Future Media Technologies: From 3D Video to Sensory Experience

--tutorial to be held at ACM Multimedia 2010, Oct 25th, Morning, Florence, Italy

--download as PDF

For registration details, please consult ACM Multimedia 2010 Web site!

Instructors: Karsten Müller (Fraunhofer/Heinrich-Hertz-Institut, Berlin) and Christian Timmerer (Klagenfurt University, Austria)

Abstract: The past decade has witnessed a significant increase in the research efforts around the Quality of Experience (QoE) which is generally referred to as a human-centric paradigm for the Quality of a Service (QoS) as perceived by the (end) user. As it puts the end user in the center stage, it may have various dimensions and one dimension recently gained momentum is 3D video. Another dimension aims at going beyond 3D and promises advanced user experience through sensory effects, both introduced briefly in the following.

3D Video: Stereo and Multi-View Video Technology
3D related media technologies have recently developed from pure research-oriented work towards applications and products. 3D content is now being produced on a wider scale and first 3D applications have been standardized, such as multi-view video coding for 3D Blu Ray disks. This part of the tutorial starts with an overview on 3D in the form of stereo video based systems, which are currently being commercialized. Here, stereo formats and associated coding are introduced. This technology is used for 3D cinema applications and mobile 3DTV environments. For the latter, user requirements and profiling will be introduced as a form to assess user quality of experience. For 3D home entertainment, glasses-free multi-view displays are required, as more than one user will watch 3D content. For such displays, the current stereo solutions need to be extended. Therefore, new activities in 3D video are introduced. These 3D solutions will develop a generic 3D video format with color and supplementary geometry data, e.g. depth maps, and associated coding and rendering technology for any multi-view display, independent of the number of views. As such technology is also developed in international consortia, the most prominent, like the 3D@HOME consortium, the EU 3D, Immersive, Interactive Media Cluster and the 3D video activities in ISO-MPEG are introduced.

Advanced User Experience through Sensory Effects
This part of the tutorial addresses a novel approach for increasing the user experience – beyond 3D – through sensory effects. The motivation behind this work is that the consumption of multimedia assets may stimulate also other senses than vision or audition, e.g., olfaction, mechanoreception, equilibrioception, or thermoception that shall lead to an enhanced, unique user experience. This could be achieved by annotating the media resources with metadata (currently defined by ISO/MPEG as part of the MPEG-V standard) providing so-called sensory effects that steer appropriate devices capable of rendering these effects (e.g., fans, vibration chairs, ambient lights, perfumer, water sprayers, fog machines, etc.). In particular, we will review the concepts and details of the forthcoming MPEG-V standard and present our prototype architecture for the generation, transport, decoding and use of sensory effects. Furthermore, we will present details and results of a series of formal subjective quality assessments which confirm that the concept of sensory effects is a vital tool for enhancing the user experience.

Course Outline
Introduction and Overview (~10min.)
3D Video: Stereo and Multi-View Video Technology (~90min)

• User Requirements for 3D video technologies
• Stereo Video solutions for 3D cinema and mobile applications
• User experience and profiling for mobile 3DTV
• Format description and coding for stereo and multi-view video technology
• Depth-enhanced 3D video solutions for 3D home entertainment

Advanced User Experience through Sensory Effects (~90min)

• MPEG-V: context and objectives including an overview of all parts.
• In depth review of Part 3 of MPEG-V entitled Sensory Information.
• A test-bed for the quality of multimedia experience evaluation of sensory effects and demonstration.
• How to improve the Quality of Experience through sensory effects? Results from first subjective experiments.
• Conclusions and future work.

Intended Audience
Intermediate, specifically, graduate students and researchers interested in 3D video and sensory experiences.

Biography of Presenters

Karsten Müller received the Dipl.-Ing. and Dr.-Ing. degree from the Technical University of Berlin, Germany, in 1997 and 2006 respectively. In 1993 he spent one year studying Electronics and Communication Engineering at Napier University of Edinburgh/Scotland, including a half-year working period at Integrated Communication Systems Inc. in Westwick near Cambridge/England. In this working period he developed software for voice mail systems and statistical analysis of caller data. In 1996 he joined the Heinrich-Hertz-Institute (HHI) Berlin, Image Processing Department, were he is a project coordinator for European projects in the field of 3D media technology. He also co-chairs the European 3D Media Cluster, which serves as a contact gateway for information exchange between the associated European projects and international 3D media activities.
His research interests include motion and disparity estimation, 3D media representation and coding, 3D graphics-based scene reconstruction with multi-texture surfaces, and 3D metadata and content description. He has been actively involved in MPEG activities, standardizing the multi-view description for MPEG-7, the view-dependent multi-texturing methods for the 3D scene representation in MPEG4-AFX and contributing to the multi-view video coding process in MPEG4-MVC and 3D Video. In recent projects he was involved in research and development of traffic surveillance systems and visualization of multiple-view video, 3D scene reconstruction, object segmentation, tracking and 3D reconstruction, 3D scene and object representation and interactive user navigation in 3D environments. Currently, he is a Project Manager for European projects in the field of 3D video technology and multimedia content description. He is senior member of the IEEE.

Christian Timmerer received his M.Sc. (Dipl.-Ing.) in January 2003 and his Ph.D. (Dr.techn.) in June 2006 (for research on the adaptation of scalable multimedia content in streaming and constraint environments) both from the Klagenfurt University. He joined the Klagenfurt University in 1999 and is currently a Assistant Professor (Ass.-Prof.) at the Department of Information Technology (ITEC) – Multimedia Communication Group. His research interests include the transport of multimedia content, multimedia adaptation in constrained and streaming environments, distributed multimedia adaptation, and QoS/QoE. He has published more than 50 papers (incl. book chapters and tutorials) in these areas and the general chair of WIAMIS2008. Finally, he is an editorial board member of the Encyclopedia of Multimedia, ACM/Springer International Journal on Multimedia Tools and Applications (MTAP), and associate editor for IEEE Computer Science Computing Now.
He has been actively participating in several EC-funded projects, notably the FP6-IST-DANAE (2004-2006), FP6-IST-ENTHRONE (2006-2008), FP7-ICT-P2P-Next (2008-2012), and FP7-ICT-ALICANTE (2010-2013) projects. Dr. Timmerer participated in the work of ISO/MPEG for several years, notably as the head of the Austrian delegation, coordinator of several core experiments, co-chair of several ad-hoc groups, and as an editor for Parts 7 and 8 of MPEG-21, Digital Item Adaptation and Reference Software for which he received ISO/IEC certificates. His current contributions are in the area of the MPEG Extensible Middleware (MXM), MPEG-V (Media Context and Control), Advanced IPTV Terminal (AIT), and MPEG Media Transport (MMT) for which (i.e., MXM and MPEG-V) he also serves as an editor. Publications and MPEG contributions can be found under http://research.timmerer.com.

MPEG Press Release: Unified Speech and Audio Coding Achieves Major Milestone

--full press release available here.

Geneva, CH – The 93rd MPEG meeting was held in Geneva, Switzerland from the 26th to the 30th of July 2010.

Highlights of the 93rd Meeting

• USAC progresses to Committee Draft
• MPEG-V collection of six new parts for information exchange with virtual worlds is completed
• AIT reaches first major standard’s development milestone
• MPEG launches standard for the carriage of multimedia data over HTTP
• New amendment enables efficient representation of 3D meshes with multiple attributes
• Calls for Proposals for MMT and Composite Font Standard issued

Details for each highlight can be found in the full press release.

Digging Deeper – How to Contact MPEG

Communicating the large and sometimes complex array of technology that the MPEG Committee has developed is not a simple task. Experts, past and present, have contributed a series of white-papers and vision documents that explain each of these standards individually. The repository is growing with each meeting, so if something you are interested is not there yet, it may appear there shortly – but you should also not hesitate to request it. You can start your MPEG adventure at: http://mpeg.chiariglione.org/technologies.htm.

MPEG news: a report from the 93rd meeting, Geneva, Switzerland

Here comes a longer version of my "less than 140 chars summary" I've posted to Twitter :-)

Table of contents:

1. MPEG-V reached Final Draft International Standard (FDIS) after three years of development.
2. HTTP streaming of MPEG Media has been evaluated to Dynamic Adaptive Streaming over HTTP (DASH) working draft using 3GPP's Adaptation HTTP Streaming (AHS) as starting point.
3. A Call for Proposals for MPEG Media Transport (MMT) has been issued which will be evaluated in January 2011.

1. MPEG-V reached Final Draft International Standard (FDIS) after three years of development

MPEG-V parts (ISO/IEC 23005): see also previous posts related to MPEG-V

• ISO/IEC 23005-1: Architecture. This part is more like a technical reports describing the system architecture, standardization areas, and possible use cases as well as concrete instantiations of the architecture for various application domains.
• ISO/IEC 23005-2: Control information. This part provides description formats for describing device capabilities, user preferences, and device commands required in the context of the Representation of Sensory Effects (RoSE) framework, a possible instantiation of the MPEG-V architecture.
• ISO/IEC 23005-3: Sensory information. Together with the description formats defined in part 2 of MPEG-V it allows for a complete instantiation of the RoSE framework. In particular, it defines description formats enabling the representation of sensory effects (e.g., wind, vibration, explosion, etc.) that can be rendered on specific devices (e.g., fans, vibrating chairs, add'l light sources, etc.), hence, increasing the overall user experience.
• ISO/IEC 23005-4: Virtual world object characteristics. Part 4 of MPEG-V provides means to describe the characteristics of virtual world objects following an XML notation (btw. XML is also used by the other parts) that shall enable the interoperable exchange of such objects among virtual worlds of different kinds.
• ISO/IEC 23005-5: Data formats for interaction devices. N/A
• ISO/IEC 23005-6: Common types and tools. Defines basic data types and tools used by the other parts of MPEG-V.
• ISO/IEC 23005-7: Conformance and reference software (currently at FCD level). This part comprises the reference software as MPEG Extensible Middleware (MXM) engines and provides means for conformance testing.

Details concerning MPEG-V are discussed here.

2. Dynamic Adaptive Streaming over HTTP (DASH)

I had the honor to chair the evaluation of responses submitted to the HTTP Streaming of MPEG Media AhG. We had 15 submissions from 20 organizations (companies/institutions/universities) and the 3GPP's Adaptive HTTP Streaming has been selected as a starting point. In particular, MPEG will standardize a Manifest File (MF), a Delivery Format (DF), and means for easy conversion from/to existing File Formats (FF) and Transport Streams (TS) as depicted in the presentation below.

More details will be provided once the MPEG DASH standard has become more mature. Details concerning DASH are discussed here.

Call for Proposals for MPEG Media Transport (MMT)

A Call for Proposals for MPEG Media Transport (MMT) has been issued which will be evaluated in January 2011. All details can be found here and are not repeated here. Details concerning MMT are discussed here.

11th Multimedia Metadata Community Workshop on Interoperable Social Multimedia Applications (WISMA 2010)

On May 19-20, 2010 I've attended the 11th Multimedia Metadata Community Workshop on Interoperable Social Multimedia Applications (WISMA 2010) in Barcelona, Spain. The proceedings are available online on CEUR-WS.org and Twitter stream in case you'd like to review it.

I had two presentation which I'd like to provide here. The first one was on A Metadata Model for Peer-to-Peer Media Distribution:

Abstract: In this paper we describe a metadata solution for a Peer-to-Peer (P2P) content distribution system termed NextShare. We outline the key motivating factors for our approach, detail the overall generic architecture we have developed and present the workflow for delivering metadata through Peer-to-Peer based content distribution. The paper also presents the metadata model we have developed and we describe in detail how all the content can be packetized and distributed using NextShare. Finally, a description of the core and optional metadata attributes which may be utilized within the system is provided.

A Metadata Model for Peer-to-Peer Media Distribution

View more presentations from Klagenfurt University.

The second presentation provided an answer for the following question: Are Sensory Effects ready for the World Wide Web?

Abstract. The World Wide Web (WWW) is one of the main entry points to access and consume Internet content in various forms. In particular, the Web browser is used to access different types of media (i.e., text, image, audio, and video) and on some platforms is the only way to access the vast amount of information on the Web. Recently, it has been proposed to stimulate also other senses than vision or audition while consuming multimedia content through so-called sensory effects, with the aim to increase the user’s Quality of Experience (QoE). The effects are represented as Sensory Effects Metadata (SEM) which is associated to traditional multimedia content and is rendered (synchronized with the media) on sensory devices like fans, vibration chairs, lamps, etc. In this paper we provide a principal investigation of whether the sensory effects are ready for the WWW and, in anticipation of the result, we propose how to embed sensory effect metadata within Web content and the synchronized rendering thereof.

Are Sensory Effects ready for the World Wide Web?

View more presentations from Klagenfurt University.

WIAMIS'10: Increasing the User Experience of Multimedia Presentations with Sensory Effects

Here comes the abstract of our paper that has been accepted for publication at WIAMIS'10. Please join us there in case you'd like to get in touch we us!

Markus Waltl, Christian Timmerer, Hermann Hellwagner, Increasing the User Experience of Multimedia Presentations with Sensory Effects, accepted for publication at 11th International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS’10), Desenzano del Garda, Italy, April 12-14, 2010.

Abstract:
The term Universal Multimedia Experience (UME) has gained momentum and is well recognized within the research community. As this approach puts the user into the center stage, additional complexity is added to the overall quality assessment problem which calls for a scientific framework to capture, measure, quantify, judge, and explain the user experience. In previous work we have proposed the annotation of multimedia content with sensory effect metadata that can be used to stimulate also other senses than vision or audition. In this paper we report first results obtained from subjective tests in the area of sensory effects attached to traditional multimedia presentations such as movies that shall lead to an enhanced, unique, and worthwhile user experience.

See here the list of accepted papers at WIAMIS'10.

MPEG news: a report from the 90th meeting in Xi'an, China

The 90th MPEG meeting in Xi’an, China is coming up with some very interesting news which are briefly highlighted here. First and, I think, most importantly, the timeline for the new MPEG/ITU-T video coding format has been discussed and it seems the final Call for Proposals (CfP) will be ready in January 2010. A draft CfP is available now and hopefully will be also publicly available if they solve all the editing issues until early November. This means that the proposals will be evaluated in April 2010 (note: this will be a busy meeting as a couple of other calls need to be evaluated too; see later). The CfP defines five classes of test sequences with the following characteristics (number of sequences available in brackets):

• Class A with 2560x1600 cropped from 4Kx2K (2);
• Class B with 1920x1080p at 24/50-60 fps (5);
• Class C with 832x480 WVGA (4);
• Class D with 416x240 WQVGA (4); and
• Class E with 1280x720p at 50-60fps (3).

For classes B, C, and E subjective tests will be performed whereas classes A and D will be only evaluated objectively using PSNR. The reason for evaluating A and D using objective measurements is due to its insignificant subjective differences with B and C respectively. Finally, they’re still discussing about the actual common nickname name of the standard as it seems some are not happy with high-performance video coding but that’s yet another story…

Second, 3D video coding is still a major topic in MPEG but you probably need to wait yet another year until a Call for Proposals will be issued. That is, a 3DV standard will be probably available around the beginning of 2013 at the earliest. The major issue right now is the availability of content – as usual – and different device manufacturer standards with respect to 3D video.

The third major topic at this meeting was around AIT and MMT, two acronyms you shall become more familiar in the future. The former is referred to as Advanced IPTV Terminal (AIT) and aims to develop an ecosystem for media value chains and networks. Therefore, basic (atomic) services will be defined including protocols (payload formats) to enable users to call these services, Application Programming Interfaces to access services, and bindings to specific programming languages. Currently, 30 of these basic services are foreseen which can be clustered in services pertaining to the identification, authentication, description, storage, adaptation, posting, packaging, delivery, presentation, interaction, aggregation, management, search, negotiation, and transaction. The timeline is similar as for HVC which means that proposals will be evaluated in April 2010. The latter is referred to as MPEG Media Transport (MMT) and basically aims to become a successor of the well-known MPEG-2 Transport Stream. Currently, two topics are explored for which also requirements have been formulated. The first topic covers adaptive, progressive transport and the second topic is in the area of cross-layer design. Further topics where this activity might look into are hybrid delivery and conversational services. As for HVC and AIT, the proposals are going to be evaluated in April 2010. However, in order to further refine this possible new work item, MPEG will held a workshop in January 2010 on the Wednesday during the Kyoto meeting focusing on “adaptive progressive transport” and “cross-layer design”.

However, MPEG is looking forward to a very busy meeting in April 2010 which by the way will be held in Dresden, Germany.

Another issue that has been discussed in Xi’an was (again) the development of a royalty free codec within MPEG. While some might say that within MPEG, trying to establish a royalty free codec is a first step towards failure, others argue that MPEG-1 is already royalty free, for MPEG-2 most patents expire in 2011, the Internet community is requesting this (e.g., IETF established coded group and Google has chosen On2, a royalty free codec), and, finally, MPEG-4 Part 10 royalty free baseline basically failed. Thus, maybe (or hopefully) this is the right time for a royalty free codec within MPEG and who can predict the future? Anyway, there’s some activity going on in this area and if you’re interested, stay tuned…

Finally, I’d like to note that MPEG-V (Media Context & Control) and MPEG-U (Rich Media User Interface) are progressing smoothly and both going hand in hand towards its finalization. This meeting, the FCDs have been approved which forms a major milestone as this was the last chance for substantial new contributions. One such input was related to advanced user interaction like the Wiimote, etc. which will become part of MPEG-V but used also by MPEG-U. Hence, one might argue merging these two standards into one single standard called MPEG-W (i.e., U+V=W) and a wedding ceremony could be performed at the next meeting in Kyoto with Geishas as witnesses … why not? Please raise your voice now or be silent forever!

W3C Relaunches Multimodal Interaction Working Group

W3C is pleased to announce the relaunch of the Multimodal Interaction Working Group to develop technology that enables users to use their preferred modes of interaction with the Web. Deborah Dahl (Invited Expert) chairs the group which is chartered to develop open standards to adapt to device, user and environmental conditions, and to allow multiple modes of Web interaction including GUI, speech, vision, pen, gestures, haptic interfaces, sensor data, etc. W3C Members may use this form to join the Working Group. Read about the Multimodal Interaction Activity.

Very interesting as this WG aims bringing the Emotion Markup Language to Candidate Recommendation. So far the Elements of an EmotionML 1.0 have been identified as a W3C Incubator Group Report. Interestingly, ISO/IEC MPEG also has emotions within the scope of its MPEG-V work item. I wonder whether both groups could benefit from each other, the one way or the other ... let me know what is your opinion.

MPEG-V: Media Context and Control

MPEG-V: Media Context and Control

View more presentations from Christian Timmerer.

The committee drafts of parts 1-4 of MPEG-V are publicly available:

• Part 1: Architecture
• Part 2: Control Information
• Part 3: Sensory Information
• Part 4: Avatar Information

Related document (e.g., context/objectives, requirements, etc.) can be found in the here under "General".

Recently, I've presented a paper at QoMEx2009 which is related to MPEG-V.

MPEG Global Conference points the way to Ultra HD online services

Source: http://www.chiariglione.org/mpeg/meetings/london09/london_press.htm

London meeting sees significant improvement in compression for High Performance Video Coding

London, United Kingdom – The 89th MPEG meeting was held in London, United Kingdom from the 29th of June to the 3rd of July 2009.

Highlights of the 89th Meeting

Responses for Evidence Evaluated for HVC
During its 89th meeting, MPEG evaluated responses that were received on the Call for Evidence on High-Performance Video Coding (HVC), issued to obtain evidence of video coding technology providing compression capability clearly higher than that provided by the existing AVC standard (ITU-T H.264 | ISO/IEC 14496-10). Significant gains in compression were found when an assessment was made based on information brought by the contributors. A subjective comparison was performed in a blind test with a set of video test sequences encoded by the AVC High Profile at matching rate points. Gains were demonstrated for test cases ranging from resolutions as low as 412x240 pixels (Wide QVGA) up to resolutions for ultra-high definition. MPEG has therefore concluded that the development of the next generation of video compression technology is to be started with the issuing of a formal Call for Proposals by the next meeting.

AVC Extended with New Profiles for Baseline and MVC Technologies

At the 89th meeting, the AVC standard (ITU-T H.264 | ISO/IEC 14496-10) was further extended with the issuing of a Final Draft Amendment (FDAM) ballot containing the specification of two new profiles and new supplemental enhancement information. The first of the new profiles is the Constrained Baseline Profile, which forms the maximally-interoperable set of coding tools from the most widely deployed of existing profiles (the Baseline and High Profiles). The second new profile is a special case of multivew video coding (MVC) called the Stereo High Profile. The Stereo High profile enables all of the coding tools of the High Profile along with inter-view prediction capability for two-view (stereo) video applications such as 3D entertainment video.

Additionally, a new supplemental enhancement information (SEI) message has been defined for AVC. This new message – called the frame packing arrangement SEI message – enables the encoder to indicate to the decoder how to extract two distinct views of a video scene from a single decoded frame. The message also serves as a way to support stereo-view video in applications that require full compatibility with prior decoder designs that are not capable of supporting the new Stereo High Profile.

MPEG Promotes Technologies to link Real and Virtual Worlds

At its 88th meeting, MPEG had published a new call for proposals (N10526) with updated requirements (N10235) for an extension of the Media Context and Control project.

The technical contributions related to haptic and tactile devices, emotions, and virtual goods received at its 89th meeting have enabled MPEG to build a complete framework for defining haptic properties on top of virtual objects and to control haptic devices. This is now part of ISO/IEC 23005 or MPEG-V, a standard (formerly called Information Exchange with Virtual Worlds) providing a global framework and associated data representations to enable the interoperability between different virtual worlds (e.g. a digital content provider of a virtual world, a game with the exchange of real currency, or a simulator) and between virtual worlds and the real world (sensors, actuators, robotics, travel, real estate, or other physical systems).

MPEG Progresses Media Context and Control Project

MPEG has also advanced to the Committee Draft stage four parts of MPEG-V. The first part describes the architecture of the standard. The second part, “Control Information”, provides metadata representation of device capabilities and user preferences to be used for the information exchange between a controlling device and the real actuator or sensors. The third part, “Sensory Information”, provides metadata to represent sensory effects such as temperature, wind, vibration, fog, and more. The fourth part, “Avatar Characteristics”, provides metadata to commonly represent information about Avatars for the exchange of virtual characters between virtual worlds.

MPEG Hosts MXM Developer’s Day

The first MXM Developer’s Day workshop has been hosted by MPEG during its 89th meeting. The workshop featured demonstrations by companies and organisations that are developing MXM standards and applications. MXM, currently at its Final Committee Draft stage, provides specifications of APIs and an open source implementation (released under the BSD licence) to access various MPEG standards for easy deployment of applications. In this workshop detailed information about the APIs currently under standardization has been provided and several interesting demonstrations with the potential to create new business opportunities have also been presented. More information about this workshop can be found at http://mxm.wg11.sc29.org.

Rich Media User Interface Moves toward Completion

At its 89th meeting, MPEG has also advanced MPEG Rich Media UI (ISO/IEC 23007 or MPEG-U), to the Committee Draft stage. MPEG-U standardizes widget packaging, delivery, representation and communication formats. In its current draft, MPEG-U adopts and extends the W3C widget representation to provide a complete framework that can be used also in a non-Web based environment without a browser. Additionally, this standard enables communication among widgets on the same device or different devices, and other applications to better support connected environments.

Visual Signatures Enable New Applications

MPEG’s Visual Signatures define the world’s first standardized tools for content-based identification of any visual content even in very large databases, e.g. on the web. These tools enable a range of new applications including semantic linking, library management, metadata association (e.g. title, photographer, director, etc.) and content usage tracking. In the same way that a fingerprint or signature identifies a person, a Visual Signature is a compact descriptor uniquely representing either an image or video. The descriptor is derived directly from analysis of the visual content and is robust to heavy compression and editing.

The Image Signature and Video Signature are two separate amendments to MPEG-7. Collectively the two amendments are referred to as the MPEG-7 Visual Signatures. At the London meeting, the Video Signature advanced to the Proposed Draft Amendment (PDAM) stage with a target completion date of July 2010. The Image Signature was published as an ISO/IEC standard in April 2009.

Mobile Services to Be Enhanced by New BIFS Profile
At this meeting, MPEG advanced the new BInary Format for Scenes (BIFS) profile to the Committee Draft stage by incorporating additional nodes and technologies submitted as responses to the Call for Proposals for new BIFS technologies. The requirements for this profile (provided in N10567) originated from organizations of various industries and SDOs for digital radio and mobile television broadcasting. This profile will enable the development of more efficient and enhanced interactive services for mobile broadcasting services including digital radio or mobile television on small handheld devices. Moreover, it is backward compatible with Core2D@Level1 which is widely adopted by the industry.

Contact MPEG

Digging Deeper Once Again
Communicating the large and sometimes complex array of technology that the MPEG Committee has developed is not a simple task. The experts past and present have contributed a series of white-papers that explain each of these standards individually. The repository is growing with each meeting, so if something you are interested is not there yet, it may appear there shortly – but you should also not hesitate to request it. You can start your MPEG adventure at: http://www.chiariglione.org/mpeg/mpeg-tech.htm

Ends

Further Information
Future MPEG meetings are planned as follows:
No. 90, Xian, CN, 26-30 October, 2009
No. 91, Kyoto, JP, 18-22 January, 2010
For further information about MPEG, please contact:
Dr. Leonardo Chiariglione (Convener of MPEG, Italy)
Via Borgionera, 103
10040 Villar Dora (TO), Italy
Tel: +39 011 935 04 61
Email: mailto:leonardo@chiariglione.org
or
Dr. Arianne T. Hinds
Ricoh | IBM InfoPrint Solutions Company
6300 Diagonal Highway, MS 04N
Boulder, CO 80301, USA
Tel +1 720 663 3565
Email: arianne.hinds@infoprint.com

This press release and other MPEG-related information can be found on the MPEG homepage:
http://www.chiariglione.org/mpeg
The text and details related to the Calls mentioned above (together with other current Calls) are in the Hot News section, http://www.chiariglione.org/mpeg/hot_news.htm. These documents include information on how to respond to the Calls.
The MPEG homepage also has links to other MPEG pages which are maintained by the MPEG subgroups. It also contains links to public documents that are freely available for download by those who are not MPEG members. Journalists that wish to receive MPEG Press Releases by email should contact Dr. Arianne T. Hinds using the contact information provided above.

MPEG news: a report from the 89th meeting in London, UK

A lot of interesting things happened at this meeting, notably the MXM Developer's Day, the Modern Media Transport workshop, MPEG-V and MPEG-U have been promoted to committee draft, and for MPEG High-performance Video Coding (HVC) enough evidence has been provided in order to start working towards a Call for Proposals (CfP).

The MXM Developer's Day was a great success with 45+ participants and all presentations are publicly available. Leonardo presented the MXM Vision while Filippo and Marius concentrated on the MXM Architecture and API respectively. This introductory session was followed by practical examples and demonstrations:

• Fully Interoperable Streaming of Media Resources in Heterogeneous Environments
• Including MPEG-4 3D graphics in your application
• An MXM-based application for sharing protected content

The workshop on Modern Media Transport (MMT) had even more participants (80+) and was clustered into two session. Session one was focusing on industry practice and presentations where given on how MPEG-2 TS and MP4 is being used. Furthermore, the DVB activity in the area of IPTV and InternetTV was presented. All the presentations will be publicly available through the MPEG Web site. The conclusion was that although MPEG-2 TS / MP4 is heavily used, it has some drawbacks due to their popularity. That is, MPEG-2 TS is running out of code points which is an issue. On the other hand, if MPEG is going to standardize something new, it has been recognized that it has to be to substantially better than what exists on the market with a high demand of backwards-compatibility to MPEG-2 TS. The issue will be further studied and stay tuned!

MPEG-V also known as Media Context and Control has promoted four parts to committee draft. The four parts are as follows:

• Part 1: Architecture
• Part 2: Control Information
• Part 3: Sensory Information
• Part 4: Avatar Characteristics

I've provided an overview during the final plenary and the presentation is accessible here.

MPEG-U is about Widgets and has been promoted to committee draft also. It seems to be an interesting activity which has a relationship to W3C's Widget activity. It will be interesting to see how these two standards co-exist.

Finally, the call for evidence for High-performance Video Coding (HVC) provided the following result: "Yes, we have enough evidence about improved compression technology (compared to AVC HP)". Thus, MPEG started working towards a call for proposals and a time schedule has been created. Furthermore, the future collaboration between MPEG and VCEG has been discussed.

That's it for now but I'll provide more details on the individual topics later. Please stay tuned!