Showing posts with label high-performance video coding. Show all posts
Showing posts with label high-performance video coding. Show all posts

Thursday, February 11, 2010

Heil The New xVC Heir: HVC

For the record and also as an update, here is my guest post at RADVISION's Video over Enterprise blog.
"The HVC efforts have already begun. Based on the current timeline, one can expect the new standard to be available around the end of 2012/beginning of 2013. This may seem far away, but as many video infrastructure products have a 2 year design process, this is very relevant to today’s design efforts.
And so we are looking forward to a new and very exciting xVC episode, and it will be very interesting to see how the new standard evolves and whether it will fit today’s expectations. Thus, stay tuned!"


The latest information about High-performance Video Coding (HVC) can be found at the MPEG Web site and here are links to the publicly available documents:
Most interestingly, the joint CfP on HVC - joint between MPEG and ITU-T - and the vision and requirements documents. If you're interested in the upcoming discussions around HVC you may join the mailing list or even the #HVC Twitter stream (alt) ;-)

Monday, February 8, 2010

MPEG targets next generation teleconferencing and personalized audio in its newest technology

Kyoto, Japan – The 91st MPEG meeting was held in Kyoto, Japan from the 18th to the 22nd of January 2010.

Highlights of the 91st Meeting
  • Advanced technology enables bitrate-efficient teleconferencing and personalized audio
  • MPEG and ITU-T SG16 form Joint Collaborative Team (JCT) for High Performance Video Coding
  • Creating solutions for Advanced IPTV Terminals
  • MPEG Extensible Middleware (MXM) standard debuts to facilitate easy access to MPEG tools across a global market
  • MPEG creates efficient and flexible standard for 3D graphics compression
  • MPEG Media Transport (MMT) workshop targets requirements and technologies for streaming of MPEG content
Wanna know more/details? Please check out the official press release!

    Thursday, January 28, 2010

    Overview of Selected Current MPEG Activities

    --this covers a report from the 91st MPEG meeting in Kyoto, Japan

    Previously, I've always provided a written report but this time it comes in form of a presentation (slideshow) - enjoy!

    Monday, November 9, 2009

    Draft Call for Proposals on High-Performance Video Coding (HVC)

    --excerpted from here!

    Purpose: A new generation of video compression technology that has substantially higher compression capability than the existing AVC (ITU-T H.264 | ISO/IEC 14496-10) standard is targeted (see [1][2] for background information).

    Timeline:
    • 2009/11/06: Draft CfP
    • 2009/11/06: Availability of all test materials defined in Draft CfP (download information can be requested from contact persons)
    • 2009/11/30: Availability of anchors
    • 2010/01/15: Deadline for Pre-registration (mandatory): one of the contact persons (see Section 9) must be notified An invoice for the testing fee will be sent after pre-registration. Additional logistic information will also be sent to proponents by this date.
    • 2010/01/22: Final Call for Proposals (Note: The Final Call may contain minor updates)
    • 2010/02/15: Formal registration and payment of the fee
    • 2010/02/22: Coded test material available at the test site (Note: People who formally registered will receive instructions regarding how to submit the coded materials)
    • 2010/03/15: Subjective assessment starts at this date or sooner
    • 2010/04/12: Registration of documents describing the proposals with contact persons
    • 2010/04/13: Submission of documents to contact persons (Note: Contact persons will compile all proposal documents into one or more input documents and upload)
    • 2010/04/16: Cross-checking of bitstreams and binary decoders (participation mandatory for proponents)
    • 2010/04/17: Subjective test results available within standardization body
    • 2010/04/17-23: Evaluation of proposals at standardization meeting (Note: Proponents are requested to attend this standardization meeting)
    Anticipated tentative timeline after CfP
    (refers to first version of the new standard, which may be extended later):
    • Core experiments begin 2010/04
    • Test model 2010/10
    • Final standard approval 2012/07
    Test classes:
    • Class A: Cropped areas of size 2560x1600 taken from the following sequences (frame rates unchanged): First 5 s of "Traffic" (4096x2048p 30 fps), "PeopleOnStreet" (3840x2160p 30 fps).
    • Class B: 1920x1080p 24 fps: "ParkScene", "Kimono" and 1920x1080p 50-60 fps: "Cactus", "BasketballDrive", "BQTerrace"
    • Class C: 832x480p 30-60 fps (WVGA): "BasketballDrill", "BQMall", "PartyScene", "RaceHorses"
    • Class D: 416x240p 30-60 fps (WQVGA): "BasketballPass", "BQSquare", "BlowingBubbles", "RaceHorses"
    • Class E: 1280x720p 60fps: "Vidyo1", "Vidyo3" and "Vidyo4"
    For further information about constraint sets, target rate points, and anchors, please go to directly the Draft CfP.

    Test sites and fees:
    The proposals submission material for Classes B, C and E will be evaluated by means of a formal subjective assessment process. The tests will be conducted at FUB (Test Coordinator, Rome, Italy), EBU (Geneva, Switzerland) and EPFL (Lausanne, Switzerland).

    Proponents will be charged a fee per submitted algorithm proposal. Such fee will be a flat charge for each proposal to cover the logistic cost (without any profit). The fee is non-refundable after the formal registration deadline has passed. The fee is estimated to be in the range of 4000 to 6000 Euros. The exact cost can only be determined after the number of submitted proposals is known, but over-payments will be reimbursed.

    Additional requirements on submissions can be found here.

    References:
    [1] VCEG-AL96, Draft Requirements for Next Generation Video Coding, July 2009
    [2] ISO/IEC JTC1/SC29/WG11, Vision and Requirements for High-Performance Video Coding, MPEG Document N10361, February 2009

    Saturday, October 31, 2009

    MPEG news: a report from the 90th meeting in Xi'an, China

    The 90th MPEG meeting in Xi’an, China is coming up with some very interesting news which are briefly highlighted here. First and, I think, most importantly, the timeline for the new MPEG/ITU-T video coding format has been discussed and it seems the final Call for Proposals (CfP) will be ready in January 2010. A draft CfP is available now and hopefully will be also publicly available if they solve all the editing issues until early November. This means that the proposals will be evaluated in April 2010 (note: this will be a busy meeting as a couple of other calls need to be evaluated too; see later). The CfP defines five classes of test sequences with the following characteristics (number of sequences available in brackets):
    • Class A with 2560x1600 cropped from 4Kx2K (2);
    • Class B with 1920x1080p at 24/50-60 fps (5);
    • Class C with 832x480 WVGA (4);
    • Class D with 416x240 WQVGA (4); and
    • Class E with 1280x720p at 50-60fps (3).
    For classes B, C, and E subjective tests will be performed whereas classes A and D will be only evaluated objectively using PSNR. The reason for evaluating A and D using objective measurements is due to its insignificant subjective differences with B and C respectively. Finally, they’re still discussing about the actual common nickname name of the standard as it seems some are not happy with high-performance video coding but that’s yet another story…

    Second, 3D video coding is still a major topic in MPEG but you probably need to wait yet another year until a Call for Proposals will be issued. That is, a 3DV standard will be probably available around the beginning of 2013 at the earliest. The major issue right now is the availability of content – as usual – and different device manufacturer standards with respect to 3D video.

    The third major topic at this meeting was around AIT and MMT, two acronyms you shall become more familiar in the future. The former is referred to as Advanced IPTV Terminal (AIT) and aims to develop an ecosystem for media value chains and networks. Therefore, basic (atomic) services will be defined including protocols (payload formats) to enable users to call these services, Application Programming Interfaces to access services, and bindings to specific programming languages. Currently, 30 of these basic services are foreseen which can be clustered in services pertaining to the identification, authentication, description, storage, adaptation, posting, packaging, delivery, presentation, interaction, aggregation, management, search, negotiation, and transaction. The timeline is similar as for HVC which means that proposals will be evaluated in April 2010. The latter is referred to as MPEG Media Transport (MMT) and basically aims to become a successor of the well-known MPEG-2 Transport Stream. Currently, two topics are explored for which also requirements have been formulated. The first topic covers adaptive, progressive transport and the second topic is in the area of cross-layer design. Further topics where this activity might look into are hybrid delivery and conversational services. As for HVC and AIT, the proposals are going to be evaluated in April 2010. However, in order to further refine this possible new work item, MPEG will held a workshop in January 2010 on the Wednesday during the Kyoto meeting focusing on “adaptive progressive transport” and “cross-layer design”.

    However, MPEG is looking forward to a very busy meeting in April 2010 which by the way will be held in Dresden, Germany.

    Another issue that has been discussed in Xi’an was (again) the development of a royalty free codec within MPEG. While some might say that within MPEG, trying to establish a royalty free codec is a first step towards failure, others argue that MPEG-1 is already royalty free, for MPEG-2 most patents expire in 2011, the Internet community is requesting this (e.g., IETF established coded group and Google has chosen On2, a royalty free codec), and, finally, MPEG-4 Part 10 royalty free baseline basically failed. Thus, maybe (or hopefully) this is the right time for a royalty free codec within MPEG and who can predict the future? Anyway, there’s some activity going on in this area and if you’re interested, stay tuned…

    Finally, I’d like to note that MPEG-V (Media Context & Control) and MPEG-U (Rich Media User Interface) are progressing smoothly and both going hand in hand towards its finalization. This meeting, the FCDs have been approved which forms a major milestone as this was the last chance for substantial new contributions. One such input was related to advanced user interaction like the Wiimote, etc. which will become part of MPEG-V but used also by MPEG-U. Hence, one might argue merging these two standards into one single standard called MPEG-W (i.e., U+V=W) and a wedding ceremony could be performed at the next meeting in Kyoto with Geishas as witnesses … why not? Please raise your voice now or be silent forever!

    Monday, February 9, 2009

    Vision and Requirements for High-Performance Video Coding (HVC)

    The following text comprises the 'Vision and Requirements for High-Performance Video Coding (HVC)' approved by MPEG's Video and Requirements Sub-groups with the full reference: ISO/IEC JTC1/SC29/WG11/N10361, Lausanne, Switzerland, February 2009. Also publicly available here.

    Introduction - Vision

    This document presents the vision, goals and requirements for the High-Performance Video Coding (HVC) (note: this name is tentative) coding standard. This document will evolve in the next few months as the project takes shape and also as a result of input from potential users.

    A large quantity of video material is already distributed in digital over broadcast channels, digital networks and packaged media. More and more of this material will be distributed with increased resolution and quality demand.

    Technology evolution will soon make possible the capture and display of video material with a quantum leap in quality (temporal and spatial resolution, color fidelity, amplitude resolution). Networks are already finding it difficult to carry HDTV resolution and data rates economically to the end user. Therefore, further data rate increase will put additional pressure on the networks. For example:
    • High-definition (HD) devices (displays and cameras) are affordable for consumer usage today, while the currently available internet and broadcast network capacity is not sufficient to transfer large amount of HD content economically. While this situation may change slowly over time, the next generation of ultra-HD (UHD) contents and devices, such as 4Kx2K displays for home cinema applications and digital cameras, are already appearing on the horizon.
    • For mobile terminals, video quality using resolutions such as QCIF at low frame rates and low bit rates today is largely unacceptable. While the overall data rate will increase with the evolution of 3G/LTE and 4G networks, also the number of users increases simultaneously with their quality demand. Anticipating that lightweight HD resolutions such as 720p or even beyond will be introduced in the mobile sector to provide similar perceptual quality as for the home applications, lack of sufficient data rates as well as the prices to be paid for transmission will remain a problem for the long term.
    MPEG has concluded that video bitrate (when current compression technology is used) will go up faster than the network infrastructure will be able to carry economically, both for wireless and wired networks. Therefore a new generation of video compression technology that has sufficiently higher compression capability than the existing AVC standard in its best configuration (the High Profile), is needed. A study has been started on the feasibility of HVC, which is mainly intended for high quality applications, in particular expecting
    • Performance improvements in terms of coding efficiency at higher resolution,
    • Applicability to entertainment-quality services such as HD mobile, home cinema and Ultra High Definition (UHD) TV.
    MPEG plans to develop HVC following its general philosophy and goals [1]:
    • A requirement to be supported by HVC should refer to at least one specific application
    • Minimum number of conformance points defined in terms of profiles and levels
    • Definition of a new HVC standard that is not backwards compatible with an existing MPEG standard should be justified by sufficient performance gain
    • Complexity scalability in encoder and decoder: (1) Possible asymmetry of encoder and decoder processing complexity; (2)Full specification of decoding, preferably with no mismatch; (3)Scalability between amount of encoder processing and achievable quality
    Requirements

    Compression Performance

    A substantially greater bitrate reduction over MPEG-4 AVC High Profile is required for the target application(s); at no point of the entire bitrate range shall HVC be worse than existing standard(s).

    Subjective visually lossless compression shall be supported.

    Picture Formats

    The HVC development effort will focus on a set of rectangular picture formats that will include all commonly used picture formats, ranging at least from VGA to 4Kx2K, and potentially extending to QVGA and 8Kx4K.

    Picture formats of arbitrary size shall also be supported, within limits specific to each Level. The HVC codec shall support at least the same range of picture formats supported by the MPEG-4 AVC syntax.

    Color Spaces and Color Sampling

    a) The YCbCr color space 4:2:0, 8 bits per component shall be supported
    b) YCbCr/RGB 4:4:4 should be supported.
    c) Higher bit depth up to 14 bits per component should be supported

    Note: Further information is needed on whether support for 4:2:2 color space is needed for progressively scanned video due to digital video standards used in studios. Same for wide gamut color.

    Frame Rates

    In general typical frame rates of 24 to 60 fps shall be supported.
    The HVC codec shall be at least as flexible in terms of frame rate as the AVC syntax which supports up to 172 frames per second (fps).

    Scanning Methods

    Support for progressive scanning shall be required for all Profiles and Levels.

    Complexity

    HVC complexity shall allow for feasible implementation within the constraints of the available technology at the expected time of usage.

    HVC should be capable of trading-off complexity and compression efficiency, by having
    • an operating point with significant decrease in complexity compared to AVC but with better compression efficiency than AVC,
    • an operating point with increase complexity and commensurate increase in compression performance.

    Parallel processing should be possible.

    Note: Complexity includes: Power consumption, computational power, memory bandwidth etc.

    Low Delay

    HVC should be capable of operating in a low delay mode.

    Note: HVC should be capable of trading-off complexity, compression efficiency and delay.

    Random Access and “Trick Modes”

    The standard shall support random access to certain positions of a stored video stream, and allow fast channel switching in the case of multi-channel services.

    Pause, fast forward, normal speed reverse, and fast reverse access to a stored video bitstream should be supported.

    Intra-only coding of video frames should be supported.

    Error Resilience

    Video bitstream segmentation and packetization methods for the target networks shall be developed. The video layer should be designed in a way such that relevant error resilience measures can effectively be applied at the network layer for networks needing error recovery. Proper balance of increase in complexity, loss in coding efficiency and benefits achieved by the error resilience measures at the coding layer for various networks should be achieved.

    Buffer Models

    Buffer models, including hypothetical reference decoders (HRDs), shall be specified for target applications.

    Interface to System Layers

    The HVC codec shall be designed to permit efficient adaptation and integration with the target system and delivery layers.

    Scalable Video Coding

    The design of the initial phase shall be such that it will possible to add scalable coding tools on top of the single layer system under consideration initially.

    Definitions

    This section defines terms used within the context of this document.

    Conformance Point: A conformance point is a specification of a particular Profile, at a certain Level, at which conformance can be tested. Conformance Points establish normative parts of the HVC codec standard. A HVC encoder operating at a given Profile and Level shall not generate an output which exceeds the decoding capabilities of a decoder compliant with the same Profile and Level. A HVC decoder operating at a given Profile and Level shall correctly decode the output of any encoder that is compliant with the same Profile and Level. For each Profile there shall be at least one Level.

    Profile: A Profile is a set of algorithmic tools, representing a particular tradeoff of performance and resource consumption, supporting the requirements of a particular set of applications. Applications which require similar tradeoffs between these parameters should use the same Profile.

    Level: A Level describes performance parameters within each Profile, which describe lower limits on decoder capability. Performance parameters typically include maximum picture size, macroblocks per second, bitrate, frame rate, buffer sizes and similar parameters.

    References

    [1] “The MPEG vision”, ISO/IEC JTC1/SC 29/WG11/N10412, Lausanne, CH, February 2009.


    Friday, February 6, 2009

    MPEG news: a report from the 87th meeting in Lausanne, Switzerland

    MPEG’s high-performance video coding (HVC) standard is evolving and currently targets mobile devices, IPTV, and Ultra-HD. However, the trade-off between coding efficiency and codec complexity is still driving the thresholds which has been set at this meeting, well, at least initially. For low-complexity HVC is seeking for 25% gain in coding efficiency and for full-/increased-complexity the threshold is set to 50% gain in coding efficiency (cf. goal for AVC standardization). The application scenarios range from shared Ultra-HD to personalized experiences. The latter is targeting a viewing distance of 0.5*h (i.e., 50cm) and for personal use only. A vision document (N10361) has been issued at this meeting and interested parties are requested to join the Ad-hoc Group (AhG) for further discussions (N10371). One open issue with HVC is whether and how audio will adapt to these new developments, probably with high-performance audio coding (HAC) or can HE-AAC already complement HVC? Anyway, short viewing distance ultimately leads to short listening distances and a wide angle for the sound stage...

    3D video coding (3DVC) has already a history within MPEG. A vision document has been produced here stating that MPEG’s version of 3D video shall be compatible with existing standards, mono/stereo devices, and existing/planned infrastructure. While the former seems to be very clear, i.e., backwards compatibility to AVC is desirable in the same way as it was for SVC, the latter needs some more discussions. How can one be compatible with planned infrastructures?

    In the area of advanced IPTV terminal (AIT) we had a productive meeting with ITU-T SG16 Q.13. The objectives look very promising and aim to increase the user experience thanks to the usage of latest media coding, transport, and processing technologies. An AhG has been set up to discuss the details. Again, AhGs are open to the public and everyone can join and contribute.

    The information exchange with virtual worlds (MPEG-V) has merged with the representation of sensory effects (RoSE) and will be now jointly developed under the umbrella of MPEG-V. However, it was sought that ‘information exchange with virtual worlds’ is no longer appropriate and they’re seeking for a new name. Any inputs are welcome and are discussed within the AhG that is open to everybody. Thus, join this exciting activity now! In its current working draft the following sensory effects are defined which shall create an enhanced, immersive user experience: various light effects, temperature, wind, vibration, water sprayer, perfumer/scent, fog, window blind/shadow, sound, and color correction.

    Finally, MPEG got another Emmy, the "Technology and Engineering Emmy Award 2007-2008" for MPEG-4 AVC from the National Academy of Television Arts & Sciences (NATAS). It seems that success within MPEG is from now on measured in terms of number of awards received ;-) And last but not least, if you’ve been ever interested in the MPEG vision, you may find it here.

    Friday, October 17, 2008

    What are the new challenges in video coding standardization? (part two)

    The results of the workshop will be available as output document N10174 with the main conclusions as follows:
    • Larger formats (resolutions) are becoming more and more popular.
    • Video bitrate - especially uncompressed - is (far) ahead the technologies for economic network transmission for both wired and wireless networks, i.e., bitrate >(>) bandwidth.
    • Next generation of video compression technology is needed with higher compression capabilities than the AVC standard.
    However, there's a need for more evidence that goes beyond the initial optimism where 35% bitrate reduction was reported for 1080p. Therefore, MPEG will adopt its usual process which consists of:
    1. Collecting requirements mainly targeting compression performance and higher resolutions
    2. Call for test material (N10176) seeking for video sequence with VGA, 1080p, or 4Kx2K resolution captured with current camera equipment.
    3. Call for evidence to let the industry (in particular, the national bodies) respond to the idea of developing a High-performance Video Coding (HVC) standard.
    I'll provide references to the above mentioned documents as soon as they become available! Interested parties are invited to discuss issues related to HVC as part of an Ad-hoc Group (AhG) with the following mandates:
    1. To further discuss vision, applications and requirements of high-performance video coding.
    2. To distribute the Call for Test Sequences and assist the video chair in collecting and evaluating new test sequences for the upcoming Call for Evidence.
    3. To further discuss and develop conditions and testing methodologies for the Draft Call for Evidence.
    Reference:

    Tuesday, October 14, 2008

    What are the new challenges in video coding standardization? (part one)

    At today's workshop on new challenges in video coding standardization, MPEG tried to figure out what's beyond AVC, SVC, and MVC. Even before the workshop started, a new acronym was found for this envisaged standard: HVC which stands for High-Performance Video Coding. In my view this already provides a rough direction where MPEG is heading to, i.e., higher resolutions (beyond 4K a.k.a. UD which is four times full HD) and higher framerates (beyond 120Hz up to 180/200/240Hz). Interestingly, displays that can handle such resolutions and framerate will be available around 2012 which would require to start video coding standardization for that right now. Another issue that needs to be considered is the viewing angle with such huge display sizes. Currently, the distance between display and viewer is about three times the display height which results in a viewing angle of 33° using a 40'' TV set. With a 80'' device the viewing angle increases to 63° that needs to be considered both by the codec and the display, of course. However, it is not clear which kind of application (domain) MPEG is targeting for this possible future video coding standard.

    Of course, there's much to say bout that and I'll report on that later during the week.

    Finally, I also learned another acronym (note: MPEG is full of acronyms) which is ODS that stands for Other Digital Stuff.