The original blog post can be found at the Bitmovin Techblog and has been modified/updated here to focus on and highlight research aspects. Additionally, this version of the blog post will be also posted at ACM SIGMM Records.
The 128th MPEG meeting concluded on October 11, 2019 in Geneva, Switzerland with the following topics:
- Low Complexity Enhancement Video Coding (LCEVC) Promoted to Committee Draft
- 2nd Edition of Omnidirectional Media Format (OMAF) has reached the first milestone
- Genomic Information Representation – Part 4 Reference Software and Part 5 Conformance Promoted to Draft International Standard
In this report we will focus on video coding aspects (i.e., LCEVC) and immersive media applications (i.e., OMAF). At the end, we will provide an update related to adaptive streaming (i.e., DASH and CMAF).
Low Complexity Enhancement Video Coding
Low Complexity Enhancement Video Coding (LCEVC) has been promoted to committee draft (CD) which is the first milestone in the ISO/IEC standardization process. LCEVC is part two of MPEG-5 or ISO/IEC 23094-2 if you prefer the always easy-to-remember ISO codes. We introduced MPEG-5 already in previous posts and LCEVC is about a standardized video coding solution that leverages other video codecs in a manner that improves video compression efficiency while maintaining or lowering the overall encoding and decoding complexity.
The LCEVC standard uses a lightweight video codec to add up to two layers of encoded residuals. The aim of these layers is correcting artefacts produced by the base video codec and adding detail and sharpness for the final output video.
The target of this standard comprises software or hardware codecs with extra processing capabilities, e.g., mobile devices, set top boxes (STBs), and personal computer based decoders. Additional benefits are the reduction in implementation complexity or a corresponding expansion in spatial resolution.
LCEVC is based on existing codecs which allows for backwards-compatibility with existing deployments. Supporting LCEVC enables “softwareized” video coding allowing for release and deployment options known from software-based solutions which are well understood by software companies and, thus, opens new opportunities in improving and optimizing video-based services and applications.
Research aspects: in video coding, research efforts are mainly related to coding efficiency and complexity (as usual). However, as MPEG-5 basically adds a software layer on top of what is typically implemented in hardware, all kind of aspects related to software engineering could become an active area of research.
Omnidirectional Media Format
The scope of the Omnidirectional Media Format (OMAF) is about 360° video, images, audio and associated timed text and specifies (i) a coordinate system, (ii) projection and rectangular region-wise packing methods, (iii) storage of omnidirectional media and the associated metadata using ISOBMFF, (iv) encapsulation, signaling and streaming of omnidirectional media in DASH and MMT, and (v) media profiles and presentation profiles.
At this meeting, the second edition of OMAF (ISO/IEC 23090-2) has been promoted to committee draft (CD) which includes
- support of improved overlay of graphics or textual data on top of video,
- efficient signaling of videos structured in multiple sub parts,
- enabling more than one viewpoint, and
- new profiles supporting dynamic bitstream generation according to the viewport.
360° video is certainly a vital use case towards a fully immersive media experience. Devices to capture and consume such content are becoming increasingly available and will probably contribute to the dissemination of this type of content. However, it is also understood that the complexity increases significantly, specifically with respect to large-scale, scalable deployments due to increased content volume/complexity, timing constraints (latency), and quality of experience issues.
Research aspects: understanding the increased complexity of 360° video or immersive media in general is certainly an important aspect to be addressed towards enabling applications and services in this domain. We may even start thinking that 360° video actually works (e.g., it's possible to capture, upload to YouTube and consume it on many devices) but the devil is in the detail in order to handle this complexity in an efficient way to enable seamless and high quality of experience.
DASH and CMAF
The 4th edition of DASH (ISO/IEC 23009-1) will be published soon and MPEG is currently working towards a first amendment which will be about (i) CMAF support and (ii) event processing model. An overview of all DASH standards is depicted in the figure below, notably part one of MPEG-DASH referred to as media presentation description and segment formats.
The 2nd edition of the CMAF standard (ISO/IEC 23000-19) will become available very soon and MPEG is currently reviewing additional tools in the so-called technologies under considerations document as well as conducting various explorations. A working draft for additional media profiles is also under preparation.
Research aspects: with CMAF, low-latency supported is added to DASH-like applications and services. However, the implementation specifics are actually not defined in the standard and subject to competition (e.g., here). Interestingly, the Bitmovin video developer reports from both 2018 and 2019 highlight the need for low-latency solutions in this domain.
At the ACM Multimedia Conference 2019 in Nice, France I gave a tutorial entitled “A Journey towards Fully Immersive Media Access” which includes updates related to DASH and CMAF. The slides are available here.
Outlook 2020
Finally, let me try giving an outlook for 2020, not so much content-wise but events planned for 2020 that are highly relevant for this column:- MPEG129, Jan 13-17, 2020, Brussels, Belgium
- DCC 2020, Mar 24-27, 2020, Snowbird, UT, USA
- MPEG130, Apr 20-24, 2020, Alpbach, Austria
- NAB 2020, Apr 08-22, Las Vegas, NV, USA
- MIPR 2020, Apr 09-10, Shenzhen, Guangdong, China
- ICASSP 2020, May 4-8, 2020, Barcelona, Spain
- QoMEX 2020, May 26-28, 2020, Athlone, Ireland
- MMSys 2020, Jun 8-11, 2020, Istanbul, Turkey
- IMX 2020, June 17-19, 2020, Barcelona, Spain
- IVMSP 2020, June 23-26, 2020, Nafplio, Greece
- MPEG131, Jun 29 - Jul 3, 2020, Geneva, Switzerland
- NetSoft, QoE Mgmt Workshop, Jun 29 - Jul 3, 2020, Ghent, Belgium
- ICME 2020, Jul 6-10, London, UK
- ATHENA summer school, Jul 13-17, Klagenfurt, Austria
No comments:
Post a Comment