Monday, October 14, 2019

Happy World Standards Day 2019 - Video Standards Create a Global Stage

Today on October 14, we celebrate the World Standards Day, "the day honors the efforts of the thousands of experts who develop voluntary standards within standards development organizations" (SDOs). Many SDOs such as W3CIETF, ITU, ISO (incl. JPEG and MPEG) celebrate this with individual statements, highlighting the importance of standards and interoperability in today's information and communication technology landscape. Interestingly, this year's topic for the World Standards Day within ISO is about video standards creating a global stage. Similarly, national bodies of ISO provide such statements within their own country, e.g., the A.S.I. statement can be found here (note: in German). I have also blogged about the World Standards Day in 2017.

HEVC Emmy located at ITU-T, Geneva, CH (Oct'19).
The numbers for video content created, distributed (incl. delivery, streaming, ...), processed, consumed, etc. increases tremendously and, actually, more than 60 percent of today's world-wide internet traffic is attributed to video streaming. For example, almost 700,000 hours of video are watched on Netflix and 4.5 million videos are viewed on YouTube within a single internet minute in 2019. Videos are typically compressed (or encoded) prior to distribution and are decompressed (or decoded) before rendering on potentially a plethora of heterogeneous devices. Such codecs (portmanteau of coder-decoder) are subject to standardization and with AVC and HEVC (jointly developed by ISO/IEC MPEG and ITU-T VCEG) we have two successful standards which even have been honored with Primetime Engineering Emmy Awards (see one of them in the picture).

Within Austria, Bitmovin has been awarded with the Living Standards Award in 2017 for its contribution to the MPEG-DASH standard, which enables dynamic adaptive streaming over HTTP. This standard -- the 4th edition is becoming available very soon -- is now heavily deployed and has been adopted within products and services such as Netflix, Amazon Prime Video, YouTube, etc.

Standardization can be both source for and sink of research activities, i.e., development of efficient algorithms conforming to existing standards or research efforts leading to new standards. One example of such research efforts just recently started at the Institute of Information Technology (ITEC) at Alpen-Adria-Universität Klagenfurt (AAU) as part of the ATHENA (AdapTive Streaming over HTTP and Emerging Networked MultimediA Services) project. The aim of this project is to research and develop novel paradigms, approaches, (prototype) tools and evaluation results for the phases (i) multimedia content provisioning (video coding), (ii) content delivery (video networking), (iii) content consumption (player) in the media delivery chain, and (iv) end-to-end aspects, with a focus on, but not being limited to, HTTP Adaptive Streaming (HAS).

The SDO behind these standards is MPEG (officially ISO/IEC JTC 1/SC 29/WG 11), which has a proven track record of producing very successful standards (not only those mentioned as examples above) and its future is currently discussed within its parent body (SC 29). A possible MPEG future is described here, which suggests upgrading the current SC 29 working groups to sub-committees (SCs), specifically to spin-off a new SC that basically covers MPEG while the remaining WG (JPEG) arises within SC 29. This proposal of MPEG and JPEG as SC is partially motivated by the fact that both WGs work on a large set of standardization projects, actually developed by its subgroups. Thus, elevating both WGs (JPEG & MPEG) to SC level would only reflect the current status quo but would also preserve two important brands for both academia and industry. Further details can be found at

Tuesday, September 10, 2019

2019 Global Internet Phenomena Report: more than 60 percent is Video Streaming

Source: Sandvine, Sep 10, 2019.
The 2019 Global Internet Phenomena Report has been published on September 10, 2019 and is available here.  I've previously posted about this in 2015, 2018, and Feb 2019 (related to mobile). Thus, it's also interesting to compare this report with what has been posted previously, specifically with respect to the 2018 report...

The 2019 global report reveals that video streaming now covers more than 60% of the internet traffic (see figure on the right) but only a small increase compared to last year (+2.9). We may question whether the 80% (or more) will be reached by 2022 as predicted by some reports (note: I assume this one here is meant).  The question is whether 4K or 8K will help to make the predictions become reality; we will see pretty soon.

Interestingly, Netflix' application traffic share decreased by 2.3 percentage points and is now about 12.6% while other HTTP media streaming traffic reached 12.8%. "Operator IPTV" increased to 7.2% with +2.8 compared to last year. For example, in Americas "Operator IPTV" has even a higher downstream application traffic share (15%) than Netflix (12.87%).

From a European perspective, we see QUIC among the top 10 of "EMEA: Downstream Application Traffic Share" with 3.1% but, unfortunately, the report does not provide further details about what that actually means.

The "Spotlight: Streaming Video Traffic Share" is shown in the figure below and reveals that both Netflix and YouTube have a higher traffic share in EMEA than Americas and "Operator IPTV" is only mentioned in Americas and not at all in EMEA or APAC.

Source: Sandvine, Sep 10, 2019.

As mentioned in the beginning, the full report is available here -- covering also other aspects -- but it confirms that global video traffic share increases but probably with smaller steps than anticipated some years ago.

Monday, September 9, 2019

Video Developer Report 2019

... and Bitmovin did it again; published the 2019 Video Developer Report last week. I've briefly reported about it last year here. Interestingly, this year 542 people from 108 countries participated (vs. 456 from over 67 countries last year).

The biggest challenges seem to be latency (54%) and playback on all devices (41%). Other challenges (>20%) are related to DRM, CDN, user engagement with video, and ads in general.

Last year I've also shared the codec usage and it's probably interesting to compare these numbers with this year's results as shown below. Interestingly, the numbers (for 'planning to implement') are a bit lower compared to last year which could be explained by a more conservative approach from developers or simply by the fact that more people responded to the survey with a greater diversity in terms of different countries.

Current Video Codec Usage and Plans to Implement in next 12 Months.
The actual video codec usage compares to last year's report as follows: AVC (-1), HEVC (+1), VP9 (+/- 0), AV1 (+1).

Another interesting aspect is the usage of streaming formats and plans to implement them within the next 12 months as shown below. Comparing with last year's report (available here), we can observe the following major changes: HLS (-3), MPEG-DASH (-3), RTMP (-2), Smooth Streaming (+2), Progressive Streaming (-1), MPEG-CMAF (+2), HDS (-4).

Current Streaming Formats and Plans to Implement in next 12 Months.

In general, one can observe that the adoption of new formats are happening at a slower pace than expected and I am wondering what this means for the new video coding formats coming up like VVC et al. (note: these are results from a public survey with different participants compared to last years which need to be taken into account when comparing results over years).

For more details, the full report can be downloaded for free from here.

Thursday, September 5, 2019

ACMMM'19: Docker-Based Evaluation Framework for Video Streaming QoE in Broadband Networks

Docker-Based Evaluation Framework for Video Streaming QoE in Broadband Networks
(Demo Paper)

[PDF] (coming soon; slides to be provided later)

Cise Midoglu (Simula), Anatoliy Zabrovskiy (AAU), Özgü Alay (Simula), Daniel Hölbling-Inzko (Bitmovin), Carsten Griwodz (Univ. of Oslo), Christian Timmerer (AAU/Bitmovin)

Abstract: Video streaming is one of the top traffic contributors in the Internet and a frequent research subject. It is expected that streaming traffic will grow 4-fold for video globally and 9-fold for mobile video between 2017 and 2022. In this paper, we present an automatized measurement framework for evaluating video streaming QoE in operational broadband networks, using headless streaming with a Docker-based client, and a server-side implementation allowing for the use of multiple video players and adaptation algorithms. Our framework allows for integration with the MONROE testbed and Bitmovin Analytics, which bring on the possibility to conduct large-scale measurements in different networks, including mobility scenarios, and monitor different parameters in the application, transport, network, and physical layers in real-time.

Keywords: adaptive streaming, network measurements, OTT video analytics, QoE

Wednesday, August 21, 2019

MPEG news: a report from the 127th meeting, Gothenburg, Sweden

The original blog post can be found at the Bitmovin Techblog and has been modified/updated here to focus on and highlight research aspects. Additionally, this version of the blog post will be also posted at ACM SIGMM Records.

MPEG News Archive

Plenary of the 127th MPEG Meeting in Gothenburg, Sweden.
The 126th MPEG meeting concluded on March 29, 2019 in Geneva, Switzerland with the following topics:
  • Versatile Video Coding (VVC) enters formal approval stage, experts predict 35-60% improvement over HEVC
  • Essential Video Coding (EVC) promoted to Committee Draft
  • Common Media Application Format (CMAF) 2nd edition promoted to Final Draft International Standard
  • Dynamic Adaptive Streaming over HTTP (DASH) 4th edition promoted to Final Draft International Standard
  • Carriage of Point Cloud Data Progresses to Committee Draft
  • JPEG XS carriage in MPEG-2 TS promoted to Final Draft Amendment of ISO/IEC 13818-1 7th edition
  • Genomic information representation – WG11 issues a joint call for proposals on genomic annotations in conjunction with ISO TC 276/WG 5
  • ISO/IEC 23005 (MPEG-V) 4th Edition – WG11 promotes the Fourth edition of two parts of “Media Context and Control” to the Final Draft International Standard (FDIS) stage

The corresponding press release of the 127th MPEG meeting can be found here:

Versatile Video Coding (VVC)

The Moving Picture Experts Group (MPEG) is pleased to announce that Versatile Video Coding (VVC) progresses to Committee Draft, experts predict 35-60% improvement over HEVC.

The development of the next major generation of video coding standard has achieved excellent progress, such that MPEG has approved the Committee Draft (CD, i.e., the text for formal balloting in the ISO/IEC approval process).

The new VVC standard will be applicable to a very broad range of applications and it will also provide additional functionalities. VVC will provide a substantial improvement in coding efficiency relative to existing standards. The improvement in coding efficiency is expected to be quite substantial – e.g., in the range of 35–60% bit rate reduction relative to HEVC although it has not yet been formally measured. Relative to HEVC means for equivalent subjective video quality at picture resolutions such as 1080p HD or 4K or 8K UHD, either for standard dynamic range video or high dynamic range and wide color gamut content for levels of quality appropriate for use in consumer distribution services. The focus during the development of the standard has primarily been on 10-bit 4:2:0 content, and 4:4:4 chroma format will also be supported.

The VVC standard is being developed in the Joint Video Experts Team (JVET), a group established jointly by MPEG and the Video Coding Experts Group (VCEG) of ITU-T Study Group 16. In addition to a text specification, the project also includes the development of reference software, a conformance testing suite, and a new standard ISO/IEC 23002-7 specifying supplemental enhancement information messages for coded video bitstreams. The approval process for ISO/IEC 23002-7 has also begun, with the issuance of a CD consideration ballot.

Research aspects: VVC represents the next generation video codec to be deployed in 2020+ and basically the same research aspects apply as for previous generations, i.e., coding efficiency, performance/complexity, and objective/subjective evaluation. Luckily, JVET documents are freely available including the actual standard (committee draft), software (and its description), and common test conditions. Thus, researcher utilizing these resources are able to conduct reproducible research when contributing their findings and code improvements back to the community at large. 

Essential Video Coding (EVC)

MPEG-5 Essential Video Coding (EVC) promoted to Committee Draft

Interestingly, at the same meeting as VVC, MPEG promoted MPEG-5 Essential Video Coding (EVC) to Committee Draft (CD). The goal of MPEG-5 EVC is to provide a standardized video coding solution to address business needs in some use cases, such as video streaming, where existing ISO video coding standards have not been as widely adopted as might be expected from their purely technical characteristics.

The MPEG-5 EVC standards includes a baseline profile that contains only technologies that are over 20 years old or are otherwise expected to be royalty-free. Additionally, a main profile adds a small number of additional tools, each providing significant performance gain. All main profile tools are capable of being individually switched off or individually switched over to a corresponding baseline tool. Organizations making proposals for the main profile have agreed to publish applicable licensing terms within two years of FDIS stage, either individually or as part of a patent pool.

Research aspects: Similar research aspects can be described for EVC and from a software engineering perspective it could be also interesting to further investigate this switching mechanism of individual tools or/and fall back option to baseline tools. Naturally, a comparison with next generation codecs such as VVC is interesting per se. The licensing aspects itself are probably interesting for other disciplines but that is another story...

Common Media Application Format (CMAF)

MPEG ratified the 2nd edition of the Common Media Application Format (CMAF)

The Common Media Application Format (CMAF) enables efficient encoding, storage, and delivery of digital media content (incl. audio, video, subtitles among others), which is key to scaling operations to support the rapid growth of video streaming over the internet. The CMAF standard is the result of widespread industry adoption of an application of MPEG technologies for adaptive video streaming over the Internet, and widespread industry participation in the MPEG process to standardize best practices within CMAF.

The 2nd edition of CMAF adds support for a number of specifications that were a result of significant industry interest. Those include
  • Advanced Audio Coding (AAC) multi-channel;
  • MPEG-H 3D Audio;
  • MPEG-D Unified Speech and Audio Coding (USAC);
  • Scalable High Efficiency Video Coding (SHVC);
  • IMSC 1.1 (Timed Text Markup Language Profiles for Internet Media Subtitles and Captions); and
  • additional HEVC video CMAF profiles and brands.
This edition also introduces CMAF supplemental data handling as well as new structural brands for CMAF that reflects the common practice of the significant deployment of CMAF in industry. Companies adopting CMAF technology will find the specifications introduced in the 2nd Edition particularly useful for further adoption and proliferation of CMAF in the market.

Research aspects: see below (DASH).

Dynamic Adaptive Streaming over HTTP (DASH)

MPEG approves the 4th edition of Dynamic Adaptive Streaming over HTTP (DASH)

The 4th edition of MPEG-DASH comprises the following features:
service description that is intended by the service provider on how the service is expected to be consumed;
  • a method to indicate the times corresponding to the production of associated media;
  • a mechanism to signal DASH profiles and features, employed codec and format profiles; and
  • supported protection schemes present in the Media Presentation Description (MPD).
It is expected that this edition will be published later this year. 

Research aspects: CMAF 2nd and DASH 4th edition come along with a rich feature set enabling a plethora of use cases. The underlying principles are still the same and research issues arise from updated application and service requirements with respect to content complexity, time aspects (mainly delay/latency), and quality of experience (QoE). The DASH-IF awards the excellence in DASH award at the ACM Multimedia Systems conference and an overview about its academic efforts can be found here. For example, see here our recent research on bandwidth prediction in low-latency chunked streaming. Additionally, our tutorial at ACM Multimedia 2019 about a journey towards fully immersive media access reviews state of the art in this area and how it could be extended enabling 6DoF HAS services through point cloud compression.

Carriage of Point Cloud Data

MPEG progresses the Carriage of Point Cloud Data to Committee Draft

At its 127th meeting, MPEG has promoted the carriage of point cloud data to the Committee Draft stage, the first milestone of ISO standard development process. This standard is the first one introducing the support of volumetric media in the industry-famous ISO base media file format family of standards.

This standard supports the carriage of point cloud data comprising individually encoded video bitstreams within multiple file format tracks in order to support the intrinsic nature of the video-based point cloud compression (V-PCC). Additionally, it also allows the carriage of point cloud data in one file format track for applications requiring multiplexed content (i.e., the video bitstream of multiple components is interleaved into one bitstream).

This standard is expected to support efficient access and delivery of some portions of a point cloud object considering that in many cases that entire point cloud object may not be visible by the user depending on the viewing direction or location of the point cloud object relative to other objects. It is currently expected that the standard will reach its final milestone by the end of 2020.

Research aspects: MPEG's Point Cloud Compression (PCC) comes in two flavors, video- and geometric-based but still requires to be packaged into file and delivery formats. MPEG's choice here is the ISO base media file format and the efficient carriage of point cloud data is characterized by both functionality (i.e., enabling the required used cases) and performance (such as low overhead).

MPEG 2 Systems/Transport Stream

JPEG XS carriage in MPEG-2 TS promoted to Final Draft Amendment of ISO/IEC 13818-1 7th edition

At its 127th meeting, WG11 (MPEG) has extended ISO/IEC 13818-1 (MPEG-2 Systems) – in collaboration with WG1 (JPEG) – to support ISO/IEC 21122 (JPEG XS) in order to support industries using still image compression technologies for broadcasting infrastructures. The specification defines a JPEG XS elementary stream header and specifies how the JPEG XS video access unit (specified in ISO/IEC 21122-1) is put into a Packetized Elementary Stream (PES). Additionally, the specification also defines how the System Target Decoder (STD) model can be extended to support JPEG XS video elementary streams.

Genomic information representation

WG11 issues a joint call for proposals on genomic annotations in conjunction with ISO TC 276/WG 5

The introduction of high-throughput DNA sequencing has led to the generation of large quantities of genomic sequencing data that have to be stored, transferred and analyzed. So far WG 11 (MPEG) and ISO TC 276/WG 5 have addressed the representation, compression and transport of genome sequencing data by developing the ISO/IEC 23092 standard series also known as MPEG-G. They provide a file and transport format, compression technology, metadata specifications, protection support, and standard APIs for the access of sequencing data in the native compressed format.

An important element in the effective usage of sequencing data is the association of the data with the results of the analysis and annotations that are generated by processing pipelines and analysts. At the moment such association happens as a separate step, standard and effective ways of linking data and meta information derived from sequencing data are not available.

At its 127th meeting, MPEG and ISO TC 276/WG 5 issued a joint Call for Proposals (CfP) addressing the solution of such problem. The call seeks submissions of technologies that can provide efficient representation and compression solutions for the processing of genomic annotation data.

Companies and organizations are invited to submit proposals in response to this call. Responses are expected to be submitted by the 8th January 2020 and will be evaluated during the 129th WG 11 (MPEG) meeting. Detailed information, including how to respond to the call for proposals, the requirements that have to be considered, and the test data to be used, is reported in the documents N18648, N18647, and N18649 available at the 127th meeting website ( For any further question about the call, test conditions, required software or test sequences please contact: Joern Ostermann, MPEG Requirements Group Chair ( or Martin Golebiewski, Convenor ISO TC 276/WG 5 (

ISO/IEC 23005 (MPEG-V) 4th Edition

WG11 promotes the Fourth edition of two parts of “Media Context and Control” to the Final Draft International Standard (FDIS) stage

At its 127th meeting, WG11 (MPEG) promoted the 4th edition of two parts of ISO/IEC 23005 (MPEG-V; Media Context and Control) standards to the Final Draft International Standard (FDIS). The new edition of ISO/IEC 23005-1 (architecture) enables ten new use cases, which can be grouped into four categories: 3D printing, olfactory information in virtual worlds, virtual panoramic vision in car, and adaptive sound handling. The new edition of ISO/IEC 23005-7 (conformance and reference software) is updated to reflect the changes made by the introduction of new tools defined in other parts of ISO/IEC 23005. More information on MPEG-V and its parts 1-7 can be found at

Finally, the unofficial highlight of the 127th MPEG meeting we certainly found while scanning the scene in Gothenburg on Tuesday night...

Monday, August 12, 2019

ACMMM'19: Towards 6DoF HTTP Adaptive Streaming Through Point Cloud Compression

Towards 6DoF HTTP Adaptive Streaming Through Point Cloud Compression

[PDF] (coming soon; slides to be provided later)

Jeroen van der Hooft, Tim Wauters, Filip De Turck (Ghent University - imec), Christian Timmerer, and Hermann Hellwagner (Alpen-Adria-Universität Klagenfurt)

Abstract: The increasing popularity of head-mounted devices and 360° video cameras allows content providers to offer virtual reality video streaming over the Internet, using a relevant representation of the immersive content combined with traditional streaming techniques. While this approach allows the user to freely move her head, her location is fixed by the camera’s position within the scene. Recently, an increased interest has been shown for free movement within immersive scenes, referred to as six degrees of freedom. One way to realize this is by capturing objects through a number of cameras positioned in different angles, and creating a point cloud which consists of the location and RGB color of a significant number of points in the three-dimensional space. Although the concept of point clouds has been around for over two decades, it recently received increased attention by ISO/IEC MPEG, issuing a call for proposals for point cloud compression. As a result, dynamic point cloud objects can now be compressed to bit rates in the order of 3 to 55 Mb/s, allowing feasible delivery over today’s mobile networks. In this paper, we propose PCC-DASH, a standards-compliant means for HTTP adaptive streaming of scenes comprising multiple, dynamic point cloud objects. We present a number of rate adaptation heuristics which use information on the user’s position and focus, the available bandwidth, and the client’s buffer status to decide upon the most appropriate quality representation of each object. Through an extensive evaluation, we discuss the advantages and drawbacks of each solution. We argue that the optimal solution depends on the considered scene and camera path, which opens interesting possibilities for future work.

Keywords: HTTP adaptive streaming, MPEG-DASH, immersive video, point clouds, MPEG V-PCC, rate adaptation

Tuesday, July 30, 2019

ACM MMSys 2020 Research Track - Call for Papers

ACM MMSys 2020 Research Track - Call for Papers
June 8-11, 2020, Istanbul, Turkey


The ACM Multimedia Systems Conference (MMSys) provides a forum for researchers to present and share their latest research findings in multimedia systems. While research about specific aspects of multimedia systems are regularly published in the various proceedings and transactions of the networking, operating systems, real-time systems, databases, mobile computing, distributed systems, computer vision, and middleware communities, MMSys aims to cut across these domains in the context of multimedia data types. This provides a unique opportunity to investigate the intersections and the interplay of the various approaches and solutions developed across these domains to deal with multimedia data types.

MMSys is a venue for researchers who explore:
  • Complete multimedia systems that provide a new kind of multimedia experience or system whose overall performance improves the state-of-the-art through new research results in more than one component, or
  • Enhancements to one or more system components that provide a documented improvement over the state-of-the-art for handling continuous media or time-dependent services.
Such individual system components include:
  • Operating systems
  • Distributed architectures and protocols
  • Domain languages, development tools and abstraction layers
  • Using new architectures or computing resources for multimedia
  • New or improved I/O architectures or I/O devices, innovative uses, and algorithms for their operation
  • Representation of continuous or time-dependent media
  • Metrics and measurement tools to assess performance
This touches aspects of many hot topics including but not limited to: content preparation and (adaptive) delivery systems, High Dynamic Range (HDR), games, virtual/augmented/mixed reality, 3D video, immersive systems, plenoptics, 360-degree video, volumetric video delivery, multimedia Internet of Things (IoT), multi and many-core, GPGPUs, mobile multimedia and 5G, wearable multimedia, peer-to-peer (P2P), cloud-based multimedia, cyber-physical systems, multi-sensory experiences, smart cities, Quality of Experience (QoE).

We encourage submissions in the following focus areas
  • Machine learning and statistical modeling for video streaming
  • Volumetric media: from capture to consumption
  • Fake media and tools for preventing illegal broadcasts
Refer to the Web site for more info.

Important Dates
  • Submission deadline: January 10, 2020 (firm deadline)
  • Acceptance notification: March 16, 2020
  • Camera-ready deadline: April 17, 2020
  • Online submission:
  • Submission format: 6-12 pages, using ACM style format (double-blind)
  • Reproducibility: Obtain an ACM reproducibility badge by making datasets and code available (Authors will be contacted to make their artifacts available after paper acceptance)
General Chairs
TPC Chairs
Submission Information
  • Papers should be between 6-12 pages long (in PDF format) prepared in the ACM style and written in English. MMSys papers enable authors to present entire multimedia systems or research work that builds on considerable amounts of earlier work in a self-contained manner. MMSys papers are published in the ACM Digital Library. The papers are double-blind reviewed.
  • All submissions will be peer-reviewed by at least three TPC members. All papers will be evaluated for their scientific quality. Authors will have a chance to submit their rebuttals before online discussions among the TPC members.
ACM SIGMM has a tradition of publishing open datasets (MMSys) and open source projects (ACM Multimedia). MMSys 2020 will continue to support scientific reproducibility, by implementing the ACM reproducibility badge system. All accepted papers will be contacted by the Reproducibility Chair, inviting the authors to make their dataset and code available, and thus, obtaining an ACM badge (visible at the ACM DL). The additional material will be published as Appendixes, with no effect on the final page count for papers.