Tuesday, March 12, 2019

QoMEX'19: Tile-based Streaming of 8K Omnidirectional Video: Subjective and Objective QoE Evaluation

Tile-based Streaming of 8K Omnidirectional Video: Subjective and Objective QoE Evaluation

[PDF/Slides will be posted after the event]

Raimund Schatz (AIT Austrian Institute of Technology), Anatoliy Zabrovskiy (Alpen-Adria Universität Klagenfurt), Christian Timmerer (Alpen-Adria Universität Klagenfurt, Bitmovin Inc.)

Abstract: Omnidirectional video (ODV) streaming applications are becoming increasingly popular. They enable a highly immersive experience as the user can freely choose her/his field of view within the 360-degree environment. Current deployments are fairly simple but viewport-agnostic which inevitably results in high storage/bandwidth requirements and low Quality of Experience (QoE). A promising solution is referred to as tile- based streaming which allows to have higher quality within the user’s viewport while quality outside the user’s viewport could be lower. However, empirical QoE assessment studies in this domain are still rare. Thus, this paper investigates the impact of different tile-based streaming approaches and configurations on the QoE of ODV. We present the results of a lab-based subjective evaluation in which participants evaluated 8K omnidirectional video QoE as influenced by different (i) tile-based streaming approaches (full vs. partial delivery), (ii) content types (static vs. moving camera), and (iii) tile encoding quality levels determined by different quantization parameters. Our experimental setup is characterized by high reproducibility since relevant media delivery aspects (including the user’s head movements and dynamic tile quality adaptation) are already rendered into the respective processed video sequences. Additionally, we performed a complementary objective evaluation of the different test sequences focusing on bandwidth efficiency and objective quality metrics. The results are presented in this paper and discussed in detail which confirm that tile-based streaming of ODV improves visual quality while reducing bandwidth requirements.

Index Terms: Omnidirectional Video, Tile-based Streaming, Subjective Testing, Objective Metrics, Quality of Experience

Acknowledgment: This work was supported in part by the Austrian Research Promotion Agency (FFG) under the Next Generation Video Streaming project "PROMETHEUS".

Thursday, February 21, 2019

Mobile data traffic report and forecast 2017-2022

On Feb 18, 2019 both Sandvine and Cisco released their mobile data traffic report and forecast 2018-2022 respectively.

Starting with the Sandvine 2019 Mobile Internet Phenomena Report which features mobile data of global (except significant portions of China and India) traffic share for applications with respect to downstream, upstream, and connections.

The main message is "YouTube is the global leader with over 35% of worldwide mobile traffic, dwarfing Netflix’s 15% share in the Global Report." Looking at the global application traffic share for downstream we have YouTube (37.04%), Facebook video (2.53%) and Netflix (2.44%); in total around 42% is video (compared to almost 58% in the global report from October 2018). The top applications are shown in the figure below.
Source: Sandvine 2019 Mobile Internet Phenomena Report, Feb 18, 2019.
This report also highlights QoE and packet loss which basically focuses on throughput, latency, and packet loss that then leads to a so-called ScoreCard as shown below. However, streaming video here is not considered as a delay-sensitive application which might be true for video on demand but could look different for live service, specifically with respect to the start-up delay and delay compared to traditional TV services.
Source: Sandvine 2019 Mobile Internet Phenomena Report, Feb 18, 2019.
Cisco's Visual Networking Index: Global Mobile Data Traffic Forecast Update, 2017–2022 measured that mobile video traffic accounted for 59% of total mobile data traffic in 2017, which means more than half of all mobile data traffic, similarly to almost 58% in the Sandvine global report from October 2018. However, please note the 42% in the Sandvine mobile report although Sandvine didn't have a specific number for video only.

Interestingly, Cisco predicts that nearly 79% of the world’s mobile data traffic will be video by 2022, it will increase 9-fold between 2017 and 2022. Furthermore, mobile video will grow at a CAGR of 55% between 2017 and 2022 which is higher than the overall average mobile traffic CAGR of 46%. Of the 77 exabytes per month crossing the mobile network by 2022, nearly 61 exabytes will be due to video (see figure below).
Source: Cisco VNI Mobile, Feb 18, 2019.
In any case, these reports confirm that video is already responsible for the majority of data traffic worldwide for both mobile and fixed-network access; and it will continue to grow...

Monday, February 18, 2019

MPEG news: a report from the 125th meeting, Marrakesh, Morocco

The original blog post can be found at the Bitmovin Techblog and has been modified/updated here to focus on and highlight research aspects. Additionally, this version of the blog post will be also posted at ACM SIGMM Records.


The 125th MPEG meeting concluded on January 18, 2019 in Marrakesh, Morocco with the following topics:
  • Network-Based Media Processing (NBMP) – MPEG promotes NBMP to Committee Draft stage
  • 3DoF+ Visual – MPEG issues Call for Proposals on Immersive 3DoF+ Video Coding Technology
  • MPEG-5 Essential Video Coding (EVC) – MPEG starts work on MPEG-5 Essential Video Coding
  • ISOBMFF – MPEG issues Final Draft International Standard of Conformance and Reference software for formats based on the ISO Base Media File Format (ISOBMFF)
  • MPEG-21 User Description – MPEG finalizes 2nd edition of the MPEG-21 User Description
The corresponding press release of the 125th MPEG meeting can be found here. In this blog post I’d like to focus on those topics potentially relevant for over-the-top (OTT), namely NBMP, EVC, and ISOBMFF.

Network-Based Media Processing (NBMP)

The NBMP standard addresses the increasing complexity and sophistication of media services, specifically as the incurred media processing requires offloading complex media processing operations to the cloud/network to keep receiver hardware simple and power consumption low. Therefore, NBMP standard provides a standardized framework that allows content and service providers to describe, deploy, and control media processing for their content in the cloud. It comes with two main functions: (i) an abstraction layer to be deployed on top of existing cloud platforms (+ support for 5G core and edge computing) and (ii) a workflow manager to enable composition of multiple media processing tasks (i.e., process incoming media and metadata from a media source and produce processed media streams and metadata that are ready for distribution to a media sink). The NBMP standard now reached Committee Draft (CD) stage and final milestone is targeted for early 2020.

In particular, a standard like NBMP might become handy in the context of 5G in combination with mobile edge computing (MEC) which allows offloading certain tasks to a cloud environment in close proximity to the end user. For OTT, this could enable lower latency and more content being personalized towards the user’s context conditions and needs, hopefully leading to a better quality and user experience.

For further research aspects please see one of my previous posts

MPEG-5 Essential Video Coding (EVC)

MPEG-5 EVC clearly targets the high demand for efficient and cost-effective video coding technologies. Therefore, MPEG commenced work on such a new video coding standard that should have two profiles: (i) royalty-free baseline profile and (ii) main profile, which adds a small number of additional tools, each of which is capable, on an individual basis, of being either cleanly switched off or else switched over to the corresponding baseline tool. Timely publication of licensing terms (if any) is obviously very important for the success of such a standard.

The target coding efficiency for responses to the call for proposals was to be at least as efficient as HEVC. This target was exceeded by approximately 24% and the development of the MPEG-5 EVC standard is expected to be completed in 2020.

As of today, there’s the need to support AVC, HEVC, VP9, and AV1; soon VVC will become important. In other words, we already have a multi-codec environment to support and one might argue one more codec is probably not a big issue. The main benefit of EVC will be a royalty-free baseline profile but with AV1 there’s already such a codec available and it will be interesting to see how the royalty-free baseline profile of EVC compares to AV1.

For a new video coding format we will witness a plethora of evaluations and comparisons with existing formats (i.e., AVC, HEVC, VP9, AV1, VVC). These evaluations will be mainly based on objective metrics such as PSNR, SSIM, and VMAF. It will be also interesting to see subjective evaluations, specifically targeting OTT use cases (e.g., live and on demand).

ISO Base Media File Format (ISOBMFF)

The ISOBMFF (ISO/IEC 14496-12) is used as basis for many file (e.g., MP4) and streaming formats (e.g., DASH, CMAF) and as such received widespread adoption in both industry and academia. An overview of ISOBMFF is available here. The reference software is now available on GitHub and a plethora of conformance files are available here. In this context, the open source project GPAC is probably the most interesting aspect from a research point of view.

Thursday, January 31, 2019

Post-Doctoral Research Positions in "Adaptive Streaming over HTTP and Emerging Networked Multimedia Services"

The Institute of Information Technology at the Alpen-Adria-Universität Klagenfurt invites applications for:

Post-Doctoral Research Positions (100% employment) 
within the Christian Doppler (CD) Pilot Laboratory ATHENA 
Adaptive Streaming over HTTP and Emerging Networked Multimedia Services” 

at the Faculty for Technical Sciences. The monthly salary for these positions is according to the standard salaries of the Austrian collective agreement, min. € 3,803.90 pre-tax (14x yearly) (Uni-KV: B1 lit.b, http://www.aau.at/en/uni-kv). Expected start date of employment is May 1st 2019.

ATHENA stands for Adaptive Streaming over HTTP and Emerging Networked Multimedia Services and has been jointly proposed as a CD Laboratory (https://www.cdg.ac.at/) by the Institute of Information Technology (ITEC; http://itec.aau.at) at Alpen-Adria-Universität Klagenfurt (AAU) and Bitmovin GmbH (https://bitmovin.com) to address current and future research and deployment challenges of HTTP Adaptive Streaming (HAS) and emerging streaming methods.

AAU (ITEC) has been working on adaptive video streaming for more than a decade, has a proven record of successful research projects and publications in the field, and has been actively contributing to MPEG standardization for many years, including MPEG-DASH; Bitmovin is a video streaming software company founded by ITEC researchers in 2013 and has developed highly successful, global R&D and sales activities and a world-wide customer base since then.

The aim of ATHENA CD Lab is to research and develop novel paradigms, approaches, (prototype) tools and evaluation results for the areas (1) multimedia content provisioning (i.e., video coding), (2) content delivery (i.e., multimedia networking) and (3) content consumption (i.e., HAS player aspects) in the media delivery chain as well as for (4) end-to-end aspects, with a focus on, but not being limited to, HTTP Adaptive Streaming (HAS). The new approaches and insights are to enable Bitmovin to build innovative applications and services to account for the steadily increasing and changing multimedia traffic on the Internet. In addition, according to the CD Lab model of “application-oriented basic research”, the goal is to publish the results in international, high quality professional journals and conference proceedings.

Your profile:
  • Doctoral degree of Technical Science in the field of Computer Science or Electrical Engineering, completed at a domestic or foreign university (with good final degrees);
  • Interest and experience in at least on of the following areas: (1) multimedia content provisioning (i.e., video coding) and (2) end-to-end aspects, with a focus on, but not being limited to, HTTP Adaptive Streaming (HAS);
  • Excellent English skills, both in written and oral form. 
Desirable qualifications include:
  • Excellent programming skills, especially C, C++, Java, JavaScript;
  • Relevant international and practical work experience;
  • Social and communicative competences and ability to work in a team;
  • Experience with university teaching and research activities.
The working language and the research program are in English. There is no need to learn German for this position unless the applicant wants to participate in undergraduate teaching which is optional.

Submit all relevant documents, including copies of all school certificates and performance records, by email to:

Dr. Christian Timmerer
Institute of Information Technology, Alpen-Adria-Universität Klagenfurt
Universitätsstraße 65 – 67, 9020 Klagenfurt, Austria
Email: christian(dot)timmerer(at)itec(dot)aau(dot)at
URL: http://blog.timmerer.comhttp://itec.aau.at/

Klagenfurt, situated at the beautiful Lake Wörthersee – one of the largest and warmest alpine lakes in Europe – has nearly 100.000 inhabitants. Being a small city, with a Renaissance-style city center reflecting 800 years of history and with Italian influence, Klagenfurt is a pleasant place to live and work. The university is located only about 1.5 kilometers east of Lake Wörthersee and about 3 kilometers west of the city enter.


Wednesday, January 30, 2019

Doctoral Student Positions in "Adaptive Streaming over HTTP and Emerging Networked Multimedia Services"

The Institute of Information Technology at the Alpen-Adria-Universität Klagenfurt invites applications for:

Doctoral Student Positions (100% employment) 
within the Christian Doppler (CD) Pilot Laboratory ATHENA 
Adaptive Streaming over HTTP and Emerging Networked Multimedia Services” 

at the Faculty for Technical Sciences. The monthly salary for these positions is according to the standard salaries of the Austrian collective agreement, min. € 2,864.50 pre-tax (14x yearly) (Uni-KV: B1, http://www.aau.at/en/uni-kv). Expected start date of employment is May 1st 2019.

ATHENA stands for Adaptive Streaming over HTTP and Emerging Networked Multimedia Services and has been jointly proposed as a CD Laboratory (https://www.cdg.ac.at/) by the Institute of Information Technology (ITEC; http://itec.aau.at) at Alpen-Adria-Universität Klagenfurt (AAU) and Bitmovin GmbH (https://bitmovin.com) to address current and future research and deployment challenges of HTTP Adaptive Streaming (HAS) and emerging streaming methods.

AAU (ITEC) has been working on adaptive video streaming for more than a decade, has a proven record of successful research projects and publications in the field, and has been actively contributing to MPEG standardization for many years, including MPEG-DASH; Bitmovin is a video streaming software company founded by ITEC researchers in 2013 and has developed highly successful, global R&D and sales activities and a world-wide customer base since then.

The aim of ATHENA CD Lab is to research and develop novel paradigms, approaches, (prototype) tools and evaluation results for the areas (1) multimedia content provisioning (i.e., video coding), (2) content delivery (i.e., multimedia networking) and (3) content consumption (i.e., HAS player aspects) in the media delivery chain as well as for (4) end-to-end aspects, with a focus on, but not being limited to, HTTP Adaptive Streaming (HAS). The new approaches and insights are to enable Bitmovin to build innovative applications and services to account for the steadily increasing and changing multimedia traffic on the Internet. In addition, according to the CD Lab model of “application-oriented basic research”, the goal is to publish the results in international, high quality professional journals and conference proceedings.

Your profile:
  • Master or diploma degree of Technical Science in the field of Computer Science or Electrical Engineering, completed at a domestic or foreign university (with good final degrees);
  • Interest and experience in one or more of the above identified areas, namely (1) multimedia content provisioning (i.e., video coding), (2) content delivery (i.e., multimedia networking) and (3) content consumption (i.e., HAS player aspects) in the media delivery chain as well as (4) end-to-end aspects, with a focus on, but not being limited to, HTTP Adaptive Streaming (HAS);
  • Excellent English skills, both in written and oral form. 
Desirable qualifications include:
  • Excellent programming skills, especially C, C++, Java, JavaScript;
  • Relevant international and practical work experience;
  • Social and communicative competences and ability to work in a team;
  • Experience with university teaching and research activities.
The working language and the research program are in English. There is no need to learn German for this position unless the applicant wants to participate in undergraduate teaching which is optional.

Submit all relevant documents, including copies of all school certificates and performance records, by email to:

Dr. Christian Timmerer
Institute of Information Technology, Alpen-Adria-Universität Klagenfurt
Universitätsstraße 65 – 67, 9020 Klagenfurt, Austria
Email: christian(dot)timmerer(at)itec(dot)aau(dot)at
URL: http://blog.timmerer.com, http://itec.aau.at/

Klagenfurt, situated at the beautiful Lake Wörthersee – one of the largest and warmest alpine lakes in Europe – has nearly 100.000 inhabitants. Being a small city, with a Renaissance-style city center reflecting 800 years of history and with Italian influence, Klagenfurt is a pleasant place to live and work. The university is located only about 1.5 kilometers east of Lake Wörthersee and about 3 kilometers west of the city enter.

Sunday, December 23, 2018

What happened in multimedia communication in 2018?

In January 2018 I wrote a blog post entitled "What to care about in multimedia communication in 2018?" and I think it's worth looking back to see what actually happened with respect to next generation video coding formats and adaptive streaming techniques.

In April 2018, the responses to the call for proposals for the next standard in video compression have been evaluated and a first working draft and test model for the Versatile Video Coding (VVC) standard have been approved. At this point already, some proposals demonstrated compression efficiency gains of typically 40% or more when compared to using HEVC. Currently, working draft 3 and test model 3 of VVC (VTM 3) are available and we may certainly expect compression efficiency gains well-beyond the targeted 50% for the final standard. An overview about VVC can be found here (by C. Feldmann) and here (by M. Wien). The licensing issues have been acknowledged and, thus, the Media Coding Industry Forum (MC-IF) has been established.

At the beginning of 2018, everyone was also very curious about AOMedia and AV1. Version 1 of the specification has been finally become available and in the meantime it is implemented/deployed on both content provisioning/encoding (e.g. Bitmovin) and content consumption/decoding (e.g., Chrome, Firefox). In this context, we also published a multi-codec DASH dataset comprising AVC, HEVC, VP9, and AV1 (VVC will be added at a later stage). In general, however, we are entering the era of multiple video codecs deployed in products and services whereby this trend is also confirmed by Bitmovin's latest video developer survey.

MPEG-DASH 3rd edition has been approved and is awaiting publication but I expect this to happen in 2019 though. An overview of the MPEG-DASH status is shown in the figure below.
In this context, the DASH-IF produced various vital assets such as interoperability guidelines (latest v4.3, content protection, ATSC 3.0, SAND), test vectors, conformance tools, and a reference client. For informative aspects of MPEG-DASH such as the bitrate adaptation schemes the interested reader is referred to our survey. This survey gives an overview about existing techniques (see figure below) and also outlines future research. It is available for free for everyone (open access).


Finally, I mentioned a couple of scientific events in 2018 including QoMEX, MMSys (NOSSDAV, PV), ICME, ICIP, PCS, and MIPR. I have attended all of the them (except PCS), each showing advances in their respective field. These events are probably worth to attend also in 2019 but I will certainly blog about this early next year. However, I'd like to hear your opinion of what happened in 2018 and what we may expect in 2019...

Friday, December 14, 2018

Christian Doppler Research Association approves ATHENA project proposal

ATHENA stands for Adaptive Streaming over HTTP and Emerging Networked Multimedia Services and has been jointly proposed by the Institute of Information Technology (ITEC; http://itec.aau.at) at Alpen-Adria-Universität Klagenfurt (AAU) and Bitmovin GmbH (https://bitmovin.com) to address current and future research and deployment challenges of HTTP adaptive steaming (HAS) and emerging streaming methods.

AAU (ITEC) has been working on adaptive video streaming for more than a decade, has a proven record of successful research projects and publications in the field, and has been actively contributing to MPEG standardization for many years, including MPEG-DASH; Bitmovin is a video streaming software company founded by ITEC researchers in 2013 and has developed highly successful, global R&D and sales activities and a world-wide customer base since then. 

The aim of ATHENA is to research and develop novel paradigms, approaches, (prototype) tools and evaluation results for the areas (1) multimedia content provisioning (i.e., video coding), (2) content delivery (i.e., multimedia networking) and (3) content consumption (i.e., HAS player aspects) in the media delivery chain as well as for (4) end-to-end aspects, with a focus on, but not being limited to, HTTP Adaptive Streaming (HAS). The new approaches and insights are to enable Bitmovin to build innovative applications and services to account for the steadily increasing and changing multimedia traffic on the Internet.

The project has been approved by the Christian Doppler Research Association as a CD pilot laboratory -- the first such kind of project at Alpen-Adria-Universität Klagenfurt -- with a duration of two years including a five year extension after successful review after the first two years (i.e., seven years in total). Thus, stay tuned for details and yes, I'm hiring PhD students for the areas above (detailed job description will be published soon).