One-Year-ATHENA: Happy Birthday

One year ago in October 2019 we started the Christian Doppler Labor ATHENA at the University of Klagenfurt which is funded by the Christian Doppler Research Association and Bitmovin. The aim of ATHENA is to research and develop novel paradigms, approaches, (prototype) tools, and evaluation results for the phases

• multimedia content provisioning,
• content delivery, and
• content consumption in the media delivery chain as well as for
• end-to-end aspects, with a focus on, but not being limited to, HTTP Adaptive Streaming (HAS).

https://athena.itec.aau.at/

Prior to starting ATHENA, we were confronted with hiring a team from scratch (i.e., 2 post-docs and 6 PhD students), which kept us very busy but eventually, we found the best team for ATHENA. Within a few months only, we managed to have the first ATHENA publication accepted at an international conference and we prepared for the first conference travels. Unfortunately, during this period COVID-19 reached us and we (all) experienced a major disruption in our daily life. I am very proud of my entire team about how we dealt with this unpleasant situation.

In this first year of ATHENA, we got 12* papers accepted for publication at international conferences/journals, which makes one publication per month on average; a truely remarkable result for a one year old project (out of seven years in total). Finally, I would like to thank our (inter)national collaborators for joining one or the other effort within ATHENA.

The list of ATHENA publications after this first year is as follows:

1. “From Capturing to Rendering: Volumetric Media Delivery With Six Degrees of Freedom“, IEEE Communication Magazine.
2. “Fast Multirate Encoding for HTTP Adaptive Streaming Using Machine Learning“, IEEE International Conference on Visual Communications and Image Processing (VCIP) 2020.
3. “Relevance-Based Compression of Cataract Surgery Videos Using Convolutional Neural Networks,” ACM International Conference on Multimedia 2020.
4. “QUALINET White Paper on Definitions of Immersive Media Experience (IMEx)”, European Network on Quality of Experience in Multimedia Systems and Services.
5. “Scalable High Efficiency Video Coding based HTTP Adaptive Streaming over QUIC Using Retransmission“, ACM SIGCOMM 2020 Workshop on Evolution, Performance, and Interoperability of QUIC (EPIQ 2020).
6. “Towards View-Aware Adaptive Streaming of Holographic Content,” 2020 IEEE International Conference on Multimedia & Expo Workshops (ICMEW).
7. “H2BR: An HTTP/2-based Retransmission Technique to Improve the QoE of Adaptive Video Streaming“, In Proceedings of the 25th ACM Workshop on Packet Video (PV ’20).
8. “CAdViSE: Cloud-based Adaptive Video Streaming Evaluation Framework for the Automated Testing of Media Players“, In Proceedings of the 11th ACM Multimedia Systems Conference (MMSys ’20).
9. “Objective and Subjective QoE Evaluation for Adaptive Point Cloud Streaming“, In 2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX).
10. “Performance Analysis of ACTE: A Bandwidth Prediction Method for Low-Latency Chunked Streaming“, ACM Transactions on Multimedia Computing, Communications, and Applications
11. “On Optimizing Resource Utilization in AVC-based Real-time Video Streaming“, 2020 IEEE Conference on Network Softwarization.
12. “Multi-Period Per-Scene Optimization for HTTP Adaptive Streaming“, 2020 IEEE International Conference on Multimedia and Expo (ICME).
13. “Fast Multi-Rate Encoding for Adaptive HTTP Streaming“, 2020 Data Compression Conference (DCC).

* ... the QUALINET white paper is not published at any conference/journal and that's why it's excluded from the counting. Nevertheless, this white paper is an important piece of work from QUALINET from various contributors working in the area of immersive media experiences.

Bitmovin and the University of Klagenfurt Collaborate on Innovative Video Transmission Technology

Bitmovin, a world leader in online video technology, is teaming up with the University of Klagenfurt and the Austrian Federal Ministry of Digital and Economic Affairs (BMDW) in a multi-million Euro research project to uncover techniques that will enhance the video streaming experiences of the future.

© aau/Waschnig

The joint project establishes a dedicated research team to investigate potential new tools and methodologies for encoding, transport and playback of live and on-demand video using the HTTP Adaptive Streaming protocol that is widely used by online video and TV providers. The resulting findings will help empower the creation of next-generation solutions for higher quality video experiences at lower latency, while also potentially reducing storage and distribution costs.

Margarete Schramböck, Federal Minister for Digital and Economic Affairs, sees great potential for the future in the development of technologies of this kind: “Video represents 60% of the Internet data volume and, correspondingly, the potential for optimization and resource conservation is enormous. At the same time, the Christian Doppler Laboratory contributes to the development of high-tech in Carinthia, secures jobs and draws qualified personnel to the region. A win-win situation for companies, science and society.”

Fierce competition increases the need for innovation

“The partnership with the University of Klagenfurt allows us to investigate the basic building blocks of video delivery in greater detail. This will help us to remain in pole position in the years ahead”, as Christopher Müller, CTO at Bitmovin states. Christian Timmerer, Associate Professor at the Institute of Information Technology (ITEC) at the University of Klagenfurt and Laboratory Director, goes on to explain: “Increasing competition between online video providers will accelerate the need for innovation. We continuously strive to maintain the optimum balance between cost, quality of user experience and increasing complexity of content.”

Ministry of Economic Affairs provides support through the Christian Doppler Research Association

The Christian Doppler Laboratory ATHENA is jointly funded by Bitmovin and the Christian Doppler Research Association, whose primary public sponsor is the Federal Ministry of Digital and Economic Affairs. The budget for 7 years of research is approx. 4.5 million Euros, with the public sector providing roughly 2.7 million of this total. Martin Gerzabek, President of the Christian Doppler Research Association, sees great potential for cooperation between science and industry, as in this case: “ATHENA is our first Christian Doppler Laboratory at the University of Klagenfurt. We are very pleased about the expansion of our funding model, which facilitates cooperation between outstanding science and innovative companies on an equal footing. We congratulate the University of Klagenfurt on this great success and confidently look forward to further CD labs and JR centres in the region.”

According to Oliver Vitouch, Rector of the University of Klagenfurt, “ATHENA offers a fantastic opportunity for further pioneering developments in global leading-edge technologies. Video streaming has permeated our everyday lives; most of us use it on a daily basis. This lab of the future is an ideal blend of research and innovation”. In Klagenfurt, members of the Institute for Information Technology have been working on the development of innovative multimedia technology for around 20 years. Bitmovin, which operates on a global scale and maintains sites on three continents today,
originally began its operations in Klagenfurt: The three founders (Stefan Lederer CEO, Christopher Müller CTO, and Christian Timmerer CIO) first collaborated on the development of the MPEG-DASH video streaming standard during their time at the University of Klagenfurt. This standard is currently used by YouTube, Netflix, ORF-TVThek, Flimmit and many more besides.

© aau/Waschnig

About Bitmovin

Bitmovin was founded in 2013 by Stefan Lederer, Christopher Müller, and Christian Timmerer as a spinoff of the University of Klagenfurt, where they both worked on the standardization of MPEG-DASH, a major standard for video streaming, during their time as students. The start-up company found its first home in the neighbouring Lakeside Science & Technology Park. Today, the company provides the world’s most powerful products for highly efficient video streaming on the Internet. Large, international customers such as the BBC or Hulu Japan rely on solutions developed in Carinthia.

Since participating in the renowned Y Combinator programme in the USA, the official corporate headquarters are located in San Francisco. However, the two locations in Austria remain the centres of excellence for research and development – not least due to the strong ties to the University of Klagenfurt. Over the course of two financing rounds in 2016 and 2018, the company was able to secure over 40 million dollars in venture capital from international investors. Most recently, Bitmovin was granted up to 20 million euros by the European Investment Bank to finance research and development as well as investments in sales and marketing in the coming years. Market-oriented, forward-looking product development and research at the cutting edge earn Bitmovin awards time and again, such as the “Phoenix” start-up prize in 2016, one of the most prestigious start-up prizes in Austria, with which the Austria Wirtschaftsservice GmbH (aws), the Austrian Research Promotion Agency (FFG) and the Federation of Austrian Industries (IV) recognise outstanding research achievements and innovative product ideas.

About Christian Doppler Laboratories

In Christian Doppler Laboratories, application-oriented basic research is carried out at a high level, which involves outstanding scientists cooperating with innovative companies. The Christian Doppler Research Association is internationally regarded as a best-practice example for the promotion of this type of cooperation. Christian Doppler Laboratories are jointly financed by the public sector and the participating companies. The primary public sponsor is the Federal Ministry of Digital and Economic Affairs (BMDW).

About the University of Klagenfurt

Since its foundation in 1970, the University of Klagenfurt has successfully established itself as one of six state universities with a broad range of subjects in Austria. More than 11,600 students pursue their studies and research at the University of Klagenfurt; around 2000 of these are international students. Approximately 1,500 employees strive for top quality in teaching and research. According to the QS World University Rankings (“Top 50 under 50”) the university belongs to the 150 best young universities worldwide. In the Times Higher Education World University Rankings 2020, which endeavour to rank the top 1,400 universities across the globe, it placed in the 301-350 range. In the discipline of Computer Science, the University of Klagenfurt achieved third place among Austrian universities in the 201-250 range. One of the university’s key research strengths lies in “networked and autonomous systems”.

Happy World Standards Day 2019 - Video Standards Create a Global Stage

Today on October 14, we celebrate the World Standards Day, "the day honors the efforts of the thousands of experts who develop voluntary standards within standards development organizations" (SDOs). Many SDOs such as W3C, IETF, ITU, ISO (incl. JPEG and MPEG) celebrate this with individual statements, highlighting the importance of standards and interoperability in today's information and communication technology landscape. Interestingly, this year's topic for the World Standards Day within ISO is about video standards creating a global stage. Similarly, national bodies of ISO provide such statements within their own country, e.g., the A.S.I. statement can be found here (note: in German). I have also blogged about the World Standards Day in 2017.

HEVC Emmy located at ITU-T, Geneva, CH (Oct'19).

The numbers for video content created, distributed (incl. delivery, streaming, ...), processed, consumed, etc. increases tremendously and, actually, more than 60 percent of today's world-wide internet traffic is attributed to video streaming. For example, almost 700,000 hours of video are watched on Netflix and 4.5 million videos are viewed on YouTube within a single internet minute in 2019. Videos are typically compressed (or encoded) prior to distribution and are decompressed (or decoded) before rendering on potentially a plethora of heterogeneous devices. Such codecs (portmanteau of coder-decoder) are subject to standardization and with AVC and HEVC (jointly developed by ISO/IEC MPEG and ITU-T VCEG) we have two successful standards which even have been honored with Primetime Engineering Emmy Awards (see one of them in the picture).

Within Austria, Bitmovin has been awarded with the Living Standards Award in 2017 for its contribution to the MPEG-DASH standard, which enables dynamic adaptive streaming over HTTP. This standard -- the 4th edition is becoming available very soon -- is now heavily deployed and has been adopted within products and services such as Netflix, Amazon Prime Video, YouTube, etc.

Standardization can be both source for and sink of research activities, i.e., development of efficient algorithms conforming to existing standards or research efforts leading to new standards. One example of such research efforts just recently started at the Institute of Information Technology (ITEC) at Alpen-Adria-Universität Klagenfurt (AAU) as part of the ATHENA (AdapTive Streaming over HTTP and Emerging Networked MultimediA Services) project. The aim of this project is to research and develop novel paradigms, approaches, (prototype) tools and evaluation results for the phases (i) multimedia content provisioning (video coding), (ii) content delivery (video networking), (iii) content consumption (player) in the media delivery chain, and (iv) end-to-end aspects, with a focus on, but not being limited to, HTTP Adaptive Streaming (HAS).

The SDO behind these standards is MPEG (officially ISO/IEC JTC 1/SC 29/WG 11), which has a proven track record of producing very successful standards (not only those mentioned as examples above) and its future is currently discussed within its parent body (SC 29). A possible MPEG future is described here, which suggests upgrading the current SC 29 working groups to sub-committees (SCs), specifically to spin-off a new SC that basically covers MPEG while the remaining WG (JPEG) arises within SC 29. This proposal of MPEG and JPEG as SC is partially motivated by the fact that both WGs work on a large set of standardization projects, actually developed by its subgroups. Thus, elevating both WGs (JPEG & MPEG) to SC level would only reflect the current status quo but would also preserve two important brands for both academia and industry. Further details can be found at http://mpegfuture.org/.

Video Developer Report 2019

... and Bitmovin did it again; published the 2019 Video Developer Report last week. I've briefly reported about it last year here. Interestingly, this year 542 people from 108 countries participated (vs. 456 from over 67 countries last year).

The biggest challenges seem to be latency (54%) and playback on all devices (41%). Other challenges (>20%) are related to DRM, CDN, user engagement with video, and ads in general.

Last year I've also shared the codec usage and it's probably interesting to compare these numbers with this year's results as shown below. Interestingly, the numbers (for 'planning to implement') are a bit lower compared to last year which could be explained by a more conservative approach from developers or simply by the fact that more people responded to the survey with a greater diversity in terms of different countries.

Current Video Codec Usage and Plans to Implement in next 12 Months.

The actual video codec usage compares to last year's report as follows: AVC (-1), HEVC (+1), VP9 (+/- 0), AV1 (+1).

Another interesting aspect is the usage of streaming formats and plans to implement them within the next 12 months as shown below. Comparing with last year's report (available here), we can observe the following major changes: HLS (-3), MPEG-DASH (-3), RTMP (-2), Smooth Streaming (+2), Progressive Streaming (-1), MPEG-CMAF (+2), HDS (-4).

Current Streaming Formats and Plans to Implement in next 12 Months.

In general, one can observe that the adoption of new formats are happening at a slower pace than expected and I am wondering what this means for the new video coding formats coming up like VVC et al. (note: these are results from a public survey with different participants compared to last years which need to be taken into account when comparing results over years).

For more details, the full report can be downloaded for free from here.

Video Developer Survey 2019: Call for Participation

After the success of 2018 and 2017 editions of the Video Developer Survey, Bitmovin is calling once again for your participation in this years survey, which is accessible here:

https://www.surveymonkey.com/r/2019developersurvey_int

The goal is to learn from you about the video codecs, formats and platforms used in your organization and how you see the tech evolving in the next year. 

An example outcome from last years edition is shown below, which illustrates the planned video codec usage in the next 12 months compared to the 2017 report.

Planned video codec usage in the next 12 months 2017 vs. 2018.

Christian Doppler Research Association approves ATHENA project proposal

ATHENA stands for Adaptive Streaming over HTTP and Emerging Networked Multimedia Services and has been jointly proposed by the Institute of Information Technology (ITEC; http://itec.aau.at) at Alpen-Adria-Universität Klagenfurt (AAU) and Bitmovin GmbH (https://bitmovin.com) to address current and future research and deployment challenges of HTTP adaptive steaming (HAS) and emerging streaming methods.

AAU (ITEC) has been working on adaptive video streaming for more than a decade, has a proven record of successful research projects and publications in the field, and has been actively contributing to MPEG standardization for many years, including MPEG-DASH; Bitmovin is a video streaming software company founded by ITEC researchers in 2013 and has developed highly successful, global R&D and sales activities and a world-wide customer base since then. 

The aim of ATHENA is to research and develop novel paradigms, approaches, (prototype) tools and evaluation results for the areas (1) multimedia content provisioning (i.e., video coding), (2) content delivery (i.e., multimedia networking) and (3) content consumption (i.e., HAS player aspects) in the media delivery chain as well as for (4) end-to-end aspects, with a focus on, but not being limited to, HTTP Adaptive Streaming (HAS). The new approaches and insights are to enable Bitmovin to build innovative applications and services to account for the steadily increasing and changing multimedia traffic on the Internet.

The project has been approved by the Christian Doppler Research Association as a CD pilot laboratory -- the first such kind of project at Alpen-Adria-Universität Klagenfurt -- with a duration of two years including a five year extension after successful review after the first two years (i.e., seven years in total). Thus, stay tuned for details and yes, I'm hiring PhD students for the areas above (detailed job description will be published soon).

AAU and Bitmovin presenting IEEE ICIP 2018

The IEEE International Conference in Image Processing (ICIP) is with more than 1,000 attendees one of the biggest conferences of the Signal Processing Society. At ICIP'18, Anatoliy (AAU) and myself (AAU/Bitmovin) attended with the following presentations:

On Monday, October 8, I was on the panel of the Young Professional Networking Event (together with Amy Reibman and Sheila Hemami) sharing my experiences with all attendees. See one picture here.

On Tuesday, October 9, I presented at the Innovation Program talking about "Video Coding for Large-Scale HTTP Adaptive Streaming Deployments: State of the Art and Challenges Ahead".

Video Coding for Large-Scale HTTP Adaptive Streaming Deployments: State of the Art and Challenges Ahead from Christian Timmerer

On Wednesday, October 10, Anatoliy presented our joint AAU/Bitmovin paper about "A Practical Evaluation of Video Codecs for Large-Scale HTTP Adaptive Streaming Services". Abstract: The number of bandwidth-hungry applications and services is constantly growing. HTTP adaptive streaming of audio- visual content accounts for the majority of today’s internet traffic. Although the internet bandwidth increases also constantly, audio-visual compression technology is inevitable and we are currently facing the challenge to be confronted with multiple video codecs. This paper provides a practical evaluation of state of the art video codecs (i.e., AV1, AVC/libx264, HEVC/libx265, VP9/libvpx-vp9) for large- scale HTTP adaptive streaming services. In anticipation of the results, AV1 shows promising performance compared to established video codecs. Additionally, AV1 is intended to be royalty free making it worthwhile to be considered for large scale HTTP adaptive streaming services.

A Practical Evaluation of Video Codecs for Large-Scale HTTP Adaptive Streaming Services from Christian Timmerer

Acknowledgment: This work was supported in part by the Austrian Research Promotion Agency (FFG) under the Next Generation Video Streaming project “PROMETHEUS”.

2018 Video Developer Survey

Bitmovin's 2018 Video Developer Survey reveals interesting details about

• streaming formats (MPEG-DASH, HLS, CMAF, etc.),
• video codecs (AVC, HEVC, VP9, AV1),
• audio codecs (AAC, MP3, Dolby, etc.),
• encoding preferences (hardware, software on-premise/cloud),
• players (open source, commercial, in-house solution),
• DRM,
• monetization model,
• ad standard/technology, and
• what are the biggest problems experienced today (DRM, ad-blocker, ads in general, server-side ad insertion, CDN issues, broadcast delay/latency, getting playback to run on all devices)

For example, the figure below illustrates the planned video codec usage in the next 12 months compared to the 2017 report.

Planned video codec usage in the next 12 months 2017 vs. 2018.

In total, 456 survey submissions from over 67 countries have been received and included into the report, which can be downloaded here for free.

360-degree (live) adaptive streaming with RICOH THETA S and Bitmovin

Recently I got the RICOH THETA S 360-degree camera and I asked myself how to setup a (live) adaptive streaming session using the Bitmovin cloud encoding and HTML5 player. I quickly found some general guidelines on the internet but before providing step-by-step instructions one has to consider the following:

• Update the firmware of your RICOH THETA S by downloading the basic app, start it (while the camera is connected via USB) and go to File -> Firmware Update... and follow the steps on the screen. It's pretty easy and mine got updated from v1.11 to v1.82.
• Think about a storage solution for your files generated by the Bitmovin cloud encoding and possible options are FTP, Amazon S3, Google Cloud Storage, and Dropbox. I used Amazon S3 for this setup which provides a bucket name, "AWS Access Key", and "AWS Secret Key".
• Setup a basic web site and make sure it works with the Bitmovin HTML5 player for video on demand services with the content hosted on the previously selected storage solution (i.e., avoid any CORS issues). In my setup I used Wordpress and the Bitmovin Wordpress plugin which makes it very easy...

Step 1: Follow steps 1-4 from here.

Follow steps 1-4 from the general guidelines. Basically, install the live-streaming app, register the device, and install/configure OBS. Enable the live streaming on the RICOH THETA S and within OBS use the "Custom Streaming Server" of the "Stream" settings. That basically connects the RICOH THETA S with OBS on your local computer. The next step is forwarding this stream to the Bitmovin cloud encoding service for DASH/HLS streaming.

Step 2: Create a new Bitmovin Output

1. Login to the Bitmovin portal and go to Encoding -> Outputs -> Create Output
2. Select Amazon S3 and use any “Output Profile name”, e.g., ricoh-livestream-test
3. Enter the name of your Bucket from Amazon S3
4. The prefix is not needed
5. Select any “Host-Region” (preferably one close to where you are)
6. Enter the ”AWS Access Key" and the “AWS Secret Key” from Amazon S3
7. Make sure the "Create Public S3 URLs" checkbox is enabled

An example screenshot is shown below.

Finally, click the “+” sign to create it and if everything is correct, the output will be created, otherwise an error message will be shown. In such a case, make sure the bucket name and keys are correct as provided when creating a bucket on Amazon S3.

Step 3: Create a new Bitmovin Livestream

1. Login to the Bitmovin portal and go to Live (beta) -> Create Livestream
2. Select "Encoding-Profile": bitcodin fullHD is sufficient (4K not needed as the device provides only fullHD)
3. Select "Output-Profile": select the output you’ve created in previous step (ricoh-livestream-test)
4. Add a "Livestream-Name" (any string works here), e.g., ricoh-livestream-test
5. Add a "Stream-Key" (any string works here), e.g., ricohlivestreamtest
6. Click "Create Live Stream", an "Important Notice" shows up & click "Create Live Stream"
7. Wait (could take some time, you may reload the page or go to the "Overview") for RTMP PUSH URL to be used in OBS

An example screenshot is shown below which displays the RTMP PUSH URL, Stream Key, MPD URL, and HLS URL to be used in the next steps.

The next step is to start streaming in OBS which provides the live stream from the RICOH THETA S to the Bitmovin cloud encoding.

Step 4: Start Streaming in OBS

1. Go to OBS -> Settings
2. In section "Stream", select "Custom Streaming Server"
3. Enter the RTMP PUSH URL from Bitmovin in the "URL" field of OBS
4. Enter the Stream Key from Bitmovin in the "Stream key" field of OBS
5. Click "OK" and then click "Start Streaming" in OBS

An example screenshot is shown below and if everything works fine OBS will stream to the Bitmovin cloud encoding  service.

The final step is setting up the HTML5 player..

Step 5: Setup the HTML5 Player

Basically follow the instructions here or in my case I simply used Wordpress and the Bitmovin Wordpress plugin.

1. Go to the Bitmovin WP plugin
2. Select "Add New Video"
3. Enter any name/title of the new video
4. In the "Video" section, enter the "DASH URL" and "HLS URL" from the Bitmovin livestream provided in step 3 (i.e., the MPD URL and the HLS URL)
5. In the "Player" section, select latest stable (in my case this was latest version 7)
6. In the "VR" section, select startup mode "2d" and leave the rest as is

An example screenshot is shown below.

Finally, click on "Publish" in Wordpress which will give you a shortcut code to be placed (copy/paste) into your site or post and you're done...!

The setup during the live streaming session is shown in the screenshot below. The RICOH THETA S on the right is mounted on a tripod and connected via USB. My MacBook Pro runs OBS (see display on the right) which streams it to the Bitmovin cloud encoding and also shows the live streaming session within a browser (see display on the left) using the Bitmovin HTML5 player.

A similar approach can be used for video on demand content but in such a case you don't need OBS as you simply encode your content using the Bitmovin cloud encoding, transfer it to your web server, and use the HTML5 player for the actual streaming.

Open Positions at Bitmovin

About working with Bitmovin

Bitmovin, a YCombinator company, is a fast growing privately owned technology leader, located in Klagenfurt am Wörthersee in Austria and in California. Our company is leading in research and development of cutting-edge multimedia services like, e.g.,Dynamic Adaptive Streaming over HTTP (DASH), HTTP Live Streaming (HLS), multimedia systems and multimedia cloud services.

We believe our employees are the most valuable assets of our company. They drive Bitmovin’s success with their knowledge, experience and passion. Therefore we provide a high degree of freedom to our employees to initiate projects and take on responsibility, while paying a very competitive salary above the average.

Working at Bitmovin is international, fast-paced, fun and challenging. We’re looking for talented, passionate and inspired people who want to change the way media is consumed online. Join us to develop, sell and market the world’s fastest video encoding service.

Sales and Marketing

• Director Sales US [Full-Time, San Francisco, US]
• Marketing Professional [Full-Time, San Francisco, USA], [Full-Time, Klagenfurt, Austria]
• Sales Representative [Full-Time, Klagenfurt, Austria]
• Sales Development / Demand Generation [Full-Time, Klagenfurt, Austria]
• Growth Hacker / Online Marketing [Full-Time, Klagenfurt, Austria], [Full-Time, San Francisco, USA]
• Account Executive [Full-Time, San Francisco, US], [Full-Time, Klagenfurt, EU]

Software and Development

• Software Engineer – Adaptive Streaming Player [Full-Time, Klagenfurt, Austria]
• Software Engineer – Cloud Encoding Service [Full-Time, Klagenfurt, Austria]
• DevOps Engineer [Full-Time, Klagenfurt, Austria]
• Solution Architect [Full-Time, San Francisco, USA], [Full-Time, Klagenfurt, Austria]

Admin and Finance

• Office Management / Secretary [Full-Time, Klagenfurt, Austria]
• Business Intelligence, Controlling & Finance [Full-Time, Klagenfurt, Austria]

Real-Time Entertainment now accounts for >70% of the Internet Traffic

Sandvine's Global Internet Phenomena Report (December 2015 edition) reveals that real-time entertainment (i.e., streaming video and audio) traffic now accounts for more than 70% of North American downstream traffic in the peak evening hours on fixed access networks (see Figure 1). Interestingly, five years ago it accounted only for less than 35%.

Netflix is mainly responsible for this with a share of >37% (i.e., more than the total five years ago) but already had a big share in 2011 (~32%) and didn't "improve" that much. Second biggest share is coming from YouTube with roughly 18%.

I'm using these figures within my slides to motivate that streaming video and audio is a huge market - opening a lot of opportunities for research and innovation - and it's interesting to see how the Internet is being used. In most of these cases, the Internet is used as is, without any bandwidth guarantees and clients adapt themselves to what's available in terms of bandwidth. Service providers offer the content in multiple versions (e.g., different bitrates, resolution, etc.) and each version is segmented to which clients can adapt both at the beginning and also during the session. This principle is known as over-the-top adaptive video streaming and a standardized representation format is available known as Dynamic Adaptive Streaming over HTTP (DASH) under ISO/IEC 23009. Note that the adaptation logic is not part of the standard and open a punch of possibilities in terms of research and engineering.

Both Netflix and YouTube adopted the DASH format which is now natively supported by modern Web browsers thanks to the HTML5 Media Source Extensions (MSE) and even digital rights management is possible due to Encrypted Media Extensions (EME). All one needs is a client implementation that is compliant to the standard - the easy part; the standard is freely available - and adapts to the dynamically changing usage context while maximizing the Quality of Experience (QoE) - the difficult part. That's why we at bitmovin thought to setup a grand challenge at IEEE ICME 2016 in Seattle, USA with the aim to solicit contributions addressing end-to-end delivery aspects which improve the QoE while optimally utilising the available network infrastructures and its associated costs. This includes the content preparation for DASH, the content delivery within existing networks, and the client implementations. Please feel free to contribute to this exciting problem and if you have further questions or comments, please contact us here.

HEVC, AOMedia, MPEG, and DASH

Ultra-high definition (UHD) displays are available for quite some time and in terms of video coding the MPEG-HEVC/H.265 standard was designed to support these high resolutions in an efficient way. And it does, with a performance gain of more than twice as much as its predecessor MPEG-AVC/H.264. But it all comes with costs - not only in terms of coding complexity at both encoder and decoder - especially when it comes to licensing. The MPEG-AVC/H.264 licenses are managed by MPEG LA but for HEVC/H.265 there are two patent pools available which makes its industry adoption more difficult than it was for AVC.

HEVC was published by ISO in early 2015 and in the meantime MPEG started discussing about future video coding using its usual approach of open workshops inviting experts from companies inside and outside of MPEG. However, now there’s the Alliance for Open Media (AOMedia) promising to provide "open, royalty-free and interoperable solutions for the next generation of video delivery” (press release). A good overview and summary is available here which even mentions that a third HEVC patent pool is shaping up (OMG!).

Anyway, even if AOMedia’s "media codecs, media formats, and related technologies” are free like in “free beer” it’s still not clear whether it will taste anything good. Also, many big players are not part of this alliance and could (easily) come up with some patent claims at a later stage jeopardising the whole process (cf. what happened with VP9). In any case, AOMedia is certainly disruptive and together with other disruptive media technologies (e.g., PERSEUS although I have some doubts here) might change the media coding landscape, not clear whether it will be a turn to the better though...

Finally, I was wondering how does this all impact DASH, specifically as MPEG LA recently announced that they want to establish a patent pool for DASH although major players have stated some time ago not to charge anything for DASH (wrt licensing). In terms of media codecs please note that DASH is codec agnostic and it can work with any codec, also those not specified within MPEG and we’ve shown it works some time ago already (using WebM). The main problem is, however, which codecs are supported on which end user devices and how to access them with which API (like HMTL5 & MSE). For example, some Android devices support HEVC but not through HMTL5 & MSE which makes it more difficult to integrate with DASH.

Using MPEG-DASH with HMTL5 & MSE is currently the preferred way how to deploy DASH, even the DASH-IF’s reference player (dash.js) is assuming HTML5 & MSE and companies like bitmovin are offering bitdash following the same principles. Integrating new codecs on the DASH encoding side like on bitmovin’s bitcodin cloud-based transcoding-as-a-service isn’t a big deal and can be done very quickly as soon as software implementations are available. Thus, the problem is more on the plethora of heterogeneous end user devices like smart phones, tablets, laptops, computers, set-top-boxes, TV sets, media gateways, gaming consoles, etc. and their variety of platforms and operating systems.

Therefore, I’m wondering whether AOMedia (or whatever will come in the future) is a real effort changing the media landscape to the better or just another competing standard to choose from … but on the other side, as Andrew S. Tanenbaum has written already in his book on computer networks, “the nice thing about standards is that you have so many to choose from.”

Over-the-Top Content Delivery: State of the Art and Challenges Ahead at ICME 2015

As stated in my MPEG report from Warsaw I attended ICME'15 in Torino to give a tutorial -- together with Ali Begen -- about over-the-top content delivery. The slides are available as usual and embedded here...

Over the Top Content Delivery: State of the Art and Challenges Ahead from Christian Timmerer

If you have any questions or comments, please let us know. The goal of this tutorial is to give an overview about MPEG-DASH and also selected informative aspects (e.g., workflows, adaptation, quality, evaluation) not covered in the standard. However, it should not be seen as a tutorial on the standard as many approaches presented here can be also applied on other formats although MPEG-DASH seems to be the most promising from those available. During the tutorial we ran into interesting questions and discussions with the audience and I could also show some live demos from bitmovin using bitcodin and bitdash. Attendees were impressed about the maturity of the technology behind MPEG-DASH and how research results find their way into actual products available on the market.

If you're interested now, I'll give a similar tutorial -- with Tobias Hoßfeld -- about "Adaptive Media Streaming and Quality of Experience Evaluations using Crowdsourcing" during ITC27 (Sep 7, 2015, Ghent, Belgium) and bitmovin will be at IBC2015 in Amsterdam.

One Year of MPEG

In my last MPEG report (index) I’ve mentioned that the 112th MPEG meeting in Warsaw was my 50th MPEG meeting which roughly accumulates to one year of MPEG meetings. That is, one year of my life I've spend in MPEG meetings - scary, isn't it? Thus, I thought it’s time to recap what I have done in MPEG so far featuring the following topics/standards where I had significant contributions:

• MPEG-21 - The Multimedia Framework 
• MPEG-M - MPEG extensible middleware (MXM), later renamed to multimedia service platform technologies 
• MPEG-V - Information exchange with Virtual Worlds, later renamed to media context and control
• MPEG-DASH - Dynamic Adaptive Streaming over HTTP

MPEG-21 - The Multimedia Framework

I started my work with standards, specifically MPEG, with Part 7 of MPEG-21 referred to as Digital Item Adaptation (DIA) and developed the generic Bitstream Syntax Description (gBSD) in collaboration with SIEMENS which allows for a coding-format independent (generic) adaptation of scalable multimedia content towards the actual usage environment (e.g., different devices, resolution, bitrate). The main goal of DIA was to enable the Universal Media Access (UMA) -- any content, anytime, anywhere on any device -- and also motivated me to start this blog. I also wrote a series of blog entries on this topic: O Universal Multimedia Access, Where Art Thou? which gives an overview about this topic and basically is also what I’ve done in my Ph.D. thesis. Later I helped a lot in various MPEG-21 parts including its dissemination and documented where it has been used. In the past, I saw many forms of Digital Items (e.g., iTunesLP was one of the first) but unfortunately the need for a standardised format is very low. Instead, proprietary formats are used and I realised that developers are more into APIs than formats. The format comes with the API but it’s the availability of an API that attracts developers and makes them to adopt a certain technology. 

MPEG-M

The lessons learned from MPEG-21 was one reason why I joined the MPEG-M project as it was exactly the purpose to create an API into various MPEG technologies, providing developers a tool that makes it easy for them to adopt new technologies and, thus, new formats/standards. We created an entire architecture, APIs, and reference software to make it easy for external people to adopt MPEG technologies. The goal was to hide the complexity of the technology through simple to use APIs which should enable the accelerated development of components, solutions, and applications utilising digital media content. A good overview about MPEG-M can found on this poster.

MPEG-V

When MPEG started working on MPEG-V (it was not called like that in the beginning), I saw it as an extension of UMA and MPEG-21 DIA to go beyond audio-visual experiences by stimulating potentially all human senses. We created and standardised an XML-based language that enables the annotation of multimedia content with sensory effects. Later the scope was extended to include virtual worlds which resulted in the acronym MPEG-V. It also brought me to start working on Quality of Experience (QoE) and we coined the term Quality of Sensory Experience (QuASE) as part of the (virtual) SELab at Alpen-Adria-Universität Klagenfurt which offers a rich set of open-source software tools and datasets around this topic on top of off-the-shelf hardware (still in use in my office).

MPEG-DASH

The latest project I’m working on is MPEG-DASH where I’ve also co-founded bitmovin, now a successful startup offering fastest transcoding in the cloud (bitcodin) and high quality MPEG-DASH players (bitdash). It all started when MPEG asked me to chair the evaluation of call for proposals on HTTP streaming of MPEG media. We then created dash.itec.aau.at that offers a huge set of open source tools and datasets used by both academia and industry worldwide (e.g., listed on DASH-IF). I think I can proudly state that this is the most successful MPEG activity I've been involved so far... (note: a live deployment can be found here which shows 24/7 music videos over the Internet using bitcodin and bitdash).

DASH and QuASE are also part of my habilitation which brought me into the current position at Alpen-Adria-Universität Klagenfurt as Associate Professor. Finally, one might ask the question, was it all worth spending so much time for MPEG and at MPEG meetings. I would say YES and there are many reasons which could easily results in another blog post (or more) but it’s better to discuss this face to face, I'm sure there will be plenty of possibilities in the (near) future or you come to Klagenfurt, e.g., for ACM MMSys 2016 ...

MPEG news: a report from the 110th meeting, Strasbourg, France

This blog post is also available at SIGMM records.

The 110th MPEG meeting was held at the Strasbourg Convention and Conference Centre featuring the following highlights:

• The future of video coding standardization
• Workshop on media synchronization
• Standards at FDIS: Green Metadata and CDVS
• What's happening in MPEG-DASH?

Additional details about MPEG's 110th meeting can be also found here including the official press release and all publicly available documents.

The Future of Video Coding Standardization

MPEG110 hosted a panel discussion about the future of video coding standardization. The panel was organized jointly by MPEG and ITU-T SG 16's VCEG featuring Roger Bolton (Ericsson), Harald Alvestrand (Google), Zhong Luo (Huawei), Anne Aaron (Netflix), Stéphane Pateux (Orange), Paul Torres (Qualcomm), and JeongHoon Park (Samsung).

As expected, "maximizing compression efficiency remains a fundamental need" and as usual, MPEG will study "future application requirements, and the availability of technology developments to fulfill these requirements". Therefore, two Ad-hoc Groups (AhGs) have been established which are open to the public:

• AHG on Future Video Coding Technology [email][subscription]
• AHG on industry needs for Future Video Coding [email][subscription]

The presentations of the brainstorming session on the future of video coding standardization can be found here.

Workshop on Media Synchronization

MPEG101 also hosted a workshop on media synchronization for hybrid delivery (broadband-broadcast) featuring six presentations "to better understand the current state-of-the-art for media synchronization and identify further needs of the industry".

• An overview of MPEG systems technologies providing advanced media synchronization, Youngkwon Lim, Samsung
• Hybrid Broadcast - Overview of DVB TM-Companion Screens and Streams specification, Oskar van Deventer, TNO
• Hybrid Broadcast-Broadband distribution for new video services :  a use cases perspective, Raoul Monnier, Thomson Video Networks
• HEVC and Layered HEVC for UHD deployments, Ye Kui Wang, Qualcomm
• A fingerprinting-based audio synchronization technology, Masayuki Nishiguchi, Sony Corporation
• Media Orchestration from Capture to Consumption, Rob Koenen, TNO

The presentation material is available here. Additionally, MPEG established an AhG on timeline alignment (that's how the project is internally called) to study use cases and solicit contributions on gap analysis and also technical contributions [email][subscription].

Standards at FDIS: Green Metadata and CDVS

My first report on MPEG Compact Descriptors for Visual Search (CDVS) dates back to July 2011 which provides details about the call for proposals. Now, finally, the FDIS has been approved during the 110th MPEG meeting. CDVS defines a compact image description that facilitates the comparison and search of pictures that include similar content, e.g. when showing the same objects in different scenes from different viewpoints. The compression of key point descriptors not only increases compactness, but also significantly speeds up, when compared to a raw representation of the same underlying features, the search and classification of images within large image databases. Application of CDVS for real-time object identification, e.g. in computer vision and other applications, is envisaged as well.

Another standard reached FDIS status entitled Green Metadata (first reported in August 2012). This standard specifies the format of metadata that can be used to reduce energy consumption from the encoding, decoding, and presentation of media content, while simultaneously controlling or avoiding degradation in the Quality of Experience (QoE). Moreover, the metadata specified in this standard can facilitate a trade-off between energy consumption and QoE. MPEG is also working on amendments to the ubiquitous MPEG-2 TS ISO/IEC 13818-1 and ISOBMFF ISO/IEC 14496-12 so that green metadata can be delivered by these formats.

What's happening in MPEG-DASH?

MPEG-DASH is in a kind of maintenance mode but still receiving new proposals in the area of SAND parameters and some core experiments are going on. Also, the DASH-IF is working towards new interoperability points and test vectors in preparation of actual deployments. When speaking about deployments, they are happening, e.g., a 40h live stream right before Christmas (by bitmovin, a top-100 company that matters most in online video). Additionally, VideoNext was co-located with CoNEXT'14 targeting scientific presentations about the design, quality and deployment of adaptive video streaming. Webex recordings of the talks are available here. In terms of standardization, MPEG-DASH is progressing towards the 2nd amendment including spatial relationship description (SRD), generalized URL parameters and other extensions. In particular, SRD will enable new use cases which can be only addressed using MPEG-DASH and the FDIS is scheduled for the next meeting which will be in Geneva, Feb 16-20, 2015. I'll report on this within my next blog post, stay tuned..