Delivering Traditional and Omnidirectional Media

This tutorial will be given at the following events:

• IEEE ICME 2018, July 23-27, 2018, San Diego, USA

Delivering Traditional and Omnidirectional Media from Christian Timmerer

Abstract

Universal media access as proposed in the late 90s is now closer to reality. Users can generate, distribute and consume almost any media content, anywhere, anytime and with/on any device. A major technical breakthrough was the adaptive streaming over HTTP resulting in the standardization of MPEG-DASH, which is now successfully deployed in most platforms. The next challenge in adaptive media streaming is virtual reality applications and, specifically, omnidirectional (360°) media streaming.

This tutorial first presents a detailed overview of adaptive streaming of both traditional and omnidirectional media, and focuses on the basic principles and paradigms for adaptive streaming. New ways to deliver such media are explored and industry practices are presented. The tutorial then continues with an introduction to the fundamentals of communications over 5G and looks into mobile multimedia applications that are newly enabled or dramatically enhanced by 5G.

A dedicated section in the tutorial covers the much-debated issues related to quality of experience. Additionally, the tutorial provides insights into the standards, open research problems and various efforts that are underway in the streaming industry.

Learning Objectives

Upon attending this tutorial, the participants will have an overview and understanding of the following topics:

• Principles of HTTP adaptive streaming for the Web/HTML5
• Principles of omnidirectional (360) media delivery
• Content generation, distribution and consumption workflows
• Standards and emerging technologies, new delivery schemes in the adaptive streaming space
• Measuring, quantifying and improving quality of experience
• Fundamental technologies of 5G
• Features and services enabled or enhanced by 5G
• Current and future research on delivering traditional and omnidirectional media

Table of Contents

Part I: Streaming (Presented by Dr. Begen and Dr. Timmerer)

• Survey of well-established streaming solutions (DASH, CMAF and Apple HLS)
• HTML5 video and media extensions
• Multi-bitrate encoding, and encapsulation and encryption workflows
• Common issues in scaling and improving quality, multi-screen/hybrid delivery
• Acquisition, projection, coding and packaging of 360 video
• Delivery, decoding and rendering methods
• The developing MPEG-OMAF and MPEG-I standards

Part II: Communications over 5G (Presented by Dr. Ma and Dr. Begen)

• 5G fundamentals: radio access and core network
• Multimedia signal processing and communications
• Emerging mobile multimedia use cases
• Detailed analysis for selected use cases
• Improving QoE

Speakers

Ali C. Begen recently joined the computer science department at Ozyegin University. Previously, he was a research and development engineer at Cisco, where he has architected, designed and developed algorithms, protocols, products and solutions in the service provider and enterprise video domains. Currently, in addition to teaching and research, he provides consulting services to industrial, legal, and academic institutions through Networked Media, a company he co-founded. Begen holds a Ph.D. degree in electrical and computer engineering from Georgia Tech. He received a number of scholarly and industry awards, and he has editorial positions in prestigious magazines and journals in the field. He is a senior member of the IEEE and a senior member of the ACM. In January 2016, he was elected as a distinguished lecturer by the IEEE Communications Society. Further information on his projects, publications, talks, and teaching, standards and professional activities can be foundhttp://ali.begen.net

Liangping Ma is with InterDigital, Inc., San Diego, CA. He is an IEEE Communication Society Distinguished Lecturer focusing on 5G technologies and standards, video communication and cognitive radios. He is an InterDigital delegate to the 3GPP New Radio standards. His current research interests include various aspects about ultra-reliable and low-latency communication, such as channel coding, multiple access and resource allocation. Previously, he led the research on Quality of Experience (QoE) driven system optimization for video streaming and interactive video communication. Prior to joining InterDigital in 2009, he was with San Diego Research Center and Argon ST (acquired by Boeing), where he led research on cognitive radios and wireless sensor networks and served as the principal investigators of two projects supported by the Department of Defense and the National Science Foundation, respectively. He is the co-inventor of more than 40 patents and the author/co-author of more than 50 journal and conference papers. He has been the Chair of the San Diego Chapter of the IEEE Communication Society since 2014. He received his PhD from University of Delaware in 2004 and his B.S. from Wuhan University, China, in 1998.

Christian Timmerer received his M.Sc. (Dipl.-Ing.) in January 2003 and his Ph.D. (Dr.techn.) in June 2006 (for research on the adaptation of scalable multimedia content in streaming and constrained environments) both from the Alpen-Adria-Universität (AAU) Klagenfurt. He joined the AAU in 1999 (as a system administrator) and is currently an Associate Professor at the Institute of Information Technology (ITEC) within the Multimedia Communication Group. His research interests include immersive multimedia communications, streaming, adaptation, quality of experience, and sensory experience. He was the general chair of WIAMIS 2008, QoMEX 2013 and MMSys 2016, and has participated in several EC-funded projects, notably DANAE, ENTHRONE, P2P-Next, ALICANTE, SocialSensor, COST IC1003 QUALINET and ICoSOLE. He also participated in ISO/MPEG work for several years, notably in the area of MPEG-21, MPEG-M, MPEG-V, and MPEG-DASH where he also served as a standard editor. In 2012, he co-founded Bitmovin to provide professional services around MPEG-DASH where he currently holds the position of the Chief Innovation Officer (CIO).

What to care about in multimedia communication in 2018?

In the past days/weeks you may have witness a high number of forecasts/predictions for 2018, like this one here. I'm not so good at predictions and we all learned these days to be careful about speculation. Thus, my focus here is on things to care about in 2018.

MPEG and VCEG are working towards a new video coding standard (naming, number scheme yet to be defined) and the call for proposals is out. Responses will be evaluated by the 122nd MPEG meeting in April 2018 (San Diego, CA, USA) and a new standard is expected to be available in late 2020. The main focus of the CfP is (i) 360-degree omnidirectional video, (ii) high-dynamic range (HDR), (iii) wide colour gamut (WCG), and (iv) conventional standard-dynamic-range camera content. The goal is -- simple, as usual -- compress digital video content, i.e., twice as much as you did before with the same video quality, e.g., as HEVC, or get higher quality with the same number of bits (or a combination thereof). Initial, preliminary results indicate this goal is feasible and everyone is looking forward to the MPEG meeting in April; certainly a place to be.

In addition to what MPEG/VCEG is doing, the Alliance for Open Media gained significant attention with its AV1 codec, first demos are available, and recently also Apple joined AOM. AV1 is not longer controlled by a single company, and, thus, it is becoming a real alternative in the video coding landscape, specifically for the streaming market. The nice thing, it's open source and royalty-free! In other words, one should not neglect AV1 and I think we will see many, hopefully good news in 2018.

On this topic, you might be interested in reading this and this.

What about DASH in 2018? We will see a 3rd edition of MPEG-DASH, the DASH-IF will further work on interoperability points, and I expect further convergence of DASH and HLS towards CMAF. However, I also expect minor changes for the main, common use cases utilizing the core technology of HTTP adaptive streaming. Changes, if any, will be transparent to most of us. On the other hand, immersive media and user engagement will become more and more important as more services are delivered over the top leading to more content becoming available to end users, thus, increasing competition among providers, vendors, etc. As a consequence, (a) content, (b) quality, and (c) costs will be important aspects, whereby (a+c) are "easy to sell" but (b) is still difficult to quantify (and "sell") with many open issues to solve in the (near) future.

Before drifting off to forecasts and predictions, I'd like to conclude with a list of scientific events in 2018 which are worthwhile to attend:

• QoMEX -- Int'l Conf. on Quality of Multimedia Experience -- will be hosted in Sardinia, Italy from May 29-31, which is THE conference to be for QoE of multimedia applications and services. Submission deadline is January 15/22, 2018.
• MMSys -- Multimedia Systems Conf. -- and specifically Packet Video, which will be on June 12 in Amsterdam, The Netherlands. Packet Video is THE adaptive streaming scientific event 2018. Submission deadline is March 1, 2018.
• Additionally, you might be interested in ICME (July 23-27, 2018, San Diego, USA; I'm part of a tutorial there;), ICIP (October 7-10, 2018, Athens, Greece; specifically in the context of video coding), and PCS (June 24-27, 2018, San Francisco, CA, USA; also in the context of video coding).
• The DASH-IF academic track hosts special events at MMSys (Excellence in DASH Award) and ICME (DASH Grand Challenge).
• MIPR -- 1st Int'l Conf. on Multimedia Information Processing and Retrieval -- will be in Miami, Florida, USA from April 10-12, 2018. It has a broad range of topics including networking for multimedia systems as well as systems and infrastructures.