A Tutorial on Immersive Video Delivery: From Omnidirectional Video to Holography

 A Tutorial on Immersive Video Delivery: From Omnidirectional Video to Holography

IEEE Communications Surveys and Tutorials

[PDF]

Jeroen van der Hooft (Ghent University, Belgium), Hadi Amirpour (AAU, Austria), Maria Torres Vega (KU Leuven, Belgium), Yago Sanchez (Fraunhofer/HHI), Raimund Schatz (AIT, Austria), Thomas Schierl (Fraunhofer/HHI, Germany), and Christian Timmerer (AAU, Austria)

Abstract: Video services are evolving from traditional two-dimensional video to virtual reality and holograms, which offer six degrees of freedom to users, enabling them to freely move around in a scene and change focus as desired. However, this increase in freedom translates into stringent requirements in terms of ultra-high bandwidth (in the order of Gigabits per second) and minimal latency (in the order of milliseconds). To realize such immersive services, the network transport, as well as the video representation and encoding, have to be fundamentally enhanced. The purpose of this tutorial article is to provide an elaborate introduction to the creation, streaming, and evaluation of immersive video. Moreover, it aims to provide lessons learned and to point at promising research paths to enable truly interactive immersive video applications toward holography.

Keywords—Immersive video delivery, 3DoF, 6DoF, omnidirectional video, volumetric video, point clouds, meshes, light fields, holography, end-to-end systems

J. van der Hooft, H. Amirpour, M. Torres Vega, Y. Sanchez, R. Schatz, T. Schierl, C. Timmerer, "A Tutorial on Immersive Video Delivery: From Omnidirectional Video to Holography," in IEEE Communications Surveys & Tutorials, vol. 25, no. 2, pp. 1336-1375, Secondquarter 2023, doi: 10.1109/COMST.2023.3263252.

Immersive Video Delivery: From Omnidirectional Video to Holography from Alpen-Adria-Universität

QUALINET White Paper on Definitions of Immersive Media Experience (IMEx)

QUALINET announces its recent White Paper on Definitions of Immersive Media Experience (IMEx).

It is online available here https://arxiv.org/abs/2007.07032 for free.

With the coming of age of virtual/augmented reality and interactive media, numerous definitions, frameworks, and models of immersion have emerged across different fields ranging from computer graphics to literary works. Immersion is oftentimes used interchangeably with presence as both concepts are closely related. However, there are noticeable interdisciplinary differences regarding definitions, scope, and constituents that are required to be addressed so that a coherent understanding of the concepts can be achieved. Such consensus is vital for paving the directionality of the future of immersive media experiences (IMEx) and all related matters.

The aim of this white paper is to provide a survey of definitions of immersion and presence which leads to a definition of immersive media experience (IMEx). The Quality of Experience (QoE) for immersive media is described by establishing a relationship between the concepts of QoE and IMEx followed by application areas of immersive media experience. Influencing factors on immersive media experience are elaborated as well as the assessment of immersive media experience. Finally, standardization activities related to IMEx are highlighted and the white paper is concluded with an outlook related to future developments. 

This White Paper is a contribution by QUALINET, the European Network on Quality of Experience in Multimedia Systems and Services (http://www.qualinet.eu/) to the discussions related to Immersive Media Experience (IMEx). It is motivated by the need for definitions around this term to foster a deeper understanding of ideas and concepts originating from multidisciplinary groups but with a joint interest in multimedia experiences. Thus, this white paper has been created mainly with such multimedia experiences in mind but may be also used beyond.

The QUALINET community aims at extending the notion of network-centric Quality of Service (QoS) in multimedia systems, by relying on the concept of Quality of Experience (QoE). The main scientific objective is the development of methodologies for subjective and objective quality metrics taking into account current and new trends in multimedia communication systems as witnessed by the appearance of new types of content and interactions. QUALINET (2010-2014 as COST Action IC1003) meets once a year collocated with QoMEX (http://qomex.org/) to coordinate its activities around 4 Working Groups (WGs): (i) research, (ii) standardization, (iii) training, and (iv) innovation.

List of Authors and Contributors

Andrew Perkis (andrew.perkis@ntnu.no, editor), Christian Timmerer (christian.timmerer@itec.uni-klu.ac.at, editor), Sabina Baraković, Jasmina Baraković Husić, Søren Bech, Sebastian Bosse, Jean Botev, Kjell Brunnström, Luis Cruz, Katrien De Moor, Andrea de Polo Saibanti, Wouter Durnez, Sebastian Egger-Lampl, Ulrich Engelke, Tiago H. Falk, Jesús Gutiérrez, Asim Hameed, Andrew Hines, Tanja Kojic, Dragan Kukolj, Eirini Liotou, Dragorad Milovanovic, Sebastian Möller, Niall Murray, Babak Naderi, Manuela Pereira, Stuart Perry, Antonio Pinheiro, Andres Pinilla, Alexander Raake, Sarvesh Rajesh Agrawal, Ulrich Reiter, Rafael Rodrigues, Raimund Schatz, Peter Schelkens, Steven Schmidt, Saeed Shafiee Sabet, Ashutosh Singla, Lea Skorin-Kapov, Mirko Suznjevic, Stefan Uhrig, Sara Vlahović, Jan-Niklas Voigt- Antons, Saman Zadtootaghaj.

How to reference this white paper

Perkis, A., Timmerer, C., et al., “QUALINET White Paper

on Definitions of Immersive Media Experience (IMEx)”, European Network on Quality of Experience in Multimedia Systems and Services, 14th QUALINET meeting (online), May 25, 2020.

Alternatively, you may export the citation from arXiv: https://arxiv.org/abs/2007.07032.

QoMEX'17 Review: Down the Rabbit Hole - Immersive Experience

During QoMEX 2017 in Erfurt, Germany we had a special session entitled "Down the Rabbit Hole" which I have introduced here already. The papers of the special session will appear soon in IEEEXplore but together with my co-organizers of this special session -- Raimund Schatz and Judith Redi -- we also wanted to run the special session in a special way. Therefore, we asked authors to prepare concise and thought-provoking paper presentations (~15min incl. Q&A -- paper title, presenter, picture, key words below) to save some time for a panel discussion. Surprisingly, it worked very well and the special session turned out to be worthwhile and informative. In order to keep the audience connected and involved we posted a single slide of all panelists (i.e., paper presenters) which was shown all the time (see below).

The discussion was centered around the question "what is your understanding of a fully immersive experience" which revealed interesting aspects and finally resulted in the main challenge how to quantify immersive experience. In this context, Mr. T. (only those who've been at QoMEX and in this session know why he is called Mr. T. -- join us next time and we will explain you what's behind) raised an interesting idea to interpret the Turing test for immersive experience. That is, fully or truly immersive experience is achieved if a human is no longer aware that she/he actually interacts with cyber-physical systems. I think this statement sets the bar (high) but definitely worth to consider.

Finally, I'd like to thank all presenters/panelists for an amazing special session at QoMEX'17 but the journey is not yet over. I'll be attending ACM MMSys and IEEE ICME presenting/discussing various aspects of immersive experiences; also at the MPEG meeting in Torino which will be dedicated to standardization aspects of immersive experiences.

Also big big thanks to the conference organizers, the team around the general chair Alexander Raake
(TU Ilmenau, Germany), for hosting such a wonderful event! Hope seeing you all next year for QoMEX 2018.

Feel free to test/play around with Bitmovin solutions for VR/360-degree streaming and if you have a RICOH THETA S check out my blog post how to setup a live streaming session.

Come and join us on the journey down the rabbit hole which eventually will lead to wonderland.