Colloquium on Quality of Experience in Multimedia Systems and Services

November 23, 2012, 09:00-13:00

Alpen-Adria-Universität Klagenfurt, Austria
Seminar Room of Lakeside Labs

Chair: Christian Timmerer

Multimedia is a ubiquitous part of modern life; content is delivered to a wide variety of devices both fixed and mobile, the consumption environment can vary dramatically, and users increasingly demand fulfilling interaction with their media. Evaluation, both objective and subjective, of separate audio, video, or image content is well established, but there are substantial challenges in assessing users’ experiences of mixed content. This is exacerbated by the diversity of devices, interfaces, consumption environments, and content prevalent in today’s market. Increasingly, evaluation approaches must include psychological, perceptual, and contextual dimensions to truly represent and measure the users’ experiences. Understanding and modelling how users react to and perceive content and applications will inform and improve new applications and lead to better user experiences.

In this context the Förderverein Technische Fakultät and COST Action IC1003 QUALINET invites interested parties, specifically researchers, practitioners, and students to the Colloquium on Quality of Experience in Multimedia Systems and Services. Additionally, the colloquium will hold as part of the research topic "Human Centered Computing and Design" established at he Alpen-Adria-Universität Klagenfurt.

We are happy to announce the program as follows with distinguished speakers who are all in leading positions of the COST Action IC1003 QUALINET.

Program

09:00-09:15: Welcome notice and introduction

09:15-10:00: Keynote on "Quality of Experience: Past, Present and Future trends" by Touradj Ebrahimi, EPFL

10:00-10:30: Quality Prediction for Speech-based Telecommunication Services by Sebastian Möller, Quality and usability lab, Telekom Innovation Labs, TU Berlin

10:30-11:00: Quality of Experience of 3DTV and Beyond by Patrick Le Callet, Univ. of Nantes

11:00-11:15: Coffee break

11:15-11:45: Influence of Chromatic Information on QoE by Antonio Pinheiro, U.B.I., Portugal

11:45-12:15: Bitstream and hybrid-based video quality assessment for IPTV monitoring by Savvas Argyropoulos, Telekom Innovation Labs, TU Berlin

12:15-12:45: Machine Learning for objective QoE assessment: Science, Myths and a look to the future by Judith Redi, TU Delft.

12:45-13:15: Overview of Qualinet multimedia databases by Karel Fliegel, Czech Technical University in Prague

13:15: Closing and lunch

Registration
The registration to this colloquium is free of charge and includes workshop attendance, coffee break, and lunch. However, you are required to send an email to Melissa.Aichholzer@aau.at (cc: christian.timmerer@itec.aau.at).

Location
Lakeside Labs with detailed location information incl. nearby hotels here. Additionally, a Google map with the most important point of interests can be found here.

Related links:

• Förderverein Technische Fakultät
• COST Action IC1003 QUALINET
• 5th International Workshop on Quality of Multimedia Experience 2013
• Alpen-Adria-Universität Klagenfurt
• Lakeside Labs

Abstracts and biographies of distinguished speakers

Titel: Quality of Experience : Past, Present and Future Trends

Presenter: Touradj Ebrahimi - EPFL

Abstract: With recent progress in networks, devices and interfaces, the traditional ways media has been produced, distributed and consumed have gone through major changes. This paradigm shift is expected to continue, giving birth to new products, services and business models, well beyond those implemented or even imagined today. One constant in all this change is the continuing pursuit of higher, richer and better user experience. Quality of Experience is believed by many to provide the right framework to achieve this goal. 

This talk will start by providing a historical perspective of quality and how it has been evaluated in conventional and even current multimedia systems. We will then discuss different definitions of Quality of Experience and some of the models based on such definitions. The talk will then provide a number of illustrative examples of QoE definitions and models in concrete situations, namely, mobile 3D and mobile media consumption. We will then conclude with a discussion on majors trends and challenges in QoE. 

CV: Touradj Ebrahimi is currently Professor at EPFL heading its Multimedia Signal Processing Group. He is also adjunct Professor with the Center of Quantifiable Quality of Service at Norwegian University of Science and Technology (NTNU). In 2001, he was the first to suggest the notion of Quality of Experience (QoE) be used in multimedia communication systems, and since has produced more than 50 publications on this topic. Prof. Ebrahimi is a co-founder of the International Workshop on Quality of Multimedia Experience (QoMEX), and serves as the chair of the European COST Action IC1003 (Qualinet), on quality of experience in multimedia systems and services. His other research interests include still, moving, and 3D image processing and coding, visual information security (rights protection, watermarking, authentication, data integrity, steganography), new media, and human computer interfaces (smart vision, brain computer interface). He is the author or the co-author of more than 200 research publications, and holds 14 patents. Prof. Ebrahimi is a member of IEEE, SPIE, ACM and IS&T.

Titel: Quality Prediction for Speech-based Telecommunication Services

Presenter: Sebastian Möller, Quality and usability lab, Telekom Innovation Labs, TU Berlin

Abstract: Speech-based services come in an ever increasing variety, and it becomes more and more difficult for system designers and service operators to ensure a sufficiently high quality, usability, and finally acceptance of these services. Whereas the ultimate way to measure quality is by asking test users, such empirical tests require a substantial effort in terms of time and money, and are thus commonly limited to an absolute minimum. One way to cope with this problem is to estimate the quality which will be perceived by the user on the basis of physically measurable quantities, such as speech signals, or parameters describing a spoken interaction.

In this talk, we will present approaches for prediction the quality and usability of speech-based services taking three common application examples: (1) Speech transmission services where overall quality as well as individual quality dimensions can be estimated from transmitted and reference speech signals; (2) synthesized-speech services where no reference is available but can be built artificially with the help of prior knowledge; as well as (3) spoken dialogue services where user behavior needs to be predicted, with the help of probabilities and rules. Small examples will underline the usefulness but also the limitations of the currently available approaches.

CV: Sebastian Möller was born in 1968 and studied electrical engineering at the universities of Bochum (Germany), Orléans (France) and Bologna (Italy). From 1994 to 2005, he held the position of a scientific researcher at the Institute of Communication Acoustics (IKA), Ruhr-University Bochum, and worked on speech signal processing, speech technology, communication acoustics, as well as on speech communication quality aspects. Since June 2005, he works at Deutsche Telekom Laboratories, TU Berlin. He was appointed Professor at TU Berlin for the subject "Quality and Usability" in April 2007, and heads the "Quality and Usability Lab" at Telekom Innovation Laboratories.

He received a Doctor-of-Engineering degree at Ruhr-University Bochum in 1999 for his work on the assessment and prediction of speech quality in telecommunications. In 2000, he was a guest scientist at the Institut dalle Molle d'Intélligence Artificielle Perceptive (IDIAP) in Martigny (Switzerland) where he worked on the quality of speech recognition systems. He gained the qualification needed to be a professor (venia legendi) at the Faculty of Electrical Engineering and Information Technology at Ruhr-University Bochum in 2004, with a book on the quality of telephone-based spoken dialogue systems. In September 2008, we worked as a Visiting Fellow at MARCS Auditory Laboratories, University of Western Sydney (Australia) on the evaluation of avatars. In November 2011, he was Visiting Professor at the Universidad de Granada (Spain), and from Februar to April 2012 Visiting Professor at the Ben Gurion University of the Negev in Be'er Sheva (Israel). He is currently Adjunct Professor at the University of Canberra (Australia). His most recent book on "Quality Engineering" was published in 2010.

Sebastian Möller was awarded the GEERS prize in 1998 for his interdisciplinary work on the analysis of infant cries for early hearing-impairment detection, the ITG prize of the German Association for Electrical, Electronic & Information Technologies (VDE) in 2001, the Lothar-Cremer prize of the German Acoustical Association (DEGA) in 2003, a Heisenberg fellowship of the German Research Foundation (DFG) in 2005, and the Johann Philipp Reis prize in 2009. Since 1997, he has taken part in the standardization activities of the International Telecommunication Union (ITU-T) on transmission performance of telephone networks and terminals. He is currently acting as a Rapporteur for question Q.8/12.

Titel: Quality of Experience of 3DTV and Beyond

Presenter: Patrick Le Callet, Univ. of Nantes

Abstract: S-3DTV (Stereoscopic-3DTV), HDR (High Dynamic range), 4K are recent considered technologies to improve the viewer experience. Nevertheless, such technologies are really useful only if properly tuned to increase the Quality of Experience (QoE). For instance, S-3DTV (stereoscopic-3DTV) is supposed to provide enhanced depth perception to viewer but might affect other factors of user experience. Subjective assessment is the most direct way to assess quality of experience (QoE). However, conventional assessment methods are not sufficient to evaluate the QoE of S-3DTV. This talk aims to review the challenge of QoE assessment through use cases related to S-3DTV and HDR. Methodologies to evaluate QoE with observers will be discussed, including environment conditions, scales and display characterization.

CV: Patrick LE CALLET is full professor at Ecole polytechnique de l’université de Nantes (Engineer School) in the Electrical Engineering and the Computer Science department where. Since 2006, he is the head of the Image and VideoCommunication lab at CNRS IRCCyN, a group of more than 35 researchers. He is mostly engaged in research dealing with the application of human vision modeling in image and video processing. His current centers of interest are 3D image and video quality assessment, watermarking techniques and visual attention modeling and applications. He is co-author of more than 150 publications and communications and co-inventor of 13 international patents on these topics. He has coordinated and is currently managing for IRCCyN several National or European collaborative research programs representing grants of more than 2,5 million euros. He is serving in VQEG (Video Quality Expert Group) where is co-chairing the "HDR Group" and "3DTV" activities. He is currently serving as associate editor for IEEE transactions on Circuit System and Video Technology, SPIE Journal of Electronic Imaging and SPRINGER EURASIP Journal on Image and Video Processing.

Title: Influence of Chromatic Information on QoE

Presenter: Antonio Pinheiro, U.B.I., Portugal

Abstract: The low sensibility revealed by humans to the chromatic information is well known since a long time. That fact was always explored in color multimedia systems! In particular, the early analog color TV systems always had a very reduced amount of chromatic information compared with the amount of luminance information. In this talk, an evaluation of the influence of the chromatic information on QoE of still images is presented. For that, a set of images used for testing are represented in the CIE 1976 (L*a*b*) color space because it is partially uniform and highly related with the human perceptual notion of color. After clustering the image colors, a random error of known magnitude is given to each color cluster on the chromatic components a* and b*, keeping L* constant. With this method images are not degraded by artefacts induced by color discontinuities. A calibrated true color system is used for evaluation of the individuals’ sensibility to color changes. Using the Medium Opinion Score the sensibility to color changes was evaluated and further modelled. As expected, a very low sensibility to chromatic changes was revealed. Finally, objective models for chromatic errors are discussed and the relation between the quality and the emotional state of the subjects is considered.

CV: António M. G. Pinheiro is a lecture at U.B.I. (University of Beira Interior), Covilha, Portugal. After graduated in IST, Portugal in 1988 he worked as Assistant Lecture in U.B.I. In 1998 joined the University of Essex, U.K., where, he received the PhD in Electronic Systems Engineering in 2002 under the supervision of Prof. Mohammed Ghanbari. He is a Portuguese representative in the European Union Action COST IC1003, QUALINET – European Network on Quality of Experience in Multimedia Systems and Services and was a Portuguese representative of COST – 292, Semantic Multimodal Analysis of Digital Media. He is also the coordinator of the research Group Electronic and Computer Systems of the Remote Sensing Unit. Antonio Pinheiro has lectured several courses on Signal and Image Processing and on Digital Systems Design and Microprocessors Architectures and Applications.

Title: Bitstream and hybrid-based video quality assessment for IPTV monitoring

Presenter: Savvas Argyropoulos, Telekom Innovation Labs, TU Berlin

Abstract: The proliferation of IPTV services during the last years has necessitated the development of objective video quality assessment methods. It has become evident that network parameters which define the Quality of Service (Qos) are not sufficient to estimate the service quality perceived by the user, typically referred to as Quality of Experience (QoE). This talk will present an objective no-reference bitstream and hybrid video quality assessment method and provide details on the algorithms which are employed to evaluate different types of degradations, such as compression artefacts, packet loss impairments, playback freezing and re-buffering .

CV: Savvas Argyropoulos received the Diploma and Ph.D. degrees in electrical and computer engineering from the Aristotle University of Thessaloniki (AUTH), Thessaloniki, Hellas, in 2004 and 2008, respectively. He is currently a research scientist at Deutsche Telekom Innovation Laboratories (T-Labs), in Berlin, Germany. His research interests include video quality assessment, video coding and transmission, distributed source coding, and perceptual video coding.

Title: Machine Learning for objective QoE assessment: Science, Myths and a look to the future

Presenter: Judith Redi, TU Delft

Abstract: Machine learning has been recently shown to be a very promising tool to support automated QoE assessment. Its ability to mimic highly non-linear, complex phenomena, such as user experience and quality judgment, is extremely appealing for the implementation of on-line, accurate QoE control systems. Nevertheless, applying Machine Learning to QoE is an high risk, high gain approach: if misused, it can lead to poorly flexible and unreliable systems. Key to the attainment of all the gain without risks is a profound understanding of the advantages and limitations that characterize learning machines. Even more important is a strong knowledge of the phenomenon to be mimicked, that is, the user experience.

In this talk, theoretical background, applicative tips and practical examples will be reviewed, with the aim of drawing guidelines for the successful application of Machine Learning to objective QoE assessment.

CV: Judith Redi is Assistant Professor at Delft University of Technology, department of Intelligent Systems, since 2010. She obtained her PhD from the University of Genoa (Italy) in 2010, with a thesis on learning machines for objective image quality assessment, final result of a project on visual quality in displays funded by Philips research. After receiving the award for the best ICT thesis from University of Genoa, she worked as a Post-Doc at Eurecom (France) focusing on image analysis and computer vision. At TU Delft, she works on image and video understanding towards the maximization of the quality of multimedia experiences, for which she was awarded an NWO Veni grant in 2012. She is coordinator of the Qualinet (COST IC1003) Industrial Forum and Management Committee member for the Qualinet COST action.

Title: Overview of Qualinet multimedia databases

Presenter: Karel Fliegel, Czech Technical University in Prague

Abstract: Success of current and future developments in QoE largely depends on availability of proper multimedia content. One of the goals within the interests of the COST Action IC1003 QUALINET and its Working Group 4 (Databases and Validation) is to create rich and internationally recognized database of content of different sorts and take the steps to make it accessible to all researchers. In this talk we will give an overview of current status of this effort with brief description of currently available databases and explanation on how these databases can be used for QoE evaluations and benchmarking. Moreover, we will focus on the Qualinet Databases online tool as Qualinet’s main resource for sharing of the datasets among Qualinet members and scientific community.

CV: Karel Fliegel (born in 1979) received his Ing. (M.Sc.) degree in electrical engineering, specialization in audiovisual technology, from the Czech Technical University in Prague, Czech Republic in 2004 and Ph.D. from the same university in 2011. Currently he holds a position of assistant professor in the Multimedia Technology Group (MMTG) at the CTU in Prague. His research interests include image processing, imaging systems, image and video compression, QoE in multimedia systems. He is a deputy leader in WG 4 (Databases and Validation) of COST Action IC1003 QUALINET.