Monday, October 14, 2024

Happy World Standards Day 2024

As we celebrate World Standards Day, it's important to recognize the monumental advancements the MPEG community has made over the past year. These achievements continue to influence multimedia standards worldwide, playing a crucial role in ensuring seamless, high-quality digital experiences.

  1. ISO Base Media File Format (8th Edition): This standard has been pivotal for media streaming applications, particularly for formats like DASH (Dynamic Adaptive Streaming over HTTP) and CMAF (Common Media Application Format). The latest update facilitates more seamless media switching and continuous presentation, optimizing the user experience across different devices.
  2. Neural Network Compression (2nd Edition): With AI technologies rapidly evolving, MPEG's neural network compression standard addresses the need for efficient storage and inference in multimedia systems. The second edition enhances reference software, providing robust tools for handling complex neural networks in applications such as image and video processing.
  3. Low Latency, Low Complexity LiDAR Coding: As industries like autonomous vehicles and smart cities expand, this standard addresses the need for efficient and real-time processing of LiDAR data. The MPEG community has developed compression techniques that maintain low latency and complexity, enabling faster decision-making for autonomous systems.
  4. MPEG-DASH (6th Edition): The 6th edition of MPEG-DASH brings exciting improvements in adaptive streaming. Key updates include support for new CMCD parameters for better content management and a background mode that allows players to receive updates without disrupting media playback. These advancements significantly enhance streaming efficiency and flexibility.
  5. Video Coding for Machines (VCM): A significant addition this year has been the introduction of Video Coding for Machines. This emerging standard focuses on machine vision applications, where efficient encoding and decoding are crucial for machine learning tasks such as object detection and recognition. This innovation caters to the increasing integration of machine-based analytics in multimedia systems.
  6. Immersive Media and Volumetric Video: MPEG’s work on volumetric video coding and standards for immersive media continues to push the boundaries of AR/VR technologies. This ensures that immersive content can be delivered across various platforms with improved consistency and performance.

These highlights exemplify MPEG's commitment to fostering innovation through multimedia standards, shaping the future of digital content. On this World Standards Day, let’s celebrate the efforts that keep the digital ecosystem thriving!

Friday, September 27, 2024

ACM Mile-High Video Conference 2025: Call for Contributions


MHV 2025: ACM Mile-High Video Conference 2025
Call for Contributions
February 18-20, 2025, The Cable Center, Denver, Colorado
https://www.mile-high.video/

Monday, September 16, 2024

MPEG news: a report from the 147th meeting

This blog post is based on the MPEG press release and has been modified/updated here to focus on and highlight research aspects. This version of the blog post will also be posted at ACM SIGMM Records.

The 147th MPEG meeting was held in Sapporo, Japan from 15-19 July 2024, and the official press release can be found here. It comprises the following highlights:
  • ISO Base Media File Format*: The 8th edition was promoted to Final Draft International Standard, supporting seamless media presentation for DASH and CMAF.
  • Syntactic Description Language: Finalized as an independent standard for MPEG-4 syntax.
  • Low-Overhead Image File Format*: First milestone achieved for small image handling improvements.
  • Neural Network Compression*: Second edition for conformance and reference software promoted.
  • Internet of Media Things (IoMT): Progress made on reference software for distributed media tasks.
* … covered in this blog post and expanded with possible research aspects.

8th edition of ISO Base Media File Format

The ever-growing expansion of the ISO/IEC 14496-12 ISO base media file format (ISOBMFF) application area has continuously brought new technologies to the standards. During the last couple of years, MPEG Systems (WG 3) has received new technologies on ISOBMFF for more seamless support of ISO/IEC 23009 Dynamic Adaptive Streaming over HTTP (DASH) and ISO/IEC 23000-19 Common Media Application Format (CMAF) leading to the development of the 8th edition of ISO/IEC14496-12.

The new edition of the standard includes new technologies to explicitly indicate the set of tracks representing various versions of the media presentation of a single media for seamless switching and continuous presentation. Such technologies will enable more efficient processing of the ISOBMFF formatted files for DASH manifest or CMAF Fragments.

Research aspects: The central research aspect of the 8th edition of ISOBMFF, which “will enable more efficient processing,” will undoubtedly be its evaluation compared to the state-of-the-art. Standards typically define a format, but how to use it is left open to implementers. Therefore, the implementation is a crucial aspect and will allow for a comparison of performance. One such implementation of ISOBMFF is GPAC, which most likely will be among the first to implement these new features.

Low-Overhead Image File Format

ISO/IEC 23008-12 image format specification defines generic structures for storing image items and sequences based on ISO/IEC 14496-12 ISO base media file format (ISOBMFF). As it allows the use of various high-performance video compression standards for a single image or a series of images, it has been adopted by the market quickly. However, it was challenging to use it for very small-sized images such as icons or emojis. While the initial design of the standard was versatile and useful for a wide range of applications, the size of headers becomes an overhead for applications with tiny images. Thus, Amendment 3 of ISO/IEC 23008-12 low-overhead image file format aims to address this use case by adding a new compact box for storing metadata instead of the ‘Meta’ box to lower the size of the overhead.

Research aspects: The issue regarding header sizes of ISOBMFF for small files or low bitrate (in the case of video streaming) was known for some time. Therefore, amendments in these directions are appreciated while further performance evaluations are needed to confirm design choices made at this initial step of standardization.

Neural Network Compression

An increasing number of artificial intelligence applications based on artificial neural networks, such as edge-based multimedia content processing, content-adaptive video post-processing filters, or federated training, need to exchange updates of neural networks (e.g., after training on additional data or fine-tuning to specific content). For this purpose, MPEG developed a second edition of the standard for coding of neural networks for multimedia content description and analysis (NNC, ISO/IEC 15938-17, published in 2024), adding syntax for differential coding of neural network parameters as well as new coding tools. Trained models can be compressed to at least 10-20% for several architectures, even below 3%, of their original size without performance loss. Higher compression rates are possible at moderate performance degradation. In a distributed training scenario, a model update after a training iteration can be represented at 1% or less of the base model size on average without sacrificing the classification performance of the neural network.

In order to facilitate the implementation of the standard, the accompanying standard ISO/IEC 15938-18 has been updated to cover the second edition of ISO/IEC 15938-17. This standard provides a reference software for encoding and decoding NNC bitstreams, as well as a set of conformance guidelines and reference bitstreams for testing of decoder implementations. The software covers the functionalities of both editions of the standard, and can be configured to test different combinations of coding tools specified by the standard.

Research aspects: The reference software for NNC, together with the reference software for audio/video codecs, are vital tools for building complex multimedia systems and its (baseline) evaluation with respect to compression efficiency only (not speed). This is because reference software is usually designed for functionality (i.e., compression in this case) and not performance.

The 148th MPEG meeting will be held in Kemer, Türkiye, from November 04-08, 2024. Click here for more information about MPEG meetings and their developments.

Wednesday, August 7, 2024

University assistant predoctoral (all genders welcome) (in German: Universitätsassistent:in)

The University of Klagenfurt, with approximately 1,500 employees and over 12,000 students, is located in the Alps-Adriatic region and consistently achieves excellent placements in rankings. The motto “per aspera ad astra” underscores our firm commitment to the pursuit of excellence in all research, teaching, and university management activities. The principles of equality, diversity, health, sustainability, and compatibility of work and family life serve as the foundation for our work at the university.

The University of Klagenfurt is pleased to announce the following open position at the Department of Information Technology at the Faculty of Technical Sciences with an expected starting date of November 4, 2024:

University assistant predoctoral (all genders welcome) (in German: Universitätsassistent:in)

within the Ada Lovelace Programme (project title: Streaming of Holographic Content and its Impact on the Quality of Experience).

  • Level of employment: 100 % (40 hours/week)
  • Minimum salary: € 50,103.20 per annum (gross); Classification according to collective agreement: B1 
  • Contract duration: 4 years
  • Application deadline: by September 11, 2024
  • Reference code: 348/24

Tasks and responsibilities:

  • Autonomous scientific work, including the publication of research articles in the fields of coding and streaming of holographic content, Quality of Experience (QoE), and behavioural sciences
  • Conducting independent scientific research with the aim of submitting a dissertation and acquiring a doctoral degree in technical sciences
  • Teaching exercises and lab courses (e.g., in the computer science Bachelor’s or/and Master’s programme)
  • Participating in research projects of the department, especially within the Ada Lovelace Programme (Streaming of Holographic Content and its Impact on the Quality of Experience)
  • Mentoring students
  • Assisting in public relations activities, science to public communication, and extra-curricular events of the department and the faculty

Prerequisites for the appointment:

  • Completed Diploma or Master’s degree from a recognized university in the field of computer science, information and communications engineering, electrical engineering, or related fields. The completion of this degree must be fulfilled no later than two weeks before the starting date; hence, the last possible deadline for meeting this requirement is October 20, 2024
  • Strong background in one or more of the following fields: multimedia systems (i.e., video/holographic content coding/streaming, Quality of Experience) and empirical research methods (i.e., statistical methods, interdisciplinary research with behavioural sciences)
  • Fluent in written and spoken English
  • Programming experience in multimedia systems

Additional desired qualifications:

  • Experience with scientific publications or presentations
  • Experience in interdisciplinary research projects, ideally in the behavioural sciences, as the project involves empirical research
  • Excellent ability to work with teams
  • Scientific curiosity and enthusiasm for research in multimedia systems and empirical research
The doctoral student will be co-supervised by Christian Timmerer, Heather Foran, and Hadi Amirpour.

Our offer:

This position serves the purposes of the vocational and scientific education of graduates of Master’s or Diploma degree programmes and sets the goal of completing a Doctoral degree / a Ph.D. in Technical Sciences. Therefore, applications by persons who have already completed a subject-specific doctoral degree or a subject-relevant Ph.D. program cannot be considered. 

The employment contract is concluded for the position of university assistant (predoctoral) and stipulates a starting salary of € 3,578.80 gross per month (14 times a year; previous experience deemed relevant to the job can be recognized in accordance with the collective agreement). 

The University of Klagenfurt also offers:

  • Personal and professional advanced training courses, management, and career coaching
  • Numerous attractive additional benefits, see also https://jobs.aau.at/en/the-university-as-employer/
  • Diversity- and family-friendly university culture
  • The opportunity to live and work in the attractive Alps-Adriatic region with a wide range of leisure activities in the spheres of culture, nature, and sports

The application:

If you are interested in this position, please apply in English by providing the following documents:

  • Letter of application/cover letter including motivation statement for the given position
  • Curriculum vitae (with clear information about the degrees, including date/place/grade, the experience acquired, the thesis title, the list of publications (if any), and any other relevant information)
  • Copy of the degree certificates and transcripts of the courses
  • Any certificates that can prove the fulfilment of the required and additional qualifications listed above (e.g., the submission of the final thesis if required by the degree programme, copy of publications, programming skills certificates, language skills certificates, etc.)
  • Final thesis or other study-related written work (like seminar reports) or excerpts thereof
  • If an applicant has not received the Diploma or Master’s degree by the application deadline, the applicant should provide a declaration, written either by a supervisor or by the candidate themselves, on the feasibility of finishing the Diploma or Master’s degree by October 30, 2024 at the latest. 

To apply, please select the position with the reference code 348/24 in the category “Scientific Staff” using the link “Apply for this position” in the job portal at jobs.aau.at/en/.

Candidates must furnish proof that they meet the required qualifications by October 20, 2024 at the latest.

For further information on this specific vacancy, please contact Univ.-Prof. DI Dr. Christian Timmerer (christian.timmerer@aau.at). General information about the university as an employer can be found at https://jobs.aau.at/en/the-university-as-employer/. At the University of Klagenfurt, recruitment and staff matters are accompanied not only by the authority responsible for the recruitment procedure but also by the Equal Opportunities Working Group and, if necessary, by the Representative for Disabled Persons.

The University of Klagenfurt aims to increase the proportion of women and, therefore, invites explicitly qualified women to apply for the position. Where the qualification is equivalent, women will be given preferential consideration. 

People with disabilities or chronic diseases, who fulfill the requirements, are particularly encouraged to apply. 

Travel and accommodation costs incurred during the application process will not be refunded. Translations into other languages shall serve informational purposes only. Solely the version advertised in the University Bulletin (Mitteilungsblatt) shall be legally binding.

Monday, July 15, 2024

Successful 5-year Evaluation of Christian Doppler Laboratory ATHENA

The Christian Doppler (CD) Laboratory ATHENA was established in October 2019 to tackle current and future research and deployment challenges of HTTP Adaptive Streaming (HAS) and emerging streaming methods. The goal of CD laboratories is to conduct application-oriented basic research, promote collaboration between universities and companies, and facilitate technology transfer. They are funded through a public-private partnership between companies and the Christian Doppler Research Association, which is funded by the Ministry for Digital and Economic Affairs and the National Foundation for Research, Technology, and Development (Nationalstiftung für Forschung, Technologie und Entwicklung (FTE)). ATHENA is supported by Bitmovin as a company partner.

The CD laboratories have a duration of seven years and undergo rigorous scientific review after two and five years. This spring, the CD lab ATHENA completed its 5-year evaluation, and we have just received official notification from the CDG that we have successfully passed the review. Consequently, it is time to briefly outline the main achievements during this second phase (i.e., years 2 to 5) of the CD lab ATHENA.

Before exploring the achievements, it’s important to highlight the ongoing relevance of research in video streaming, given its dominance in today’s Internet usage. The January 2024 Sandvine Internet Phenomena report revealed that video streaming accounts for 68% of fixed/wired Internet traffic and 64% for mobile Internet traffic. Specifically, Video on Demand (VoD) represents 54% of fixed/wired and 57% of mobile traffic, while live streaming contributes to 14% of fixed/wired and 7% of mobile traffic. The major services in this domain include YouTube and Netflix, each commanding more than 10% of the overall Internet traffic, with TikTok, Amazon Prime, and Disney+ also playing significant roles.

ATHENA is structured into four work packages, each with distinct objectives as detailed below:

  1. Content provisioning: Primarily involves video encoding for HAS, quality-aware encoding, learning-based encoding, and multi-codec HAS.
  2. Content delivery: Addresses HAS issues by utilizing edge computing, exchanging information between CDN/SDN and clients, providing network assistance for clients, and evaluating corresponding utilities.
  3. Content consumption: Focuses on bitrate adaptation schemes, playback improvements, context and user awareness, and studies on Quality of Experience (QoE).
  4. End-to-end aspects: Offers a comprehensive view of application and transport layer enhancements, Quality of Experience (QoE) models, low-latency HAS, and learning-based HAS.

During the 2nd phase of ATHENA’s work, we achieved significant results, including publications in respected academic journals and conferences. Specifically, our publications were featured in key multimediasignal processingcomputer networks & wireless communication, and computing systems venues, as categorized by Google Scholar under engineering and computer science. Some of the notable publications include IEEE Communications Surveys & Tutorials (impact factor: 35.6), IEEE Transactions on Image Processing (10.6), IEEE Internet of Things Journal (10.6), IEEE Transactions on Circuits and Systems for Video Technology (8.4), and IEEE Transactions on Multimedia (7.3).

Furthermore, we focused on technology transfer by submitting 16 invention disclosures, resulting in 13 patent applications (including provisionals). Collaborating with our company partner, we obtained 6 granted patents. Additionally, we’re pleased to report on the progress of our spin-off projects, as well as the funding secured for two FFG-funded projects named APOLLO and GAIA, and an EU Horizon Europe-funded innovation action called SPIRIT.

The ATHENA team was also active in organizing scientific events such as workshops, special sessions, and special issues at IEEE ICME, ACM MM, ACM MMSys, ACM CoNEXT, IEEE ICIP, PCS, and IEEE Network. We also contributed to reproducibility in research through open source tools (e.g., Video Complexity Analyzer and LLL-CAdViSE) and datasets (e.g., Video Complexity Dataset and Multi-Codec Ultra High Definition 8K MPEG-DASH Dataset) among others.

We also note our contributions to the applications of AI in video coding & streaming, for example in video coding and video streaming as follows:

A major outcome of the second phase is the successful defense of the inaugural cohort of PhD students:

Two postdoctoral scholars have reached a significant milestone on their path toward habilitation

During the second phase, each work package produced excellent publications in their domain, briefly highlighted in the following. Content provisioning (WP-1) focuses mainly on video coding for HAS (43 papers) and immersive media coding for streaming (4 papers). The former can be further subdivided into the following topic areas:

  • Video complexity: spatial and temporal feature extraction (4 papers)
  • Compression efficiency improvement of individual representations (1 paper)
  • Encoding parameter prediction for HAS (9 papers)
  • Efficient bitrate ladder construction (4 papers)
  • Fast multi-rate encoding (3 papers)
  • Data security and data hiding (7 papers)
  • Energy-efficient video encoding for HAS (4 papers)
  • Advancing video quality evaluation (7 papers)
  • Datasets (4 papers)

Content delivery (WP-2) dealt with SDN/CDN assistance for HAS, edge computing support for HAS, and network-embedded media streaming support, resulting in 21 papers. Content consumption (WP-3) worked on QoE enhancement mechanisms at client-side and QoE- and energy-aware content consumption (11 papers). Finally, end-to-end Aspects (WP-4) produced 15 papers in the area of end-to-end QoE improvement in multimedia video streaming. We reported 94 papers published/accepted for the ATHENA 5-year evaluation.

In this context, it is also important to highlight the collaboration within ATHENA, which has resulted in joint publications across various work packages (WPs) and with other ITEC members. For example, collaborations with Prof. Schöffmann (FWF-funded project OVID), FFG-funded projects APOLLO/GAIA, and EU-funded project SPIRIT. In addition, we would like to acknowledge our international collaborators, such as Prof. Hongjie He from Southwest Jiaotong University, Prof. Patrick Le Callet from the University of Nantes, Prof. Wassim Hamidouche from the Technology Innovation Institute (UAE), Dr. Sergey Gorinsky from IMDEA, Dr. Abdelhak Bentaleb from Concordia University, Dr. Raimund Schatz from AIT, and Prof. Pablo Cesar from CWI. We are also pleased to report the successful technology transfers to Bitmovin, particularly CAdViSE (WP-4) and WISH ABR (WP-3). Regular “Fun with ATHENA” meetups and Break-out Groups are utilized for in-depth discussions about innovations and potential technology transfers.

Over the next two years, the ATHENA project will prioritize the development of deep neural network/AI-based image and video coding within the context of HAS. This includes energy- and cost-aware video coding for HAS, immersive video coding such as volumetric video and holography, as well as Quality of Experience (QoE) and energy-aware content consumption for HAS (including energy-efficient, AI-based live video streaming) and generative AI for HAS.

Thanks to all current and former ATHENA team members: Samira Afzal, Hadi Amirpour, Jesús Aguilar Armijo, Emanuele Artioli, Christian Bauer, Alexis Boniface, Ekrem Çetinkaya, Reza Ebrahimi, Alireza Erfanian, Reza Farahani, Mohammad Ghanbari (late), Milad Ghanbari, Mohammad Ghasempour, Selina Zoë Haack, Hermann Hellwagner, Manuel Hoi, Andreas Kogler, Gregor Lammer, Armin Lachini, David Langmeier, Sandro Linder, Daniele Lorenzi, Vignesh V Menon, Minh Nguyen, Engin Orhan, Lingfeng Qu, Jameson Steiner, Nina Stiller, Babak Taraghi, Farzad Tashtarian, Yuan Yuan, and Yiying Wei. Finally, thanks to ITEC support staff Martina Steinbacher, Nina Stiller, Margit Letter, Marion Taschwer, and Rudolf Messner.

We also would like to thank the Christian Doppler Research Association for continuous support, organizing the review, and the reviewer for constructive feedback!

Monday, July 1, 2024

HTTP Adaptive Streaming – Quo Vadis? (2024)

Telecom Seminar Series at TII, Jun 27, 2024, 04:00 PM Dubai

Abstract: Video traffic on the Internet is constantly growing; networked multimedia applications consume a predominant share of the available Internet bandwidth. A major technical breakthrough and enabler in multimedia systems research and of industrial networked multimedia services certainly was the HTTP Adaptive Streaming (HAS) technique. This resulted in the standardization of MPEG Dynamic Adaptive Streaming over HTTP (MPEG-DASH) which, together with HTTP Live Streaming (HLS), is widely used for multimedia delivery in today’s networks. Existing challenges in multimedia systems research deal with the trade-off between (i) the ever-increasing content complexity, (ii) various requirements with respect to time (most importantly, latency), and (iii) quality of experience (QoE). Optimizing towards one aspect usually negatively impacts at least one of the other two aspects if not both. This situation sets the stage for our research work in the ATHENA Christian Doppler (CD) Laboratory (Adaptive Streaming over HTTP and Emerging Networked Multimedia Services; https://athena.itec.aau.at/), jointly funded by public sources and industry. In this talk, we will present selected novel approaches and research results of the first year of the ATHENA CD Lab’s operation. We will highlight HAS-related research on (i) multimedia content provisioning (machine learning for video encoding); (ii) multimedia content delivery (support of edge processing and virtualized network functions for video networking); (iii) multimedia content consumption and end-to-end aspects (player-triggered segment retransmissions to improve video playout quality); and (iv) novel QoE investigations (adaptive point cloud streaming). We will also put the work into the context of international multimedia systems research. 

Thursday, June 6, 2024

Video Streaming: Then, Now, Future

I'm happy to share my slides from my public/inaugural lecture at the University of Klagenfurt on June 5, 2022.

  • Title: "Video Streaming: Then, Now, Future"
  • June 5, 2024, 17:00, University of Klagenfurt, Hörsaal 2
In my public lecture, I provide insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. I'm also presenting provocative contributions of his own that have significantly influenced the industry. I conclude by looking at future challenges and invite the audience to join in a discussion (e.g., in the comments below).