MPEG news: a report from the 149th meeting

This blog post is based on the MPEG press release and has been modified/updated here to focus on and highlight research aspects. This version of the blog post will also be posted at ACM SIGMM Records.

MPEG News Archive

The 149th MPEG meeting took place in Geneva, Switzerland, from January 20 to 24, 2025. The official press release can be found here. MPEG promoted three standards (among others) to Final Draft International Standard (FDIS), driving innovation in next-generation, immersive audio and video coding, and adaptive streaming:

• MPEG-I Immersive Audio enables realistic 3D audio with six degrees of freedom (6DoF).
• MPEG Immersive Video (Second Edition) introduces advanced coding tools for volumetric video.
• MPEG-DASH (Sixth Edition) enhances low-latency streaming, content steering, and interactive media.

This blog post focuses on these new standards/editions based on the press release and amended with research aspect relevant for the ACM SIGMM community.

MPEG-I Immersive Audio

At the 149th MPEG meeting, MPEG Audio Coding (WG 6) promoted ISO/IEC 23090-4 MPEG-I immersive audio to Final Draft International Standard (FDIS), marking a major milestone in the development of next-generation audio technology.

MPEG-I immersive audio is a groundbreaking standard designed for the compact and highly realistic representation of spatial sound. Tailored for Metaverse applications, including Virtual, Augmented, and Mixed Reality (VR/AR/MR), it enables seamless real-time rendering of interactive 3D audio with six degrees of freedom (6DoF). Users can not only turn their heads in any direction (pitch/yaw/roll) but also move freely through virtual environments (x/y/z), creating an unparalleled sense of immersion.

True to MPEG’s legacy, this standard is optimized for efficient distribution – even over networks with severe bitrate constraints. Unlike proprietary VR/AR audio solutions, MPEG-I Immersive Audio ensures broad interoperability, long-term stability, and suitability for both streaming and downloadable content. It also natively integrates MPEG-H 3D Audio for high-quality compression.

The standard models a wide range of real-world acoustic effects to enhance realism. It captures detailed sound source properties (e.g., level, point sources, extended sources, directivity characteristics, and Doppler effects) as well as complex environmental interactions (e.g., reflections, reverberation, diffraction, and both total and partial occlusion). Additionally, it supports diverse acoustic environments, including outdoor spaces, multiroom scenes with connecting portals, and areas with dynamic openings such as doors and windows. Its rendering engine balances computational efficiency with high-quality output, making it suitable for a variety of applications.

Further reinforcing its impact, the upcoming ISO/IEC 23090-34 Immersive audio reference software will fully implement MPEG-I immersive audio in a real-time framework. This interactive 6DoF experience will facilitate industry adoption and accelerate innovation in immersive audio. The reference software is expected to reach FDIS status by April 2025.

With MPEG-I immersive audio, MPEG continues to set the standard for the future of interactive and spatial audio, paving the way for more immersive digital experiences.

Research aspects: Research can focus on optimizing the streaming and compression of MPEG-I immersive audio for constrained networks, ensuring efficient delivery without compromising spatial accuracy. Another key area is improving real-time 6DoF audio rendering by balancing computational efficiency and perceptual realism, particularly in modeling complex acoustic effects like occlusions, reflections, and Doppler shifts for interactive VR/AR/MR applications.

MPEG Immersive Video (Second Edition)

At the 149th MPEG meeting, MPEG Video Coding (WG 4) advanced the second edition of ISO/IEC 23090-12 MPEG immersive video (MIV) to Final Draft International Standard (FDIS), marking a significant step forward in immersive video technology.

MIV enables the efficient compression, storage, and distribution of immersive video content, where multiple real or virtual cameras capture a 3D scene. Designed for next-generation applications, the standard supports playback with six degrees of freedom (6DoF), allowing users to not only change their viewing orientation (pitch/yaw/roll) but also move freely within the scene (x/y/z). By leveraging strong hardware support for widely used video formats, MPEG immersive video provides a highly flexible framework for multi-view video plus depth (MVD) and multi-plane image (MPI) video coding, making volumetric video more accessible and efficient.

With the second edition, MPEG continues to expand the capabilities of MPEG immersive video, introducing a range of new technologies to enhance coding efficiency and support more advanced immersive experiences. Key additions include:

• Geometry coding using luma and chroma planes, improving depth representation
• Capture device information, enabling better reconstruction of the original scene
• Patch margins and background views, optimizing scene composition
• Static background atlases, reducing redundant data for stationary elements
• Support for decoder-side depth estimation, enhancing depth accuracy
• Chroma dynamic range modification, improving color fidelity
• Piecewise linear normalized disparity quantization and linear depth quantization, refining depth precision

The second edition also introduces two new profiles: (1) MIV Simple MPI profile, allowing MPI content playback with a single 2D video decoder, and (2) MIV 2 profile, a superset of existing profiles that incorporates all newly added tools.

With these advancements, MPEG immersive video continues to push the boundaries of immersive media, providing a robust and efficient solution for next-generation video applications.

Research aspects: Possible research may explore advancements in MPEG immersive video to improve compression efficiency and real-time streaming while preserving depth accuracy and spatial quality. Another key area is enhancing 6DoF video rendering by leveraging new coding tools like decoder-side depth estimation and geometry coding, enabling more precise scene reconstruction and seamless user interaction in volumetric video applications.

MPEG-DASH (Sixth Edition)

At the 149th MPEG meeting, MPEG Systems (WG 3) advanced the sixth edition of MPEG-DASH (ISO/IEC 23009-1 Media presentation description and segment formats) by promoting it to the Final Draft International Standard (FDIS), the final stage of standards development. This milestone underscores MPEG’s ongoing commitment to innovation and responsiveness to evolving market needs.

The sixth edition introduces several key enhancements to improve the flexibility and efficiency of MPEG-DASH:

• Alternative media presentation support, enabling seamless switching between main and alternative streams
• Content steering signaling across multiple CDNs, optimizing content delivery
• Enhanced segment sequence addressing, improving low-latency streaming and faster tune-in
• Compact duration signaling using patterns, reducing MPD overhead
• Support for Common Media Client Data (CMCD), enabling better client-side analytics
• Nonlinear playback for interactive storylines, expanding support for next-generation media experiences

With these advancements, MPEG-DASH continues to evolve as a robust and scalable solution for adaptive streaming, ensuring greater efficiency, flexibility, and enhanced user experiences across a wide range of applications.

Research aspects: While advancing MPEG-DASH for more efficient and flexible adaptive streaming has been subject to research for a while, optimizing content delivery across multiple CDNs while minimizing latency and optimizing QoE remains an open issue. Another key area is enhancing interactivity and user experiences by leveraging new features like nonlinear playback for interactive storylines and improved client-side analytics through Common Media Client Data (CMCD).

The 150th MPEG meeting will be held online from March 31 to April 04, 2025. Click here for more information about MPEG meetings and their developments.

MPEG news: a report from the 148th meeting

This blog post is based on the MPEG press release and has been modified/updated here to focus on and highlight research aspects. This version of the blog post will also be posted at ACM SIGMM Records.

MPEG News Archive

The 148th MPEG meeting took place in Kemer, Türkiye, from November 4 to 8, 2024. The official press release can be found here and includes the following highlights:

• Point Cloud Coding: AI-based point cloud coding & enhanced G-PCC
• MPEG Systems: New Part of MPEG DASH for redundant encoding and packaging, reference software and conformance of ISOBMFF, and a new structural CMAF brand profile
• Video Coding: New part of MPEG-AI and 2nd edition of conformance and reference software for MPEG Immersive Video (MIV)
• MPEG completes subjective quality testing for film grain synthesis using the Film Grain Characteristics SEI message

148th MPEG Meeting, Kemer, Türkiye, November 4-8, 2024.

Point Cloud Coding

At the 148^th MPEG meeting, MPEG Coding of 3D Graphics and Haptics (WG 7) launched a new AI-based Point Cloud Coding standardization project. MPEG WG 7 reviewed six responses to a Call for Proposals (CfP) issued in April 2024 targeting the full range of point cloud formats, from dense point clouds used in immersive applications to sparse point clouds generated by Light Detection and Ranging (LiDAR) sensors in autonomous driving. With bit depths ranging from 10 to 18 bits, the CfP called for solutions that could meet the precision requirements of these varied use cases.

Among the six reviewed proposals, the leading proposal distinguished itself with a hybrid coding strategy that integrates end-to-end learning-based geometry coding and traditional attribute coding. This proposal demonstrated exceptional adaptability, capable of efficiently encoding both dense point clouds for immersive experiences and sparse point clouds from LiDAR sensors. With its unified design, the system supports inter-prediction coding using a shared model with intra-coding, applicable across various bitrate requirements without retraining. Furthermore, the proposal offers flexible configurations for both lossy and lossless geometry coding.

Performance assessments highlighted the leading proposal’s effectiveness, with significant bitrate reductions compared to traditional codecs: a 47% reduction for dense, dynamic sequences in immersive applications and a 35% reduction for sparse dynamic sequences in LiDAR data. For combined geometry and attribute coding, it achieved a 40% bitrate reduction across both dense and sparse dynamic sequences, while subjective evaluations confirmed its superior visual quality over baseline codecs.

The leading proposal has been selected as the initial test model, which can be seen as a baseline implementation for future improvements and developments. Additionally, MPEG issued a working draft and common test conditions.

Research aspects: The initial test model, like those for other codec test models, is typically available as open source. This enables both academia and industry to contribute to refining various elements of the upcoming AI-based Point Cloud Coding standard. Of particular interest is how training data and processes are incorporated into the standardization project and their impact on the final standard.

Another point cloud-related project is called Enhanced G-PCC, which introduces several advanced features to improve the compression and transmission of 3D point clouds. Notable enhancements include inter-frame coding, refined octree coding techniques, Trisoup surface coding for smoother geometry representation, and dynamic Optimal Binarization with Update On-the-fly (OBUF) modules. These updates provide higher compression efficiency while managing computational complexity and memory usage, making them particularly advantageous for real-time processing and high visual fidelity applications, such as LiDAR data for autonomous driving and dense point clouds for immersive media.

By adding this new part to MPEG-I, MPEG addresses the industry's growing demand for scalable, versatile 3D compression technology capable of handling both dense and sparse point clouds. Enhanced G-PCC provides a robust framework that meets the diverse needs of both current and emerging applications in 3D graphics and multimedia, solidifying its role as a vital component of modern multimedia systems.

MPEG Systems Updates

At its 148^th meeting, MPEG Systems (WG 3) worked on the following aspects, among others:

• New Part of MPEG DASH for redundant encoding and packaging
• Reference software and conformance of ISOBMFF
• A new structural CMAF brand profile

The second edition of ISO/IEC 14496-32 (ISOBMFF) introduces updated reference software and conformance guidelines, and the new CMAF brand profile supports Multi-View High Efficiency Video Coding (MV-HEVC), which is compatible with devices like Apple Vision Pro and Meta Quest 3.

The new part of MPEG DASH, ISO/IEC 23009-9, addresses redundant encoding and packaging for segmented live media (REAP). The standard is designed for scenarios where redundant encoding and packaging are essential, such as 24/7 live media production and distribution in cloud-based workflows. It specifies formats for interchangeable live media ingest and stream announcements, as well as formats for generating interchangeable media presentation descriptions. Additionally, it provides failover support and mechanisms for reintegrating distributed components in the workflow, whether they involve file-based content, live inputs, or a combination of both.

Research aspects: With the FDIS of MPEG DASH REAP available, the following topics offer potential for both academic and industry-driven research aligned with the standard's objectives (in no particular order or priority):

• Optimization of redundant encoding and packaging: Investigate methods to minimize resource usage (e.g., computational power, storage, and bandwidth) in redundant encoding and packaging workflows. Explore trade-offs between redundancy levels and quality of service (QoS) in segmented live media scenarios.
• Interoperability of live media Ingest formats: Evaluate the interoperability of the standard's formats with existing live media workflows and tools. Develop techniques for seamless integration with legacy systems and emerging cloud-based media workflows.
• Failover mechanisms for cloud-based workflows: Study the reliability and latency of failover mechanisms in distributed live media workflows. Propose enhancements to the reintegration of failed components to maintain uninterrupted service.
• Standardized stream announcements and descriptions: Analyze the efficiency and scalability of stream announcement formats in large-scale live streaming scenarios. Research methods for dynamically updating media presentation descriptions during live events.
• Hybrid workflow support: Investigate the challenges and opportunities in combining file-based and live input workflows within the standard. Explore strategies for adaptive workflow transitions between live and on-demand content.
• Cloud-based workflow scalability: Examine the scalability of the REAP standard in high-demand scenarios, such as global live event streaming. Study the impact of cloud-based distributed workflows on latency and synchronization.
• Security and resilience: Research security challenges related to redundant encoding and packaging in cloud environments. Develop techniques to enhance the resilience of workflows against cyberattacks or system failures.
• Performance metrics and quality assessment: Define performance metrics for evaluating the effectiveness of REAP in live media workflows. Explore objective and subjective quality assessment methods for media streams delivered using this standard.

The current/updated status of MPEG-DASH is shown in the figure below.

MPEG-DASH status, November 2024.

Video Coding Updates

In terms of video coding, two noteworthy updates are described here:

• Part 3 of MPEG-AI, ISO/IEC 23888-3 – Optimization of encoders and receiving systems for machine analysis of coded video content, reached Committee Draft Technical Report (CDTR) status
• Second edition of conformance and reference software for MPEG Immersive Video (MIV). This draft includes verified and validated conformance bitstreams and encoding and decoding reference software based on version 22 of the Test model for MPEG immersive video (TMIV). The test model, objective metrics, and some other tools are publicly available at https://gitlab.com/mpeg-i-visual.

Part 3 of MPEG-AI, ISO/IEC 23888-3: This new technical report on "optimization of encoders and receiving systems for machine analysis of coded video content" is based on software experiments conducted by JVET, focusing on optimizing non-normative elements such as preprocessing, encoder settings, and postprocessing. The research explored scenarios where video signals, decoded from bitstreams compliant with the latest video compression standard, ISO/IEC 23090-3 – Versatile Video Coding (VVC), are intended for input into machine vision systems rather than for human viewing. Compared to the JVET VVC reference software encoder, which was originally optimized for human consumption, significant bit rate reductions were achieved when machine vision task precision was used as the performance criterion.

The report will include an annex with example software implementations of these non-normative algorithmic elements, applicable to VVC or other video compression standards. Additionally, it will explore the potential use of existing supplemental enhancement information messages from ISO/IEC 23002-7 – Versatile supplemental enhancement information messages for coded video bitstreams – for embedding metadata useful in these contexts.

Research aspects: (1) Focus on optimizing video encoding for machine vision tasks by refining preprocessing, encoder settings, and postprocessing to improve bit rate efficiency and task precision, compared to traditional approaches for human viewing. (2) Examine the use of metadata, specifically SEI messages from ISO/IEC 23002-7, to enhance machine analysis of compressed video, improving adaptability, performance, and interoperability.

Subjective Quality Testing for Film Grain Synthesis

At the 148^th MPEG meeting , the MPEG Joint Video Experts Team (JVET) with ITU-T SG 16 (WG 5 / JVET) and MPEG Visual Quality Assessment (AG 5) conducted a formal expert viewing experiment to assess the impact of film grain synthesis on the subjective quality of video content. This evaluation specifically focused on film grain synthesis controlled by the Film Grain Characteristics (FGC) supplemental enhancement information (SEI) message. The study aimed to demonstrate the capability of film grain synthesis to mask compression artifacts introduced by the underlying video coding schemes.

For the evaluation, FGC SEI messages were adapted to a diverse set of video sequences, including scans of original film material, digital camera noise, and synthetic film grain artificially applied to digitally captured video. The subjective performance of video reconstructed from VVC and HEVC bitstreams was compared with and without film grain synthesis. The results highlighted the effectiveness of film grain synthesis, showing a significant improvement in subjective quality and enabling bitrate savings of up to a factor of 10 for certain test points.

This study opens several avenues for further research:

• Optimization of film grain synthesis techniques: Investigating how different grain synthesis methods affect the perceptual quality of video across a broader range of content and compression levels.
• Compression artifact mitigation: Exploring the interaction between film grain synthesis and specific types of compression artifacts, with a focus on improving masking efficiency.
• Adaptation of FGC SEI messages: Developing advanced algorithms for tailoring FGC SEI messages to dynamically adapt to diverse video characteristics, including real-time encoding scenarios.
• Bitrate savings analysis: Examining the trade-offs between bitrate savings and subjective quality across various coding standards and network conditions.

---

The 149th MPEG meeting will be held in Geneva, Switzerland from January 20-24, 2025. Click here for more information about MPEG meetings and their developments.

Happy World Standards Day 2024

As we celebrate World Standards Day, it's important to recognize the monumental advancements the MPEG community has made over the past year. These achievements continue to influence multimedia standards worldwide, playing a crucial role in ensuring seamless, high-quality digital experiences.

1. ISO Base Media File Format (8th Edition): This standard has been pivotal for media streaming applications, particularly for formats like DASH (Dynamic Adaptive Streaming over HTTP) and CMAF (Common Media Application Format). The latest update facilitates more seamless media switching and continuous presentation, optimizing the user experience across different devices.
2. Neural Network Compression (2nd Edition): With AI technologies rapidly evolving, MPEG's neural network compression standard addresses the need for efficient storage and inference in multimedia systems. The second edition enhances reference software, providing robust tools for handling complex neural networks in applications such as image and video processing.
3. Low Latency, Low Complexity LiDAR Coding: As industries like autonomous vehicles and smart cities expand, this standard addresses the need for efficient and real-time processing of LiDAR data. The MPEG community has developed compression techniques that maintain low latency and complexity, enabling faster decision-making for autonomous systems.
4. MPEG-DASH (6th Edition): The 6th edition of MPEG-DASH brings exciting improvements in adaptive streaming. Key updates include support for new CMCD parameters for better content management and a background mode that allows players to receive updates without disrupting media playback. These advancements significantly enhance streaming efficiency and flexibility.
5. Video Coding for Machines (VCM): A significant addition this year has been the introduction of Video Coding for Machines. This emerging standard focuses on machine vision applications, where efficient encoding and decoding are crucial for machine learning tasks such as object detection and recognition. This innovation caters to the increasing integration of machine-based analytics in multimedia systems.
6. Immersive Media and Volumetric Video: MPEG’s work on volumetric video coding and standards for immersive media continues to push the boundaries of AR/VR technologies. This ensures that immersive content can be delivered across various platforms with improved consistency and performance.

These highlights exemplify MPEG's commitment to fostering innovation through multimedia standards, shaping the future of digital content. On this World Standards Day, let’s celebrate the efforts that keep the digital ecosystem thriving!

ACM Mile-High Video Conference 2025: Call for Contributions

MHV 2025: ACM Mile-High Video Conference 2025
Call for Contributions
February 18-20, 2025, The Cable Center, Denver, Colorado
https://www.mile-high.video/

ACM Mile-High Video (MHV) is a flagship industry-oriented technical conference in the area of video technologies, which has been successfully running in Denver, Colorado, starting from 2016. ACM MHV 2025 welcomes contributions from both industry and academia to share real-world problems and solutions as well as novel approaches and innovations from content production to consumption. ACM MHV 2025 will provide a unique opportunity to view the interplay of the industry and academia in the area of video technologies.

ACM MHV contributions are solicited in, but not limited to, the following areas:

• Content production, encoding, and packaging

• Encoding for broadcast, mobile, and OTT (incl. using AI/ML in encoding),

• New and emerging audio, image, and video codecs (incl. point cloud coding, light field coding, holography coding, etc.)

• Edge, network, and cloud-based coding

• Perceptually optimized objective quality metrics

• Quality assessment models and tools, and user experience studies

• Storage applications for video processing and streaming

• Accessibility

• HDR

• Video workflows

• Virtualized headends, cloud-based workflows for production and distribution

• Redundancy and resilience in content origination

• Ingest protocols

• Ad insertion

• Content delivery and security

• Developments in transport protocols and new delivery paradigms

• Protection for OTT distribution and tools against piracy

• Analytics

• Streaming technologies

• Adaptive streaming and transcoding

• Low latency

• Player, playback, and QoE developments

• Content discovery, promotion, and recommendation systems

• Protocol and Web API improvements and innovations for streaming video

• Industry trends

• 3D/XR video (NeRF, Gaussian splatting, etc.)

• Scalable and multi-view video coding deployments

• Video and audio coding for machines

• Cloud gaming and gaming streaming

• Provenance, content authentication, and deepfakes

• Energy management in video compression and streaming

• Edge computing tools and applications

• Hardware for content encoding, storage, and distribution

• Standards and interoperability

• New and developing standards in the media and delivery space

• Interoperability guidelines

Reasons for academics to submit to and attend the ACM Mile-High Video 2025 event:

• Networking with Industry Leaders: Engage with top minds from both academia and industry, fostering collaborations and partnerships in cutting-edge video technology.
• High Visibility: Present your research to a global audience, including key industry players and decision-makers.
• Innovative Content: Showcase your work in emerging areas such as metaverse, AI-driven video technologies, and advanced codecs.
• Research Impact: Influence the direction of next-gen video technology and its applications across industries.

Participating in ACM MHV 2025 is a prime opportunity to boost academic careers by connecting theoretical research with real-world innovations.

Persuasive reasons for industry professionals to submit and attend ACM Mile-High Video 2025:

• Access to Cutting-Edge Research: Stay at the forefront of video technology with insights into innovations in streaming, codecs, and content delivery.
• Collaborate with Academia: Engage with academic researchers to explore potential partnerships that can accelerate R&D and technological breakthroughs.
• Showcase Industry Leadership: Present your company’s pioneering solutions, fostering brand recognition and thought leadership in the global video tech  community.
• Talent and Recruitment: Discover emerging talent and ideas that could impact your organization’s future.

Participating drives industry growth through shared knowledge and collaboration!

Prospective speakers are invited to submit an extended abstract (one page ~400 words + references) that will be peer-reviewed by the ACM MHV technical program committee (TPC) for relevance, timeliness, and technical correctness.

The authors of the accepted extended abstracts will be invited to optionally submit a full-length paper (up to six pages + references) for possible inclusion into the conference proceedings. These papers must be original work (i.e., not published previously in a journal or conference) and will also be peer-reviewed by the ACM MHV TPC.

Accepted extended abstracts and full-length papers will be presented at the ACM MHV conference, and, optionally, will be published at the conference proceedings within the ACM Digital Library.

All prospective ACM authors are subject to all ACM Publications Policies, including ACM's new Publications Policy on Research Involving Human Participants and Subjects.

How to Submit an Extended Abstract

Prospective authors are invited to submit one-page extended abstracts here 

Important Dates

• Extended abstract submission deadline: Oct. 31, 2024 AoE (firm deadline)

• Notification of extended abstract acceptance: Nov. 27, 2024 AoE

• Optional: Full-length paper submission deadline: Dec. 16, 2024 AoE

• Notification of full-length paper acceptance: Jan. 22, 2025 AoE

• Camera-ready submission (extended abstracts/full-length papers) deadline: Jan. 31, 2025 AoE

ACM MHV 2025 Program Chairs

• Christian Timmerer (AAU; christian.timmerer AT aau.at)

• Dan Grois (AnyAI; dgrois AT acm.org)

• Gwendal Simon (Synamedia; gsimon AT synamedia.com)

• Jill Boyce (Nokia; jill.boyce AT nokia.com)

• Yuriy Reznik (Brightcove; yreznik AT brightcove.com)

MPEG news: a report from the 147th meeting

This blog post is based on the MPEG press release and has been modified/updated here to focus on and highlight research aspects. This version of the blog post will also be posted at ACM SIGMM Records.

MPEG News Archive

The 147th MPEG meeting was held in Sapporo, Japan from 15-19 July 2024, and the official press release can be found here. It comprises the following highlights:

• ISO Base Media File Format*: The 8th edition was promoted to Final Draft International Standard, supporting seamless media presentation for DASH and CMAF.
• Syntactic Description Language: Finalized as an independent standard for MPEG-4 syntax.
• Low-Overhead Image File Format*: First milestone achieved for small image handling improvements.
• Neural Network Compression*: Second edition for conformance and reference software promoted.
• Internet of Media Things (IoMT): Progress made on reference software for distributed media tasks.

* … covered in this blog post and expanded with possible research aspects.

8th edition of ISO Base Media File Format

The ever-growing expansion of the ISO/IEC 14496-12 ISO base media file format (ISOBMFF) application area has continuously brought new technologies to the standards. During the last couple of years, MPEG Systems (WG 3) has received new technologies on ISOBMFF for more seamless support of ISO/IEC 23009 Dynamic Adaptive Streaming over HTTP (DASH) and ISO/IEC 23000-19 Common Media Application Format (CMAF) leading to the development of the 8th edition of ISO/IEC14496-12.

The new edition of the standard includes new technologies to explicitly indicate the set of tracks representing various versions of the media presentation of a single media for seamless switching and continuous presentation. Such technologies will enable more efficient processing of the ISOBMFF formatted files for DASH manifest or CMAF Fragments.

Research aspects: The central research aspect of the 8th edition of ISOBMFF, which “will enable more efficient processing,” will undoubtedly be its evaluation compared to the state-of-the-art. Standards typically define a format, but how to use it is left open to implementers. Therefore, the implementation is a crucial aspect and will allow for a comparison of performance. One such implementation of ISOBMFF is GPAC, which most likely will be among the first to implement these new features.

Low-Overhead Image File Format

ISO/IEC 23008-12 image format specification defines generic structures for storing image items and sequences based on ISO/IEC 14496-12 ISO base media file format (ISOBMFF). As it allows the use of various high-performance video compression standards for a single image or a series of images, it has been adopted by the market quickly. However, it was challenging to use it for very small-sized images such as icons or emojis. While the initial design of the standard was versatile and useful for a wide range of applications, the size of headers becomes an overhead for applications with tiny images. Thus, Amendment 3 of ISO/IEC 23008-12 low-overhead image file format aims to address this use case by adding a new compact box for storing metadata instead of the ‘Meta’ box to lower the size of the overhead.

Research aspects: The issue regarding header sizes of ISOBMFF for small files or low bitrate (in the case of video streaming) was known for some time. Therefore, amendments in these directions are appreciated while further performance evaluations are needed to confirm design choices made at this initial step of standardization.

Neural Network Compression

An increasing number of artificial intelligence applications based on artificial neural networks, such as edge-based multimedia content processing, content-adaptive video post-processing filters, or federated training, need to exchange updates of neural networks (e.g., after training on additional data or fine-tuning to specific content). For this purpose, MPEG developed a second edition of the standard for coding of neural networks for multimedia content description and analysis (NNC, ISO/IEC 15938-17, published in 2024), adding syntax for differential coding of neural network parameters as well as new coding tools. Trained models can be compressed to at least 10-20% for several architectures, even below 3%, of their original size without performance loss. Higher compression rates are possible at moderate performance degradation. In a distributed training scenario, a model update after a training iteration can be represented at 1% or less of the base model size on average without sacrificing the classification performance of the neural network.

In order to facilitate the implementation of the standard, the accompanying standard ISO/IEC 15938-18 has been updated to cover the second edition of ISO/IEC 15938-17. This standard provides a reference software for encoding and decoding NNC bitstreams, as well as a set of conformance guidelines and reference bitstreams for testing of decoder implementations. The software covers the functionalities of both editions of the standard, and can be configured to test different combinations of coding tools specified by the standard.

Research aspects: The reference software for NNC, together with the reference software for audio/video codecs, are vital tools for building complex multimedia systems and its (baseline) evaluation with respect to compression efficiency only (not speed). This is because reference software is usually designed for functionality (i.e., compression in this case) and not performance.

The 148th MPEG meeting will be held in Kemer, Türkiye, from November 04-08, 2024. Click here for more information about MPEG meetings and their developments.

University assistant predoctoral (all genders welcome) (in German: Universitätsassistent:in)

The University of Klagenfurt, with approximately 1,500 employees and over 12,000 students, is located in the Alps-Adriatic region and consistently achieves excellent placements in rankings. The motto “per aspera ad astra” underscores our firm commitment to the pursuit of excellence in all research, teaching, and university management activities. The principles of equality, diversity, health, sustainability, and compatibility of work and family life serve as the foundation for our work at the university.

The University of Klagenfurt is pleased to announce the following open position at the Department of Information Technology at the Faculty of Technical Sciences with an expected starting date of November 4, 2024:

University assistant predoctoral (all genders welcome) (in German: Universitätsassistent:in)

within the Ada Lovelace Programme (project title: Streaming of Holographic Content and its Impact on the Quality of Experience).

• Level of employment: 100 % (40 hours/week)
• Minimum salary: € 50,103.20 per annum (gross); Classification according to collective agreement: B1 
• Contract duration: 4 years
• Application deadline: by September 11, 2024
• Reference code: 348/24

Tasks and responsibilities:

• Autonomous scientific work, including the publication of research articles in the fields of coding and streaming of holographic content, Quality of Experience (QoE), and behavioural sciences
• Conducting independent scientific research with the aim of submitting a dissertation and acquiring a doctoral degree in technical sciences
• Teaching exercises and lab courses (e.g., in the computer science Bachelor’s or/and Master’s programme)
• Participating in research projects of the department, especially within the Ada Lovelace Programme (Streaming of Holographic Content and its Impact on the Quality of Experience)
• Mentoring students
• Assisting in public relations activities, science to public communication, and extra-curricular events of the department and the faculty

Prerequisites for the appointment:

• Completed Diploma or Master’s degree from a recognized university in the field of computer science, information and communications engineering, electrical engineering, or related fields. The completion of this degree must be fulfilled no later than two weeks before the starting date; hence, the last possible deadline for meeting this requirement is October 20, 2024
• Strong background in one or more of the following fields: multimedia systems (i.e., video/holographic content coding/streaming, Quality of Experience) and empirical research methods (i.e., statistical methods, interdisciplinary research with behavioural sciences)
• Fluent in written and spoken English
• Programming experience in multimedia systems

Additional desired qualifications:

• Experience with scientific publications or presentations
• Experience in interdisciplinary research projects, ideally in the behavioural sciences, as the project involves empirical research
• Excellent ability to work with teams
• Scientific curiosity and enthusiasm for research in multimedia systems and empirical research

The doctoral student will be co-supervised by Christian Timmerer, Heather Foran, and Hadi Amirpour.

Our offer:

This position serves the purposes of the vocational and scientific education of graduates of Master’s or Diploma degree programmes and sets the goal of completing a Doctoral degree / a Ph.D. in Technical Sciences. Therefore, applications by persons who have already completed a subject-specific doctoral degree or a subject-relevant Ph.D. program cannot be considered. 

The employment contract is concluded for the position of university assistant (predoctoral) and stipulates a starting salary of € 3,578.80 gross per month (14 times a year; previous experience deemed relevant to the job can be recognized in accordance with the collective agreement). 

The University of Klagenfurt also offers:

• Personal and professional advanced training courses, management, and career coaching
• Numerous attractive additional benefits, see also https://jobs.aau.at/en/the-university-as-employer/
• Diversity- and family-friendly university culture
• The opportunity to live and work in the attractive Alps-Adriatic region with a wide range of leisure activities in the spheres of culture, nature, and sports

The application:

If you are interested in this position, please apply in English by providing the following documents:

• Letter of application/cover letter including motivation statement for the given position
• Curriculum vitae (with clear information about the degrees, including date/place/grade, the experience acquired, the thesis title, the list of publications (if any), and any other relevant information)
• Copy of the degree certificates and transcripts of the courses
• Any certificates that can prove the fulfilment of the required and additional qualifications listed above (e.g., the submission of the final thesis if required by the degree programme, copy of publications, programming skills certificates, language skills certificates, etc.)
• Final thesis or other study-related written work (like seminar reports) or excerpts thereof
• If an applicant has not received the Diploma or Master’s degree by the application deadline, the applicant should provide a declaration, written either by a supervisor or by the candidate themselves, on the feasibility of finishing the Diploma or Master’s degree by October 30, 2024 at the latest. 

To apply, please select the position with the reference code 348/24 in the category “Scientific Staff” using the link “Apply for this position” in the job portal at jobs.aau.at/en/.

Candidates must furnish proof that they meet the required qualifications by October 20, 2024 at the latest.

For further information on this specific vacancy, please contact Univ.-Prof. DI Dr. Christian Timmerer (christian.timmerer@aau.at). General information about the university as an employer can be found at https://jobs.aau.at/en/the-university-as-employer/. At the University of Klagenfurt, recruitment and staff matters are accompanied not only by the authority responsible for the recruitment procedure but also by the Equal Opportunities Working Group and, if necessary, by the Representative for Disabled Persons.

The University of Klagenfurt aims to increase the proportion of women and, therefore, invites explicitly qualified women to apply for the position. Where the qualification is equivalent, women will be given preferential consideration. 

People with disabilities or chronic diseases, who fulfill the requirements, are particularly encouraged to apply. 

Travel and accommodation costs incurred during the application process will not be refunded. Translations into other languages shall serve informational purposes only. Solely the version advertised in the University Bulletin (Mitteilungsblatt) shall be legally binding.

Successful 5-year Evaluation of Christian Doppler Laboratory ATHENA

The Christian Doppler (CD) Laboratory ATHENA was established in October 2019 to tackle current and future research and deployment challenges of HTTP Adaptive Streaming (HAS) and emerging streaming methods. The goal of CD laboratories is to conduct application-oriented basic research, promote collaboration between universities and companies, and facilitate technology transfer. They are funded through a public-private partnership between companies and the Christian Doppler Research Association, which is funded by the Ministry for Digital and Economic Affairs and the National Foundation for Research, Technology, and Development (Nationalstiftung für Forschung, Technologie und Entwicklung (FTE)). ATHENA is supported by Bitmovin as a company partner.

The CD laboratories have a duration of seven years and undergo rigorous scientific review after two and five years. This spring, the CD lab ATHENA completed its 5-year evaluation, and we have just received official notification from the CDG that we have successfully passed the review. Consequently, it is time to briefly outline the main achievements during this second phase (i.e., years 2 to 5) of the CD lab ATHENA.

Before exploring the achievements, it’s important to highlight the ongoing relevance of research in video streaming, given its dominance in today’s Internet usage. The January 2024 Sandvine Internet Phenomena report revealed that video streaming accounts for 68% of fixed/wired Internet traffic and 64% for mobile Internet traffic. Specifically, Video on Demand (VoD) represents 54% of fixed/wired and 57% of mobile traffic, while live streaming contributes to 14% of fixed/wired and 7% of mobile traffic. The major services in this domain include YouTube and Netflix, each commanding more than 10% of the overall Internet traffic, with TikTok, Amazon Prime, and Disney+ also playing significant roles.

ATHENA is structured into four work packages, each with distinct objectives as detailed below:

1. Content provisioning: Primarily involves video encoding for HAS, quality-aware encoding, learning-based encoding, and multi-codec HAS.
2. Content delivery: Addresses HAS issues by utilizing edge computing, exchanging information between CDN/SDN and clients, providing network assistance for clients, and evaluating corresponding utilities.
3. Content consumption: Focuses on bitrate adaptation schemes, playback improvements, context and user awareness, and studies on Quality of Experience (QoE).
4. End-to-end aspects: Offers a comprehensive view of application and transport layer enhancements, Quality of Experience (QoE) models, low-latency HAS, and learning-based HAS.

During the 2nd phase of ATHENA’s work, we achieved significant results, including publications in respected academic journals and conferences. Specifically, our publications were featured in key multimedia, signal processing, computer networks & wireless communication, and computing systems venues, as categorized by Google Scholar under engineering and computer science. Some of the notable publications include IEEE Communications Surveys & Tutorials (impact factor: 35.6), IEEE Transactions on Image Processing (10.6), IEEE Internet of Things Journal (10.6), IEEE Transactions on Circuits and Systems for Video Technology (8.4), and IEEE Transactions on Multimedia (7.3).

Furthermore, we focused on technology transfer by submitting 16 invention disclosures, resulting in 13 patent applications (including provisionals). Collaborating with our company partner, we obtained 6 granted patents. Additionally, we’re pleased to report on the progress of our spin-off projects, as well as the funding secured for two FFG-funded projects named APOLLO and GAIA, and an EU Horizon Europe-funded innovation action called SPIRIT.

The ATHENA team was also active in organizing scientific events such as workshops, special sessions, and special issues at IEEE ICME, ACM MM, ACM MMSys, ACM CoNEXT, IEEE ICIP, PCS, and IEEE Network. We also contributed to reproducibility in research through open source tools (e.g., Video Complexity Analyzer and LLL-CAdViSE) and datasets (e.g., Video Complexity Dataset and Multi-Codec Ultra High Definition 8K MPEG-DASH Dataset) among others.

We also note our contributions to the applications of AI in video coding & streaming, for example in video coding and video streaming as follows:

• Fast Multi-Rate Encoding with Machine Learning (using Convolutional Neural Networks (CNNs))
• LiDeR: Lightweight video Super Resolution for mobile devices (using Deep Neural Networks (DNNs))
• Blind Visual Quality Assessment Using Vision Transformers
• Video Complexity Analysis (VCA) and optimizations for per-title encoding (using Linear Regression, Random Forest, and XGBoost models)
• DeepStream: Video streaming enhancements using compressed deep neural networks (using Deep Neural Networks (DNNs))
• ECAS-ML: Edge-assisted adaptive bitrate switching (using Long Short-Term Memory (LSTM))
• Quality Optimization of Live Streaming Services over HTTP with Reinforcement Learning (RL)

A major outcome of the second phase is the successful defense of the inaugural cohort of PhD students:

• Dr. Alireza Erfanian: “Optimizing QoE and Latency of Video Streaming using Edge Computing and In-Network Intelligence”, May 25, 2023
• Dr. Ekrem Çetinkaya: “Video Coding Enhancements for HTTP Adaptive Streaming using Machine Learning”, June 7, 2023
• Dr. Minh Nguyen: “Policy-driven Dynamic HTTP Adaptive Streaming Player Environment”, June 30, 2023
• Dr. Jesús Aguilar Armijo: “Multi-access Edge Computing for Adaptive Video Streaming”, July 10, 2023
• Dr. Reza Farahani: “Network-Assisted Delivery of Adaptive Video Streaming Services through CDN, SDN, and MEC”, August 22, 2023
• Dr. Vignesh V Menon: “Content-adaptive Video Coding for HTTP Adaptive Streaming”, January 15, 2024
• Dr. Babak Taraghi, “End-to-end Quality of Experience Evaluation for HTTP Adaptive Streaming”, July 10, 2024

Two postdoctoral scholars have reached a significant milestone on their path toward habilitation

• Dr. Hadi Amirpour, “Video Coding for Efficient HTTP Adaptive Streaming”, February 8, 2024
• Dr. Farzad Tashtarian, “How to Optimize Dynamic Adaptive Video Streaming? Challenges and Solutions”, February 27, 2023 & “End-to-End Adaptive Video Streaming Optimization”, June 26, 2024

During the second phase, each work package produced excellent publications in their domain, briefly highlighted in the following. Content provisioning (WP-1) focuses mainly on video coding for HAS (43 papers) and immersive media coding for streaming (4 papers). The former can be further subdivided into the following topic areas:

• Video complexity: spatial and temporal feature extraction (4 papers)
• Compression efficiency improvement of individual representations (1 paper)
• Encoding parameter prediction for HAS (9 papers)
• Efficient bitrate ladder construction (4 papers)
• Fast multi-rate encoding (3 papers)
• Data security and data hiding (7 papers)
• Energy-efficient video encoding for HAS (4 papers)
• Advancing video quality evaluation (7 papers)
• Datasets (4 papers)

Content delivery (WP-2) dealt with SDN/CDN assistance for HAS, edge computing support for HAS, and network-embedded media streaming support, resulting in 21 papers. Content consumption (WP-3) worked on QoE enhancement mechanisms at client-side and QoE- and energy-aware content consumption (11 papers). Finally, end-to-end Aspects (WP-4) produced 15 papers in the area of end-to-end QoE improvement in multimedia video streaming. We reported 94 papers published/accepted for the ATHENA 5-year evaluation.

In this context, it is also important to highlight the collaboration within ATHENA, which has resulted in joint publications across various work packages (WPs) and with other ITEC members. For example, collaborations with Prof. Schöffmann (FWF-funded project OVID), FFG-funded projects APOLLO/GAIA, and EU-funded project SPIRIT. In addition, we would like to acknowledge our international collaborators, such as Prof. Hongjie He from Southwest Jiaotong University, Prof. Patrick Le Callet from the University of Nantes, Prof. Wassim Hamidouche from the Technology Innovation Institute (UAE), Dr. Sergey Gorinsky from IMDEA, Dr. Abdelhak Bentaleb from Concordia University, Dr. Raimund Schatz from AIT, and Prof. Pablo Cesar from CWI. We are also pleased to report the successful technology transfers to Bitmovin, particularly CAdViSE (WP-4) and WISH ABR (WP-3). Regular “Fun with ATHENA” meetups and Break-out Groups are utilized for in-depth discussions about innovations and potential technology transfers.

Over the next two years, the ATHENA project will prioritize the development of deep neural network/AI-based image and video coding within the context of HAS. This includes energy- and cost-aware video coding for HAS, immersive video coding such as volumetric video and holography, as well as Quality of Experience (QoE) and energy-aware content consumption for HAS (including energy-efficient, AI-based live video streaming) and generative AI for HAS.

Thanks to all current and former ATHENA team members: Samira Afzal, Hadi Amirpour, Jesús Aguilar Armijo, Emanuele Artioli, Christian Bauer, Alexis Boniface, Ekrem Çetinkaya, Reza Ebrahimi, Alireza Erfanian, Reza Farahani, Mohammad Ghanbari (late), Milad Ghanbari, Mohammad Ghasempour, Selina Zoë Haack, Hermann Hellwagner, Manuel Hoi, Andreas Kogler, Gregor Lammer, Armin Lachini, David Langmeier, Sandro Linder, Daniele Lorenzi, Vignesh V Menon, Minh Nguyen, Engin Orhan, Lingfeng Qu, Jameson Steiner, Nina Stiller, Babak Taraghi, Farzad Tashtarian, Yuan Yuan, and Yiying Wei. Finally, thanks to ITEC support staff Martina Steinbacher, Nina Stiller, Margit Letter, Marion Taschwer, and Rudolf Messner.

We also would like to thank the Christian Doppler Research Association for continuous support, organizing the review, and the reviewer for constructive feedback!