SVC, MANE, and the Future of the Internet

What have Scalable Video Coding (SVC), Media-Aware Network Elements (MANEs) and the Future of the Internet in common? Well, in this article, I'm trying to highlight this...

Scalable Video Coding

Scalability in video coding has a long history which started with MPEG-2 (Wikipedia) introducing temporal, spatial, signal-to-noise ratio (SNR) scalability, and data partitioning. However, as it successor MPEG-4, it lacks of coding efficiency compared to a single layer coded bitstream. With the emergence of the Advanced Video Coding (AVC) standard - a joint development between ISO/MPEG and ITU-T (H.264) referred to as Joint Video Team (JVT) - that gained momentum (i.e., AVC is THE state-of-the-art video coding standard with which one has to compare), the JVT tried it again and started working on Scalable Video Coding with the aim to introduce at most 10% coding overhead ... and this goal has been reached! SVC offers three (main) scalability dimensions:

• temporal: frames per second, e.g., 30fps -> 15 fps
• spatial: resolution, e.g., HTDV (1920x1080) -> SDTV (720×576)
• SNR: quality, e.g., 32dB -> 28 dB

The main advantage of SVC is that it allows for extraction of low-quality (i.e., temporal, spatial, SNR) bitstreams by simply discarding portions of the original bitstream whereby the base layer is always compliant to AVC. In the past, we've done some measurements concerning the adaptation of SVC bitstreams utilizing a generic vs. specific approach.

Media-Aware Network Element

A Media-Aware Network Element (MANE) as defined by the IETF RFC3984 is referred to as a network element, such as a middlebox or application layer gateway that is capable of parsing certain aspects of the RTP payload headers or the RTP payload and reacting to the contents. MANEs - in its current state - are tightly aligned with SVC and, thus, allow the adaptation of SVC bitstreams within the network as shown in the figure below (taken from here).
The current status of the RTP payload format for SVC bitstreams can be found here. With SVC and its RTP payload format, it's very simple to deploy an adaptation engine within the network (i.e., the MANE) that adapts the video content according to different terminal capabilities and network characteristics (e.g., HDTV, SDTV, mobile devices).

The Future Internet

The term "Future Internet" comes from the future Internet conference that was held in spring 2008 in Bled, Slovenia and refers to the European Future Internet Portal which serves as the central forum for European activities and discussions on the future of the Internet. Within the Bled Declaration the main challenges towards the Future Internet have been highlighted, among them capabilities for supporting the creation, sharing, locating and delivery of new-media content which indeed is the result of research and development in the area of SVC and MANE respectively. EC-funded projects that are active supporters of the Bled Declaration and utilize SVC and MANE respectively are, for example, P2P-Next and SEA. However, there's still a long, difficult path ahead with a lot of challenges to be addressed, specifically in the area of Quality of Servce/Experience and, thus, adaptation of multimedia content (but that's another story...).

The next event is the Future of the Internet conference in Prague, in May 2009.

Joint Master Thesis Projects between STMicroelectronics and Klagenfurt University

• Visual quality optimization techniques applied to rate-control in SVC
• Streaming of scalable video bitstreams on variable bandwidth channels
• Development of motion compensated filtering techniques for noise reduction in digital video sequences generated by CMOS sensors

For further information, please contact me!

Visual quality optimization techniques applied to rate-control in SVC
Reference: Tea Anselmo (STM), Daniele Alfonso (STM), Christian Timmerer (UNIKLU), Hermann Hellwagner (UNIKLU)
Description: In digital video coding, the main aim of bit-rate control is to provide a bit-stream with a final data rate as close as possible to a predefined target one, by simply adapting the quantization parameter to the video sequence complexity. However, in case of low bit-rate applications or highly complex sequences, the bit-rate control would impose a too high quantization step thus inevitably compromising the visual quality of coded pictures.
In the depicted case, it is convenient the application of a Frame Skipping strategy. The Frame Skipping technique adaptively skips whole coded pictures, so the encoder could allocate more bits for other frames so as to perform higher image quality and reduce rate criticisms without losing bits. Moreover, the human visual system is particularly sensitive to quantization artefacts and frame rate discontinuity, therefore skipping frame selection must balance quality and rate to avoid visual fluctuation of image quality. This thesis work implies:

• Study and documentation on frame skipping techniques in literature.
• Development and integration of the Frame Skipping technique into the STMicroelectronics proprietary Constant Bit-Rate (CBR) control algorithm. The STMicroelectronics encoder is a C++ software model of the new MPEG SVC (Scalable Video Coding) standard. The implementation will be carried out using GNU C/C++ program languages and will be executed in a general purpose computational platform (PC).
• Finally, extensive objective and subjective evaluations must be done to test the algorithm performance.

Streaming of scalable video bitstreams on variable bandwidth channels
Reference: Daniele Alfonso (STM), Christian Timmerer (UNIKLU), Hermann Hellwagner (UNIKLU)
Description: The new video coding standard ITU-T/MPEG Scalable Video Coding (SVC) extends the H.264/AVC functionalities with new effective improvements particularly devoted to layered coding in terms of temporal, spatial and quality scalability. A scalable bitstream can contain different representations of the same video sequence, with different temporal or spatial resolutions and quality granularity, all encapsulated according to the standard coding hierarchy. In particular, with the MGS (Medium Grain Scalability) each picture can have multiple quality representations, each one with an increasing associated bit-rate, thus providing the possibility to select different compromises between quality and rate. In this way, SVC enables the transmission and decoding of partial bit-streams to adapt instantaneously to channel conditions, for example in case of video streaming applications as video conferencing services.
The final aim of this thesis work is the study, development and performance evaluation of a transmission system based on the SVC standard and devoted to video streaming applications on variable bandwidth channels. The activity implies:

• outline a set of significant encoding scenarios to be tested;
• channel modelling and simulation;
• extensive objective and subjective evaluations on decoder side of algorithm performance.
  

Development of motion compensated filtering techniques for noise reduction in digital video sequences generated by CMOS sensors
Reference: Daniele Alfonso (STM), Christian Timmerer (UNIKLU), Hermann Hellwagner (UNIKLU)
Description: Noise reduction is a fundamental aspect for every digital video process since it produces two main benefits: it improves human visual perception and increases compression efficiency of digital video coding systems. Today the most diffused video standard is the H.264/AVC (Advanced Video Coding), which implements a hybrid coding approach by exploiting both temporal and spatial redundancies to achieve efficient compression ratios. The noise, actually, causes high frequency spectral contributions in both spatial and temporal domains, thus preventing the coding process from fully exploitation of temporal and spatial samples correlation. The main targets of this thesis work are:

• noise model shaping of most diffused CMOS sensors for digital cameras and camcorders;
• development of a noise reduction method combining spatial and temporal filtering, also known as MCTF (Motion Compensated Temporal Filtering), employing STMicroelectronics proprietary algorithm implementation;
• integration of the de-noising model as spatio-temporal prefiltering stage of HDTV resolution sequences, according to the H.264/AVC coding standard;
• extensive objective and subjective evaluation of algorithm performance.
  

ISMW 2009 Deadline Extension until January 20

We've extended the deadline for ISMW 2009 to be aligned with the mother conference, i.e, Int'l Wireless Communications and Mobile Computing Conference (IWCMC2009).

Important Dates

Paper Submission Deadline: December 20, 2008January 20, 2009

Paper Acceptance Notification: March 25, 2009

Camera-ready Paper Submissions: April 15, 2009

Registration Deadline for Authors: April 15, 2009

Please register and upload your paper via EDAS.

Technik LIVE @ Klagenfurt University

Dec 19, 2008: We had a couple of workshops here at Klagenfurt University where we invited local schools - in and around Klagenfurt. Our group was also there with workshop demonstrating the amBX technology that is used in an internal project here and also lead to a couple of MPEG contributions.

A video of this event is available on YouTube. Note, it's in German only...

TV anywhere, anytime ...

When reading the article on TechCrunch about watching cable TV on your iPhone, brings me back to the vision of UMA - my first blog entry here. In fact, with Sling Media's Set-Top Box (STB) you can stream TV/video content to your computer within your house. Now they'll announced to approach mobile devices at this week’s Macworld:

SlingPlayer Mobile gives consumers their entire home TV experience, including local channels, local sports teams, video on demand, pay per view, etc. Any program that you can watch on your sofa back home, you can now watch via your iPhone using a standard network connection. In addition, SlingPlayer Mobile for iPhone users can also control their home digital video recorder (DVR) to watch recorded shows, pause, rewind, and fast forward live TV, or even queue new recordings while away from home.

The application will be submitted to Apple some time in Q1 but on their homepage the SlingPlayer Mobile for Blackberry is already available as public beta right now - also for Windows Smartphone, Windows Mobile Pocket PC, Palm, and Symbian. Well, one could state that Apple's iPhone is the last one in the queue but I leave this debate open to everyone...

For me, this means that watching "my TV content" anywhere and anytime is becoming reality. However, I don't have a Sling Media STB and I also wonder about the legal aspects in European countries, etc. Furthermore, I would be interested whether it is possible to stream any content - not only TV content - to your mobile devices as film industry is starting to attach Digital Copies to their DVDs and Blu-Ray Discs respectively.