Video Complexity Analyzer (VCA) v1.0

Release of Video Complexity Analyzer (VCA) version 1.0 open-source software.

The primary objective of VCA is to become the best spatial (E) and temporal (h) complexity predictor for every frame/video segment/video sequence, which aids in predicting encoding parameters for applications like scene-cut detection and online per-title encoding. VCA leverages x86 SIMD and multi-threading optimizations for effective performance. While VCA is primarily designed as a video complexity analyzer library, a command-line executable is provided to facilitate testing and development. We expect VCA to be utilized in many leading video encoding solutions in the coming years.

VCA is available as an open-source library, published under the GPLv3 license. For more details, please visit the online software documentation here. The source code can be found here.

A heatmap of the spatial (E) and temporal (h) complexity is shown below.

Heatmap of spatial complexity (E)

Heatmap of temporal complexity (h)

 

 

 

 

 

 

 

 

 

A performance comparison (fps) of VCA (with different levels of threading enabled) compared to Spatial Information/Temporal Information (SITI) is shown below.

Further information about possible VCA applications can be found at https://athena.itec.aau.at/, e.g. (list to-be-continued;),

• Live-PSTR: Live Per-title Encoding for Ultra HD Adaptive Streaming
• OPTE: Online Per-title Encoding for Live Video Streaming
• CODA: Content-aware Frame Dropping Algorithm for High Frame-rate Video Streaming
• VQEG NORM talk on Video Quality Analyzer
• INCEPT: INTRA CU Depth Prediction for HEVC
• Efficient Content-Adaptive Feature-based Shot Detection for HTTP Adaptive Streaming

Open Source Scalable Video Coding (SVC) Software

After DVB and ATSC announced to consider Scalable Video Coding (SVC) within their standards, I thought it would be interesting to blog about SVC software that is publicly available, especially because SVC has found its way already into video conferencing products (e.g., Vidyo, RADVISION, GIPS, SPIRIT DSP). Currently, I'm aware of the following open source SVC software:

• SVC Reference Software (JSVM software) which focuses on functionality rather than performance. Most of the people use this for research purposes as the reference codec.
• The P2P-Next consortium provides its SVC software (encoder/decoder) as open source under LGPL which comprises an optimized version of the JSVM for both encoding and decoding.
  
• The Open SVC Decoder has been released recently - also under LGPL - and provides an alternative to the above mentioned implementations. Interestingly, it provides an integration for the Core Pocket Media Player (TCPMP) and Mplayer. Further information can be found on their Wiki.

In case you think I missed anything, don't hesitate to comment...

Nokia releases the source code of its MVC implementation

Nokia announced today open availability of its MVC source code implementing H.264/MPEG-4 MVC, a leading multi-view video coding standard created jointly by the Telecommunication Standardization Sector of ITU and the Moving Picture Experts Group, MPEG of ISO/IEC. The source code is fully optimized for different uses in mobile devices and includes a player for the Maemo environment, making it easy for developers to take full advantage of the source code in an open, expandable ecosystem.

MVC is the newest addition to highly successful and widely deployed H.264/AVC family of video codecs serving multi-view and 3D video usage. It provides superior network robustness and compression performance for delivering 3D video by e.g. taking advantage of the inter-view dependencies of the different visual channels. In addition, its backwards compatibility with H.264/AVC codecs makes it widely interoperable in environments having both 2D and 3D capable devices.

Nokia acknowledges finalization of the MVC specification and is proud to contribute to the MVC community with this software release. “We are all very excited about this technology and the way it adds a whole new dimension to visual communication and entertainment. We hope this donation helps the developer community to take advantage of the full potential of the MVC technology and stimulates the creation of innovative applications and services”, says Jyri Huopaniemi, Director of Nokia Research Center Tampere laboratory.

The software is available for download at: http://research.nokia.com/research/mobile3D. The implementation has also been showcased at Nokia World 2008 using a modified Nokia N800 device with an integrated auto-stereoscopic display for natural real-time 3D video playback.

CfP: MPEG Extensible Middleware (MXM) Developer's Day

1st International MPEG Extensible Middleware Developer's Day (MXMDevDay) 2009
30 June, 2009, Queen Mary University London, UK

Details will appear soon at the MXM Web site under http://mxm.wg11.sc29.org/.

The purpose of this event is to share with the software developers community the state of the art and the prospects of the MPEG Extensible Middleware (MXM), a standard designed to promote the extended use of digital media content through increased interoperability and accelerated development of components, solutions and applications.

The MXM standard will promote the creation of a global market of applications, devices, and MXM components. As a result, we expect that innovative business models will be easily deployed.

The MXM developers day is particularly targeted to people:
— doing research and development of digital media architectures
— developing digital media applications and services
— developing digital media middleware
— interested in new opportunities in digital media
— developing and distributing open source software

Date & Venue
Tuesday, 30 June 2009, 9:00 – 13:00
Room 328, Francis Bancroft Building, Queen Mary University of London, Mile End Road, London E1 4NS

Programme
Session 1 (09:00-09:45): Opening, Chair: Panos Kudumakis
The MXM Vision, Leonardo Chiariglione

Session 2 (09:45-11:15): MXM Overview, Chair: Christian Timmerer
MXM Architecture, Filippo Chiariglione
MXM API, Marius Preda

Coffee Break (11:15-11:45)

Session 3 (11:45-13:00): Software Development
MXM Reference Software
Selected MXM Applications

Registration
The event is free of charge but possible attendees shall register to this event via the MXM Web Site at http://mxm.wg11.sc29.org/.

Chairs

• Filippo Chiariglione
  CEDEO.net, Italy, filippo(at)cedeo(dot)net
• Panos Kudumakis
  Queen Mary University, London, UK, panos(dot)kudumakis(at)elec(dot)qmul(dot)ac(dot)uk
• Marius Preda
  Institut TELECOM, France, Marius(dot)Preda(at)it-sudparis(dot)eu
• Christian Timmerer
  Klagenfurt University, Austria, christian(dot)timmerer(at)itec(dot)uni-klu(dot)ac(dot)at

ACM Multimedia 2008: Day 3

The open source track was a great success thanks to its contributors presenting

• Network-Integrated Multimedia Middleware (NMM)**
• LIRe: Lucene Image Retrieval (An Extensible Java CBIR Library)
• GpuCV (An OpenSource GPU-Accelerated Framework for Image Processing and Computer Vision)
• An Open Source Software Framework for DVB-* Transmission: OpenCaster and FATCAPS
  
• FOBS: An Open Source Object-Oriented Library for Accessing Multimedia Content

** ... awarded with the ACM Multimedia Open Source Award.

The panel discussion was about multimedia education: can we find unity in diversity? All panelists concluded that there's a lack of good, comprehensive text books. Furthermore, we should also think about a multimedia curriculum which should be multi-disciplinary, i.e., should have modules from computer science, electrical engineering, and "arts" (e.g., production tools). See also here...