MPEG's Advanced IPTV Terminal (AIT) Ready for Take Off?

During its 90th meeting, MPEG remarkably revised and improved the requirements for the Advanced IPTV Terminal (AIT) standard. Please see a previous post for an overview. The main objective of AIT is as follows and taken from N11070.

The AIT standard will support the service providers’ drive to deploy innovative multimedia services ... by identifying a set of “Basic Services” and defining the corresponding set of protocols and APIs to enable any user in an AIT value chain to access those services in an interoperable fashion.

Currently, requirements for 30 of these basic services have been defined which can be clustered in services pertaining to the identification, authentication, description, storage, adaptation, posting, packaging, delivery, presentation, interaction, aggregation, management, search, negotiation, and transaction. Furthermore, basic services can be grouped to so-called "Aggregated Services" enabling one to offer a bundle of services to someone else.

Interestingly, the requirements document also defines some "standard-related requirements" which are as follows:

1. The Standard shall be based on technologies from Recommendations | International Standards unless no Recommendation | International Standard provides a solution satisfying the Requirements
2. The Standard shall only reference, not specify technologies. If new technologies will be required they will be specified by appropriate standards. However, some “glue” specification may be needed for integrating the technologies
3. The Standard shall specify (a) protocols to enable users to call services, (b) Application Programming Interfaces to access services, (c) bindings to specific programming languages, and (d) informative examples of AIT devices (server and client).

In other words, MPEG will only make references to existing standards and if something is missing, will specify amendments in the respective existing standards and reference them from within AIT.

Okay, so what's now the list of basic services and their definitions. Note that each of the services have individual requirements and also a section about what is being requested by proponents including a list of candidate technologies developed by MPEG.

1. Identify User: A Service that identifies Users in an AIT value chain
2. Authenticate User: A Service that Authenticate Users in an AIT value chain
3. Describe User: A Service that allows Description of Users of an AIT value chain
4. Identify Content: A Service that allows Identification of Content in an AIT value chain
5. Authenticate Content: A Service that allows Authentication of Content in an AIT value chain
6. Describe Content: A Service that allows Users associate Metadata to Content in an AIT value chain
7. Store Content: A Service that allows Users in an AIT value chain physically saves Content
8. Adapt Content: A Service that allows Users in an AIT value chain to process Content, e.g. conversion from one format to another, on the fly, …
9. Post Content: A Service that lets Users of an AIT value chain make other Users aware of their Content including a way to access it, e.g. as putting URI in an EPG or in a web page
10. Package Content: A Service that allows Users of an AIT value chain to make Content ready for delivery, e.g. creating MPEG-21 file for a certain Content for download
11. Deliver Content: A Service that allows Users to transfer Content between Users of an AIT value chain
12. Present Content: A Service that allows Users to experience Content, e.g. when a Device decodes video into a sequence of images displayed on the screen
13. Interact with Content: A Service that allows Users interact with Content
14. Identify Service: A Service that allows Users to assign a unique Identifier to Services of an AIT value chain
15. Authenticate Service: A Service that allows Users to confirm the identity of a Service
16. Describe Service: A Service that allows Users to associate metadata to Services of an AIT value chain
17. Aggregate Service: A Service that allows Users to combine different Services at the technical level to an Aggregated Service of an AIT value chain
18. Identify Device: A Service that allows Users to assign a unique Identifier to Devices of an AIT value chain
19. Authenticate Device: A Service that allows Users to confirm the identity of a Device
20. Describe Device: A Service that allows Users to describe Devices of an AIT value chain
21. Manage License: A Service that handles the life cycle of a Licence for Users in an AIT value chain, e.g., to create, issue, authenticate (i.e., verify integrity and authenticity of the Licence), authorize (the Rights in), revoke and expire a Licence.
22. Manage Tool: A Service that handles the life cycle of a Tool in an AIT value chain, e.g., to create, identify, describe, deliver, install, authenticate, authorize (use of), uninstall and update a Tool
23. Manage Event: A Service that caters to the life cycle of Events in an AIT value chain, e.g., to request and report Events
24. Identify Group: A Service that allows Users to assign a unique Identifier to Groups of an AIT value chain
25. Describe Group: A Service that allows Users to describe Groups of an AIT value chain
26. Authenticate Group: A Service that allows Users to confirm the identity of a Group
27. Manage Group: A Service that allows Users to manage life cycle of groups of entities such as Users, Device, Contents, or Services, in an AIT value chain
28. Search Entity: A Service that locate Entities such as Users, Device, Contents, or Services, in an AIT value chain satisfying some criteria
29. Negotiate Entity: A Service that helps Users of an AIT value chain achieve an agreement on the terms of use of an Entity, e.g., rights and conditions to use Content, Service (e.g., Delivery, Adaptation)
30. Transact Entity: A Service that allows Users in an AIT value chain to interface with Payment and Cashing systems

Note that the terms used in the basic services are defined in the Annex of N11063.

MPEG Media Transport is taking shape

Achievements at the 90th MPEG meeting in Xi’an, China with respect to MPEG Media Transport (MMT):

• Context & Objectives;
• Ad-hoc Group; and
• Future workshop in January 2010 on the Wednesday during the Kyoto meeting focusing on “adaptive progressive transport” and “cross-layer design”.

The first document, context & objectives, describes the scope of the standard based – as usual – on a number of use cases. These are adaptivity, delivery, QoS/QoE-awareness, convergence, and transparency which are briefly highlighted in the following:

• Adaptivity is referred to adaptability of multimedia content towards the usage environment context in which the content is being used. Ultimately, this includes the end user and his environment but also the delivery and other areas.
• Delivery differentiates between (traditional) IPTV, progressive download, and peer-to-peer delivery.
• QoS/QoE-awareness refers to several optimization techniques which roughly can be clustered into application-level and cross-layer optimizations. The former is further divided into retransmission and buffer management as well as forward error correction (FEC). The latter can be categorized into bottom-up, top-down, and integrated approaches well-known from the research literature.
• Convergence includes topics related to network convergence, i.e., media-aware devices, and service convergence.
• Transparency is mainly referred to as transparency with respect to different content protection schemes and rights management systems.
  

The current requirements document, however, focuses on two main areas which are adaptive delivery and cross-layer optimization. A couple of general requirements (i.e., IMHO a wish list we all pretty much agree on) precede the requirements pertaining to these two selected topics which are listed below followed by my comments:

Anyway, all the technologies that will be submitted as a response to a call for proposals – most likely this will happen in April 2010 – will have to show evidence that they are better than their predecessors (i.e., MPEG-2 TS) but it is pretty much unclear how this evaluation is going to be performed.

Some people argue that the functionality of adaptive, progressive download could be achieved with (minor) extensions of the MP4 file format but is something the industry definitely needs right now. Therefore, solutions for this part of MMT shall be standardized without any delay, i.e., at maximum warp speed! In my view, such a requirement should be included in the list above although it is kind of obvious…

Cross-layer (XL) optimization is a research topic which is around for quite some time with pretty cool results. However, the results mainly stem from XL stuff being done at the lower layers, i.e., link layer and so on. So what MPEG can offer is to standardize an interface – parameter names & semantics – to XL optimization modules whereas these modules should be out-of-scope and left open for industry competition. A kind of interface that could be used for this purpose is Supplemental Enhancement Information (SEI) – originally standardized during the course of the AVC (MPEG-4 Part 10 | H.264) developments – which nowadays can be included in MPEG-2 TS, file formats based on the ISO Base Media File Format, and also RTP. Thus, what is needed is simply a registration authority for SEI messages in the context of XL optimization. However, in order to represent advanced XL models, further standardization might be needed but we have already demonstrated how this could be accomplished existing MPEG-21 Digital Item Adaptation tools [1][2][3]. As in the case of SEI message, the only missing link is some kind of registration authority for parameter names and semantics which have not been considered when the standard has been finally approved – that was already in 2003.

Finally, I’d like to mention the Ad-hoc Group (AhG) where discussions between meetings take place and the workshop in January 2010 on the Wednesday during the Kyoto meeting focusing on “adaptive progressive transport” and “cross-layer design”. Details about the workshop you can find here. Note that usually such a workshop is open to everybody, i.e., you don’t need to be a member of ISO/MPEG in order to attend this workshop.

References:
[1] I. Kofler, C. Timmerer, H. Hellwagner, and T. Ahmed: “Towards MPEG-21-based Cross-layer Multimedia Content Adaptation”, Proc. 2nd International Workshop on Semantic Media Adaptation and Personalization (SMAP 2007), London, UK, Dec. 2007.
[2] C. Timmerer, V. H. Ortega, J. M. González, and A. León: “Measuring Quality of Experience for MPEG-21-based Cross-Layer Multimedia Content Adaptation”, to appear in Proceedings of the 1st ACS/IEEE International Workshop on Wireless Internet Services (WISe’08), Doha, Qatar, April 1-4, 2008.
[3] I. Kofler, J. Seidl, C. Timmerer, H. Hellwagner, I. Djama and T. Ahmed, “Using MPEG-21 for cross-layer multimedia content adaptation”, Journal on Signal, Image and Video Processing, Springer, vol. 2, no. 4, Dec. 2008.

MPEG news: a report from the 90th meeting in Xi'an, China

The 90th MPEG meeting in Xi’an, China is coming up with some very interesting news which are briefly highlighted here. First and, I think, most importantly, the timeline for the new MPEG/ITU-T video coding format has been discussed and it seems the final Call for Proposals (CfP) will be ready in January 2010. A draft CfP is available now and hopefully will be also publicly available if they solve all the editing issues until early November. This means that the proposals will be evaluated in April 2010 (note: this will be a busy meeting as a couple of other calls need to be evaluated too; see later). The CfP defines five classes of test sequences with the following characteristics (number of sequences available in brackets):

• Class A with 2560x1600 cropped from 4Kx2K (2);
• Class B with 1920x1080p at 24/50-60 fps (5);
• Class C with 832x480 WVGA (4);
• Class D with 416x240 WQVGA (4); and
• Class E with 1280x720p at 50-60fps (3).

For classes B, C, and E subjective tests will be performed whereas classes A and D will be only evaluated objectively using PSNR. The reason for evaluating A and D using objective measurements is due to its insignificant subjective differences with B and C respectively. Finally, they’re still discussing about the actual common nickname name of the standard as it seems some are not happy with high-performance video coding but that’s yet another story…

Second, 3D video coding is still a major topic in MPEG but you probably need to wait yet another year until a Call for Proposals will be issued. That is, a 3DV standard will be probably available around the beginning of 2013 at the earliest. The major issue right now is the availability of content – as usual – and different device manufacturer standards with respect to 3D video.

The third major topic at this meeting was around AIT and MMT, two acronyms you shall become more familiar in the future. The former is referred to as Advanced IPTV Terminal (AIT) and aims to develop an ecosystem for media value chains and networks. Therefore, basic (atomic) services will be defined including protocols (payload formats) to enable users to call these services, Application Programming Interfaces to access services, and bindings to specific programming languages. Currently, 30 of these basic services are foreseen which can be clustered in services pertaining to the identification, authentication, description, storage, adaptation, posting, packaging, delivery, presentation, interaction, aggregation, management, search, negotiation, and transaction. The timeline is similar as for HVC which means that proposals will be evaluated in April 2010. The latter is referred to as MPEG Media Transport (MMT) and basically aims to become a successor of the well-known MPEG-2 Transport Stream. Currently, two topics are explored for which also requirements have been formulated. The first topic covers adaptive, progressive transport and the second topic is in the area of cross-layer design. Further topics where this activity might look into are hybrid delivery and conversational services. As for HVC and AIT, the proposals are going to be evaluated in April 2010. However, in order to further refine this possible new work item, MPEG will held a workshop in January 2010 on the Wednesday during the Kyoto meeting focusing on “adaptive progressive transport” and “cross-layer design”.

However, MPEG is looking forward to a very busy meeting in April 2010 which by the way will be held in Dresden, Germany.

Another issue that has been discussed in Xi’an was (again) the development of a royalty free codec within MPEG. While some might say that within MPEG, trying to establish a royalty free codec is a first step towards failure, others argue that MPEG-1 is already royalty free, for MPEG-2 most patents expire in 2011, the Internet community is requesting this (e.g., IETF established coded group and Google has chosen On2, a royalty free codec), and, finally, MPEG-4 Part 10 royalty free baseline basically failed. Thus, maybe (or hopefully) this is the right time for a royalty free codec within MPEG and who can predict the future? Anyway, there’s some activity going on in this area and if you’re interested, stay tuned…

Finally, I’d like to note that MPEG-V (Media Context & Control) and MPEG-U (Rich Media User Interface) are progressing smoothly and both going hand in hand towards its finalization. This meeting, the FCDs have been approved which forms a major milestone as this was the last chance for substantial new contributions. One such input was related to advanced user interaction like the Wiimote, etc. which will become part of MPEG-V but used also by MPEG-U. Hence, one might argue merging these two standards into one single standard called MPEG-W (i.e., U+V=W) and a wedding ceremony could be performed at the next meeting in Kyoto with Geishas as witnesses … why not? Please raise your voice now or be silent forever!

Streaming Movies to Playstation 3

The New York Times reports that in the future - next month - movies will be streamed to Sony's Playstation 3 (PS3). The movies (and TV shows) are actually coming from Netflix giving this company another distribution channel. Therefore, Sony offers its own Internet movie store available (movies for about $3-4) and in order to access the Netflix service a monthly subscription of $8.99 or more is needed. The actual implementation is not very elegant as it comes as a "DVD-by-mail". That is, one needs a special disk from Netflix (free of charge) and users need to pop it into their PS3 each time they would like to access the service. This situation should change once the PS3 software gets updated which will happen next year.

Anyway, it seems that the PS3 is becoming yet another STB-ish device... excellent!

First impressions from Xi'an, China

After having arrived in Xi'an, China on Saturday afternoon, my first experience was already at Beijing airport with a Chinese Red Bull can (see figure left) and I've to say that the taste is somewhat different to what is served back at home. The term "different" seems to be something which can be appropriately applied to whole China? Well, let's see what's coming next.

My hotel is very nice and (very) luxury - depending on what you're usually used to. However, although I've a room on the 15th floor (out of 18) I do not have a clear view of the city (see figure right). Thus, I wonder whether the Chinese language has any sign for respirable dust...

Another experience was my first dinner together with some colleagues from MPEG. We simple entered an arbitrary restaurant that looked nice from outside and ordered something by picture. Yes, "order by picture" which is similar to "query by example" where one also gets something similar to what is represented in the example picture (see figure below ;-)
Finally, the people here are very nice and friendly although sometimes communication turns out a major issue resulting in strange misunderstandings. For examples, yesterday evening we asked for cold drinks and just got ice cubes as we had our drinks already on the table, or trying to order fried rice as shown in the figure below, hmmm...
The fun will continue tonight, I assume ... so stay tuned!

An Important Step Towards UMA/UME

In my view such an API is an important step towards realizing the objectives behind universal multimedia access/experience after the beginning of standardization work around CC/PP & UAProf, MPEG-21 DIA, etc.

in Bezug auf: Device APIs Requirements (auf Google Sidewiki anzeigen)

SAMT 2009: Call for Participation

-----------------------------------------------------------------------
SAMT 2009
4th International Conference on Semantic and Digital Media Technologies
-----------------------------------------------------------------------

2-4 December 2009
Graz, Austria

The 4th International Conference on Semantic and Digital Media
Technologies (SAMT 2009) targets at narrowing the large disparity
between the low-level descriptors that can be computed automatically
from multimedia content and the richness and subjectivity of semantics
in user queries and human interpretations of audiovisual media - The
Semantic Gap.

The preliminary program of SAMT 2009 can be found at http://www.samt2009.org/program
Next to the full and short papers of the main conference SAMT offers:

* Keynotes

- Ricardo Baeza-Yates, Yahoo! Research Barcelona

- Stefan Rüger, Open University

* Industry day with presentations about how semantic multimedia
technologies are put into practice

* Tutorials

- Web of Data in the Context of Multimedia
Bernhard Haslhofer, Bernhard Schandl, Andreas Langegger, Wolfgang
Halb and Tobias Buerger

- MPEG Metadata for Context-Aware Multimedia Applications
Christian Timmerer and Mathias Lux

- A Semantic Multimedia Web: Create, Annotate, Present and Share
your Media
Lynda Hardman and Raphaël Troncy

* Workshops

- Workshop on Semantic Multimedia Database Technologies http://semudate2009.fim.uni-passau.de/

- 3rd Workshop on Learning the Semantics of Audio Signals http://lsas2009.dke-research.de/

Registration is open at http://www.samt2009.org/registration - early
registration ends on Nov. 1.