Cesar Teixeira	Iran Abrão	Eduardo Barrére	Maria da Graça Pimentel
Departamento de Computação			Departamento de Computação
Universidade Federal de São Carlos			Universidade de São Paulo/ São Carlos
{iran,barrere,cesar}@dc.ufscar.br			mgp@icmsc.sc.usp.br

This paper presents an architecture aimed at supporting distributed hypermedia applications focusing on the interchange of multimedia objects over the Internet, their processing and presentation. The model is based on DAVIC proposals for interactive TV, which means that MHEG-5 is adopted as the representation standard for multimedia objects and DSM-CC as the real time protocol for data delivering.

Keywords: DAVIC (Digital Audio-Visual Council), MHEG-5 (Multimedia and Hypermedia information coding Expert Group), DSM-CC (Digital Storage Media Command and Control), Multimedia Objects, Real Time Delivery.

The distribution of data and processing is a strong tendency as a solution to attend the requirements of new applications, as it is the case with hypermedia systems [1], which also impose the adoption of standards for representing and interchanging multimedia information.

The great improvement in the number of machines connected to the global network Internet, the widespread use and the increasing amount of new applications developed over such network have contributed to the evolution of hypermedia systems [1][2][4]. Nevertheless, several technical limitations are to be overcome in order that the Internet is able to satisfactory handle hyperdocuments that include remote multimedia data. Multimedia data, in the context of this paper, is defined as a temporal and spatial meaningful composition of objects of several media ¾ including at least one continuous media ¾ combined with rules for managing such a composition as a function of the occurrence of events such as user interaction, for example.

Among the diversity of services running on the Internet, the World Wide Web (WWW) has great popularity and is quite useful. This service allows the interchange of information among local or remote platforms, usually involving different operating systems.

In the WWW, information transverses the network as a file encoded by a standard language, the HyperText Markup Language ¾ HTML [5]. A web browser is a software that, running in the client workstation, offers facilities for navigation in the web and is responsible for retrieving an presenting information as well as managing user interaction. A closer view of a HTML document shows simple text with some special formatting and anchoring tags. Such a simplicity has contributed to the great acceptability of HTML. However, its drawback is the of representing multimedia objects that impose temporal constraints among its continuous media components, although improvements in the language would contribute to overcome such limitations.

Similarly, the protocol used for managing the transfer of HTML documents is also quite simple. HyperText Transfer Protocol (HTTP), the protocol used by the clients and servers in the WWW, was not designed for real time delivery of continuous media. As a consequence, multimedia applications in the web are limited to those which may accept previous download of continuous media for later presentation.

A totally distinct initiative to promote the development of distributed hypermedia applications is being taken by the Digital Audio-Visual Council ¾ DAVIC [6][7][8]. This is an international consortium of many companies interested in the establishment of standards and rules to regulate developments related to hypermedia systems aiming at dealing with real time delivery attending requests on demand, as required by applications such as Interactive-TV.

DAVIC proposals are based in sharing the same infrastructure used for cable television to solve its networking necessities, since the main focus is directed to applications retated to the television area, such as video-on-demand, for example.

Two good points of the DAVIC proposals are (a) the adoption of MHEG-5 [11] as the standard for the representation and interchange of multimedia objects and (b) the use of DSM-CC [9] as the communication protocol. The MHEG-5 standard, differently from HTML, was specifically designed to allow inter-media synchronization and on demand interchange of multimedia objects. The protocol DSM-CC is used for controlling interactive multimedia sessions as well as the resources involved; it is suitable for the real time delivery required by some applications involving continuous media.

The architecture proposed in this work take the DAVIC proposals for interactive-TV as reference. The authors believe that improvements on the Internet, resulting from the widespread use of high performance networks, will allow solutions based on such an architecture for applications demanding both delivery of continuous media in real time and inter-media synchronization.

This paper is organized as follows: a brief overview of DAVIC, DSM-CC and MHEG-5 is presented in Session 2; a description of the architecture proposed is given in Section 3; Section 4 discusses some related work and Section 5 presents the conclusion.

Figure 1 illustrates a simplified version of the architecture proposed by DAVIC for Interactive-TV. Three systems are emphasized:

• CPS (Content Provider System): the CPS system owns or is licensed to sell content to the service provider; its purpose is to load content into the Service Provider System.
• SPS (Service Provider System): the SPS is a collection of system blocks that accept process and present information for delivery to a Service Consumer System. It is responsible for attending client requests (consumers) for multimedia information.
• SCS (Service Consumer System): the SCS is responsible for the primary functions that allow a human or machine consumer service to interact with the SPS. Usually these functions are implemented in a Set Top Box (STB), which consists of the hardware and software needed for the integration of a TV-set (or VCR), user controls (remote control device) and the network.

S1 to S4 in Figure 1 indicate the data-flow on different levels of communication between the Service Consumer System and the Server Provider System.

DAVIC adopts the DSM-CC protocol in flows S2 and S3 (information in end-user and session levels, respectively). Multimedia objects provided by SPS to the SCS conform to the MHEG-5 standard.

Figure 1 - Simplified DAVIC architecture

DSM-CC [?reference?] provides high-level protocols for the high-level control of applications such as video-on-demand, home-shopping, etc. DSM-CC is independent of the transport level, which means that is does not demand a pure MPEG-2 Transport Stream Network to support such applications ¾ although high-speed networks are recommended.

The following areas are covered by DSM-CC [10]:

• Network Session and Resource Control
• Configuration of a Client
• Downloading to a Client
• VCR-like control of video stream
• Generic Interactive Application Services
• Generic Broadcast Application Services

DSM-CC is based on a global Client/Service and Network entities model. It does not specify how each entity must be implemented, instead it specifies the constraints applied to the User-to-User (U-U) and User-to-Network (U-N) interfaces.

The download and configuration capabilities of DSM-CC are quite important for systems with low-end clients such as set top boxes. For instance, a client can always be configured with the most recent version of software available at the server.

MHEG ¾ Multimedia and Hypermedia information coding Expert Group ¾ is an ISO/IEC standard developed in order to formalize the definition of the semantic and the syntax of multimedia objects, which are to be interchanged among interactive multimedia applications in a Client/Server architecture. MHEG aims at guarantying interoperability among heterogeneous platforms.

Part five of the MHEG standard, MHEG-5, is dedicated to applications with minimal resources, such as interactive-TV, and defines a reduced set of efficient primitives for the specification of multimedia objects suitable for such applications. Even basic terminals ¾ presenting memory and processing capability limitations ¾ may be used to run engines able to decode and present MHEG-5 objects.

The object-orientated paradigm is used to define the MHEG-5 class hierarchy. Multimedia applications in the context of MHEG-5 are event-driven. They are composed of a web of scenes and link objects which defines how navigation among scenes and actions must be executed when some specified events occur; events can be triggered, for instance, by a button being activated or the end of presentation of some continuous media being reached. Most applications can be developed using only the declarative code provided, although MHEG-5 also provides means to perform tasks which are better expressed in a procedural paradigm.

A scene object groups visual, audible, interaction and link objects. As a result, the application and all its scenes ¾ including related objects ¾ are designed only once and then delivered using MHEG-5 object Servers. The objects are downloaded to the Client and the MHEG-5 run-time engine is responsible for decoding, interpreting, presenting and managing them.

SMmD ¾ Distributed Multimedia Systems: Structure, Support and Applications ¾ is a project being carried out by the authors and associated researchers [3]. SMmD research focuses on the exploitation of architectures for hypermedia applications in distributed computing environments. In such a context, it was found quite suitable to use some ideas of the interactive-TV technology in order to take advantage of some of the very interesting proposals to deal with applications that require real time delivery of continuous media and inter-media synchronization. The evolution of computing networks and the perspective of high-bandwidth availability for even ordinary users may justify such option.

The basic modules of the SMmD underlying architecture are illustrated in Figure 2. The main set on the left includes the modules of the users’ workstation responsible for the access to the multimedia data, its interpretation and decoding, and its synchronization/presentation. The three modules on the right are the sources of data for users and clients.

The MHEG-5 engine is a kind of finite state machine ¾ its behavior is defined by the multimedia object currently under processed and driven by events which may result, for instance, from user interaction, or the end of the presentation of a continuous media stream, or the expiration of a timer.

Once the engine has interpreted properly a multimedia object, it commands the Presentation Control & Synchronization module in order to provide its exhibition. The contents to be exhibited may be included in the object, or be remotely located and addressed by reference in the object. In the later situation, code of local media servers must be invoked to retrieve the addressed data.

The functions of local media servers use the DSM-CC protocol in order to establish connections with equivalent parts, located in remote media servers (not shown in Figure 2). Streams of continuous media, such as audio and video, are managed in the connections while its presentation flows. Besides the exhibition of such streams, the Presentation Control & Synchronization module is responsible for intra-media and inter-media synchronization as well as for controlling user interaction.

The Presentation & Interaction Devices (PID) module is responsible for managing the exhibition and/or input devices as for instance screen, speakers, keyboard, mouse, etc. The Network Interface Unit (NIU) aggregates lower level software and hardware of the user’s workstation needed for communication with remote severs. Finally, still at the user’s side, the MHEG-5 engine also uses the DSM-CC protocol for its communications with MHEG-5 object servers.

As far as servers are concerned, Figure 2 emphasizes three types: Multimedia Object Servers, Media Servers and HTML Object Servers. Figure 2 does not show the NIU (Network Interface Unit) and DSM-CC modules which must be presented in each server.

Multimedia Object Servers are actually repository of MHEG-5 objects. Media Servers may be traditional file servers, for static media, or continuous media servers. HTML Object Servers are one-way gateways between the two worlds, the World Wide Web and the MHEG environment. They accept requests for a HTML document (properly encapsulated as MHEG-5 object requests), access the WWW to get the specified document, encapsulate the document as a MHEG-5 object and deliver it to the engine at the user’s side. Added with HTML navigation facilities in the Presentation Control & Synchronization Module, this feature allows the traditional navigation through the WWW using the interactive-TV based SMmD architecture.

Figure 2 - The SMmD architecture

Jheg (Java Mheg Engine), which model is presented in Figure 3, is the MHEG-5 engine implemented in the context of the SMmD project.

The Object Processor (OP) module is responsible by the effective interpretation of MHEG-5 objects and by the execution of related actions. When link objects are interpreted, the OP module registers them in the Link Engine module (LE), where the Link Table is maintained ¾ this table controls the mapping of LinkConditions into LinkEffects. The Object Store module (OS) is responsible for the disponibilization of prepared objects to the OP module, and the Object Decoder module (OD) decodes and prepare MHEG-5 objects.

When the MHEG-5 engine is first activated, it runs a cold start process that retrieves the first application object to be executed. From this point onwards, the whole set of MHEG-5 objects is available to the user.

Once the OP module gets the identification of the object to be processed, it requests the prepared object from the OS module. The OS module keeps a cache with scene objects prepared in advance, with the aim of minimizing access time, and objects that have been active before and as such have high probability to become active again in a short time period. If the OS module does not have the required object, it invokes the OD module to fetch, decode and prepare the desired object.

Based on the type of the objects, the OP module knows what attributes to look for in the data structure resulting from the object preparation. When processing an application or scene object, the module starts the execution of the action object specified in their OnStartUp attribute. All elementary actions are queued and processed accordingly. Link objects found by the OP module in scene or application objects are activated and registered at the Link Table.

Some events may occur as a consequence of the OP processing activities (synchronous events) or generated by processes that are asynchronous to the MHEG-5 engine, e.g. some user interaction. In the latter case, the occurrence of events is passed on to the OP through the primitive External Input( ). These external events are translated to proper events recognized by the standard and, similarly to the treatment of synchronous events, they are reported to the LE module.

Figure 3 - SMmD MHEG-5 engine model

On receiving the information that an event has occurred, the LE module checks active links of all active applications and scene objects in order to determine whether some link must be fired or not. When a link is fired, the LE module uses the primitive ProcessAction( ) to inform the OP module which action objects should be processed.

Figure 4 illustrates an MHEG-5 object of the type Application, that can be retrieved by a MHEG-5 engine through the activation of the primitives DirectoryOpen() and FileRead() in the DSM-CC module. After reaching the MHEG-5 engine, the object is processed by the OD module ¾ responsible by decoding and interpretation of the object.

After processing the information contained in the object Application, the engine ¾ using the primitives ShowText() and ShowAudio() ¾ activates the Presentation module to control the exhibition of the text and audio objects referred to in the object Application. The Presentation module then sends to the User Interface module the text object to be presented and activates the DSM-CC, using the primitives DirectoryOpen() and StreamResume(), in order to establish a connection with the Audio Server, and initiates the reception and presentation of the audio object.

The SMmD project has given attention to the implementation of the several modules of the architecture proposed to support multimedia applications, having the Internet as the communication media.

The current working version for the architecture has the Jheg module and the HTML Object Server implemented in Java; the DSM-CC module, also implemented in Java, is based on an implementation developed at the University of Massachussets at Lowell [?reference?].

Working prototypes for the Audio and Video Servers, originally developed in C++, are currently being ported to Java.

Under development are both (a) a more elaborated version of the Presentation Control & Synchronization module and (b) the implementation of scheduling mechanisms improving the handling of continuous media and the control of quality of service (QoS).

Several initiatives worldwide, including commercial ones, explore the standards MHEG-5 and DSM-CC. Even the DAVIC council, whose work focus on interactive-TV, has been developed work where DSM-CC is applied to the Internet [21]. In this context is recent work by the Alex de Jong’s research team [22][23].

This paper summarized the efforts to (a) the standardization of the representation format of multimedia objects in order to guarantee interoperability among heterogeneous platforms (MHEG-5) and (b) the establishment of protocols and application models (DSM-CC, DAVIC).

Based on those standards and on the perspective of an elevated increase of the Internet bandwidth, an architecture for supporting multimedia applications over the Internet was proposed and the current stage of its implementation presented ¾ such applications are not supported by the current largely adopted standards HTML and HTTP.

The solution of representing multimedia data conforming to MHEG-5 facilitates the authoring process, which can be made using declarative code, and demands a low complexity run time engine. The extension of the engine, in order to be able to support new objects specified using the under development standard MHEG-6 [18], will allow the execution of fairly complex multimedia applications.

The requirements demanded for controlling the interchange of continuous media data are achieved with the adoption of the DSM-CC protocol.

The Java implementations, besides having portability, allow the gradual up-dating of the (client-side) run-time software by means of downloading as often as needed.