Semantic integration of TV data and services: A survey on challenges, and approaches

 
Semantic integration of TV data and services: A survey on challenges, and approaches
Web Intelligence and Agent Systems: An International Journal 1 (2010) 1–24                                                          1
IOS Press

Semantic integration of TV data and services:
A survey on challenges, and approaches
Bassem Makni a , Stefan Dietze a and John Domingue a
a
 Knowledge Media Institute, The Open University Walton Hall, Milton Keynes, MK7 6AA, United Kingdom
E-mail: {b.makni,s.dietze,j.b.domingue}@open.ac.uk

Abstract. In this paper, we are surveying the impact of semantic Web and semantic Web services on enabling novel television
features. These novel features include being Internet based, mobile, interactive, personalised, social and semantic. Many research
efforts have contributed to extending different aspects of television delivery and consumption, with respect to content production,
metadata handling, semantic enrichment and recommendation. They adhere to the semantic Web vision for two goals: the seam-
less integration of their data and the automation of their Web services interoperation and composition. Mainly two semantic Web
services approaches are used, namely a top-down and a bottom-up approach. We study the contribution of different Semantic
Web and Semantic Web Services-based approaches to enable novel TV features.

Keywords: Next-generation TV, Semantic integration, TV data, TV services, WSMO, WSMO-lite, micro-WSMO, SAWSDL,
semantic TV content annotation, social TV

1. Introduction                                                              the novel TV features, for instance content produc-
                                                                             ers, broadcasters, social scientists, interaction design-
   The television concept has evolved and ramified                           ers, etc. interoperability is a key issue. Chan and Zeng
from its early form, of telegraph transmission of vi-                        [22] impute the reason for the proliferation of metadata
sion [38], to our contemporary perception of televi-                         schemas to the requirements differences during the de-
sion. However, this contemporary perception is be-                           sign phase with respect to intended users, subject do-
coming increasingly vague, as today television content                       main, project needs, etc. These differences are radical
is scattered over broadcast streams, Web, and private                        among the TV parties, and are reflected at two levels :
Internet Protocol television (IPTV) networks, and is
                                                                               – TV data integration
accessible, among classical sofa TV, via diverse en-
                                                                               – TV Services interoperability
hanced devices such as smart phones, tablets, Apple
TV1 and soon via Google TV2 devices. This content                            By TV data, we refer to the multimedia content and to
scattering perplexes the TV experience by increasing                         the metadata describing this content. By TV services,
the time spent in searching for relevant media. Thus,                        we refer to the operations on the multimedia content
novel TV features, including personalisation and so-                         and its metadata such as consumption, publishing and
cial networks integration are required to enhance the                        retrieval.
TV experience.                                                                  TV data integration challenge is to some extent sim-
   Since many parties, from different backgrounds and                        ilar to the integration of Web data, in terms of di-
with different concerns, are involved in materialising                       versity and distribution. And thus, we conjecture that
                                                                             the lessons learnt from the Web of data movement are
    1 http://www.apple.com/appletv/                                          valuable for TV data integration, and especially defend
    2 http://www.google.com/tv/                                              the efficacy of semantic integration of TV data.

1570-1263/10/$17.00 c 2010 – IOS Press and the authors. All rights reserved
2                                 B. Makni et al. / Semantic integration of TV data and services

   Similar challenge are posed with respect to the inte-             evolution has not yet been tackled as far as we know.
gration of services. The Semantic Web services (SWS)                 Which is our main motivation for writing this survey
research efforts have been motivated with the need                   as TV data is an interesting use case for Semantic Web
for automation of Web services related tasks such as                 technologies.
discovery, orchestration, mediation, and composition.                   To clarify the specificity of TV related services, we
The first intention was the full automation of one or                start by introducing core features of next generation
more of these tasks by providing conceptual models                   TV, implying different parties and services types (syn-
that comprehensively delineate the semantics of the                  chronous and asynchronous services, WS-* compliant
services such as Ontology Web Language for Services                  and Web API etc.). Based on the requirements which
(OWL-S) [71] and Web Services Modeling Ontology                      arise from these aspects, we survey the main Semantic
(WSMO) [94]. These efforts led to complex frame-                     Web (SW) technologies usage within the TV domain
works and tools for Semantic Web services annota-                    and the SWS approaches that we classify as top-down
tion and brokering. Even with tools and communities                  or heavyweight and bottom-up or lightweight and their
built around these SWS approaches, their complex-                    potential for supporting the above requirements. Fi-
ity hindered their uptake at a large scale for the au-               nally, we foresee the upcoming research challenges in
tomation of services tasks by the Service-Oriented Ar-               the SWS domain.
chitecture (SOA) users. Moreover, the limited emer-
gence of potential service automation scenarios, for in-
stance, based on services which allow the creation of                2. Next generation TV features
more complex orchestration or which provide analo-
gous functionalities which could be exploited via SWS                  "New technology is transforming the TV industry",
discovery, has put into question the actual potential for            says Mark Thompson, BBC CEO for Observer [83].
SWS automation.                                                      We conjecture that the biggest catalyst of this transfor-
   Hence, efforts were made to redefine the SWS no-                  mation will be SW technologies.
tion and the scope of its underlying technologies. More                In this section, we classify the new TV features and
recently, a lightweight approach has been proposed                   demonstrate in the following sections the impact of
with the aim of popularisation of SWS annotation use,                SW technologies to materialise these features.
namely with the standardisation of Semantic Annota-
tions for WSDL and XML Schema (SAWSDL) [65] to                       2.1. The integration of the Internet and television
ensure interoperability. While taking advantage from
the lessons learnt from more heavyweight approaches,                    The integration of the Internet and television was
the lightweight approaches offer a less costly way of                being realised in both ways (a) by television services,
annotating services. However, they lack support for                  like Video on Demand (VoD), becoming available over
more complex reasoning, and therefore, provide only                  the Internet and (b) by television becoming connected
limited opportunities for the automation of services re-             to the Internet via connected TV and connected set-top
lated tasks.                                                         boxes . Full integration with a built-in Internet browser
   From our experience of using SWS to broker TV re-                 in the TV set-top box will be popularised with projects
lated services, within the European project NoTube3                  such as Google TV4 and Apple TV5 . We focus on (a)
that explores television’s future in the ubiquitous Web,             way that we designate Internet based television.
two requirements for services management are preva-                     Simpson and Greenfield [98] enumerate four ramifi-
lent: a) offering a lightweight means for service anno-              cations; namely IPTV, Internet Protocol Video on De-
tation and documentation, and b) enabling service bro-               mand (IPVoD), Internet TV and Internet video; and de-
kerage through automating service discovery and or-                  fine criteria that classify them. We note that these rami-
chestration [32] .                                                   fications cross the television boundaries to video espe-
   The literature contains surveys on semantic integra-              cially for Internet video and IPVoD. Which is consis-
tion [82,13,108,33], Semantic Web Services [93,35,                   tent with the The Telecommunication Standardization
86,73] and television evolution [2,12,63,41], but the                Sector of the International Telecommunication Union
impact of Semantic Web technologies on the television
                                                                       4 http://www.google.com/tv/
    3 http://www.notube.tv/                                            5 http://www.apple.com/appletv/
B. Makni et al. / Semantic integration of TV data and services                            3

(ITU-T)6 definition of IPTV as “multimedia services                 Multicasting support Multicast addressing7 is a net-
such as television/video/ audio/text/graphics/data de-              work technology for the delivery of information to a
livered over IP based networks managed to provide the               group of destinations simultaneously using the most
required level of QoS/QoE, security, interactivity, and             efficient strategy to deliver the messages over each
reliability”.                                                       link of the network only once, creating copies only
   Thus, our survey will cover Television and Digital               when the links to the multiple destinations split. In
video, as they share many research challenges for an-               the video delivery context, multicasting is crucial be-
notation and delivery.                                              cause of the data amount that could congest the net-
                                                                    work when redundant packets flood the routers [45].
2.1.1. Internet usefulness for TV experience
                                                                    IPVoD and Internet video support only unicasting to
   The two major advantages of Internet based televi-
                                                                    allow individual play functionalities such as pause and
sion over broadcast television are (a) the built-in back
                                                                    rewind. While Internet TV uses replicated unicasting,
channel and (b) the Internet Protocol (IP) .
                                                                    in which messages are sent one by one to each client
                                                                    [45], and IPTV supports multicasting.
a) The back channel or return path carries the user’s
   feedback to the broadcaster. This is compulsory for              Delivery methods Two methods could be used for
   the new TV features such as interactivity and the                video delivery, either a HTTP based method called the
   personalisation.                                                 progressive download or a streaming method via dedi-
b) While digital video is a precisely timed and contin-             cated streaming protocols such as Real-time Transport
   uous stream and IP networks carry a loosely timed                Protocol (RTP) over User Datagram Protocol (UDP).
   collection of data fragmented into discrete packets,             The UDP stateless nature is useful for real time video
   both technologies are coupled for the following rea-             streaming because dropping packets is preferable to
   sons:                                                            waiting for delayed packets.
      – IP networking low cost owed to massive                      Digital rights management Broadly refers to a set of
        equipment production,                                       policies, techniques, and tools that guide the proper use
      – IP standardisation                                          of digital content [102]. Video is one of the main ap-
      – IP independence from the physical commu-                    plications of Digital rights management (DRM) espe-
        nication layer, which could be either wired,                cially in IPTV and IPVoD.
        wireless, 3G, or 4G based network.
                                                                    Discussion
      – IP ubiquity, i.e. support by our quotidian
                                                                       We excerpt the criteria chosen by Simpson and
        devices, like mobiles, tablets, and console
                                                                    Greenfield [98] to classify IP video in Table 1.
        games, allows the TV mobility feature.
                                                                       The novel technologies are continuously changing
                                                                    the way we consume and produce television content.
2.1.2. IP Video classification criteria
                                                                    Thus, internet video has been classified [98] and re-
   The main criteria used to classify IP video delivery
                                                                    classified [99] during the last decade by refining the
systems are:
                                                                    classification criteria to consider new technological ad-
Network type By network type we refer to network                    vances. We foresee that the discussed classification
openness, while IPTV uses private networks to deliver               will soon be made obsolete by initiatives like Hybrid
content to subscribed users, IPVoD, Internet TV and                 Broadcast Broadband TV (HbbTV)8 and Google TV9 .
Internet video are delivered via public networks typi-                 The emergence of hybrid architectures, such as
cally the Internet.                                                 HbbTV, will solve the insufficiency of the existing net-
                                                                    works infrastructure to deliver large video content, like
Quality of Service The Quality of Service (QoS) can
                                                                    High-definition television (HDTV) and 3D television
be used to assign high priority to video packets so they
                                                                    (3D-TV).
are privileged by routers. However, this is meaningless
                                                                       Furthermore, Google TV predict a full integration
in public networks such as the Internet where each ap-
                                                                    of the Internet and television by providing a built-in
plication can mark its packets as high priority, so man-
aged video delivery QoS is used only in IPTV [98].
                                                                      7 http://en.wikipedia.org/wiki/Multicast
                                                                      8 www.hbbtv.org
  6 http://www.itu.int/ITU-T/index.html                               9 http://www.google.com/tv/
4                                  B. Makni et al. / Semantic integration of TV data and services

                                                             Table 1
                                                      IP video classification
          Criterion            IPTV                 IPVoD                       Internet TV             Internet video
          Network type         Private              Public                      Public                  Public
          Quality of Service   Managed QoS          Unmanaged QoS               Unmanaged QoS           Unmanaged QoS
          Multicast support    Multicasting         Unicasting                  Replicated unicasting   Unicasting
          Delivery method      RTP over UDP         Progressive download        HTTP streaming          HTTP streaming
          Rights management    Strong with DRM      Strong often with DRM       Fairly strong           Weak or nonexistent

Internet browser within the TV, and by allowing cross-                2.2.2. Re-enabling the social aspect of TV
searching over the Internet and television content.                      Initially, watching the TV was a social activity
                                                                      where the whole family gathered around their sofa TV
2.2. Interactive TV                                                   to watch the news or the night movie and talk about it
                                                                      with friends the following day. Later on, with the in-
                                                                      creasing use of Personal video recorder (PVR), watch-
   Interactive Television (iTV) is an active watching
                                                                      ing TV became an individual activity. That reduced the
experience engaging viewers in choices and actions.
                                                                      viewers discussions about programs, which, according
This aim of making the television more dynamic and
                                                                      to the water-cooler effect [31], these discussions could
participatory is as old as the television itself. However
                                                                      be more interesting and more entertaining for the view-
the last century attempts to make the TV more inter-
                                                                      ers than the program itself.
active were not followed by the expected uptake [54].
                                                                         Social TV is defined as opportunity to interlink peo-
This was mainly due to the high costs [92] and the in-
                                                                      ple and provide communication features to create con-
trusive interfaces [11] making the interactivity cum-
                                                                      nectedness via the TV [56,21,70,109]. We define it as
bersome. Hence, Jensen [54] described the iTV as a va-                an adaptation of the social principle “It’s not what you
porware10 , which is an advertised product, often com-                know, it’s who you know” [79] to the TV context: “It’s
puter software, whose launch has not happened yet and                 not what you watch, it’s what who you know watch”.
might or might never happen. Recently new pragmatic                   And when we consider the asynchronous communica-
approaches to iTV, consisting of building on the exist-               tion, from the taxonomy of TV sociability [25], it be-
ing and lowering the challenges, produced a new airi-                 comes “It’s not what you watch, it’s what who you
ness for iTV.                                                         know watch or have watched”.
2.2.1. Interactivity types                                               The study of social interactive television is also not
   Gawlinski [43] considers the lack of an agreed                     new [21], as Wellens [111] already stated in 1979
framework for describing different types of interactiv-               that “interactive television represents means of linking
                                                                      individuals together by providing each with an elec-
ity, as one of the iTV difficulties. However, the fol-
                                                                      tronically mediated representation of the other’s voice
lowing taxonomy of TV interactivity types from Curry
                                                                      and visual presence”. However the lack of TV specific
[28] is the most agreed one:
                                                                      guidelines for interaction, enforced the use of Human
Distribution interactivity refers to controlling the                  computer interaction (HCI) techniques [44], which re-
     content delivery but not the content itself.                     sulted in a cumbersome social interaction that does not
                                                                      meet the expected seamless TV experience.
Information interactivity consists of choosing the
                                                                         Recently, reenabling the social aspect to the TV has
     delivered information such as weather or local
                                                                      gained augmenting interest, so that MIT Technology
     news.
                                                                      Review11 listed it in the ten most important emerg-
Participation interactivity involves the viewer in ac-
                                                                      ing technologies of 2010. This interest is mainly owed
     tions and choices that bring dynamic content. A
                                                                      to the new opportunities offered by virtual social net-
     typical action is voting, and a possible choice is
                                                                      works and by the commercial potential of the social
     the camera angle during a soccer game.
                                                                      TV.

    10 http://en.wikipedia.org/wiki/Vaporware                           11 http://www.technologyreview.com/tr10/
B. Makni et al. / Semantic integration of TV data and services                           5

   We distinguish two means for social interactivity in                   a range of techniques, such as recommendation
the TV context:                                                           techniques based on collaborative filtering, for
                                                                          example.
  1. Using ancillary devices: This explores the me-
                                                                     Presentation Generation and Tailoring The selec-
     dia multitasking practice where the user simul-
                                                                          tion, organization, and customisation of televi-
     taneously uses Internet and mobile phones while
     watching TV. According to Nielsen Three Screen                       sion material based on viewer queries, processed
     Report [81] survey about Television, Internet and                    programs, and viewer models.
     Mobile Usage in the US, simultaneous usage rose                 Interaction Management Adapt the human com-
     in the first quarter of 2010 by 35% to reach 60%                     puter interaction techniques to the TV context.
     of TV viewers. Comcast’s Tunerfish12 uses a web                      The human-TV interaction should include mech-
     and mobile interaction to allow friends from Twit-                   anisms for attention and dialogue management.
     ter13 and Facebook14 to share feedback about TV                 Evaluation of the user’s satisfaction with respect to
     shows. The ancillary device could also be an en-                     speed and accuracy. The speed in which the sys-
     hanced control such as a sensor-enhanced pillow                      tem is adapted to the user’s preferences and accu-
     [6].                                                                 racy in terms of precision and recall of the recom-
  2. Directly on the TV screen: Where social interac-                     mended programs.
     tion is displayed on top of the watched program
     in form of avatars [26,80] for example.                         2.4. Next-generation TV challenges

2.3. Personalised television                                           Obviously, materialising the previously discussed
                                                                     TV challenges involves many parties, from different
   Similarly to the Web content expansion phenomena,                 backgrounds and with different concerns.
the TV evolution to Internet TV has been coupled with
the scattering of TV and multimedia content, where                     1. Network experts to adapt the Internet infrastruc-
the user struggles to find a relevant content, which per-                 ture for multimedia and TV data delivery.
plexes the leisure time. Hence, the importance of the                  2. HCI specialists to define TV interactivity patterns
personalised feature within the next-generation TV.                       and human-TV interaction.
   The personalisation of TV could benefit from re-                    3. Personalisation experts to propose TV specific
search advances in recommendation systems but re-                         recommendation systems.
quires previous adaptation to TV content and TV in-                    4. Social scientists to build social networks around
teraction. Ardissono et al. [3] enumerate the following                   the TV preferences.
challenges to enable personalized television:
                                                                     And thus interoperability is a key issue for TV data
Viewer Modelling The acquisition, representation,                    integration.
    and utilization of information about viewers, such                  Guenther and Radebaugh [47] define interoperabil-
    as their characteristics (e.g., gender and age),                 ity as “the ability of multiple systems with different
    preferences, interests, beliefs, and their viewing               hardware and software platforms, data structures, and
    behaviour. This includes individual and group                    interfaces to exchange data with minimal loss of con-
    modelling.                                                       tent and functionality”. Since any loss of TV data con-
Viewer Identification The recognition of the TV viewer(s)            tent or functionalities implies degradation of reasoning
    to provide personalized services.                                capacities such as personalisation, the seamless inter-
Program Processing Implying programs segmenta-                       operability of TV data is a high level requirement.
    tion, summarisation, and indexing.                                  Chan and Zeng [22] defines three levels of interop-
Program Representation and Reasoning Modelling                       erability:
    the programs’ characteristics to measure similar-
    ities or dissimilarities between the different pro-              Schema level when different schemas are used.
    grams. Reasoning about programs can include                      Record level the same schema is used with different
                                                                         semantic interpretations of the elements.
  12 tunerfish.com                                                   Repository level when accessing the data is depen-
  13 twitter.com                                                         dant to the used repository, which hinders cross-
  14 facebook.com                                                        collection searching.
6                                   B. Makni et al. / Semantic integration of TV data and services

                                                           Table 2
                                         SW and SWS impact on TV parties interoperability
                              Interoperability level   SW     SWS      expected impact
                              Schema                   X               TV data schema interoperability
                              Record                   X               Agreed semantics
                              Repository                      X        Unified way to access TV data

The three levels of interoperability issues are om-                       We organise the following sections according to
nipresent within the different TV parties.                             each challenge and discuss the impact of SWS to solve
  In the following sections, we try to answer the fol-                 them.
lowing questions:
                                                                       3.1. Bridging the semantic gap
    – To which extent can semantic Web technologies
      enable the semantic integration of TV data?
    – How can SWS enable TV services interoperabil-                       Effective management of multimedia assets, includ-
      ity?                                                             ing content-based indexing and retrieval, impose a
    – From our experience in brokering TV services,                    deep understanding of the content at the semantic level
      what hinders the SWS uptake?                                     [23]. That could be performed either manually or auto-
                                                                       matically. On the one hand, manual semantic annota-
                                                                       tion of multimedia content suffers from subjectivity of
3. Semantic integration of TV data                                     descriptions, which hinders interoperability [58], and
                                                                       is far from being a scalable solution. On the other hand,
   Ziegler and Dittrich [114] define the semantic inte-                the automatically extracted multimedia features are
gration as “the task of grouping, combining or com-                    low-level perceptual features, faraway from the high-
pleting data from different sources by taking into ac-                 level semantic descriptions that match human cogni-
count explicit and precise data semantics in order to                  tion [58]. In order to improve Content-Based Multi-
avoid that semantically incompatible data is struc-                    media Indexing and Retrieval (CBMIR) accuracy, the
turally merged.”                                                       research efforts have shifted from designing sophisti-
   Since TV data is a multimodal data composed of                      cated low-level features extraction algorithms to bridg-
a) Multimedia content b) Structured Metadata descrip-                  ing this so-called semantic gap [67].
tions of the multimedia content c) Semi-structured                        Kompatsiaris et al. [58] classify these efforts in the
metadata with free text descriptions of the programs                   following categories: Relevance feedback [96], knowl-
embedded in Electronic program guide (EPG) for ex-                     edge based [112] and multimedia ontologies. Liu et al.
ample , the semantic integration of TV data bene-                      [67] distinguish a specific category for Web image re-
fits from research advances in each modality a) se-                    trieval, which is HTML text fusion with visual content
mantic integration and retrieval of multimedia docu-                   from Web images, that we include into the multimodal
ments, b) multimedia metadata interoperability. c) Nat-                fusion [9]. We focus on multimedia ontologies cate-
ural Language Processing (NLP) and semantic enrich-                    gory as it is the most relevant for the Semantic Web
ment research.                                                         domain.
   The main efforts of applying semantic Web tech-                        Supported by the proved effectiveness of systems
nologies for TV data integration aim a reasoning based                 with limited context of application [112], the knowl-
personalisation of TV and bringing the social aspect to                edge based approaches model the domain of applica-
TV. Thus the next-generation TV challenges are:                        tion either explicitly or implicitly.
    1. Adapting the advances of semantic multimedia                    Explicit Model based approaches uses a priori domain-
       retrieval to the TV content.                                        specific knowledge [1,103] for guiding low-level
    2. Semantic integration of the different TV related                    feature extraction, high-level descriptor deriva-
       metatdata.                                                          tion and symbolic inference [58]. Chang et al.
    3. Semantic integration of semi-structured and struc-                  [24] introduced the idea of semantic visual tem-
       tured TV data.                                                      plates to link visual features to semantics, where
    4. Reasoning based personalisation of TV content.                      each template represents a personalized view of
    5. Enabling the Social TV.                                             concepts. Prior knowledge inspired from cine-
B. Makni et al. / Semantic integration of TV data and services                            7

    matic principles is relevant also in video classifi-            videos and concluded the efficacy of concept-based ap-
    cation [17]. Lighting level differentiate low light             proaches.
    horror movies from well-lit comedies[17]. And
                                                                    3.1.2. Linked multimedia
    motion speed separates fast action movies and
                                                                       The success of the semantic Web vision has been
    sports from slow drama. Audio effects are also
    pertinent to automatically detect horror movies                 limited to a small scope in enterprises and in vir-
    [76].                                                           tual communities. This is essentially due to the scope
Implicit Uses machine learning techniques for dis-                  of their domain knowledge that eases its modelling,
    covering complex relationships and interdepen-                  added to the non maturity of ontologies merging algo-
    dencies between numerical image data and the                    rithms. That led to isolated islets of semantic Web data.
    perceptually higher level concepts [58].                        This ascertainment motivated the Linked Open Data
                                                                    (LOD) proposition [14]. This new vision defines the
Multimedia ontologies play a key role in modelling                  Semantic Web as “a technology for sharing data, just
this knowledge in a shared formalisation that allows                as the hypertext Web is for sharing documents” [15].
automatic bonding of high-level concepts from the                   The linked data movement impulsed the adoption of
model to the extracted low-level features.                          the Semantic Web vision at a large scale by linking 25
3.1.1. Multimedia ontologies                                        billion Resource Description Framework (RDF) triples
   Kompatsiaris et al. [58] motivates the usage of mul-             from 203 datasets (as of September 2010).
timedia ontologies in formalizing the multimedia se-                   The need for this impulsion was also present within
mantics as they fulfil the following requirements:                  the semantic multimedia community, and hence the
                                                                    idea of adapting the linked data principles to the multi-
  1. Persistence: Modelling the multimedia semantics                media context. Burger and Hausenblas [18] enumerate
     evolution, such as the evolution of typical desk               the following principles to interlink multimedia data:
     components, to allow usage in future applica-
     tions.                                                           1. Follow the LOD principles
  2. Consistency: Precise and non ambiguous seman-                            – Use URIs as names for things
     tic annotations are crucial for efficient reasoning
                                                                              – Use HTTP URIs so that people can look up
     about the multimedia content.
                                                                                those names.
  3. Context enabled: As multimedia objects exist in
                                                                              – When someone looks up a URI, provide
     context, modelling this context information is
                                                                                useful information, using the standards (RDF,
     beneficial for multimedia retrieval [39].
                                                                                SPARQL)
We add that                                                                   – Include links to other URIs, so that they can
  4. syntactic annotations are liable to ambiguities and                        discover more things.
     thus not interoperable nor interpretable by ma-                  2. Consider the contextual aspect to represent the se-
     chines.                                                             mantics of multimedia content.
  5. Semantic annotations refer to a knowledge for-                   3. Deploy legacy multimedia metadata formats.
     malised by an external ontology to help solv-                    4. As the need to refer fragments of multimedia
     ing ambiguities via persistent and implicit anno-                   based on space and temporal parameters is fun-
     tations.                                                            damental [105], a mechanism to specify URIs for
  6. They are also operational annotations as they are                   these fragments is mandatory.
     intended to be consumed and generated by soft-                   5. Interlinking methods are essential in order to
     ware agents.                                                        manually or (semi-) automatically interlink mul-
   Naphade et al. [78] have modelled large-scale con-                    timedia resources.
cept ontology for multimedia (LSCOM) to enable au-                  Discussion
tomatic extraction of broadcast news video. Similar                    The majority of the domains modelled by multime-
approaches to link low-level Moving Picture Experts                 dia ontologies are relevant for TV content such as :
Group-7 (MPEG-7) features to higher level concepts
include [8,40,91,10]. Hauptmann et al. [50] have com-                  – News [78,51,100,55]
pared concept-based using LSCOM ontology and text-                     – Sports [113,30]
based retrieval accuracy over a collection of news                     – Movies [29,90]
8                                B. Makni et al. / Semantic integration of TV data and services

However the existing approaches for semantic annota-                      ber stations and related communities can share.
tion of multimedia are offline [7] due to the time con-                   PBCore extends Dublin Core by adding a num-
suming phases of features extraction and classification.                  ber of elements specific to audiovisual assets that
That hinders the adoption for live programs broadcast-                    falls into three groups:
ing and reveals the need for real-time semantic anno-
                                                                             1. Content: provides descriptive metadata ele-
tation of multimedia content.
                                                                                ments.
3.2. TV-related Metadata standards                                           2. Intellectual property: provides Rights man-
                                                                                agement metadata.
   We mentioned that digital video and TV share many                         3. Instantiation: contains all technical meta-
concerns such as content annotation, and thus we                                data about the physical or digital representa-
will cover multimedia annotation standards namely                               tion of the asset such as format, media type,
MPEG-7 and Society of Motion Picture and Television                             duration etc.
Engineers (SMPTE) metatdata dictionary and TV spe-                  TV-Anytime The TV-Anytime forum17 is a worldwide
cific TV-Anytime and Public Broadcasting Metadata                       project involving vendors, broadcasters, telecom-
Dictionary (PBCore).                                                    munications companies, and the consumer elec-
MPEG-7 is a multimedia content description interface                    tronics industry, which has defined an extensive
    standardised by the International Organization                      bundle of specifications for the use of local stor-
    for Standardization (ISO) and International Elec-                   age at home in a specialized “set-top box” or in
    trotechnical Commission (IEC). The standard de-                     the TV set [27].
    fines the MPEG-7 scope by addressing applica-
                                                                    Discussion The coexistence of many metadata stan-
    tions that can be stored (on-line or off-line) or
                                                                    dards for TV is practically equivalent to the lack
    streamed (e.g. broadcast, push models on the In-
                                                                    of standards as users will again fall to using non-
    ternet), and can operate in both real-time and non
                                                                    interoperable metadata schemas. Which is inevitable
    real-time environments.
SMPTE The SMPTE Metadata Dictionary15 [95] is a                     within heterogeneous communities [22]. Thus the need
    large list of structured metadata elements grouped              for core multimedia ontologies to enable multimedia
    in the following classes: Identification, Adminis-              metadata schemas interoperability [53].
    tration, Interpretation, Parametric, Process, Rela-                Moreover, the large number of elements of the dis-
    tional, Spatio-temporal, Organisationally Regis-                cussed metadata schemas reveals their complexity, and
    tered Metadata, and Experimental Metadata. Al-                  the semantics of these elements remain implicit. For
    though it was originally designed to be encoded in              example, very different syntactic variations may be
    the Key-Length-Value (KLV) data encoding stan-                  used in multimedia descriptions with the same in-
    dard, an Extensible Markup Language (XML) se-                   tended semantics, while remaining valid MPEG-7 de-
    rialisation is available.                                       scriptions, which causes serious interoperability issues
Standard Media Exchange Framework (SMEF) The                        for multimedia processing and exchange [104]. Hence
    BBC has defined SMEF16 to support and enable                    the need for multimedia ontologies unfolding metadata
    media asset management as an end-to-end pro-                    semantics and amending their interoperability at the
    cess from commissioning to delivery to the home.                records level.
    The SMEF Data Model (SMEF-DM) provides a
                                                                    Multimedia ontologies for records interoperability
    set of definitions for the information required in
                                                                        Mai Chan and Lei Zeng [68] define the records
    production, distribution, and management of me-
                                                                        level interoperability as the “efforts intended to
    dia assets, expressed as a data dictionary and a set
                                                                        integrate the metadata records through the map-
    of Entity Relationship Diagrams.
                                                                        ping of the elements according to the semantic
PBCore The PBCore was created by the Corpora-
                                                                        meanings of these elements”.
    tion for Public Broadcasting (CPB) in the United
                                                                        The multimedia ontologies used for records level
    States to provide a simple structure that its mem-
                                                                        interoperability include:
    15 http://www.smpte-ra.org/mdd/index.html
    16 www.bbc.co.uk/guidelines/smef                                  17 http://www.tv-anytime.org/
B. Makni et al. / Semantic integration of TV data and services                            9

     Core Ontology for Multimedia (COMM) [4] was                     the similar problem of TV content overload. Gauch
          designed by re-engineering the MPEG-7                      et al. [42] define user profiling as gathering and ex-
          standard in order to discover multimedia                   ploiting some information about users in order to be
          patterns. Patterns recognition was based on                more effective. In ontology-based user profiling [74],
          two of the main patterns of Descriptive On-                the user profile is represented in terms of interesting
          tology for Linguistic and Cognitive Engi-                  concepts [46].
          neering (DOLCE) which are Descriptions
          & Situations (D&S) and Ontology of Infor-                  3.3.2. Personalised TV
          mation Objects (OIO). The typical scenario                    The most used TV personalisation techniques are
          “the decomposition of a media asset and the
          (semantic) annotation of its parts” reveals                   – Content based: uses a metric to quantify the sim-
          the two main functionalities of MPEG-7:                         ilarity between viewers’ profiles and programs
          decomposition and annotation. The decom-                        based on their content description. Similarity es-
          position consists of segmenting the multi-                      timation is time consuming [37] due to the TV
          media content based on temporal, spatial                        content amount.
          or spatio-temporal descriptors. Then these                    – Collaborative filtering: recommends programs
          segments are annotated with the MPEG-                           based on estimated similar profiles. Despite its
          7 features descriptors. Following the D&S                       effectiveness is many domains, collaborative fil-
          pattern, decomposition is a Situation (Seg-                     tering for TV programs suffers from many issues
          mentDecomposition) that satisfies a De-                         [85] such as a) first-rater problem as new pro-
          scription (SegmentationAlgorithm).                              grams are not rated enough to be recommended,
      MPEG-7 Ontology Hunter [52] reverse-engineered                      b) cold-start problem where new users did not
          a core subset of MPEG-7 specification to                        rated programs yet and no valid recommendation
          generate an RDF Schema (RDFS) ontology
                                                                          could be suggested, c) sparsity problem consists
          describing MPEG-7 elements semantics be-
                                                                          on lack of overlap between two random viewers if
          fore generating a Web Ontology Language
                                                                          they did not rated the same programs . O’Sullivan
          (OWL) version18 .
                                                                          et al. [85] focus on c) as they consider it the most
Multimedia ontologies for schemas interoperability                        stringent problem of collaborative filtering for TV
    The Multimedia Metadata Ontology (M3O) is                             program recommendation.
    a follow-up initiative to COMM based also on
                                                                        – Social filtering: similarity between profiles is
    DOLCE but not restricted to MPEG-7 and thus
                                                                          based on their friendship in social networks un-
    capable of expressing all structural information
                                                                          like estimation in collaborative filtering.
    of many multimedia metadata formats while pre-
    serving the abilities of the COMM.                               To overcome the shortcomings of each technique, TV
                                                                     recommender systems tend to use hybrid approaches
3.3. Reasoning about TV data
                                                                     [85,37,66].
   The ultimate goal behind lifting the TV data to the               Semantic personalisation of TV By semantic person-
semantic level is allowing reasoning about TV data                   alisation, we refer to both viewers’ profiles enrich-
in order to facilitate personalisation, recommendation               ment with semantics i.e. ontology-based viewer profil-
and social TV features. Besides the semantic annota-                 ing and semantic representation of the TV content.
tion of TV data, the viewers and their context should                   In SenSee framework [5], viewer profile and context
be represented at the semantic level to allow automatic
                                                                     are extended via ontologies describing time, geograph-
matching.
                                                                     ical location and TV domain knowledge. The authors,
3.3.1. Viewer profiling                                              Aroyo et al. [5], prove the advantage of ontology-based
   Since user profiling and personalisation is a well es-            TV recommendation by drawing a quantitative com-
tablished solution to the information overload problem               parison with free text approach.
[42], it stimulates viewer modelling activity to solve                  Avatar [37] recommendation is based on hybrid ap-
                                                                     proach using collaborative filtering and semantic simi-
  18 http://metadata.net/mpeg7/                                      larity between the content and the user profile. The se-
10                               B. Makni et al. / Semantic integration of TV data and services

mantic similarity is calculated according to a dedicated            tecture and that the same architecture could lead to the
TV domain ontology19 .                                              programmable Web. From the RESTful prospect, the
   The NoTube BeanCounter [106] aggregates user’s                   WS-* specifications do not complement each other but
activity from different social networks and uses the ac-            usually overlap and compete, which is confusing Web
tivity stream in TV programs recommendation. The                    Services designers. Moreover building Remote Proce-
aggregation from different Web sources illustrates one              dure Call (RPC) upon Web is counter-intuitive and
of the most important advantages of semantic tech-                  does not take advantages of the Web’s REST archi-
nologies, for instance the NoTube BeanCounter aligns                tecture. More objectively, Pautasso et al. [88] made a
the data gathered from Last.fm with the BBC pro-                    quantitative technical comparison based on architec-
grammes ontology 20 in SKOS (Simple Knowledge Or-                   tural principles and decisions and concludes that REST
ganization System) [75].                                            is well suited for basic, ad-hoc integration scenarios
                                                                    and that WS-* is more flexible and addresses advanced
3.4. Discussion                                                     quality of service requirements.

   From the user’s perspective, the semantic integra-               4.2. TV services
tion of TV data brings the freedom of choice allow-
ing the cross-collection searching over many Inter-                    It became common practice throughout the last
net video and TV repositories while preserving a cen-               decade; to expose all sorts of multimedia content and
tralised profile adapted to each collection. That is                metadata stored in one particular repository through a
materialised through the semantic interoperability be-              set of Web APIs. The motivation for this practice is to
tween the different used multimedia metadata. How-                  allow the aggregation of the multimedia content and
ever the content description is still generated manually            its tailoring to users’ requirements. Which delegates
or via crowdsourcing due to the remaining semantic-                 the adaptation of multimedia consumption to third par-
gap between the computed multimedia features and the                ties developers and exempt the multimedia providers
content concepts. Thus the semantic integration of TV               from maintaining different platforms end-points such
data is dependant to the accuracy of the content de-                as Android21 and iOS22 platforms.
scription.                                                             To name but a few, major TV broadcasters such as
                                                                    BBC expose their data via Web endpoints23 and pro-
                                                                    vide APIs to process these data, and YouTube24 data
4. Semantic integration of TV services                              API25 allows a program to search for videos, retrieve
                                                                    standard feeds, and see related content.
   Towards collaboration, the different next-generation                However, these APIs are not standardised in terms
parties adhere to the SOA to expose their services                  of inputs, outputs, nor invocation methods, and specific
and consume others’. Specifically, the NoTube SOA is                clients should be designed to each API. That raises the
based on Web services to take advantage of the well-                repository level interoperability issue. In the following
established Web architecture as a medium for services
                                                                    section, we present the major Semantic Web services
communication.
                                                                    approaches that tackle this issue generally and in TV
                                                                    context specifically.
4.1. The Web of services
                                                                    4.3. Semantic Web Services
   Mainly two approaches are used to adapt the Web to
a Web of services: the first approach proposes a stack
                                                                       Whatever the used approach RESTful or WS-* Web
of specifications to support the services requirements
                                                                    Services, performing complex operations such as ser-
such as communication, selection, security etc. giving
                                                                    vices selection, composition, or mediation requires hu-
this approach the Big Web Services name [88], the sec-
ond approach, RESTful Web Services, claims that the
                                                                      21 http://www.android.com/
success of Web is due to the maturity and ease of use
                                                                      22 http://www.apple.com/iphone/ios4/
of its Representational State Transfer (REST) archi-                  23 http://backstage.bbc.co.uk/data/Data
                                                                      24 YouTube.com
  19 http://avatar.det.uvigo.es/index-i.html                          25 http://code.google.com/apis/youtube/
  20 http://purl.org/ontology/po/                                   getting_started.html\#data_api
B. Makni et al. / Semantic integration of TV data and services                             11

man intervention. However, these operations should be                4.4.1. Capabilities representation
automated as the users required functionalities are sel-                There are two approaches to represent the capabil-
dom achieved via one Web Service; hence the need                     ities of a Web Service, the first one is based on an
for automated services composition and mediation in                  extensive ontology of functions where capabilities ad-
large scale context. Such as the Web 1.0, the software               vertisement is done by binding to a class of homoge-
agents could not reason about Web Services as they see               neous functions within a taxonomy of services such
them as inputs and outputs without any conscience if,                as flight booking, or transportation service. The sec-
for example, the received message contains a ranked                  ond approach models the flow of state transformations,
                                                                     in occurrence the flight booking service requests a de-
list of programs from a recommendation service or a
                                                                     parture and arrival cities, a departure and arrival dates
program description from an EPG service. And to au-
                                                                     and a credit card, and changes the state by decreasing
tomate services reasoning, the software agents should
                                                                     the number of available seats and withdrawing the due
process the services messages and operations at the se-              amount of money.
mantic level via shared ontologies. Which is the re-
search field of SWS [72] to solve at least one of the                4.4.2. Artificial intelligence planning
following challenges.                                                   Stuart et al. [101] define artificial intelligence plan-
                                                                     ning (AI planning) as “a kind of problem solving,
4.3.1. Discovery                                                     where an agent uses its beliefs about available actions
   Semantic Web Services discovery consists on re-                   and their consequences, in order to identify a solution
trieving the relevant services that achieve the requested            over an abstract set of possible plans.” This agent, the
functionalities. Mainly discovery is based on advertis-              planner, accepts three inputs
ing service capabilities in a centralised or distributed               1. a formalised description of the initial state of the
registry, and matchmaking these capabilities with the                     world.
requests.                                                              2. a formalised description of the agent’s goal (the
                                                                          desired behaviour).
4.3.2. Composition
                                                                       3. a formalised description of the possible actions
   We noted that complex functionalities are rarely                       that can be performed (the domain theory)
achieved via one Web Service, yet many functionali-
ties from different Web Services should be combined                  and outputs a sequence of actions that, when executed
to achieve complex requests. Most of the proposed ap-                in any world satisfying the initial state description, will
                                                                     achieve the goal. [110]
proaches for automatic Web Services composition are
inspired by the researches in cross-enterprise workflow              4.4.3. Cross-organisational workflow
and AI planning [93].                                                   A workflow is an abstraction of a business process.
                                                                     It comprises a number of logic steps (known as tasks
4.3.3. Mediation                                                     or activities), dependencies among tasks, routing rules,
   Composing different Web Services from different                   and participants [20]. When many organisations inter-
providers and which are not initially designed to coop-              vene in the business process, its abstraction is a cross-
erate raises another challenge which is Web Services                 organisational workflow that could be assimilated to
mediation. Mediation aims to adapt services outputs to               Web Services composition.
be consumed by the following service(s) in the com-
                                                                     4.4.4. Abstract State Machines
position chain.                                                         The Church-Turing thesis states that any real-world
4.3.4. Choreography                                                  computation can be translated into an equivalent com-
   The web Services choreography handles the interac-                putation involving a Turing machine. However, steps
tion between the services invoked in the composition,                number required by the machine is not bounded and
                                                                     even simple operations could be simulated by a large
from a global point of view.
                                                                     number of steps. So the importance of the Abstract
                                                                     State Machine (ASM), introduced by Gurevich [48]
4.4. Preliminaries                                                   as every sequential [49] or parallel algorithm [16] is
                                                                     behaviorally equivalent to a correspondent ASM. The
  In this section we define required notions to com-                 behaviour emulation is a set of transitions between ab-
pare the different SWS approaches.                                   stract states where
12                                      B. Makni et al. / Semantic integration of TV data and services

                                                                           chy. Grounding these types to WSDL and vice-versa
                                                                           could be defined via Extensible Stylesheet Language
                                                                           Transformations (XSLT). Preconditions and effects are
                                                                           defined via rule languages such as Rule Markup Lan-
                                                                           guage (RuleML) or OWL Rules Language (ORL) to
                                                                           define the state respectively before and after the invo-
                                                                           cation of the service. We summarise how the OWL-
                                                                           S approach tries to solve the semantic Web Services
                                                                           challenges:
                                                                           Services discovery with OWL-S For capabilities de-
                                                                           scription, OWL-S supports both services taxonomy
                                                                           and state transformation approaches. Services taxon-
             Fig. 1. Top level of the service ontology                     omy is formalised by sub-classing the Services Profiles
                                                                           which is also used to define the state transformations.
     – A state is a dictionary of (Name, Value) pairs,                     Then to match the capabilities with the request, either
       called the state signature.                                         by searching a subsumption relation between the re-
     – Transition rules define the evolution of the states                 quested service class and a class from the services tax-
       (i.e. the values change)                                            onomy, or by matching both the inputs and outputs of
                                                                           the request and the advertised services also via sub-
4.5. Semantic Web Services approaches                                      sumption relations. Fig.2 illustrates the OWL-S dis-
                                                                           covery approach.
  We distinguish two approaches that aim to solve                          Services composition with OWL-S OWL-S based
one or more of the semantic Web Services challenges,                       composition naturally falls into the AI planning sec-
which are the top down and bottom up approaches.                           tion, as OWL-S provides
4.5.1. Top down approaches                                                   1. the formalised description of the initial state of
   Top down approaches provide conceptual frame-                                the world, in terms of preconditions.
works and languages to describe the semantics of Web                         2. the formalised description of the desired goal in
Services before grounding these descriptions to the                             terms of users required outputs and effects.
services.                                                                    3. the domain theory by modelling the OWL-S
The OWL-S approach The OWL-S aims to provide                                    atomic process as an action that transforms the
building blocks for encoding rich semantic service de-                          inputs into outputs.
scriptions that builds naturally upon OWL [71]. The                        The next step is the choice of a suitable AI planning
OWL-S approach consists on an Upper Ontology for                           algorithm for Web Services composition. Oh et al. [84]
Services with three interrelated sub-ontologies: Fig. 1                    draw a decision tree in Fig 3 that pilots this choice ac-
                                                                           cording to the scale of the available services and the
Profile ontology for describing the service functional-
                                                                           complexity of composition. A simple composition in-
     ities in order to advertise the service and match it
                                                                           volves a sequential AND operator while a complex
     with the requests.
                                                                           composition is expressed by AND, OR, XOR, NOT
Process model ontology for behavioural description
                                                                           operators and by constraints.
     with the intention of service invocation, enact-
     ment, composition, monitoring and recovery.                           The WSMO approach The WSMO [94] approach
Grounding ontology bonds the process model with                            reuses the main concepts identified in the Web Service
     detailed specifications of the service from Web                       Modeling Framework (WSMF) [36] to define Seman-
     Services Description Language (WSDL).                                 tic Web Services:
Due to this enrichment of expressiveness, OWL-S ex-                        Ontologies provide the terminology used by other
tends the WSDL operations to a more abstract con-                              WSMO elements to describe the relevant aspects
struct “atomic process” that extends inputs and out-                           of the domains of discourse.
puts and introduces preconditions and effects (IOPE).                      Web services represent computational entities able
The inputs and outputs are typed following the OWL                             to provide access to services that provide some
typing system which allows binding to a class hierar-                          value in a domain. The terminology defined by
B. Makni et al. / Semantic integration of TV data and services                        13

                                             Fig. 2. Services discovery with OWL-S

                      Fig. 3. A decision tree of AI solutions for the Web Services composition problem [84]

    the ontologies is used to describe the Web ser-                  Mediators handle interoperability problems between
    vices capabilities, interfaces, and internal work-                  different WSMO elements, at the data level to re-
    ing.                                                                solve mismatches between different used termi-
                                                                        nologies, at the protocol level to ensure communi-
Goals model the user view in the Web service usage                      cation between Web services and on process level
    process in terms of requested functionalities.                      when combining Web Services.
14                                 B. Makni et al. / Semantic integration of TV data and services

                                                                      the bottom-up approach builds incrementally upon ex-
                                                                      isting Web services standards [65].
                                                                      The SAWSDL approach SAWSDL recommendation [65]
                                                                      forms the first brick upon WSDL that gears up seman-
                                                                      tic annotations of services by providing Model Refer-
                                                                      ence and Schemas Mapping extensions [60]. Where
                                                                      Model Reference is an extension attribute, sawsdl:modelReference,
          Fig. 4. WSMO taxonomy of mediators [87]
                                                                          applicable to any WSDL or XML Schema ele-
                                                                          ment to point to one or more semantic concepts,
Moreover all these concepts could have Non-Functional
                                                                          in order to describe the meaning of data or to
properties.
                                                                          specify the function of a Web service operation.
Web Services mediation with WSMO WSMO sup-
ports SWS mediation naturally via the Mediators con-                  Schemas Mapping consists of transforming data from
cept. Indeed, WSMO defines four types of mediators                        XML message format to a semantic model and
of two categories Fig. 4:                                                 vice-versa. The former transformation is called
                                                                          lifting and expressed by the sawsdl:liftingSchemaMapping
Refiners express the refinement relation between ele-                     attribute and the former is called lowering ex-
    ments                                                                 pressed by the sawsdl:loweringSchemaMapping
     OO-Mediators import ontologies and resolve                           attribute.
        possible representation mismatches between
        them.                                                         The WSMO-Lite approach As SAWSDL in itself
     GG-Mediators express goals refinement and                        does not define the semantics of Web services, but
        equivalence.                                                  offers means to link the WSDL descriptions to on-
                                                                      tologies, the Web Services Modeling Ontology Lite
Bridges enable heterogeneous elements interopera-                     (WSMO-Lite) approach [107] defines a service ontol-
    tion.
                                                                      ogy that bonds semantics to the service description via
     WG-Mediators express total or partial fulfil-                    SAWSDL attributes. These semantics are expressed by
        ment of desired goals by exposed Web Ser-                     the following WSMO-Lite ontology concepts:
        vices.
     WW-Mediators deals with heterogeneity prob-                      Ontology a subclass of owl:Ontology that defines a
        lems between Web Services that could ap-                          container for a collection of assertions about the
        pear during composition and orchestration                         information model of a service.
        tasks.                                                        ClassificationRoot defines a root class for a taxon-
                                                                          omy of services functionalities.
Web Services choreography with WSMO WSMO
                                                                      NonFunctionalParameter allows the description of
choreography inherits the core principles of ASM,
                                                                          domain specific nonfunctional properties.
namely state-based, represents a state by a signature,
models state changes by transition rules[97]. The state               Axiom sub-classed in Condition and Effect to form a
signature is expressed in terms of instances of concepts                  service capability.
or relations from a state ontology. The state change                  4.5.3. Semantic annotation of RESTful services
consists of new instance or new value for the relation                   Since there is no agreed machine-processable de-
attribute which leads to the notion of evolving ontolo-               scription language for RESTful Web services [69],
gies by analogy to the evolving algebra, the first name               the MicroWSMO [59] approach for semantic RESTful
of ASM.                                                               services builds on top of hREST [62], a microformat
4.5.2. Bottom-up approaches                                           [57] that enables the creation of machine-processable
   The bottom up approach tends to provide a more                     descriptions on top of existing HTML descriptions.
developer friendly way to semantically annotate Web                   MicroWSMO tends to provide a SAWSDL-like an-
Services. Based on the dictum that the top-down                       notation of Restful services that could be bonded to
approach assumption, that the service engineer de-                    WSMO-Lite ontology in order to provide RESTful and
scribes semantics for the service before grounding                    WSDL based services interoperability. The Figure 5
these descriptions to the services is counter-intuitive,              illustrates this relative positioning.
B. Makni et al. / Semantic integration of TV data and services                            15

                                                                       description is expressed in terms of Goals, Mediators,
                                                                       Web Services and Ontologies. These are described in
                                                                       a formal representation language, for instance, OCML
                                                                       [77]. IRS-III supports capability-based invocation: the
                                                                       request is a goal to be achieved via the following inter-
                                                                       mediated operations [19]
                                                                         1. discover potentially relevant Web services;
                                                                         2. select the set of Web services which best fit the
                                                                            incoming request;
                                                                         3. mediate any mismatches at the conceptual level;
Fig. 5. Relative positioning of WSMO-Lite and MicroWSMO [61]             4. invoke the selected Web services whilst adhering
                                                                            to any data, control flow and Web service invoca-
4.6. Semantic TV services
                                                                            tion constraints.
   Our experience of brokering TV-related services                        Given that IRS-III directly aims at automating ser-
within the NoTube project offered us a fertile ground to               vice execution related aspects, the interface covers
apply, compare, and adapt different semantic Web ser-                  choreography and orchestration descriptions. Chore-
vices approaches [32]. We introduce the specific No-                   ography addresses the communication between the
Tube challenges, the different semantic TV services                    IRS-III broker and a Web service, and is described
approaches, and their shortcomings.                                    as so-called grounding. The IRS-III grounding mech-
4.6.1. NoTube use-case                                                 anism supports REST-based, SOAP-based, and XML-
   In order to illustrate the challenges with respect to               RPC based services [64]. Grounding involves two pro-
service-related tasks, we describe one of the main use                 cesses referred to as lifting and lowering. Lowering in-
cases driven by the TV broadcast industry partners                     volves transforming input parameters at the semantic
within the NoTube project - namely, the requirement                    level to data input to the service at the syntactic level.
to provide personalized content and metadata deliv-                    Lifting involves the opposite transformation, i.e. trans-
ery to users. Here, the basic feature is the matching of               forming the data output from the service at the syn-
heterogeneous users’ profiles, e.g. including interests,               tactic level into an ontological object at the semantic
preferences, and activity data, and user contexts (e.g.                level.
current location and viewing device) to filter and de-                    At the semantic level the orchestration is repre-
liver TV content from a variety of sources. Address-                   sented by a workflow model expressed in OCML that
ing this particular use case in a service-oriented man-                describes the flow of control between Web services.
ner involves selecting, and orchestrating between nu-                  The IRS-III orchestration model supports the main
merous services that provide various functionality, for                control-flow primitives of sequence, selection, and rep-
instance, to aggregate users’ topic interests based on                 etition.
their social networking activities, retrieve EPG data
from various sources, and provide recommendations                      4.6.3. The iServe Linked Services approach
based on a dedicated algorithm. To support the highly                     iServe supports publishing service annotations as
service-oriented nature of the project, two major goals                linked data - Linked Services - expressed in terms of
need to be supported: a) support of distributed devel-                 a simple conceptual model that is suitable for both hu-
opers with lightweight service annotations, and b) sup-                man and machine consumption and abstracts from ex-
port of application automation with Semantic Web Ser-                  isting heterogeneity around service kinds and annota-
vice brokerage . In the early stage we focused on b) via               tion formalisms. Particularly iServe provides:
a top down approach, namely, IRS-III [34]. However,                       – Import of service annotations in a range of
the later need for the distribution of services annotation                  formalisms (e.g., SAWSDL, WSMO-Lite, Mi-
revealed the need for lightweight approach via iServe                       croWSMO, OWL-S) covering both WSDL ser-
[89].                                                                       vices and Web APIs;
4.6.2. The IRS-III framework                                              – Means for publishing semantic annotations of ser-
   IRS-III is a semantic execution environment that                         vices which are automatically assigned a resolv-
adopts the WSMO approach, videlicet that a service                          able HTTP URI;
You can also read
Next slide ... Cancel