ESA > Join & Share > Technology Projects > SATOPI Project

SATOPI Project

Project Title   Topic Maps as Enabling Technology for Earth Observation User Services Applications
Project Acronym   SATOPI
Contractor(s)   Space Applications Services (B)


Image Context
       How it works



Earth Observation data is valuable for understanding different environment related issues such as floods, forest fires, pollution, or earthquakes. However, in order to allocate a specific EO product (i.e. satellite image) which relates to the problem in hand, different domain knowledge data stores have to be accessed. These data stores can be multidisciplinary or interdisciplinary and are often fragmented across many organizations and projects. Accessing different data stores requires the knowledge of different user interfaces and database designs and lead to inefficiency.
For example, a researcher studying Glacial Lake Outburst Floods (GLOFs, see http://en.wikipedia.org/wiki/GLOF), a phenomenon closely linked to the recent climate changes, has first to select the sort of glacial lakes (moraine dam, supra glacier, erosion, etc.) that are of interest. Other factors related to the glacial lake should be selected as well (drainage condition, orientation, elevation etc.) Then the location of each such lake should be identified, and finally the relevant EO products should be allocated. When doing, for example, a comparison study, this process can be tedious. This inefficiency is emphasised even more when considering the impact of the weather fluctuations on the glaciers. A researcher that wants to understand the processes involved has also to navigate through the weather data in yet another database.
It is clear that integrating databases of EO products with databases containing domain knowledge provides a great advantage for domain experts and decision makers. The user has one access point to the information and can use the domain terminologies.
The example of researchers studying GLOFs is at the origin of the use case that will be implemented during the SATOPI project.
Topic Maps is an international industry standard (ISO/IEC 13250:2003) for knowledge representation and information integration. It provides the ability to store, together with the data, complex meta-data that represents the semantics &endash; that is, it lets us record the meaning of the data we store. Unlike other technologies (e.g. RDF/OWL), Topic Maps provides the ability to represent knowledge in a natural way &endash; the way humans grasp knowledge. This more natural approach is extremely powerful especially when humans must interact with information systems.
Topic Maps is the technology selected to be at the heart of the SATOPI solution.



The SATOPI project main objective is to demonstrate the novel advantages of federating EO mission products with domain specific knowledge using Topic Maps by an application targeting a specific use case &endash; Glacial Lake Outburst Flood (GLOF) phenomenon in Nepal.

In order to reach this objective, two major problems must be addressed::

  Image the access of very large data sets;
  Image the diversity of data resources.

The SATOPI project addressed these two problems by developing a software system based on the integration of the following main components:

  Image A Topic Maps engine that is capable of processing very large amounts of data. This Topic Maps engine is based on TopiEngine developed during the TopiWriter project (ESA contract number 19077/05/NL/PG).
  Image A TopiEngine query-based API (TMQL) for developers &endash; so web applications around the Topic Maps engine can be developed.
  Image A methodology for authoring domain Topic Maps Ontologies.
  Image A set of Connectors for accessing existing data sources and mapping the data into Topic Maps elements compliant with the current Ontology.
  Image User interface components based on Web technology to facilitate the creation of such applications.



The SATOPI software is a Web-based application that allows the users to search and navigate into domain knowledge using an homogeneous and responsive AJAX-enable user interface.
The Web application itself is in charge of managing the interactions with the users. It constitutes the frontend of the application. It has the double responsibility of generating the elements of the user interface and of fetching the requested data from the backend.
The backend runs a Topic Maps engine. This engine exposes an API that allows to search for data by means of Topic Maps queries. The engine gets the data from external data sources through dedicated connectors.
The following technological elements have been used to build the SATOPI application:

  Image TopiEngine: TopiEngine has been implemented by Space Applications Services under the past ESA contract 19077/05/NL/PG "TopiWriter". It complies with Topic Maps standards (TMAPI, TMQL, etc.) Open-sourced in early 2009, it is a freely customizable and extendible Topic Maps engine implemented in C/C++.
  Image Datastore Connectors are software modules implemented as extensions to TopiEngine. They are used by TopiEngine to fetch data from the various datastores. The Connectors, which expose a common API, communicate with the datastores and convert the raw data into Topic Maps constructs. TopiEngine clients only see a single Topic Maps interface: they do not know that it connects to other databases.
  Image The SATOPI solution has been implemented as a Web-based application. This AJAX-enabled application is compliant with the applicable standards: HTML, CSS, ECMAScript.
  Image The SATOPI Web application is based on the Django Web application framework. This open-source, Python-based framework, provides a template mechanism that facilitates the instant production of any type of document such as HTML pages and TMQL requests.

When all the components are put together, we obtain the following architecture diagram:



How it works

The Use Case Ontology

The definition of an ontology for a specific domain is a critical step in the course of a project that implements the Topic Maps technology.
The ontology provides a skeleton to the data, and thus also expresses limits and constraints.
The ontology defines the types of data that will be manipulated and the types of associations that can exist between those data. Any piece of information that will be used in the application must be declared as an instance of one or more of these pre-defined data types and can only be associated using those pre-defined association types. The goal was to obtain an ontology that describes the kind of questions can be answered by the application.
During the project definition phase of SATOPI, the domain experts provided a description of their domain of activity: the different types of entities, the relations that exist between those entities and the kind of information they have in hand about those entities. Descriptions were illustrated by concrete examples (e.g. the situation of the Imja Tsho lake and its parent glacier, represented in the diagram A, below.
Using these concrete examples, the topic types and the hierarchy between these types has been identified, as shown in diagram B.
Finally, each of these topic types had to be further refined: occurrence types, role types and association types have been identified for each of them (see diagram C).


Datastore Connectors

When data is requested by the client application, queries are received through the TMQL (Topic Maps Query Language) interface of TopiEngine. The queries are then decomposed in low-level TMAPI calls.
TopiEngine, knowing the ontology, is able to validate the requests and to make sure these are correctly interpreted. For example, knowing that Glacier and Lake are both sub-types of Geographical Entity, requesting for the name of all geographical entities will result in a list of glaciers and lakes names.
The Topic Maps Ontology is provided by means of an XTM document which defines the ontology using XML constructs.
The Datastore Connectors on the other hand, are configured using CTM templates, that is, using the Compact Topic Maps notation in which the data itself is replaced by placeholders and tags. Those placeholders and tags indicate how the raw data must be mapped into Topic Maps constructs.
TopiEngine forwards the TMAPI calls to all configured Datastore Connectors. The Connectors fetch the data from the datastore they are connected to and return fragments of topic map to TopiEngine.
TopiEngine dynamically merges all received Topic Maps fragments and returns the resulting structure to the client application through the TMQL interface.


The Template System

The template system of Django is used to produce two types of documents: HTML pages and fragments, and TMQL queries.
Even the simple welcome page is generated out of multiple templates. For example, the search menu that is displayed at the top of each page, can be found in a template fragment that is used by all the pages of the application.
The Web application makes use of the appropriate templates, combined with the received data, to generate the new HTML page. The chosen template depends on the type of page and the type of item to be displayed. The templates have been organized in way that permits a generic implementation of this mechanism.
The template system is also used to generate fragments of HTML pages that are requested asynchronously using AJAX.


The User Interface

The user interface of the SATOPI software is Web-based. This allows to reduce the hardware and the software requirements to the minimum, and allows the users to benefit from the system without requiring the installation of any specific software.
When the user enters the SATOPI web site, he is presented with a menu that allows him to choose how he would like to start searching for data and EO products. The menu, which is included in the header of all the pages generated by the SATOPI application, shows the list of the main object types for which a search form has been implemented: Glacier, Lake, Event and Photograph.
For each of these types, a dedicated search page is provided. The figure below shows the search page that allows the users to find glaciers using various criteria.
There is no "Submit" or "Search" button available in the search pages. Those pages are indeed fully dynamic: as soon as the user changes the value of a criterion &endash; a new value is entered, a check box or radio box is toggled or a new entry is selected in a list &endash; the list of matching items is automatically refreshed (right panel). This process is executed in the background: a "Search in progress ..." message is displayed in the page but the user can still interact with the page, setting or modifying other criteria, if necessary.


The user can obtain the detailed list of matching items by following the link provided at the top of the right hand panel.
A specific search result page has been designed for each item type. They all share the same layout: the search criteria that have been applied to obtain the matching items are summarized at the top of the page, and the items themselves are shown in the remaining of the page. The items are displayed by means of a table that also includes miscellaneous information about each item. This information is for example the geographical position of a glacier or a lake, or a satellite name and a production date for an EO product. If the amount of matching items exceeds a pre-defined limit, the table gets automatically paginated and navigation controls are displayed.


All the items identified in the search pages and search result pages are provided as hyperlinks to let the users easily reach the description page of these items.
The purpose of the description pages is to combine in a single sheet all the information that can be obtained about a particular item. The layout and the nature of the information varies from one type of item to another but in general the description pages contain the following elements: a textual description, a graphical representation (e.g. thumbnail of a satellite image), the related items (e.g. parent glacier of a lake) and the available resources (e.g. satellite images and Shapefile collections).


The page flow diagram, below, depicts the various page types the SATOPI application are serving, and the links the users can follow to navigate between those pages.


This rather simple page flow diagram has the big advantage that it does not become more complex if the decision is made to elect a new type of item as "primary" type, for which a dedicated search page, a search result page and a description page must be designed and implemented. Because these new pages would be sharing the layout of their siblings, most of the effort would reside in the preparation of the data to be displayed.
The same page flow, and the same page layouts, are also applicable to any other domain. This reduces the effort of re-applying the SATOPI solution to new use cases.



The final output of the work consists in:

  Image A Topic Maps ontology for the GLOFs domain and EO products.
  Image The SATOPI software and its user manual.
  Image The final presentation of the SATOPI project was held at ESRIN on October 22nd 2009.

Contributors to this page: andreadv .

Page last modified on Wednesday 22 of December 2010 16:08:40 CET by andreadv.