ESA > Join & Share > Technology Projects > HiProGex Project

HiProGex Project

Project Title   High Level Information Product Generation and Formatting for Specific Application Extension
Project Acronym   HiProGEx
Contractor(s)   INDRA Espacio, ACRIst, University of LEICESTER


Project Objectives
          How it Works


Objectives   Top

The High Level Product Generation Extension project main objective is to provide a demonstrator system able to generate automatically Level 3 products starting from ENVISAT PDS Level 2 products.

HiProGEx product processors will be the responsible for the generation of set of Level 3 global/regional products at different resolutions and for different temporal periods.

Products to be generated are:

  Image Global/regional averaged products derived from AATSR sensor data
    Image LST
    Image SST
    Image NDVI
    Image Anomaly product: difference between AATSR SST Level 3 and climatology files from JPL PODAC
  Image Regional averaged products derived from MERIS sensor data
    Image Over sea: aerosol optical thickness at 865 nm, epsilon
    Image Over sea/land: cloud optical thickness, cloud top pressure, total water vapour column

The main design objectives addressed by the system can be summarized in the following points:

  Image Capability to manage automatically the generation of Level 3 products from lower processing level products.
  Image Capability to incorporate new product processors not needing either to modify system software or even to stop it, addressing the Plug-and-play concept for processors.
  Image Capability to define level 3 Products parameters: area of interest, resolution, time scale and others users definable.
  Image Portability among different platforms.
  Image Capability to deploy system components in a distributed environment.
  Image Provide web access to the system for product catalogue browsing and system configuration.


Context   Top

HiProGEx is thought as a low-cost system able to generate automatically Level 3 products starting from calibrated and geocoded geophysical ENVISAT L2 AATSR and MERIS data products.

Level 2 products provides "snapshot" of geophysical information on the ground strips covered by instrument swath within time frame of several minutes.

Level 3 products are combination of mutiple scenes/orbits of Level 2 products that intersect a defined geographical area of interest over long time-frames.

Level 3 product generation requires processing of a large volume of Level 2 products and addresses aspects such as multitemporal combination of geophysical data, geocorrection, averaging and mosaicking.

Level 3 product's to be generated by HiProGEx are primarily directed at scientific users investigating the behaviour of the natural environment on any time or space scale larger than the instrument swath-width. Typical working scales ranges from regional to global ones depending on user interest.

Level 3 at global scales appraise the state of the global areas to identify areas of special interest and as input to global models of the Earth system.


Architecture   Top

HiProGEx architecture is hierarchically organized in three tiers:

  Image Presentation tier: Web access graphical user interface allowing the user to access product catalogue browse functions and to configure the system.
  Image System Production Management Tier, composed by two block of subsystems covering different functions:
    Image System configuration: Administration, Report Generation, Master Order management.
    Image Production: Product Ingestion, Product Notification, Product Scheduler, Product Control.
  Image Data/Services tier composed by data base access subsystem.

Processors generating Level 3 Products are considered as additional subsystems. Standardized interfaces with Product Control Subsystem allows for an easy plug-in of new processor onto the system.

HiProGEx architecture design allows the deployment of subsystems in different hardware platforms interconnected to a LAN.

Graphical user Interface can be accessed using a web browser from any computer connected via LAN to HiProGEx.

Underlying Technologies used to implement HiProGEx are:

  Image Java language.
  Image XML and JMS to implement Product Ingestion, Product Notification, Product Scheduler, Product Control communications.
  Image HTML/JSP/Servlet/Applet to implement graphical user Interfaces.
  Image JBDC to access database.


How it Works   Top

HiProGEx system main objective is the automatic generation of Level 3 products.

In order to distribute the processing needed to generate a Level 3 product over the time, HiProGEx provides scheduling policies to support an stepped generation of products allowing to generate intermediate products as a previous step to the generation of Level 3 products.

Product Scheduling subsystem is in charge of driving the product generation.

HiProGEx relies in two main concepts to support the management of product generation Master Order and Input Data filter.

Master orders are data structures registered in HiProGEx database defining the type of product to be generated and the temporal coverage window of it.

Input Data Filter is also data structure registered in HiProGEx database defining the area of interest for the Level 3 product to be generated and additional information user definable. Input data filter are used to select dynamically the Level 2 input products that will contribute to the generation of a given L3 product.

Product's generation work flow is implemented in the system as a pipeline of communication among production related subsystems, each of them having one and only one precise role.

The HiProGEx Product's generation work flow can be summarized in the following steps:

  Image Level 2 inputs products must be copied to an HiProGEx directory.
  Image Product Ingestion subsystem perform a polling over the directory where Level 2 input products are copied. As soon as an input product is detected, Product Ingestion will insert an entry in Product Catalogue database tables and will generate a quick-look image.
  Image Product Notification subsystem receives two type of events: ingestion of a new product and generation of new intermetiate product. These events are, in turn, broadcasted to Product Scheduling subsystem.
  Image Product Scheduling working schema is based on input product data availability, is data-driven. The subsystem reads messages notifying the ingestion/generation events of new products and based on Master Order it will:
    Image either trigger the immediate execution of a processor to generate intermediate products starting from Input Level 2 product,
    Image or trigger the generation of a Level 3 product if all intermediate product within the Level 3 product temporal coverage window have been generated.
  Image The triggering of product generation is materialized via processing order that is sent by Product Scheduling to Product Control Subsystem that will in charge of:
    Image identifying all input data needed by a processor
    Image starting processor execution
  Image Upon completion of product generation the Product Control Sytem shall:
    Image notify to product scheduling the product generation completion status
    Image notify the generation of a new product to Product Ingestion subsystem


Output   Top

As result of HiProGEx project completion two main physical elements shall be provided to ESA:

  Image A standalone demonstrator system, not be interfaced to any ESA, and able to generate Level 3 products starting from Level 2 ENVISAT PDS in an automatic way.
  Image A set of Level 3 product in HDF-EOS format.

HiProGEx shall provide system configuration functions allowing HiProGEx users, through graphical user interface application, to: support the installation of Level 3 products processors in the system and to define orders to drive the execution of product generated by new processors.

HiProGEx demonstrator system shall allow ESA to evaluate and explore a number of issues for a future operational Level 3 product generation system. In particular HiProGEx shall provide support to exploratory activities in the following fields:

  Image Functionality.
  Image Performance.
  Image Extensibility of the system: capability to add new processors.
  Image Ease of operation.
  Image Maintainability cost.

Contributors to this page: andreadv .

Page last modified on Tuesday 14 of December 2010 13:59:09 CET by andreadv.