Loading...
 
ESA > Join & Share > GPOD > GPOD SMOS L1 Processor User Manual
Print

GPOD SMOS L1 Processor User Manual

INTRODUCTION


ESA's Soil Moisture and Ocean Salinity (SMOS) mission has been designed to observe soil moisture over the Earth's landmasses and salinity over the oceans. The main objective of the G-POD SMOS Level 1 Processor Service is to provide an environment supporting early processors tests, including validation of the processor improvements supported by substantial data and processing resources.

In combining the SMOS processors with the G-POD flexibility, processing power and easy interface, the G-POD SMOS L1 testbed allows:

  • Scientists to create their own Level 1 and Level 2 products using custom configuration
  • Scientists to integrate, free of charge, their own algorithms in the G-POD environment to process SMOS
  • The SMOS processor development team to test new processor releases, seamlessly plugging them into G-POD and then running the software over a large data set
  • The SMOS quality control team to perform comparisons between the products processed in operations and products processed by G-POD, improving product quality

Document purpose

Purpose of this document is to provide a user manual for the G-POD SMOS Level 1 Processor Service, in order to help the user to exploit all the feature of the service.

Intended readership

This manual is intended to be read by an user already familiar with the G-POD web portal interface (http://gpod.eo.esa.int). If you never used G-POD, you can consult the G-POD User Manual. Minimum knowledge of the SMOS MIRAS Level 1 products is expected. For more information about SMOS L1 products, please refer to the ESA SMOS Website.

Acronyms and Abbreviations

The acronyms and abbreviations used in this document are the following ones:

Acronym Description
G-POD Grid-Processing On Demand
ESA European Space Agency
SMOS Soil Mosture and Ocean Salinity satellite
MIRAS Microwave Imaging Radiometer using Aperture Synthesis (SMOS payload)
L0 Level 0 product
L1A Level 1A product
L1B Level 1B product
L1C Level 1C product

Service implementation

SMOS Level 1 processor is constituted by 8 different sub-processors, which are used to obtain all the SMOS L1 products from L0 to L1C.

Each SMOS processor takes an input file and a set of auxiliary files. Auxiliary files are constituted by configuration information (ex. for the AUX_CNFL1P files), auxiliary data taken from remote sources (ex. the AUX_BULL_B files) or output products of previous steps (ex. for MIR_CRSD1A).

Processor steps implemented in G-POD

All the main processors, used to obtain L1A, L1B and L1C products, have been implemented in G-POD. The products not directly produced by G-POD are taken from the processing facilities. Future implementation of the remaining processor is foreseen (please ask the G-POD Team for more information).

The list of the SMOS sub-processors is reported in Table 1. The list contains a short description, the input product series and the output product series for each processor. Also, the first column of the table, reports if the processor is available or not on the G-POD platform.

Avail Codename Description Input series Output series
Y HKTM1A_ Level 1A Telemetry Processor TLM_MIRA0 TLM_MIRA1A
Y CAL_1A1_ Level 1A Calibration Processor MIR_CORx0_
MIR_UNCx0_
MIR_CRSD1A
MIR_UAVD1A
N NIRCAL__ Level 1A NIR Calibration Processor MIR_TARx0_ MIR_ANIR1A
N SCIL1A_
FLATAR_
Level 1A FTT Calibration Processor MIR_TARx0_ MIR_FTTx
Y SCL1A_ Level 1A Science Processor MIR_SC_x0_ MIR_SC_x1A
Y SCL1Bx Level 1B Science Processor MIR_SC_x1A MIR_SC_x1B
Y SCL1C_ Level 1C Science Processor MIR_SC_x1B MIR_SCxx1C
MIR_BWxx1C

Table 1: SMOS Level 1 Processors with their Input/Output product series

Image
Figure 1: Schematic representation of the SMOS processor step

G-POD runs each processing step with its own configuration baselines, orchestration rules, etc... A schematic representation of the SMOS processor step is reported in Figure 1.

As you can see from the figure, each processor step needs a set of input and auxiliary files. Input and auxiliary files are taken from what is called a "Reference".

Basically, a "reference" is a data series, for example MIR_SC_F1A, which is the L1A product series and goes in input to the L1B processor.

References are taken from the Operational Facilities or created using G-POD itself.

All the textual configuration and auxiliary files, can be taken directly from the G-POD portal (thus giving the user the possibility to make its own "custom" Level 1 files). For more information see chapter Advanced parameters.

The SMOS output can go to the G-POD Portal, to be downloaded later, directly to the user FTP server or to fill a Reference, to be used in the next processing step (ex, output of the L1B processor, goes to the L1B reference, which is used as input for the L1C processing).

Parallelization

All the processor steps in Table 1 can be parallelized to improve the performances, for example running at the same time 30 jobs one for each day of the month.

The only not parallelizable step is the "Level 1A Calibration Processor" when used to create a new MIR_CRSD1A reference, this because to run the current MIR_CRSD1A processing, you need the previous orbit data.

Note that the maximum amount of parallel jobs you can run depend on the configuration of your account and it is expressed by the "Credits" associated to your account (see the G-POD User Manual fore more information)

Users categories

There are two categories of users, the "science" and the "operator" user.

"Science users" can generate their "custom" L1 products, starting from one of the available references (Ex. Official Reprocessing, G-POD Reprocessing, etc…).

The "operator users" can write on an assigned reference, performing a complete "custom" reprocessing from L0 to L1C.

Service usage

Image
Figure 2: The SMOS Level 1 processor interface

To create your "custom" L1 product (for "science" users)

  1. Select the input dataset which corresponds to the processing step you want to run, the AOI, start and stop time of your processing and perform a query to the G-POD catalogue (as described in the G-POD User Manual
  2. Select the files to process from the list or directly form the map.
  3. Verify that the selected processor corresponds to the input/output dataset you wish to process, as reported in Table 1. If not, select the correct processor from the box and re-do the query.
  4. Select the desired processor version from the list box.
  5. If desired, press the "Use advanced configuration" button to customize your L1 product (see Advanced parameters for more information).
  6. Insert your task name, publish server, output format, etc..., as explained by the G-POD User Manual
  7. Press the "Process it" button


To perform an entire reprocessing (for "operator users"):

NOTE: It is important that the processing follows the order reported in Table 1. This because the output of the first steps is taken as input for the next ones (ex. TLM_MIRA1A and MIR_CRSD1A are necessary for MIR_SC_1A production)

For each one of the processing steps available in G-POD and reported in Table 1, perform:

  1. Select the input dataset which corresponds to the processing step you want to run, the AOI, start and stop time of your processing and perform a query to the G-POD catalogue (as described in the G-POD User Manual
  2. Select the files to process from the list or directly form the map.
  3. Select the processor version from the list box.
  4. If desired, press the "Use advanced configuration" button to customize your L1 product (see Advanced parameters for more information).
  5. Insert your task name in the top-right box
  6. Select the Reference corresponding to the output product of the step from the “Publish Server” list box
  7. Select "None" as Compression from the top-right box
  8. Press the "Process it" button.


To scheduled processing over a large time range:

NOTE: Scheduled processing is restricted. To request access to the feature, please send an email to the G-POD Team
NOTE: Scheduled processing cannot apply to the “Level 1A Calibration Processor” when the output is the MIR_CRSD1A reference, because this processor is not parallelizable so you cannot run multiple daily jobs at the same time but you need to create one single job for the entire time range.

  1. Press the "Schedule" button on top of the "SMOS L1 processor page"
  2. Select the input dataset which corresponds to the processing step you want to run
  3. Select the processor version from the list box.
  4. If desired, press the "Use advanced configuration" button to customize your L1 product (see Advanced Configuration for more information).
  5. Follow the G-POD User Manual to configure the AOI and time interval of your scheduled processing.
  6. Select the Reference corresponding to the output product of the step from the "Publish Server" list box
  7. Save the scheduler with the "Create" button.

Advanced parameters

Image
Figure 3: Advanced processing parameters

To enable the advanced processing parameters, you need to click on the "Use advanced configuration" button on the service page. A brief description of the available parameters is reported below:

Auxiliary files baseline: This is the reference used for the processing for the auxiliary and configuration files. Available references are reported in Annex A.

Orchestration baseline: Define the orchestration rule set used in the reprocessing. For now, the only avaiable orchestration rule set is the Operational one, defined by the SMOS L1OP-V3 Orchestration Baseline v2.6 15/06/2011. Custom orchestration rules can be implemented on request to the G-POD Team.

Define a custom “XXX_XXXXXX” file: With this option, it is possible to use your own configuration or auxiliary file. The feature is available only for the pure XML auxiliary files (no binary data is supported). In the case you need to use your custom binary auxiliary files, like MIR_GMATD or MIR_FTTD_, etc, you need to send them to the G-POD Team which will create a new reference baseline for you with your custom auxiliary files.

To use a custom XML file, just open the product in any text editor and copy paste the contents inside the text box corresponding to the desired product. If you wish, you can save the XML for further use, by pressing the "Add" button, specifying a file name then pressing the "Save" button.
Please note that the auxiliary files specified in the box will be used for every product, thus they will override every orchestration rules. Also, name of the saved file is not important, because the correct product name to be provided to the processor is automatically generated by the system.

Annex A Auxiliary files baselines

Auxiliary files baseline contains the following auxiliary/input product series displayed in Table 2. As reported in table 2, some of these series are overridable using the portal Advance Configuration parameters. If you wish to use your own auxiliary files which are not customizable from the portal, you can contact the G-POD Team which will create a reference for you and upload your files.

Series Overridable directly from the Portal
AUX_APDL no
AUX_APDS yes
AUX_BFP_ yes
AUX_BSCAT_ no
AUX_BULL_B no
AUX_BWGHT_ no
AUX_CNFL1P yes
AUX_DGG_ no
AUX_FAIL yes
AUX_GALAXY no
AUX_LCF_ yes
AUX_LSMASK no
AUX_MASK no
AUX_MISP no
AUX_MOONT_ no
AUX_NIR_ no
AUX_PATT no
AUX_PLM_ yes
AUX_PMS_ yes
AUX_RFI_ no
AUX_SPAR yes
AUX_SUNT no
AUX_VTEC_P no
MIR_ANIR1A no
MIR_FTTD no
MIR_FTTF__ no
MIR_GMATD_ no
MIR_JMATD_ no
MPL_ORBSCT yes
MIR_CRSD1A no
MIR_UAVD1A no
MIR_SC_L1A no
MIR_SC_L1B no
TLM_MIRA1A no

Table 2: List of the Auxiliary and configuration files constituting a Reference Baseline

The available baselines are:
Operational products Product are taken daily from the Operational chain. This baseline may not support all the different processor version at the same time, but only the processor version which are in operations at the time.
Official reprocessing This baseline contains the products and auxiliary files taken from the official reprocessing. Currently, all the 2010 data is available for the 3.46 processor.
G-POD reprocessing This baseline contains the products re-processed using G-POD joined with the auxiliary files taken from the Operational chain for the non available products. Currently this baseline contains data from 27 of November to 27 of December for the 5.02 processor.


Contributors to this page: .

Page last modified on Wednesday 28 of September 2011 17:26:57 CEST by .