ESA > Join & Share > Forums > LTDP SAFE > Processing Chain analysis trade-off



Show posts:
Jump to forum:

Processing Chain analysis trade-off

During the PDR-C collocation meeting it was agreed to improve the processing chain description in the Recommendations for Specialisation document clarifying how an EO-product is linked to information regarding the processor(s) and auxiliary file types that can be used to derive higher-level products.

The attached document “Processing Chain analysis trade-off” (PDGS-SAFE-GMV-TN-12/0195) provides the analysis and conclusions reached on this topic.

All your comments will be appreciated.

Best Regards.

Fernando Ibáñez (GMV)

Re: Processing Chain analysis trade-off


Overall this seems OK. The proposed processing chain information for SAFE 2.0 is not that different from that in SAFE 2.0 draft, there just seems to be a better, more formal organization.

A few comments:

- The example provided in Table 2-2 only considers one input and one output, so it is not enough to understand how it would look with more inputs/outputs. A more extensive example should be included in the Recommendations for Specialisations document.

- The same example, in the "Processor" column, says "Documentation reference: ATBD and DPM documents.". In general, the fields in this table should not be free text. A sentence like "ATBD and DPM documents", in terms of Long Term Data Preservation, has very little value because it will be difficult to interpret in a future context. Any references to documents should use some kind of document reference in some meaningful format. In the scope of SAFE we cannot solve the whole problem, but we should define that only document references are allowed in this field and that the mapping of these references to actual documents is outside of the scope of SAFE.

- Related to the previous item, SAFE-AUX-0040/2.0 is not an easy requirement to satisfy or to verify, because it talks about keeping enough information to generate higher-level products saying, however, that it may be enough to include a processing chain description on the control book (so, this is considered "enough information"). On the other hand, SAFE-GEN-0150/2.0 talks about "processing steps" as part of the information to be preserved. Looking at the content of the GEN_IPF_SPEC document, I think that these "processing steps" are actually at a lower level than what you are considering, i.e. they do not refer to L0->L1a as a single processing step (as your document implies, if I understand correctly), but rather to the "tasks" in the processor task table (see for example page 24, section 4.1.1). According to this, a processor can be defined as the number of composing executables/tasks and, for each of these executables/tasks, the list of input, output and intermediate file types. So, it could be argued that the "processing steps" we want to preserve are actually these (otherwise, what would be the purpose of intermediate file types?). The degree at which the information is preserved has as impact on how effective LTDP will be, but probably we also cannot afford to have everything we want to have, so there should be at least a clear understanding of what exactly is expected to be preserved on the Control Books (it should be clear in the Recommendations for Specialisations document).

- Although it is not mentioned in the document, all the proposed alternatives for PDR-C_A09 provide a clean way to handle changes in the processing chain (i.e. you only have to change it in the top-level mission specialisation document). There is a drawback in the sense that if the processing chain for any of the instruments changes you always have to change the top-level document (instead of potentially only the specific document for that instrument), but I think this is acceptable.


Re: Processing Chain analysis trade-off

Just some comments:
1)In the documents auxiliary data are listed as outputs of a processing step, it seems not an usual case. Do you have an example to justify this?

2)If in the control book there is a reference to the processor documentation it may not be needed a complete list of the intermediate processing task as listed in the processor task table but just a high level description of the whole processing.

Show posts:
Jump to forum: