ESA > Join & Share > Technology Projects > IIM-TS Project

IIM-TS Project

Project Title           Image Information Mining on Time Series
Project Acronym     IIM-TS



In recent years our ability to store large quantities of data has greatly surpassed our ability to access and meaningfully extract information from it. Emerging needs from big applications (e.g.: change detection, global monitoring, disaster management support, etc.) and the continuous increasing demand from from various international initiatives and large programmes for operational services providing reliable and timely information on changes require new methodologies and tools for extracting the relevant information from Earth Observation (EO) satellite data. Time series of EO images, alone or in combination with data from other domains, can contribute to the creation of such information and services.

The development of effective methodologies for the analysis of multi-temporal data is one of the most important and challenging issues the remote sensing community will face in the forthcoming years. Its importance and timeliness are directly related to the ever-increasing quantity of multi-temporal data provided by the numerous remote sensing satellites that orbit our planet. However, the advances in the methodologies for the analysis of multi-temporal data have been significantly under-illuminated with respect to other remote sensing data analysis topics, and the link between the end-users’ needs and the scientific community needs also to be strengthened. Moreover, the information extraction and service provision should be automated to the maximum possible extent, in order to reduce costs.

In the transition from traditional fixed-time EO data analysis to time-series image information mining, the main issues to be considered are:

  • Moving from scene understanding to process understanding
  • Developing general methods for time series analysis
  • Handling large quantities of data
  • Modelling three-dimensional phenomena (intrinsic two-dimensional aspect of images and one-dimensional aspect of time)


The objective of the IIM-TS project is to investigate automatic methods for the analysis of time series of Earth Observation images, with the purpose of identifying and locating patterns, changes and movement of recurring features, which could represent potentially useful information for specific applications. The investigation is supported by the implementation of a number of prototypes and the analysis of their operation by selected domain experts and representative users. The selected prototypes will be designed in order to be integrated in the Knowledge-centred Earth Observation (KEO) system, and will permit to analyse image time series obtained by SAR and optical sensors, of a resolution comparable to the ones in the Sentinel missions (from 1km to about 5m), also in combination with data from other domains.

The project is subdivided into three successive phases:

  • Phase 1, which aims at identifying relevant applications and selecting the most suitable algorithms for prototyping 
  • Phase 2, which aims at implementing, integrating and validating selected prototypes  
  • Phase 3, which aims at upgrading and refining the implemented prototypes following the results of validation activities

The project started on February 2007, and the Phase 1 Final Presentation has been held at ESRIN on December 2007.

Phase 2 and Phase 3 started on April 2008 and led to the implementation, integration and validation of a series of processing components into the KEO system. Details on the final IIM-TS processing environment are provided in the following presentation.


The analysis of multi-temporal images requires different steps, which follow a simple general schema:

  • Data Collection, aiming at acquiring remote-sensing imagery, suitable ground truth data, ancillary data, prior information on the specific application and end-users requirements;
  • Data Pre-processing, aiming at performing atmospheric/radiometric corrections, geometric corrections (co-registration, ortho-rectification), filtering, etc.;
  • Data Analysis, aiming at extracting the significant information (i.e. change detection, multi-temporal classification, trend analysis, etc.).

The IIM-TS project will permit to extend KIM and KEO systems capabilities to the analysis of multi-temporal series of data, allowing the extraction of relevant / interesting information from images, either through generic probabilistic techniques (for very large data volumes) or through specific processing modules(for limited data sets). The developed prototype should therefore provide:

  • A simplified core engine, built around few basic operations;
  • An efficient implementation, capable of handling large data volumes;
  • A clear data flow, for easing the implementation of more complex processing chains;
  • A user-friendly interactive interface, allowing the access to all system functionalities;
  • A modular design, permitting additional and optional operations to be carried out through external services.

The system will provide three main components:

  1. TimeSeries Preparation, aiming at composing a time series from archived data and preparing it for further processing. It includes different services: access to archived data, multi-image co-registration services, multi-image cross-calibration services, multi-image data analysis modules (Fourier and Wavelet Transform and similar);
  2. Unsupervised Analysis, aiming at locating interesting phenomena within large image time series. It includes: unsupervised binary change detection, unsupervised multi-class change detection, unsupervised multi-temporal classification, hot-spot change monitoring;
  3. SupervisedAnalysis, aiming at applying specific processing chains to limited image time series.



Figure 1: Architectural overview

Figure 1 provides an overview of the IIM-TS system architecture. In the upper part the three main components are reported in sequence (the Unsupervised Analysis is split into two sub-components, Primitive Feature Extraction and Clustering), while in the lower part some examples of services provided by external modules, permitting additional and optional operations, are represented.

The architecture of the IIM-TS system will also permit to develop and publish services into the KEO environment, which provides the basic functionalities for chaining and distributing services related to EO images exploitation (e.g. data acquisition from archives, image processing, publication of results to OGC servers).

How it works

The goals of the analysis of multi-temporal images may be different depending on the kind of data available and on the specific application considered. The most widely addressed applications are mainly related to:

  • Change Detection
  • Multi-temporal classification
  • Trend Analysis 

Change detection involves the analysis of two (or more) remote-sensing images acquired in the geographical area of interest at two (or more) different times in order to derive relevant information concerning the potential changes occurred in the earth surface between the two (or more) dates under consideration. A change can be considered as a categorical variable (class) or as a continuous variable. Authors generally distinguish between land-cover conversion, i.e. the complete replacement of one cover type by another, and land-cover modification, i.e. more subtle changes that affect the character of the land-cover without changing its overall classification. Land-cover modifications are generally more prevalent than land-cover conversions.

Multi-temporal classification involves the joint analysis of multi-temporal images for producing maps of land-cover transitions between the two considered acquisition times or the classification of the temporal signature modelled by a series of images acquired in a season (or during a year) in the investigated area.

Trend analysis in temporal series is strongly related to the analysis of land-cover modifications, according to the extraction of proper indicators of the dynamic of the land-cover, like vegetation indices, etc. This issue can be also related to forecasting applications.

In order to improve system functionalities for the analysis of multi-temporal images, a Time Series Designer GUI has been also developed and integrated within the KEO system, for helping users in preparing, browsing and analysing satellite image time series. The following figure show an example of the new user interface for Data Preparation.


Figure 2: Data Preparation


The IIM-TS project aims at developing and delivering a quasi-operational system permitting to fully exploit Image Information Mining capabilities for the analysis of image time series, either through the application of Probabilistic Information Mining techniques or through the use of specific Feature Extraction Algorithms.

During IIM-TS Phase 1 the most relevant applications have been identified and selected for prototypes implementation:

  • Binary Change Detection
  • Multi-class Change Detection
  • Trend Analysis
  • Shape Change Detection

During Phase 2 and Phase 3, the selected software prototypes have been implemented and integrated as KEO processing chains, and the graphical interface used to access the system has been upgraded to extend available functionalities for time series analysis.

Technical details on developed software prototypes and processing components integrated in KEO are available in the following documents:

The Final Presentation has been held at ESRIN on September 2009.

Contributors to this page: Michele Iapaolo .

Page last modified on Wednesday 30 of January 2013 16:56:54 CET by Michele Iapaolo.