Loading...
 
ESA > Join & Share > Technology Projects > MIR-E Project
Print

MIR-E Project

Project Title   Multiple Image Registration - Extension
Project Acronym   MIR-E
Contractor(s)   NCC - Norwegian Computing Center (Norway)

 

Image     Context
          Objectives
             Architecture
          How it works
    Output
 

Context

The study of time series of satellite images is an important task in many remote sensing applications where the objective is to study different environmental phenomena. For such applications a co-registration of the satellite images acquired at different times are important. This co-registration is often performed using a combination of manual and automatic registration techniques. However, for a multi-temporal problem where the number of images becomes large, manual correction of images is not feasible. Hence, a fully automatic procedure would be desirable.

Automatic techniques for performing each step of the process do exist, but there is no one registration technique that works equally well for all image types. The selection of the appropriate method depends on the application and the image specifics. Hence, a single registration scheme will generally not work for all different applications. Consequently there exists a large number of different automatic registration techniques, all of which essentially perform the same task, the only difference being that they are constrained to working with a very small range of images.

For a user that needs to work on different types of time series, it would be useful to have a more general tool for image registration that could be used for several applications. It has been estimated that more than 90% of the studies in remote sensing that could have used automated approaches for registration of images, did not use it. The lack of a more general tool for helping in this process, may be one of the reasons for this.

Top  


Objectives

The objective of the MIR-projects has been to develop a tool for automatic co-registration of homogeneous sequences of images, i.e. time series of images resulting from the same (or similar) sensor with comparable resolution. The tool was intended to be general. This means that it should be able to handle images from different sensors, with different contents and acquired under different conditions. No single registration scheme will generally work satisfactorily for all cases. The idea was therefore to develop a registration tool that includes a number of different methods and intelligence enabling the tool to select in each case the most appropriate methods based on image characteristics.

Such a tool was developed by Norsk Regnesentral for ESA in 2004-2005. The purpose of the software is to provide a tool for co-registration of images which is able to automatically adjust the processing to the characteristics of the image and select both the regions of the image to be used for the registration and the registration approach to be used for each region.

The Mir-Extension project (2006-2009) following this aimed at validating and improving the tool, including multiresolution strategies for handling of larger distortions and integration of the tool into the KEO framework.

Top  


Architecture

The MIR-tool is composed of several components. The diagram below gives a sketch of the basic components. The software is developed in ENVI/IDL and C/C++.

Image

Top  


How it works

The MIR tool performs the task of co-registering time series of images by registering a sensed image to a reference image. The co-registration can be performed automatically without manual intervention. For this a methodology for intelligent selection of methods based on image characteristics has been developed and combined with existing methods and tools for image matching and registration. The approach is adaptive in the way that it automatically tries to select the best registration method based on image characteristics. However, as characteristics of a remote sensing image may vary across the scene, our approach is also adaptive in the way that it tries to select the regions of the image that are best suited for registration. This also enables us to select the best registration method for each region of the image.

The approach works by dividing the pair of images to be co-registered into smaller sub-regions and extracting features from each region. Based on the extracted features the performance of each of the available methods is predicted by using a neural net. For regions with sufficiently high performance scores, the method with the best rating is used to perform a local co-registration. This results in a set of local transformations, which is used to find the global transformation. This process is illustrated in the figure below.

Image

In the following we have illustrated the performance of the adaptive image registration. This is illustrated for a pair of NOAA-AVHRR images taken over the southern part of Norway. This image pair contains differences in both cloud coverage and snow cover (red).

 

Image

In the figure below the selected regions, when using the adaptive registration on this image pair, are showed. The superimposed grid shows the division into sub-regions, and the regions marked with yellow shows the regions that were selected using two different parameter settings for region density. As can be seen from this result, the adaptive registration has selected regions over the parts that are not covered by clouds and it has also ignored homogeneous sub-regions only containing sea.

Image

The last figure illustrates which method that was selected for each region, where each sub-region is colour-coded according to which method that was applied for that region. As can be seen from this illustration, the method selected most often is method number 4. This method consists of a mutual information metric in combination with an evolutionary-based optimizer. This combination of methods is able to handle complex correspondence between image values and is also the one that is most tolerant to noisy metrics. Hence, it is well suited for an image pair like this where the differences are quite large.

 

Image

Top  


Output

The output from the project is a software that offers a general tool for co-registration of remote sensing images. The software uses a learning-based strategy where the system learns the relationship between image characteristics and performance for different registration algorithms. By applying this scheme to subimages, the approach is also made locally adaptive. This enables selection of the best registration algorithm for each region in the image, while regions unsuited for registration can be discarded.

Through this approach the software facilitates co-registration of time series of images by providing an adaptive registration with subpixel accuracy and automatic run-time selection of the best method. The software is available through the KEO environment.

The MIR final presentation was held at ESRIN on March 23, 2009.


Contributors to this page: .

Page last modified on Wednesday 15 of December 2010 16:11:57 CET by .