Preliminary adaptations

From Agri4castWiki
Revision as of 14:30, 26 February 2013 by H Eerens (talk | contribs) (Created page with '=== 3.2.1 Preliminary adaptations === ==== Flagging ==== At the end of the pre-processing, the radiometric quality indicators, and identification of cloud (shadow)-, sno…')
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

3.2.1 Preliminary adaptations Flagging

At the end of the pre-processing, the radiometric quality indicators, and identification of cloud (shadow)-, snow-covered pixels are stored in a separate image layer, the status map. Also, most of the image layers are provided in integer images.

The post-processing starts with (1) conversion to byte (except for DMP and LST) and (2) application of the status map directly in each image layer. The latter is realized by reserving the range 251 to 255 (-5 to -1) in byte (integer) images for five different flags. Not all information from the status map is retained (e.g. radiometric quality per band). The ‘no data’ flag is separated into ‘water’ and ‘boreal winter’ using a land cover classification.

Most software can only deal with one flag, but GLIMPSE, which is used for the post-processing is specifically adapted to working with multiple flags.

More information on the flags and the rescaling can be found in the section Images. Smoothing Methodology

10-daily composite images still contain perturbations due to clouds, snow and missing values. The objective of this time series analysis is to scan each pixel's profile, detect the bad values and replace them with more appropriate ones – mostly via temporal interpolation of the previous/later "good" observations. The smoothing methodology used for cleaning the NDVI and fAPAR in MCYFS follows the approach presented by Swets et al. (1999) and modified by Klisch et al. (2007). It’s a regression methodology applied on a time series through a moving window of a defined number of consecutive observations. The modification of Klisch et al. (2007) concerns the unrealistic high values before and after steep slopes in the Swets et al. (1999) methodology, and also to keep the original high values. The algorithm requires the existence of a larger number of observations (i.e. the wide interval) than the ones to be cleaned. Usually from 15 to minimum 7 observations before and after the target period are required.

The smoothing takes place only if at least the 75% of the observations in the time series are valid (no clouds, no missing data, etc…) otherwise the smoothed values are flagged as missing value. Before the real smoothing a filter is applied to observations with high differences compared to the neighbours ones or if the neighbours have flagged values. The filtering procedure is applied to all observations in the wide interval and is based on a threshold that gives the possible amplitude variation between two consecutive observations.


More information on the methodology can be found in Swets et al., 1999 and Klisch et al., 2007.

For MCYFS, the 10-daily time series of NDVI and fAPAR, coming from VGT , MODIS and NOAA-AVHRR are smoothed. This is done prior to calculation of the Long term average, Difference images, Similarity analysis in time and Cluster classification. The non-smoothed NDVI (fAPAR) has extension i (a), and the smoothed version gets extension k (b) (see Filename conventions).

The method can be applied on a past time series (back processing), but also in near real time. When applying on a past time series, the entire series is considered at once. For the near real time mode (NRT) the processing steps are described below. Application in NRT-mode:

In near real time mode, the time series of the past 36 dekads are always used for the modified SWETS-smoothing. When a new 10-daily composite arrives, the procedure is repeated on a time series of again 36 dekads, but now shifted in time with 10 days. Smoothed images of only the last 6 dekads are created. The 6th last image is now updated for the last time and is kept as final smoothed image. The 5 last dekads are updated again when a new 10-daily composite arrives. That means that the most recent smoothed scenes are only temporary versions, which will be updated five times at later stages. Over time, the chance of obtaining “good” (non-missing) values increases, so the SWETS algorithm can provide reliable estimates. And the final version only is established after 2 months or six dekads.


Three versions of the VGT-S10 NDVI of the first dekad of 2007: left=original, middle=smoothed with the SWETS algorithm, right: idem but snow flags recovered