Formats

From Agri4castWiki
Revision as of 16:23, 19 March 2013 by H Eerens (talk | contribs)
Jump to: navigation, search



Three types of deliverables are foreseen: ENVI raster images, QuickLook Maps and Databases of Regional Unmixed Means. The three formats are discussed below:

  • Images
  • Quicklooks
  • Databases with Regional Unmixed Means

Images

Image data

The image data are stored in “flat 2D” binary files, without leader/trailer bytes, and with file name extension “.IMG”. The data type can be byte, short integer, long integer or floating point. In practice, all efforts are made to save disk space by storing the images as far as possible in the most compact Byte data type (1 byte per pixel). Only for DMP the Short Integer data type is used (2 bytes per pixel). “Flat 2D” means that every image “layer” (e.g. different bands of the same registration) is stored in a separate image file. This approach makes the imagery software-independent: this generic file type can be ingested by any (commercial) image processing package.

The ancillary information (spatial, spectral, temporal characteristics) is stored in small ASCII-files with the same name as the IMG, but with a different extension. These annotation files are software specific. By default, we generate for each IMG an associated HDR-file, compatible with the widely used package IDL-ENVI. Similar annotation files can be generated for other systems. A description of the enriched ENVI-HDR file is provided in Headers.

3D multi-band images are not appropriate for the (mostly) huge images of MCYFS. As an alternative, “metafiles” are provided. These are small ASCII-files with the names of all the images which belong together for a certain application. A metafile can be “opened” as if it were a single 3D image file. Removal or addition of new layers only requires editing the metafiles, not the copying of huge image data amounts. Again, metafiles are software specific, and by default we generate “MTA-files” for ENVI and similar “VAR-files” for GLIMPSE.

Headers

All delivered imagery is directly compatible with IDL-ENVI. However, a number of non-ENVI items are added to the HDR-files. These provide additional and essential information which is not stored by ENVI (see table for an overview of the different HDR-items): (1) Temporal: the registration date and the length of the compositing period. (2) Spectral: At the level of the digital image values (V), clear distinction is made between the “significant range” (Vlo-Vhi) and all values beyond this range, which are treated as “flags” (codes to indicate special cases). To this goal, the HDR-files contain the following two additional items:

Values = { Yname, Yunit, Vlo, Vhi, Vmin, Vmax, Vint, Vslo }
  • Yname/unit= Name/Unit of the represented physical variable Y. Eg.: NDVI (-) or DMP (kgDM/ha/day).
  • Vlo/hi = Significant range of digital values to be treated as representing this physical var.
  • min/max= Observed extremes for the IMG at hand, with: Vlo ≤ Vmin ≤ Vmax ≤ Vhi.
  • Vint/slo = Intercept/slope of linear relation between V and physical Y: Y = Vint + Vslo.V.
Flags = { Vf1=Vf1,txt, Vf2=Vf2,txt, …}
  • Vfi = Flag digital number (beyond significant range Vlo-Vhi)
  • Vfi, txt = Meaning of this flag (textual)

Below, this ENVI-format with “enriched HDR-info” is called the ENVI/GLIMPSE image format.

Contents of the ENVI/GLIMPSE HDR-Files. The keywords are grouped per category.  Non-standard keywords, added by GLIMPSE are marked in grey.


More information on the ENVI/GLIMPSE format can be found in GLIMPSE.

Flags

Flags are situated outside the significant range of digital values and are used to label pixels with deviate measurements. For all remote sensing derived images in MCYFS a so-called “UNIflags” scheme was designed, which fixes the flag values and their meaning/interpretation. The system is described in the table below. It only deals with BYTE and INTEGER images for the simple reason that all images are stored in these two data types. For BYTE images, the flags are situated in the upper range of values (251-255). Hence, the significant values (Vlo-Vhi) are restricted to the range 0-250. For INTEGER images, the flags run from -1 to -5, while the significant range covers the higher positive values (0-32767).

Unified flags used in the final images (UNIflags). All images are BYTE, with exception of the Integer DMP

UnifiedFlags.png

Quicklooks

All QLs are currently generated with a special JAVA-programme. Part of the information (date, periodicity, map system, sensor,…) is read from the HDR-file of the IMG to be treated. The remainder is read from two files with “fixed specifications”, which must be prepared in advance:

  • For each sensor: a specifications file which defines the spatial aspects of the QL: layout, annotation, vector overlays (grid and admin. boundaries), logo’s, …
  • For each variable (NDVI, DMP,…) and its different states (actual values, historical mean, the various differences,…): a specific theme related file which defines the thematic aspects (colour scalings, legend,…).

The output is a PNG-file.

Databases of regional unmixed means

The output *.RUM-files are ASCII-formatted and have the following characteristics: • One line per “case”, with 12 comma-separated values per case/line. The table below gives the detailed contents. • The means and standard deviations are automatically scaled to correct physical units (e.g. “kgDM/ha/day” for DMP, “-“ for NDVI, “°C” for TS, etc.). • Missing values are labelled with an unambiguous code: -999.999. • The involved areas are expressed in relative terms (RA1/RA2) and thus independent of sensor/resolution. • Cases/lines with missing values (NpixU=0) are skipped to minimize the file size.

RUMformat.png