CMETEO

Introduction

Cmeteo stands for CORINE-meteo. The softwaretool Cmeteo can be seen an application suite to process meteo data as found in e.g. the CGMS_EUR_08.GRID_WEATHER dataset. It aggregates and translates meteo indicators according to some spatial and thematic specs. Think of meteo indicators on temperature (maximum, minimum, mean or daily), windspeed, precipitation, radiation. Also indicators as potential transpiration, evaporation may be included. In the context of MARS, the weather indicators are given per grid. These indicators will be aggregated to spatial themas like NUTS or agri environmental regions, or translated/adjusted for landcovers of interest like crops, etc. Translation will be accompanied by aggregation.

The process

The process can be described in a short way as: At the start, only once, initialise parameters to hold values that are valid for all regions and for all days in the specified interval.

• parameters according the ROI and theme: type of aggregation {plain, median}, type of interval {day, dekade}, indication to include crops {yes,no}, thresholds for area of grid {0, integer up to 99}
• get a list of all days to process
• get a list of all landcovers that are relevant for the ROI
• get a list of all cmeteo-members, being day-offsets and members of meteo-ensembles. Mostly, there is only one member and no day-offset.

Continue to process regions of level 0 belonging to the ROI:

• get a region
• get all subregions of all levels contained in that region.
• gather input for each subregion:
  • initialise an array of weather indicators for all days, all landcovers and all members of cmeteo (day offsets and member_no's).
  • for each day:
    • collect available weather indicators of all cmeteo-members for the grids contained in the subregion.
    • for each threshold:
      • if the size of area is above the threshold, summarise them per cmeteo-member and landcover. The values are weighted with the area in the grid of the landcover.
      • optionally, add aggregated weather indicators for crops, weighted for the area of the crop in the region.

As a result you find an array of aggregated weather indicators per (sub)region, landcover, day and cmeteo-member. The members of the array will be merged into the database according the rules:

• if the member does not contain any aggregation, then delete it's former representation from the database if one exists
• if the member contains an aggregation, then update it's former representation in the database if one exists
• if the member contains an aggregation, then insert it into the database if no former representation exists

In the end stop after processing all indicated regions of level 0 belonging to the ROI.

Environment

The cmeteo package is designed to operate in several environments which may differ in both input as output datasets. All these environments need to have the basic data (both input and output) available for the CMeteo process.

Software tools needed by cmeteo

Cmeteo uses functionality which is implemented in several other software tools. The major ones are listed in the following table.

tool	remarks
REGLISTS	supplies specific lists of regions directed by input parameters.
MRS_PROCESS	contains common used functionality for processing more complex programs like cmeteo.
ADD_PROCESS_LOG	procedure to send some info to a logtable.
ADD_PROCESS_METRICS	procedure to send some data, retrieved during processing, to a logtable.
update_events_archive	procedure, located in the PMB schema, to signal the successful end of processing.
DATE_GENERATOR	generic tool to generate specific lists of dates according to the input parameters.

Administration

Before the process will be executed, some checks and tasks, mainly administrative, are done.

The module checks if no other instance of cmeteo is active in it's schema and registers itself as active. At any one moment only at most one process per schema can be active. After registering, it checks if all objects are available that are needed for the selected theme (including the ROI). It also remakes some synonyms according to the demands of the theme. Before processing, it checks the availability of weather indicators for the selected interval. After processing, it collects counters of processed data and some other items and stores them into the metrics table; it signals the PMB that the process has successfully ended Finally it releases the lock on active processes.

Interface

The cmeteo package is designed to act in a batch file, or to start from the commandline. The interface consists of a set of parameters.

To get cmeteo working, you supply:

• the username (and password) of the database schema that holds the data to be processed. This user name is part of the interface
• theme, (that implies a ROI)
• spatial indicator, e.g. 'Regions', 'Agri environmental zones'

Optionally, you can supply:

• a day or an interval of days to process
• an indicator whether or not to refresh the previously aggregated weather indicators.
• a set identifiers of regions to process

Details of the interface

• The theme must be supplied in p_theme. Possible alternatives are listed in the process table of MRSMAN.
• The spatial indicator must be supplied in p_spc_res. Possible alternatives are listed in the process table of MRSMAN.
• (Optional) An interval or day to process in the set of: p_yr2start,p_dy2start,p_yr2end,p_dy2end. Cmeteo uses several possibilities to define an interval. A coarse way is to mention a year to start and a year to end. In this case all days from the first of january of the starting year to 31-st of december of the ending year will be processed. A more finegrained way is to provide a date to start and a date to end in which case only the days in between will be processed, or even one day of both dates are the same. In fact you can supply any combination of years and dates. If you don't supply any, the default interval will be choosen, which start at the first of january of the current year up to the day before today. In fact, no day in the future will be processed.
• (Optional) an indicator to refresh previous values in p_refresh, being one of 'yes', 'no'. The default is 'yes' this one is used when no value for this parameter has been supplied.
• a set identifiers of regions to process in qry_region. You may add a query or sql statement that delivers the preferred subset of regions for which you need cmeteo. The regions must be identified by their reg_map_id as found in table REGION_MAPPINGS.

The selected set must contain some or all reg_map_ids of level 0 (!) for the relevant ROI, e.g.: select rm.reg_map_id from region_mappings rm where rm.reg_level = 0 and rm.reg_code in ('LU', 'BE', 'NL'); You also may supply a limited list of ids of region mappings which must be comma separated. If you leave this parameter empty, all regions of the ROI will be processed.

Also several datasources belong to the interface of cmeteo. The most important ones are listed in the table below.

Datasets used by cmeteo:

dataset containing	example in CGMS_EUR_08	name in cmeteo	remarks
weather indicators per grid	GRID_WEATHER	CMETEO_GRID_WEATHER	input for cmeteo with standardized attribute names
weather indicators per NUTS-region	NUTS_WEATHER	CMETEO_NUTS_WEATHER CMETEO_REGION_WEATHER	input for cmeteo with standardized attribute names
weather indicators per agri-environmental region	ZONAL_WEATHER	CMETEO_REGION_WEATHER	input for cmeteo with standardized attribute names
landcovers or crops to be evaluated	CROP_LANDCOVER	CMETEO_LANDCOVERS	input for cmeteo with standardized attribute names
landcover areas intersection of grid and NUTS region	GRID_PLUS_NUTS_LANDCOVER	CMETEO_GRID_NUTS_LCV_AREAS CMETEO_GRID_LANDCOVER_AREAS	input for cmeteo with standardized attribute names
areas/surfaces for landcovers per grid and per agri-env. region	GRID_PLUS_ENS_LANDCOVER	CMETEO_GRID_ZONAL_LCV_AREAS CMETEO_GRID_LANDCOVER_AREAS	input for cmeteo, renaming columns to a standardized naming
areas/surfaces for crops per NUTS region	AGGREGATION_AREAS	CMETEO_AGGREGATION_AREAS	input for cmeteo, renaming columns to a standardized naming
weather indicators per region which are rejected		CMETEO_REJECTED_WEATHER	output for cmeteo, containing weather indicators per region that could not be stored in regular output via cmeteo_region_weather
regions and their mappings		CMETEO_REGION_MAPPINGS	input for cmeteo

Specifications of cmeteo_grid_weather

Cmeteo_region_weather is a replacement for a variety of datasets to which cmeteo need to fill up with output. The results of cmeteo must be placed into datasets with an appropriate spacial resolution like NUTS regions or agri-environmental zones. In general, datasets for NUTS-regions are pre- or postfixed with 'nuts'; datasets for agri_environmental zones are pre- or postfixed with 'zonal'. Per thema, the structure of datasets for both resolutions are very similar. In various schemas, datasets containing weather indicators do vary both in name of the set and in names of the attributes. Also the collection of the attributes may vary. However, all the sets contain the indicators that are essential to process cmeteo. As cmeteo only can use one structure for grid weather, some logic must be applied to transform the various sets into unified representations. This transformation is implemented as a view (standard selection) on each of the original datasets while renaming the attributes and eventually suppliyng neutral or valuable data for missing attributes. The dataset cmeteo_grid_weather is composed of the most wide combination of attributes of the various underlying datasets. Below a summary of some sets in various schemas and the definition of cmeteo_grid_weather domain schema: CGMS_EUR_08 CGFS_EUR_09 CGFS_EUR_09 CGMS_ASIA_08 dataset: grid_weather ecmwf_his_downscaled_data ecmwf_eps_downscaled_data ecmwf_downscaled_data cmeteo_grid_weather attributes: grid_no grid_no grid_no ecmwf_grid_no grid_no NN number(10) day mdate mdate mdate day NN date - 1) - 1) date_offset - 1) day_offset NN number(10) - 2) - 2) member_no - 2) member_no NN number(10) maximum_temperature temperature_max temperature_max temperature_max max_temperature NN number(3,1) minimum_temperature temperature_min temperature_min temperature_min min_temperature NN number(3,1) - 3) temperature_mean temperature_mean temperature_mean mean_temperature NN number(3,1) - 4) temperature_dew temperature_dew temperature_dew dew_temperature NN number(3,1) vapour_pressure vapour_pressure vapour_pressure vapour_pressure vapour_pressure NN number(4,2) windspeed wind_speed wind_speed wind_speed wind_speed NN number(5,1) rainfall precipitation precipitation precipitation precipitation NN number(4,1) e0 e0 e0 e0 e0 NN number(4,2) es0 es0 es0 es0 es0 NN number(4,2) et0 et0 et0 et0 et0 NN number(4,2) calculated_radiation radiation_global radiation_global radiation_global global_radiation NN number(6) snow_depth snow_depth snow_depth snow_depth snow_depth number(6,2) 1) If an attribute for the offset of the day isn't available, the day-offset in cmeteo grid weather will be filled with a neutral value (0). This value doesn't influence the processing. 2) For sets that doesn't contain an attribute for members, a member_no: 1 will be supplied in cmeteo grid weather. 3) If an attribute for the mean or daily temperature doesn't exist, this value will be estimated as the mean value of minimum and maximum temperature of the day. Cmeteo will process this value and produces a result for it. 4) A dummy value (0) for the dew temperature wil be supplied to cmeteo grid weather for sets lacking such an attribute. Cmeteo will process this value and produces a result for it, so precautions must be taken to supply these results to the output table.

Specifications of cmeteo_region_weather

Cmeteo_region_weather is a replacement for a variety of datasets, both for themes (e.g. cgfs-his, cgms) and for resolutions (e.g. NUTS regions, agri-environmental zones), to which cmeteo need to fill up with output. In general, datasets for NUTS-regions are pre- or postfixed with 'nuts'; datasets for agri_environmental zones are pre- or postfixed with 'zonal'. In various schemas, datasets containing weather indicators do vary both in name of the set and in names of the attributes. Also the collection of the attributes may vary. However, all the sets contain the essential indicators for which cmeteo delivers output. Per theme, the structure of datasets for both resolutions are very similar. As cmeteo only can use one structure for region weather, some logic must be applied to transform the various sets into unified representations. This transformation is implemented as a view (with the possibility to insert/update values) on each of the original datasets while renaming the attributes. As the dataset cmeteo_region_weather delivers the most wide combination of attributes of the various underlying datasets, some precautions must be taken to prevent some useless attributes being placed into the final dataset. Functionality for this is located in a technical construct (trigger) which is connected to the views. In a dynamical way cmeteo_region_weather will be linked to the proper dataset according to the theme and the appropriate spacial resolution to deliver the results of cmeteo. Below a summary of some sets in various schemas and the definition of cmeteo_region_weather domain schema: CGMS_EUR_08 CGFS_EUR_09 CGFS_EUR_09 CGMS_ASIA_08 datasets: nuts_weather ecmwf_his_downscaled_nuts ecmwf_eps_downscaled_nuts nuts_weather cmeteo_region_weather zonal_weather ecmwf_his_downscaled_zonal ecmwf_eps_downscaled_zonal attributes: reg_map_id reg_map_id reg_map_id reg_map_id reg_map_id NN number(10) landcover_id landcover_id landcover_id landcover_id landcover_id NN number(10) threshold threshold threshold threshold threshold NN number(3) day mdate mdate day day NN date - 1) - 1) date_offset - 1) day_offset NN number(10) - 2) - 2) member_no - 2) member_no NN number(10) maximum_temperature temperature_max temperature_max max_temperature max_temperature NN number(3,1) minimum_temperature temperature_min temperature_min min_temperature min_temperature NN number(3,1) daily_temperature temperature_mean temperature_mean daily_temperature mean_temperature NN number(3,1) - 3) temperature_dew temperature_dew dew_temperature dew_temperature NN number(3,1) vapour_pressure vapour_pressure vapour_pressure vapour_pressure vapour_pressure NN number(4,2) windspeed wind_speed wind_speed wind_speed wind_speed NN number(5,1) rainfall precipitation precipitation precipitation precipitation NN number(4,1) e0 e0 e0 e0 e0 NN number(4,2) es0 es0 es0 es0 es0 NN number(4,2) et0 et0 et0 et0 et0 NN number(4,2) calculated_radiation radiation_global radiation_global global_radiation global_radiation NN number(6) snow_depth snow_depth snow_depth snow_depth snow_depth number(6,2) 1) The value for the offset of the day as produced by cmeteo will not be inserted into indicated sets. 2) The value for the member as produced by cmeteo will not be inserted into indicated sets. 3) As input for this attribute was lacking cmeteo produced a dummy value (0). This value must be omitted.

Specifications of cmeteo_aggregation_areas

Cmeteo_aggregation_areas is a replacement for a some of datasets from which cmeteo need to gather input concerning sizes of surfaces. Below a summary of some sets in various schemas and the definition of cmeteo_aggregation_areas domain schema: CGMS_EUR_08 CGFS_EUR_09 CGMS_ASIA_08 dataset: aggregation_areas - 1) aggregation_areas cmeteo_aggregation_areas attributes: crop_no crop_no crop_no NN number(10) nuts_code nuts_code reg_code NN varchar2(10) year year year NN number(4) area_cultivated area_cultivated area_cultivated number source source source NN varchar2(40) day day percentage percentage 1) CGFS_EUR_09 uses the dataset of CGMS_EUR_08.

Specifications of cmeteo_grid_landcover_areas

Cmeteo_grid_landcover_areas is a replacement for a some of datasets from which cmeteo need to gather input concerning sizes of surfaces. Below a summary of some sets in various schemas and the definition of cmeteo_grid_landcover_areas domain schema: CGMS_EUR_08 CGFS_EUR_09 CGMS_ASIA_08 datasets: grid_plus_nuts_landcover - 1) grid_landcover_areas 2) cmeteo_grid_landcover_areas grid_plus_ens_landcover - 1) attributes: grid_no grid_no NN number(10) nuts_code/ens_code landcover_id landcover_id NN number(3) area area NN number(10) reg_code (only in gpel) reg_map_id reg_map_id NN number(10) 1) CGFS_EUR_09 uses the dataset of CGMS_EUR_08. 2) In CGMS_ASIA_08 a dataset has to be defined.