Software Tools

From Agri4castWiki
Jump to: navigation, search


Aggregator

This is an ORACLE package containing stored procedures developed to aggregate crop simulations to EMU, grid and regional levels and gridded weather to regional levels while differentiating for different land covers or crops. The package have to be compiled for every regional window: EUR, RUK, IND, CHN, SAM and GLO.

The core of the package is the Aggregator.aggregation procedure that select a specific aggregation procedure given the provided arguments. The arguments can be

  • Theme Crop indictors or Weather indicators (values: ‘SIM’ or ’WEATHER’)
  • Model Crop model (values: ‘WOFOST’,’WARM’,’BLAST’)
  • Meteo Meteo model (values: ‘OBS’,’HRES’,’OPE’,’HIS’,’ERA’,’ENS’,’ENSEXT’,’SEA’)
  • Crop Selected crop (values: crop number or null for all crops)
  • Start Selected start-date
  • End Selected end-date
  • From Start resolution (values: ‘STU’,’EMU’,’GRID’,’REGION_LOW’,’REGION_UPPER’)
  • To Target resolution (values: ’EMU’,’GRID’,’REGION_LOW’,’REGION_UPPER’)
  • Region Selected region (values: region code or null for all regions)
  • Continent Selected continent (values: continent code or null for all continents)
  • Regional level Select the regional level from which data must be aggregated


Two general types (themes) of aggregation procedures are distinguished:

  • Aggregate weather data from GRID resolution to all regional levels for every land cover using the information from table LINK_GRID_REGION_COVER.
  • Aggregated crop simulation results from
    • STU to EMU resolution
    • EMU to GRID resolution
    • EMU to Lowest regional level resolution
    • GRID to Lowest regional level resolution
    • Lowest regional level to Upper regional resolutions

Aggregation to the levels GRID and lowest regional level is not crop specific but based on the land cover associated with the crop parametrization (table CROP_PARAMETRIZATIONS). Area weights of these land covers are given in tables LINK_SMU_GRID_COVER and LINK_SMU_GRID_REGION_COVER. Aggregation from the lowest regional level to upper regional levels is based on crop specific area weights coming from table STAT_REGION_AREAS. Further simulation results of crop parametrization can be linked to different crops available in the statistics. For example in the regional RUK window we simulate winter wheat with parametrization 1 but we aggregate the outputs at regional level for aggregation 1 and 3, thus for winter wheat and winter barley.



AMDAC

Models of Yield Production is one of the fields covered by the Agriculture Project of the Institute for Remote Sensing Applications at the Joint Research Centre of the Commission of the European Communities in Ispra (Italy). The goal of one of the studies in this field (see operation 3.2 in the MARS Project Call for Proposals: General Conditions and Detailed Specifications of August 1990) was to provide the Agriculture Project with a software package able to perform decoding, filing and quality evaluation of actual meteorological data which are used as input for agro-meteorological models. For this purpose the Actual Meteorological Database Construction (AMDaC) package is developed by MeteoConsult (Wageningen, The Netherlands), which is described in this manual.

Red.gif

More information
AMDAC manual (pdf)

AMDAC manual Appendix A station list (pdf)


Calplat

Within the framework of the ASEMARS project, the calibration platform Calplat was developed, at Alterra, in close consultation with the Joint Research Centre (JRC) at Ispra. It was developed during 2005 and 2006, and was, in first instance, meant to calibrate the basic crop phenology parameters. Later Calplat has been extended to be able to calibrate a larger set of crop parameters for the WOFOST crop growth model and the LINGRA grass growth model, which are both part of the CGMS monitoring system.

Red.gif

More information
Calplat manual (pdf)


Calibration Manager

The calibration Manager is a Python packages that combines Python wofost (PCSE) with an open optimization tool NLopt. This makes it possible to calibrate a combination of crop parameters at the same time (e.g. TSUM1 and TSUM2, SPAN and SLATB and TDWI, etc.) using one or more target variables (e.g. day of anthesis, day of maturity, lai-max, harvest index etc.). The selected target variables are combined in a combined objective function that is optimized. Additional functionalities are added to the tool, such as normalizing target variables, assigning weights to experiments and calendars, criteria when expert knowledge is taken into account, applying additional crop masks to exclude regional observations of non agricultural areas.

Orange red.gif

More information
Calibration_Manager


CGMS

The CGMS is the combination of the WOFOST crop growth model, a relational database and a statistical yield prediction module. From 2004 onwards the development of CGMS continued in the framework of the MARSOP2, ASEMARS and MARSOP3 projects, leading to the current version CGMS 10.0.3.2. The linked document below describes version 9.2

Red.gif

More information
CGMS manual (pdf)


CGMS statistical tool

The CGMS statistical tool has been developed for JRC’s MARS project in the framework of the contract study “Actions in Support of the Enlargement of the MARS Crop Yield Forecasting System (MCYFS) Lot I (ASEMARS Lot I)”. The tool is designed for use by the crop analysts and is an improved version of the CGMS statistical module which was in use since 1994 to facilitate national and sub national crop yield forecasting. Time trend analyis of yield statistics is followed by regression or scenario analysis using biophysical indicators to explain yield statistics and search for similar years. Constructed models are used to predict yield of the current growing season.

Red.gif

More information
CGMS Statistical Tool manual (pdf)


CMETEO

The software package has been developed to aggregate grid weather to some levels of administrative and agri-environmental regions. It is a generic package that can be used with input datasets such as observed weather and forecasted weather of several models and regions of interest (ROIs) as supplied in the MARS database. For each of the input dataset the procedure is repeated using weighing methods based on occupied areas. The output of the process is merged into resulting datasets for the choosen theme and resolution.

Orange red.gif

More information
CMETEO


Control board

In order to facilitate the work of the MARS analysts the COBO (Control board) was developed. The COBO is a mixture of data warehousing and data search engines organised through a common interface based on web-portal concept. COBO represents a base tool for the analyst.


COPdate

To estimate grid specific sowing dates following sowing dates rules, the oracle package COPdate was developed

Orange red.gif

More information
COPdate


GIS interface

CGMS does not use or need a GIS to produce its results. However, a GIS is necessary for a meaningful presentation of the results, and is also indispensable for the initial creation of the database. The link to the GIS is formed by the concept of the EMU’s, grid cells, administrative regions(NUTS) and Agro-environmental zones. Meteorological data are stored in the resolutions 'grid', 'NUTS' and 'agro-environmental zones'. Simulated yields are stored in the resolutions 'EMU', 'grid' and 'NUTS'.

In the cgms database all grid cells, administrative regions and agri-environmental zones have a unique code or number that can be used to provide a link to a GIS. The EMU's are stored as a unique combination of grid number and SMU number. This unique combination can be used to provide the link to the GIS.

Red.gif

More information
GIS interface manual (pdf)


Remote Sensing Software

Orange red.gif

More information
REMOTE SENSING SOFTWARE


Image server

ECMWF weather data, aggregated for 10-daily periods and 10-daily and monthly remote sensing based indicators can be downloaded from image servers hosted by the JRC. Documentation, terms of use and download links can be found on the homepages of these image servers.


ISW

To estimate initial soil water content for crop simulations, the implemented approach is to start a water balance long before the actual crop simulation. Using this approach, recharge of the soil moisture by rainfall and water use of a pseudo crop simulation or bare soil will help to find a more representative estimate of the soil moisture level when the actual crop simulation for the year of interest starts.

Orange red.gif

More information
ISW


LTA_YIELD

LTA_YIELD is a dedicated software package that calculates long term averages of simulated crop yield for each individual geographical feature on a 10 daily base. The geographical features can be be EMU's, grid cells or adminstrative regions (NUTS).

Orange red.gif

More information
LTA_YIELD


Marsop viewer

The operational MARS services deliver and store large amounts of data. These data vary from static reference layers and input weather data originating from different supplying meteorological sources (weather stations, ECMWF) and data quick looks to the data that is generated in the various operational levels of the MARS services through downscaling, simulation of crop indicators, estimation of crop yields etcetera. The MARS viewers enable users to perform spatial and temporal analysis of these data sets in a customized way. The documentation of the Marsop3 viewer is split into two manuals: an analyst viewer version with full functionality and a web version with limited functionality.

Orange.gif

More information


PCSE

PCSE is the abbreviation for Python Crop Simulation Environment. PCSE is a Python package for building crop simulation models, in particular the crop models developed in Wageningen (Netherlands). PCSE provides the environment to implement crop simulation models, the tools for reading ancillary data (weather, soil, agromanagement) and the components for simulating biophysical processes such as phenology, respiration and evapotranspiration. PCSE also includes implementations of the WOFOST and LINTUL3.

Project Management Board

The Project Management Board (PMB) was developed to keep track of the operational processing activities. Within the MCYFS many production lines are operational. The source date come from different locations, generally in large quantities and high frequencies (daily-, decadal, monthly and seasonal updates). Different parties are involved in processing the data before they are injected in a database. And eventually most data is mirrored to a duplicate database elsewhere.

A single error could interrupt one of the production lines and may cause a complete dataset to be inaccessible (including derived results). A real-time overview of the states of the processing steps in the different production lines could help to quickly trace potential problems. This is what the PMB does. It consists of a database that stores all scheduled processing steps and their real-time status. The status of processing steps is updated by the various production lines automatically.

A web-based user interface can generate various overviews of processing steps. By default it displays the events that are delayed and not yet delivered. The user interface can also be used to manually update status information in case issues are resolved (and automatic status request can't be generated).

Red.gif

More information
PMB manual (pdf)


ReferenceWeather

Oracle package (WEATHER_OBS_LTA) to calculate average station weather.


SupitConstants

Supit constants are need to be available for all weather stations in the table WEATHER_STATION to be able to calculate solar radiation from other weather indicators as measured solar radiation is only seldom available. The SupitContstants application is developed to interpolated supit constants from the table SUPIT_REFERENCE_STATIONS to all weather stations and restuls are stored in the SUPIT_CONSTANTS table.

Orange.gif

More information
SupitConstants


Supporting software

Within the MCYFS a number of software packages are developed that support processing steps in automated production lines. Some of these software packages can also be used as a separate tool by analysts.

Orange red.gif

More information

  • REGLISTS (supplies specific lists of regions directed by input parameters)
  • DATE_GENERATOR (generic tool to generate specific lists of dates according to the input parameters)
  • PROCESS_LOGGING (procedures to send some info to a user interface)
  • PROCMAN (generic tool to assist other procedures to run in heterogeneous environments)

WOFOST Control Centre

WOFOST Control Centre (WCC) is a graphical user interface that runs the WOFOST (version 7.1.7) crop growth model. WCC facilitates selecting the production level, and input data sets on crop, soil, weather, crop calendar, hydrological field conditions, soil fertility parameters and the output options. By interactively changing model parameters and analysing the results, the tool can be used to manually calibrate a crop under local conditions. More detailed documentation and software download is available on WOFOST - WOrld FOod STudies.