View source for Interpolation of forecasted weather

__NOTOC__
{{Scientific}}
==General description==
[[File:Downscalling_eps_mon.jpg‎|Interpolation from 0.5 x 0.5 degrees grid to  25 x 25 km regular climate grid.|thumb|200px]]
Forecasted weather data, at daily timesteps, come initially on regular latitude longitude [[Meteorological data from ECMWF models#Spatial representation|grids]] in the tables {{Object|WEATHER_<MODEL>_GRID_RAW}}. The OPE model is stored on a 0.25x0.25 degrees resolution; the ENS and ENSEXT models on a 0.5x0.5 degrees resolution and the SEAS and ERA model on a 0.75x0.75 degrees resolution. The final target grids are:
* a regular 0.25 by 0.25 degrees grid used for global crop specific water balance calculations in support of the [https://mars.jrc.ec.europa.eu/asap/ ASAP system]
* a regular agriculture 25 by 25 km grid in a projected coordinate system for specific regional windows e.g. [[Interpolation of observed weather#General description|Europe]], used in the [[Crop_Simulation|crop simulation]]

To get the daily data on the target grids a downscaling method is applied.

==Downscaling==

As summarized in section [[Meteorological data from ECMWF models#Extraction of data into files|Meteorological data from ECMWF models]] forecasts for the subsets ENS, ENSEXT, SEAS and ERA are delivered into the MCYFS on coarser grids than the aspired MCYFS target grids of 0.25° or 25 kilometres. That is why scaling methods towards the target grids are applied. Aim is to have forecasts of the ECMWF subsets ENS, ENSEXT, SEAS and ERA that are as comparable and consistent as possible with the deterministic analysis and forecast OPE. The OPE analysis combines the most advanced assimilation system for observed atmospheric data with highest model grid resolution, involving the most accurate model physics, elevation and land use and soil type models. Simple interpolation methods, as inverse distance or spline, do not add information to the data. Differences in elevation, land use, land-sea-pattern are not considered. That is why in MCYFS the temperature and humidity elements, wind and radiation statistical relations between the subsets and the OPE analysis are applied. Each land grid point has its location-specific, time-dependent set of equations per subset.

'''IDW interpolation'''<br>
First data are interpolated to a regular 0.25 x 0.25 degrees grid. The spatial resolution of this grid used to be the size (approximately) on which the operational model of ECMWF was running until early 2010. We call this the OPE grid. Note that in the MCYFS database the model HIS appears as separate objects but it actually refers to the analysis part of the OPE model (the first day of the forecast depth). Therefore here only OPE is mentioned. This interpolation is applied for the ENS, ENSEXT, SEAS and ERA models data sets.

For every cell in the target OPE grid, an inverse distance interpolation (also called {{Gloshint|IDW|inverse distance weighting|IDW}}) for all weather variables is done to the 4 nearest cells of the source grid. As the ENS, ENSEXT, SEAS and ERA source grids completely surrounds the OPE target grid and covers both sea and land, the 4 nearest cells should be roughly in all directions, even for cells at the borders of the OPE target grid. The inverse distance interpolation of for instance precipitation for a single cell can be mathematically written as:
{{Hidden
|[[File:Formula IDW.JPG|200px|IDW e.g. for precipitation]]
|where:
* ''P<sub>i</sub>      : Precipitation from one of the 4 neighboring cells''
* ''d<sub>i</sub>      : Distance from the target grid cell to one of the 4 neighboring cells''
* ''Pnew               : Interpolated precipitation value''
}}

In words, the summed nearest cell precipitations / divided by the distances are divided by the summed inversed distances to give the interpolated precipitation value. Please note that distance works linearly in the used formula. A point twice as far, has half the influence. Furthermore, distances are determined in km by calculating the arc across the globe between the ECMWF model grid cell and the target grid cell and multiplying this arc with the earth’s radius.

'''Grid specific corrections'''<br>
Next, down-scaling for these models continues with grid specific corrections because of differences in elevation, land use, land-sea-pattern between the source model (ENS, ENSEXT, SEAS and ERA) and the target model (OPE). Essentially the model data are tuned to the OPE model. Main advantage is that data of the different models can be better compared and more or less consistent time-series are obtained linking reanalysis, 
SEAS, ENSEXT, ENS all around a common ‘OPE’ reference.

The grid specific equations has been derived by means of linear regression (MOS = Model Output Statistics) with the daily OPE data of at least three recent years as training set.

{|class="collapsing_table collapsible collapsed"
!Data sets used to derived the grid specific corrections
|-
|
{|class="wikitable"
!Model!!Period OPE and model training data*!!Elevation model**
|-
|ERA||Jan 2008 - Dec 2010||N400 / ALTITUDE_25
|-
|ENS||Oct 2011 – Sep 2014***||N640 / ALTITUDE_16
|-
|ENSEXT||Oct 2011 – Sep 2014***|||N640 / ALTITUDE_16
|-
|SEAS||Jan 2011 – Dec 2014****||N640 / ALTITUDE_16
|}
<nowiki>*</nowiki> hindcast model data of ENS, ENSEXT and SEAS<br>
<nowiki>**</nowiki> original elevation model of ECMWF OPE model, used to derive bias correction: Gaussian grid (and afterwards converted to a 0.25 by 0.25 degree grid) plus ALTITUDE_25 and ALTITUDE_16 columns of table {{Object|GRID_HIS}}, storing the aggregated/resampled elevation at 0.25 degree resolution)<br>
<nowiki>***</nowiki> ENS and ENSEXT data of first forecast day<br>
<nowiki>***</nowiki> each 2nd of month, 51 ensemble members, 48 runs Jan 2011 .. Dec 2014

The ERA-interim correction method was developed with OPE-data from 2008-2010 and as a consequence the method corrects for elevation differences between the elevation model ERA and the model OPE as it was in 2008-2010 (running on the N400 reduced Gaussian grid, ~ 25 km). The correction method for the other models was developed with OPE-data from 2011 - 2014 and as a consequence the method corrects for elevation differences between the elevation model ENS or ENSEXT or SEAS and the OPE model as it was in 2011 - 2014 (running on the N640 reduced Gaussian grid, ~ 16 km)  
|}


The MOS routine is used to carry out a linear regression between OPE data and the IDW-interpolated ENS, ENSEXT, SEAS or ERA data for each grid point.

{|class="collapsing_table collapsible collapsed"
!Resulting linear equation (in this case demonstrated for the ERA data set):
|-
|[[File:Formula MOS 1.JPG|350px]]
|-
|where:
* ''Y<sup>ERA-int</sup> : ERA IDW-interpolated parameter (e.g. temperature, wind) for grid box [i,j]''
* ''Y<sup>ERA-int,corr</sup> : ERA IDW-interpolated and corrected parameter for grid box [i,j]''
* ''α, β : correction coefficients (slope and intercept)
|-
| The parameter T<sub>i,j</sub> accounts for an additional seasonal correction and reads:
|-
|[[File:Formula MOS 2.JPG|350px]]
|-
|where:
* ''T<sub>1</sub> to T<sub>4</sub> : sinusoidal time functions with a period of one year''
* ''y<sub>1,i,j</sub> to y<sub>4,i,j</sub> : coefficients of time functions''
|-
| The sinusoidal time functions that were used reads:
|-
|[[File:Formula MOS 3.JPG|200px]]
|-
| With the combination of the above sine functions and coefficients, any grid-specific time correction function can be constructed. Logically the equations are most valid for the models of which the data has been used. Once the source models (SEAS, ENS and ENSEXT) develops further (different physics and spatial resolution) these equations become less accurate and need to be updated. For instance ENS and ENSEXT models changed since March 2016 (ENS day 1-10: from 32 to 18 km resolution and ENS day 11-46: from 64 to 36 km resolution) and the SEAS model changed November 2018 (80 to 35 km resolution). 

Besides, the corrections are valid for a certain OPE model. For example the corrections for ERA data are based on the OPE model that was run on a Gaussian N400 reduced grid (~25x~25km). It means that the downscaled ERA data best reflect the height model of this OPE model (column ALTITUDE_25 in table {{Object|GRID_HIS}}). 

March 2016, the OPE (HRES) model changed from 16 to 9 km resolution. Since that date OPE data, retrieved at a 0.25 degree resolution, are based on the 9 km elevation model and not on the 16 km elevation model. This could lead to inconsistencies in the time-series of OPE data. As for the main agricultural areas the differences between the 9 and 16 km elevation models, both aggregated at 0.25 degree resolution, are relative small, it is assumed that the aggregated OPE data can be represented by the 16 km based aggregated elevation model (column ALTITUDE_16 in {{Object|GRID_HIS}}).

|}

The grid specific correction is done for all elements except rainfall and snow depth as for the latter two no reliable equations could be derived. The coefficients of the equations are available in tables {{Object|GRID_SEAS_DOWN_ALGORITHMS}}, {{Object|GRID_ENS_DOWN_ALGORITHMS}}, {{Object|GRID_ENSEXT_DOWN_ALGORITHMS}} and {{Object|GRID_ERA_DOWN_ALGORITHMS}}.

The below pictures visualize the advantage of the applied downscaling method. The downscaling (middle) added information to the raw data (left). Downscaled data and the OPE analysis (right) align.  

{|class="wikitable"   
|-
|[[File:ERAInterim075Temp 20100601.png|thumb|left|300px|Daily mean air temperature on 01 June 2010 as analyzed by the 0.75° ERA-Interim.]]||[[File:ERAInterim025Temp 20100601.png|thumb|centre|300px|Same day, same source data but downscaled with grid-point- and time-specific method.]]||[[File:OPE025Temp 20100601.png|thumb|right|300px|Daily mean air temperature on 01 June 2010 as analyzed by the 0.25° deterministic ECMWF OPE.]]
|}


Finally, data available at the global OPE grid (both OPE data and downscaled ENS, ENSEXT, SEAS and ERA data), are interpolated to the 25 by 25 km agricultural grid of a specific regional window like Europe or China. The latter step is needed because:
* different projected co-ordinated systems between the global OPE and local projected grid of a regional window
* altitude differences between the global OPE grid and the local projected grid of a regional window

Data are downscaled applying an IDW interpolation and correcting for elevation differences (lapse rate). The correction factors used for the temperature and vapor pressure are respectively -0.006 (°C.m-1) and -2.5% per 100 meter increase (van der Voet et al., 1994). Therefore dewpoint temperature is first converted to actual vapour pressure while after applying the correction the actual vapour pressure is converted back into dewpoint temperature. The height models used are:
* the 25 km based height model in case of downscaled and corrected ERA data
* the 16 km based height model in case of downscaled and corrected SEAS, ENS, ENSEXT
* the 16 km based height model in case of OPE data

==Calculation additional parameters==

After down-scaling, both at the global OPE grid and the local agricultural 25 by 25 km grid, the following additional parameters are calculated at daily timestep:
* Actual vapour pressure
* Evapotranspiration (crop reference, wet bare soil and open water)

===Actual vapour pressure===
Actual vapour pressure (ea) is derived from dew point temperature (Td) by applying a standard formula for calculating the saturated vapour pressure at a specific temperature (in this case the dew point temperature).
{{Hidden
|[[File:Formula actual vapour pressure.JPG|200px|actual vapour pressure]]
|where:
*''e<sub>a</sub> : actual vapour pressure [hPa]''
*''T<sub>d</sub> : dewpoint temperature [degrees Celsius]'' 
}}

===Evapotranspiration===
In general, the evapotranspiration from a reference surface, the so-called reference crop evapotranspiration or reference evapotranspiration (ET0) can be described by the FAO Penman-Monteith ([[References|Allen et all., 1998]]). Evapotranspiration from a wet bare soil surface (ES0) and open water(E0) is calculated with the Penman formula ([[References|Penman, 1948]]). This processing is done in a similar way as for the observed weather. Therefore please follow the links here to get more details on the calculation of the [[Meteorological data from ground stations#Angot radiation|Angot radiation]] and the calculation of [[Meteorological data from ground stations#Evapotranspiration|evapotranspiration.]]

==Processing line==

ECMWF model data are delivered as described in section [[Meteorological data from ECMWF models#Extraction of data into files|meteorological data from ECMWF models]].


OPE data are directly loaded into the data base without any downscaling as the data are delivered at the global OPE grid (0.25 by 0.25 degree):

[[File:Processing-HIS-OPE-data.jpg| ]]


Down-scaling from coarser resolution global grid ECMWF data (ENS, ENSEXT, SEAS, ERA) to the global OPE grid according the IDW & grid specific correction (e.g. ENS):

[[File:Processing-ERA-ENS-ENSEXT-SEAS-data.jpg| ]]


Finally, the down-scaling from the global OPE grid to a 25 km local agricultural grid (e.g. for the European window):

[[File:Processing-OPE-ERA-ENS-ENSEXT-SEAS-data-2.JPG| ]]



In summary, the following down-scaling is applied per model and element (note B2 and C2 are only applied when down-scaling data from the global OPE grid towards a local agricultural 25 by 25 km grid): 
{|class="wikitable"
!Parameter!!ERA!!HIS!!OPE!!ENS!!ENSEXT!!SEAS
|-
|mean temperature || B1+C1+B2+C2 || B2+C2 || B2+C2 || B1+C1+B2+C2 || B1+C1+B2+C2 ||B1+C1+B2+C2
|-
|maximum temperature || B1+C1+B2+C2 || B2+C2 || B2+C2 || B1+C1+B2+C2 || B1+C1+B2+C2 || B1+C1+B2+C2
|-
|minimum temperature || B1+C1+B2+C2 || B2+C2 || B2+C2 || B1+C1+B2+C2 || B1+C1+B2+C2 || B1+C1+B2+C2
|-
|dewpoint temperature || B1+C1+B2+C2 || B2+C2 || B2+C2 || B1+C1+B2+C2 || B1+C1+B2+C2 || B1+C1+B2+C2
|-
|precipitation || B1+B2 || B2 || B2 || B1+B2 || B1+B2 || B1+B2
|-
|snow water equivalent|| B1+B2 || B2 || B2 || B1+B2 || B1+B2 || B1+B2
|-
|wind speed || B1+C1+B2 || B2 || B2 || B1+C1+B2 || B1+C1+B2|| B1+C1+B2
|-
|solar radiation || B1+C1+B2 || B2 || B2 || B1+C1+B2 || B1+C1+B2 || B1+C1+B2
|}
* B1 = IDW from coarse to global OPE grid
* B2 = IDW from global OPE grid to 25 km local agricultural grid
* C1 = grid specific correction applied after B1
* C2 = lapse rate correction because of elevation differences

==Storage of data==
With the data available on the regular global OPE (0.25 by 0.25 degree) and the local agricultural grid of 25 by 25km, data can be stored in the MCYFS tables {{Object|WEATHER_<MODEL>_GRID}} where <MODEL> is to be replaced by the abbreviation of one of the five ECMWF products (HIS, OPE, ENS, ENSEXT or SEAS).

[[Category:Weather Monitoring]]