MTS related Question and Answers v.2.0

From Wikicap - European Commission

Questions on MTS raised after the presentation on the Management Meeting

Go up to the main TG MTS page

Q MS Issue / reference MS question / remark JRC Reply
1 DE Performing the Model ETS We have to fill in the fields “executable FalsificationTest (true/false)” and the field “executabletestResult (pass/fail/inconclusive). Could you explain the meaning of these columns, perhaps with an example? What is the difference between these two columns to be filled in? The field “executable FalsificationTest” aims to indicate whether the conformance testing method used in a particular executable test is based on falsification testing or not (in which case it will be a verification testing)

The main objective of the Model Test Suite is to verify and document the presence of certain elements (features, attributes, properties) of the SUT (every individual LPIS implementation) so that the data value testing procedures of the annual LPIS quality assessment, laid down in the ETS, can be correctly performed. The current set-up of the MTS doesn’t yet apply methods involving rigorous proofs of correctness to conclusively and exhaustively demonstrate conformance of the SUT (or its individual elements), against local or common specifications and associated standards. In the context of ISO 19105 (2000), the approach/method for conformance testing adopted for the MTS is not based on verification testing, but on the so-called falsification testing. This setup was chosen for technical and economic reasons, since the complexity of the IACS and relevant standards makes the use of proof-of-correctness approach often impractical. The falsification test is a type of conformance testing method, not a characteristic of the SUT.

Contrary to the rigorous approach of a verification test, the falsification test merely looks for errors in the implementation (SUT). If errors are found, one can reasonably deduce that the implementation does not conform to the relevant specifications and standards; however, the absence of errors does not necessarily imply the opposite. The falsification test can only demonstrate non-conformance. It cannot provide absolute assurance that the implementation conforms to the relevant specifications and standards, since it does not guarantee that a particular suite of tests provides complete coverage of their content.

For that reason, for all executable tests suites in the MTS, the value of the field “executableFalsificationTest” should be set to TRUE.

The field “executabletestResult” aims to indicate the resulted verdict of conformance of a given executable test. The following values are possible:

  • Pass – This is a test verdict of conformance of the given executable test. In the context of LPIS QA and considering that the comformance test method is falsification testing, it means that the availability (or non-availability) of a certain element (feature, attribute, value type), in accordance with the specifications of the SUT, is confirmed and can be used for the ETS.
  • Fails - This is a test verdict of non-conformance of the given executable test. In the context of LPIS QA and considering that the comformance test method is falsification testing, it means that the availability (or non-availability) of a certain element (feature, attribute, value type), in accordance with the specifications of the SUT, is not confirmed and cannot be used for the ETS.
  • Inconclusive - This is a test verdict when neither a pass verdict nor a fail verdict apply. This should occur in very rare circumstances.

For example, a "pass" verdict for executable test case 1210 (checkReferenceParcel) means that there is a features type "Reference Parcel" defined in the SUT and this feature data can be used as an entry for the LPIS population and the ETS. However, it doesn't mean that the implementation of the reference parcel type in the SUT fully conforms with the relevant specifications and standards. Contrary, if there is no such feature type, the test will yield a “fail”verdict and the SUT will be non-conforming.

2 SI MTS log, SystemMetadata, VectorThematicMetadata TimePeriod - at least endPosition should not be mandatory The element “temporalExtent” is taken as defined in the INSPIRE metadata specification (2.6.1 Temporal extent). Although this time period may be expressed as an individual date instead of an interval, the need of consistency with ISO 19155 (Geographic information — Metadata) and ISO 19108 (Geographic information – Temporal Schema) would require to express the time period (TM_period) as open interval bounded by beginning and end points (instants). In the case when only an individual date is available (the acquisition can be considered as instantaneous), this single value will be put in both “beginPosition” and “endPosition”.
3 SI MTS log, SystemMetadata, OrthoMetadata How to proceed when we have numerous digital orthophoto maps (for example more than 30)? Do we need to provide a record for each of them? Point "System metadata for ortho datasets" of the TG MTS specifies that the orthoimage dataset to be reported in the system metadata is the most recent capture for a given site (area). Further in the same point, it is clarified that if two or more distinct datasets (subject to different specifications, production processes or acquired in different years) are in use for the graphical processes (application, LPIS upkeep) of the direct payments on the entire territory of the IUT, a separate metadata record for each dataset would be appropriate. In the context of the LPIS QA, the acquisition period of the orthoimage coverage is considered as key and mandatory metadata element that shall be provided through the phenomenonTime. An othophoto coverage subject to the same specifications and generated from input images captured in the same year (either calendar year or cropping season whichever is most relevant) is reported with one metadata record, even if the input images are acquired in different time during this year. The interval of time in which the input image was/were acquired by the sensor is reported through the phenomenonTime (for example from 2015-05-31 to 2015-07-15).
4 SI MTS log, SystemMetadata, OrthoMetadata elevationData maxOccurs is set to "1". Why is it not possible to have more/different options? The elevation data required in the system metadata is the one used as an input for the production of the orthoimage coverage reported in the given metadata record. If different elevation products have been used for the generation of the given orthoimage coverage, then separate OrthoMetadata records should be provided.
5 SI MTS log, SystemMetadata, OrthoMetadata It would be better to separate metadata on elevation from the metadata of orthoimagery in different metadata records. It will be considered in the next update of the TG MTS.
6 DE MTS log, SystemMetadata, OrthoMetadata There are several flights (possibly with different sensors) within a year to capture the photos for 1/3 of the territory. If there are 20 flights per year to capture the photos of 1/3 of the territory, should we then fill in 60 separate metadata records? Please refer to the answer to question 7. Any orthoimagery, acquired in the same year, with the same sensor platform (aerial film, aerial CCD or satellite), resulting in the same equivalent scale, and produced with the same elevation data, can be reported under one metadata record (providing that the other mandatory metadata elements are the same). In case of orthoimagery produced from input images with different ground sampling distances, the spatial resolution of the imageSource can be expressed either by the interval bounded by minimum and maximum values (TG Recommendation 18 from IR MD), or by the smallest equivalent scale.
7 DE MTS log, SystemMetadata, VectorThematicMetadata, TG IXIT Will we have no “theme”, if IXIT qualifier D delivers „raw“ (for FB, PB, AP)? Is there a vector metadata record necessary, in this case? Qualifier “D” of the IXIT aims to report how the MS Administration assembles the reference parcels subject to crosscheck. The value “raw” means that the data at reference parcel required for the crosscheck, is obtained directly from the delineated production block without any additional spatial operations. Your theme is the dataset of the assembled reference parcels, and should be always available, regardless the output of qualifier D. As specified in point 7.4.2 of the MTS, at least the vector metadata record for the assembled reference parcels should be provided.
8 DE MTS log, SystemMetadata, VectorThematicMetadata, TG IXIT Do we need to have always an ancillary thematic dataset (or datasets) to report, such as cadastral or land cover map? Not necessarily. It depends on the output of the IXIT. Such ancillary datasets are mostly used in LPIS based on TB or CP. Systems based on AP, FB or PB might not make use of such external data.
7 ES TG IXIT, page 4,“polychotomy questionnaire”, part A It is impossible to classify our LPIS given that the option SubCadastral Parcel is missing. Our LPIS uses external maps (the cadastral parcel), but later on we apply an internal IACS procedure in order to divide the cadastral parcel into homogeneous land cover units. Therefore, the reference parcel is not the cadastral parcel, it is the subcadastral parcel. The LPIS Core Conceptual Model (vs 2009) does include the SubCadastral parcel (see page 19). We do not understand why it is now missing in the current IXIT document. In IXIT such LPIS design implementation can be reported as follows:
1.Classifier A: “CP” – cadastral parcel

Reason: The data supplier of the graphical data for perimeter and RP ID is the cadastral institution (or other entrusted body). The polygon, used as reference parcel, represents the unit of land that was historically entered in the register.

2.Classifier B: “dedicated”

Reason: Physical borders used to “measure the agricultural land” are derived from a systematic land cover mapping project, covering all the agricultural land of the territory. The land cover mapping project is set-up by the LPIS-custodian or the PA in isolation.

3.Classifier D: “straightforward”

Reason: The maximum eligible area is derived from a straightforward (unmodified, unfiltered) spatial intersection between only perimeter and border polygons Note: In case of more complex sequence of spatial operations involved, the output for classifier D can be also “sub-parcelled”.


Questions on MTS raised after the 22nd MARS conference in Lisbon

Q MS Issue / reference MS question / remark JRC Reply
8 ES TG IXIT- phrasing of qualifier A Although our initial perimeter is derived from the cadastral parcel, Both ID and the spatial extent of the land corresponding to the reference parcel ID is defined as the outcome of qualifier D, not as the outcome of qualifier A. Please confirm that this is correct, and reprase Annex X-point 4.2 to “The primary boundary or perimeter represents the spatial extent of land corresponding to the reference parcel identifier.” Your possition is corrects, starting the RP creation process with a cadastral parcel does not imply that initial identifiers or point coordinates should be inherited by the resulting RP.

To clarify this viewpoint, we rephrase the q2 expression in the IXIT data structure from

  • "the data supplier of the graphical data for perimeter and RP ID is”

to

  • "the supplier of the graphical data for defining this perimeter and, if applicable, the initial alphanumeric data for the construction of the RP ID is”

Furthermore, please interpret the correspondent line in IXIT Data Scope, where for choice A the phrase

  • "The primary boundary or perimeter represents the spatial extent of land corresponding to the reference parcel identifier"

should be read as

  • "The primary boundary or perimeter represents the spatial extent of the unique unit of land.”
9 ES Data Model Testing Please confirm that conditional, or non-mandatory elements of the LPIS Core Model (LCM), when not found in the SUT:
  • when performing the capability tests, the value of the field executableTestResultDescription is “mapping is not applicable”
  • when performing the executable test, the value of the field Result is “pass”.
The assumption is correct; the Abstract Test Suite stipulates that “..If a correspondence is found or if there is a documented evidence that for the given IUT this element is not applicable (case of landscape features), the test will pass and the names of corresponding element(s) or evidences shall be documented in the test result. For non-mandatory element indeed:
  • The value of the field <executableTestResultDescription> should be filled in with the following text “mapping is not applicable, because……”, which means that some further explanations should be provided.
  • The value of the field <Result> is “pass”.

So, in general, we expect all MTS-related executable tests should pass, Resulted verdicts in MTS can be only: pass, fail or inconclusive (very rare and only if technical error occurs).

Note: although MTS and ETS are standalone tests, the correct and complete conduction of the ETS depends on the outcome of the MTS. It is clear that any xml/gml delivered in ETS can only be validated if there is a “pass” outcome from the element/feature mapping, resulting from a meaningful description of a corresponding SUT element in <<executableTestResultDescription> in the MTS. This will ensure the integrity between the data model applied in the SUT and the related observations collected during the ETS inspection, to allow correct inpterpretation LPIS QA results in the local context.

The integral link between the MTS and ETS is illustrated with two examples given below:

  1. If the MTS reveals that the element <landscape feature> is not present in the SUT (by running the executable test “checkLandscapeFeature”), then the ETS operator will not delineate and take into account as eligible, the area of any landscape feature during the ETS inspection. In other words, the quality measures RP landscape features (10104) and RP landscape features area (10104_2) from ETS Annex I will not be applied.
  2. If the MTS reveals that the element <anomaly> is not present in the SUT (by running the executable test “checkAnomaly”), then the ETS operator will not be able to assess whether a change of the reference parcel data from the information provided at the time of the LPIS population upload (LPISpointzerostate) is made in “in tempore non suspecto”. As a consequence, he/she will not be able to take this update into account in the ETS (see point 1.18 of ETS Annex II)
10 LU System Metadata In the Excel version of the MTS log, there are several sheets dedicated to the reporting of the thematic, vector and orthoimage metadata. Are they the same entries that have to be reported in the SystemMetadata.xml? Yes, they are identical. Please check the clarifications made in Table 5 of TG MTS Delivery – MTS package
11 - System Metadata - Orthoimagery Do we correctly presume that the values of spatial Resolution and sensor Distance are the same, since the reporting instructions for both of them refer to INSPIRE TG reguirement 27? TG requirement 27 of the INSPIRE IR MD specify only the value format of the reported spatial resolution. Please take into account also this spatial resolution (MD_Resolution) refers to the resulted orthoimage dataset only. The spatial resolution of the source image is not explicitly required in IR MD, although it can be reported in LI_Lineage. However, we explicitly require it for the MTS.
12 LU System Metadata - Orthoimagery For this campaign (2017), we’ll use orthos of 2016 flights as well orthos of 2017 flights. But these last images (2017) are not yet published. Should we describe the 2017 ortho metadata if they are not yet available for the LPIS ?

Indeed, do you need metadata of data available at the data extraction date for ETS, or matadata of all data used for the 2017 campaign, including orthophotos that are not yet available ?

Orthophoto metadata should be reported in the system metadata file of the MTS if they have been implemented in the LPIS (for the upkeep). If you used this ortho data set to update any reference parcel (within the LPIS population as submitted on the LPISQA Portal), than please report the metadata of the orthos, if you still didn't use them, than wait until they are used and than report.


Questions on MTS raised after the LPISQA Workshop in Varese

Q MS Issue / reference MS question / remark JRC Reply
13 LT MTS log, SystemMetadata, Vector ThematicMetadata How should we interpret a whole dataset and its temporal extent? Do we have to consider the update date of the oldest feature in the dataset as the begining? Or if there are unupdated features in the dataset, what date should be considered: creation or revision?

We are also confused about the definition the end position. If we understand correctly as it was responded to Slovenian question in Q&A section of MTS TG, we should specify the same value for the end postion as for the begining if the majority of features are still valid?

Practice showed that the vector dataset can be updated in a single revision project, or is being updated dynamically at parcel (polygon) level each time an anomaly triggers an update. If at least one parcel (polygon) is updated in a dataset on a specific date, that date (the last date in a dataset level) is the date of revision of the whole dataset.

It is also important to understand that the revision of each parcel on a specific date should be registered even if the parcel data remain unchanged. This date is evidence of the last verification by the Administration for particular parcel and these dates are than taken as the begin and the end position of the dataset. Begin date is the "oldest" while the end date is the "youngest" date registered for creation/revision of parcels (polygons) within one dataset.

14 PT SystemMetadata, Vector ThematicMetadata Do we have to upload MTS every year if we updated LPIS reference parcel layer that will have a new temporal extent (begin and end position)? According to the structure of the SystemMetadata.xml file, and with reference to INSPIRE, begin and end date of each product valid in the system should be reported. If any of data sets - vector and/or raster, changes due to the LPIS update procedure, and stay within the same specification, please use 'Update system metadata' edit function of the LPIS QA Portal. You can check this under MTS>Update system metadata page. Once you open the page, you will see the system metadata already reported for previous years. Edit function is editing time extent of selected record. Please follow this document on how to use this functionality.
15 Generic TG IXIT - Qualified C (Integrating eligible area from GAEC LANDSCAPE FEATURES) Could you please further clarrify the meaning of the options "inclusive" and "complementary"? The purpose of this IXIT qualifier is to provide information on the way the landscape features subject to retention (GAEC 7) are handled in the LPIS. Two modalities are possible: (1)Inclusive: their presence and area is accounted alphanumerically (as RP attribute) or spatially in dedicated spatial dataset, BUT they are not defined as reference parcels with their own RP ID; (2)Complementary: they represent individual reference parcels with their own RP ID.
16 Generic TG IXIT - Qualified A (Authoring the reference parcel PERIMETER) Could you please clarrify how to interpret correctly this clarifier in the context of the applicable RP typology? This IXIT qualifier deals with the “boundary” of the spatial object assigned to the unique reference parcel identifier. This is the property of the reference parcel most closely associated with the RP topology applicable in LPIS context, and which: (1) Identifies the actor whose role is indispensable. In most of the cases, this is the “initiator” of the RP update. (2) Defines the level of detail (granularity) of the unit of management. For example, the AP is a sub-unit of FB, while the CP can be sub-unit of BPU.


Go up to the main TG MTS page