TG MTS Analysis of results

From Wikicap - European Commission

Go up to the main TG MTS

The analysis of results shall be performed by evaluating the observed test outcome against the verdict criteria which are prescribed by the abstract test case. Although there is a clear conceptual distinction between the test campaign and the analysis phase, in the TG MTS context, the two may overlap in time. A test verdict is a statement of pass, fail or inconclusive. A (rare) verdict of inconclusive or failure needs a justification.

  • “pass verdict” means that the observed test outcome gives evidence of conformance to the conformance requirement on which the test purpose is focused, and is valid with respect to the relevant LCM element and with respect to the ICS (if provided).
  • a “fail verdict” means that the observed test outcome demonstrates non-conformance with respect to either a test purpose or at least one conformance requirement in the relevant element in LCM. In the LPIS context this means that the IUT doesn't fulfill a specific requirement from the LCM. This can be either a lack of required features type, or incompleteness of code list, or omission of required attribute.
  • an “inconclusive verdict” means that the observed test outcome produces neither a pass nor a fail verdict. This should occur only in very rare circumstances. EXAMPLE: “Test-case error”.

A justification shall be given with each fail or inconclusive verdict (in a separate document); informative messages or additional log files may also be provided.

The executable test case verdict shall be assigned to a particular test outcome using the verdict criteria relevant to that particular abstract test case. The test verdicts assigned shall then be synthesized into an overall summary for the IUT (Done later by DG JRC).

For example, a particular IUT has certain commitments to record in the LPIS landscape features subject to retention (under GAEC 7). An abstract test case could therefore be formulated as “featureTypeCompletness” test. The corresponding executable test case would be phrased as “checkLandscapeFeature”. If the LPIS (SUT) has implemented this requirement by creating a feature class “Landscape Feature” within its GIS/RDBMS then “Pass” verdict should be assigned and textual description should be provided. Example table (Table 4) is provided below. The MS LPIS would be conformant to the abstract test case “featureTypeCompletness” if the verdicts of all executable test cases within it are assigned as “Pass”.

Table 4 Analysis and documentation. Example of executable test case Nr. 1240 of Table 1.


Go up to the main TG MTS page