LPISQA 3.b Tools

From Wikicap - European Commission
version 5.3

JRC Tools

Web-application for data exchange

A secure web-application is available at: LPIS QA Web-application to facilitate and assist Member States in the LPIS QA implementation. This platform is used for:

  • upload of LPIS GML data (MS-to-JRC),
  • download of Sample Pre-selection XML data (JRC-to-MS).

To correctly use it, please refer to the sampling pre-selection procedure at: Instructions on data exchange.

JRC custom built tools

JRC offers these tools as demonstration tools only. They help to understand the GML creation process and helps to create a valid GML file from the original Member State data. The tools are in a draft version, therefore some minor problems can occure (if so, please contact the JRC team with a problem description and a bug report as a print-screen).

When an updated version of a tool is available, the corresponding link will be updated. Please, verify if you are using the latest version.

JRC Tool Description Requirements Link Date
LPIS Point Zero State A customized script that creates a valid LPIS Point Zero State GML file from a point-type shapefile. ArcMap v9.3: Tested on ArcMap version 9.3 service pack 1 LpisPointZeroStateTool - Installation 2011‑07‑15
LPIS Point Zero State A customized script that creates a valid LPIS Point Zero State GML file from a point-type shapefile. Version compatible with the gml schema v5.3 ArcMap v10.x: Tested on ArcMap version v10.0 and v10.1 LpisPointZeroStateTool - Follow the instruction of the readme.txt 2013-02-01
LPIS Polygon Zero State A customized script that creates a valid LPIS Polygon Zero State GML file from a polygon-type shapefile. BETA version, compatible with the gml schema v5.1 ArcMap v9.3: Tested on ArcMap version 9.3 service pack 1 (also ArcGis v10; SP2) Tools_v51 - Installation 2012-03-23
LPIS Polygon Zero State A customized script that creates a valid LPIS Polygon Zero State GML file from a polygon-type shapefile.Version compatible with the gml schema v5.3 ArcMap v10.x: Tested on ArcMap version 10.0 and 10.1 LpisPolygonZeroStateTool - Follow the instruction of the readme.txt 2013-01-21
ETS Inspection Measurements A customized script that creates a valid ETS Inspection Measurement GML file from a set of corresponding layers/shapefiles. BETA version, compatible with the gml schema v5.1 ArcMap v9.3: Tested on ArcMap version 9.3 service pack 1 (also ArcGis v10; SP2) Tools_v51 - Installation 2012-03-23
XML (GML) Validator An application that validates XML and GML files against their schemas. Microsoft.NET Framework 3.5 or later link JrcXmlValidator 2010-12-07

Third party commercial tools

These tools are on the commercial market and may help Member States during the implementation of the LPIS QA inspection or further analyses thereafter.

These are commercial products and the entries are provided for information only. JRC does not "certify" or "guarantee" any of these third party tools.


Tool Description Requirements More...
GDV ETS-reporter Java-based stand-alone software application that covers the process of the LPIS data quality measures (Executable Test Suite). Standalone, needs Java 1.6 installed Deutsch - English
Sinergise TopoCheck tool for spatial and meta-data validation of various datasets. It analyses the data and finds inconsistent records, problematic topologies and it also estimates an area uncertainty of each polygon. Standalone, needs Java 1.6 installed. link
Abaco QA ETS Exchange web application providing the import/export of the Commission selected sample according to ETS guidelines Any J2EE web container, Oracle Spatial 10g or 11g link
Abaco QA ETS Inspection web application managing the Quality Control workload distribution among inspectors. Provides also a tailored ETS GIS editor and the ETS scoreboard in PDF format Any J2EE web container, Oracle Spatial 10g or 11g. link
Wageningen UR - Alterra ETS Manager The ETS Manager is build as an addin for ArcGIS 9.3.1. The current version it taylored to the Dutch and Northern-Irish workflow, but can easily be adjusted according to your specific situation. Multi-user tool for the entire process of LPIS Quality Assessment. Based on file-geodatabase usage. For more detailed information please contact Inez.Woltjer@wur.nl ArcGIS 9.3.1. link

Third party free tools

Ogr2ogr

IMPORTANT: The following conversion does NOT give 100% valid GML file, some small changes are still required to tune the file: "ogr" namespace to "cap" namespace, together with "targetNamespace".

Now we are able to convert the newly created shapefile to a GML file. A tool that could be used for this purpose is ogr2ogr, from Geospatial Data Abstraction Library. Basically, one provides ogr2ogr with the input shapefile, specifies the additional fields (other than spatial ones) that are contained in the GML file, and provides the name of the GML output file:

ogr2ogr -f "GML" -a_srs "EPSG:31300" –nln “ReferenceParcel” CountryRefParcelPoints.gml CountryShapefilePoints.shp –sql “SELECT CAST(ReferenceParcelID AS character)AS rpID, CAST(ReferenceParcelArea AS float) AS referenceArea FROM CountryShapefilePoints”

The GML file shall be then zipped and shiped through the LPIS QA web-application to JRC.

FAO LCCS

Software installation setup

The software is freely distributed by FAO and comes with a self-extracting executable file, which produces the entire set of files necessary to run the setup.

The latest stable release of the LCCS (version 2), currently used in the LPIS QA, can be found here

Classification concepts and user manual

The LCCS software manual provides information on the classification concepts and the practical software use. The first part of the manual fully describes the LCCS used definitions and the conceptual basis. The second part of the manual deals with the LCCS operative use, from installation to extensive explanation of the functioning mode of each one of the program modules.

It is available on the FAO Web site: LCCS Manual.

XML Marker 1.1

XML Marker is a freeware XML Editor that uses a synchronized table-tree-and-text display to show you both the hierarchical and the tabular nature of your XML data.

It automatically produces a tabular display of any selected tag by collecting repeating attribute and tag names and then arranging them into columns. The result is a clutter-free and informative tabular display.

The tool is here available for downloading.

LPIS-QA-Reporter

In Belgium-Flanders we created a little tool to help creating some xml files for the ETS. Two of the necessary “ETS Reporting package” xml files are supported at the moment, because they are the most difficult to create manually:

  • EtsObservations.xml
  • LpisSamplePreselectionStatus.xml

The tool needs a shape file with certain mandatory columns as input (a template shape file and a description of the needed fields is included in the download). Based on this file xml files are generated... Mind: the tool doesn't do an xsd validation, so you still need to use eg. the JrcXmlValidator to validate the xmls!

BTW: This tool is provided free of charge, as is without any guarantees or warranty. The author is not responsible for any damage or losses of any kind caused by the use or misuse of the programs. The author is under no obligation to provide support, service, corrections, or upgrades to the software ;-)...

If you would like to use the tool and want to be notified of new versions or have any remarks, you can contact me here: EmailAddress PieterRoggemans.GIF...

You can download the newest version of the tool on this page: LPIS-QA-Reporter


Tips and tricks

How to install a JRC ArcGIS script

LPIS Point Zero State and LPIS Polygon Zero State tools consist of several files under a ZIP archive. They run within the ArcMap environment (built for the ArcMap version 9.3). It is a prototype that has not been largely tested yet.

To install the tools in your ArcMap component, you need to:

  1. Download the ZIP files from WikiCAP - see above links.
  2. Open ArcMap.
  3. Go to Tools>Macros>Visual Basic Editor.
  4. Delete all the files installed for the previous version of the JRC tools.
  5. Import all the files from the new installation (.cls, .bas and .frm) within Normal Project - Normal.mxt - to make it always available to ArcMap.
  6. Make sure the the following reference called "Microsoft Scripting Runtime" is checked: Visual Basic>Tools>References>Microsoft Scripting Runtime
  7. Save the project.
  8. Go to ArcMap, Tools>Customize>[Macros]>...find our Tool (indicated as Normal.ToolName.Run).
  9. Drag&drop it on your ArcMap toolbar.
  10. Click on the button you have just dropped into the toolbar or alternatively, run the tool from Tools>Macros>Macros>[Macro name]>Run.
  11. The tool's window should be opened and ready to be used.

How to run a JRC ArcGIS script (Point/Polygon Zero State)

  1. Install a tool correctly.
  2. Click on it to open its interface.
  3. Select the shapefile/layer you want to convert.
  4. Map your corresponding attribute fields.
  5. Convert the file by clicking on the Create GML File button.

The input files should be:

  • for Point Zero State and Polygon Zero State a simple-point-type or simple-polygon-type shapefile with a defined geographic coordinate system or a projected coordinate system.

The output file is a GML-file, compliant with one of the following schemas:

How to run the ETS Inspection Measurements JRC ArcGIS script

  1. Install the tool as described above.
  2. Prepare your ArcGIS project with layers/shapefiles where each layer/shapefile contains only Agriculture Land Cover Features or only Landscape Features of one geometry type (Figure 1).
  3. Prepare the required attributes in all your layers/shapefiles as indicated in Table 1.
  4. Click on the installed EtsInspectionMeasurement tool to open its interface.
  5. Click on the "Build the mapping" button.
  6. (Required) Select the layer/shapefile containing Agriculture Land Cover Features (polygons)
  7. (Required) Map the attributes from your layer/shapefile to the GML corresponding attributes
  8. Select the layer/shapefile containing Landscape Features (polygons). If you do not have landscape features mapped as polygons, leave it empty.
  9. Map required attributes
  10. Select the layer/shapefile containing Landscape Features (lines). If you do not have landscape features mapped as lines, leave it empty.
  11. Map required attributes
  12. Select the layer/shapefile containing Landscape Features (points). If you do not have landscape features mapped as points, leave it empty.
  13. Map required attributes
  14. Select the layer/shapefile containing Landscape Features (multipoints). If you do not have landscape features mapped as multipoints, leave it empty.
  15. Map required attributes
  16. Convert your data by clicking on the Create GML File button.
  17. Validate your GML file with JRC XML Validator

WikiCAP EtsInspectionMeasurements.png

Figure 1. Prepared ArcGIS project for ETS Inspection Measurements conversion. Each layer/shapefile contains only one feature type of one geometry.


Table 1. Mandatory GML attributes

Required GML attribute Description
rpID Reference Parcel Unique Identifier
landCoverFeatureID an internal unique identifier of mapped land cover types (i.e. OBJECTID)
agricultureLandCoverClassCode Land Cover Class Code from your eligibility profile
codeSpace urn:ec:lpisqa:REPORTING_YEAR:YOUR_LPIS_CODE:LOT_NUMBER:EligibilityProfile where you need to replace the bold elements with your actual data, i.e. urn:ec:lpisqa:2010:TEST-LPIS:1:EligibilityProfile


The input files should be:

  • separate ArcGIS layers or shapefiles containing only Agriculture Land Cover Features or only Landscape Features. Please note that each of the layers/shapefile may only contain features of the same geometry type (polygon/multipoligon, line/polyline, point/multipoint) and should have a defined geographic coordinate system or a projected coordinate system.

The output file is a GML-file, compliant with one of the following schema:

How to create the point representations of the parcels in ArcGIS

In order to determine the points, a command line function from ArcGIS (under the ArcInfo license) could be used: FeatureToPoint d:\workspace.mdb\parcels d:\workspace.mdb\parcels_pt INSIDE The syntax for the command is as follows: FeatureToPoint <in_features> <out_feature_class> {CENTROID | INSIDE}

Using the function (choosing INSIDE option), a new shapefile will be created. The only difference is that it will contain some point representation of the parcels instead of polygons, for each of the parcels in the original shapefile.

How to open a XML Sample pre-selection in ArcGIS

If you want to relate an XML sample pre-selection file to your ArcGIS project, you need to first open the XML file in the Excell, and then save it in the DBF format. Then, you will be able to relate it to your ArcGIS layers.

How does a correct GML look like?

Download an example: LpisPointZeroState.gml The GML file content is illustrated below:

<?xml version="1.0" encoding="utf-8" ?> 
<cap:FeatureCollection 
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
 xsi:schemaLocation="http://ec.europa.eu/dgagri/cap ftp://mars.jrc.ec.europa.eu/LPIS/Schemas/4_3_LpisPointZeroState_20101220.xsd"
 xmlns:cap="http://ec.europa.eu/dgagri/cap" xmlns:gml="http://www.opengis.net/gml"
 lpis_code="TEST-LPIS"
 lpis_lot="1"
 reporting_year="2010">
 <gml:boundedBy>
  <gml:Box srsName="EPSG:4326">
   <gml:coord>
    <gml:X>10.131254635</gml:X> 
    <gml:Y>34.055141255</gml:Y> 
   </gml:coord>
   <gml:coord>
    <gml:X>14.144205386</gml:X> 
    <gml:Y>44.831708765</gml:Y> 
   </gml:coord>
  </gml:Box>
 </gml:boundedBy>
 <gml:featureMember>
  <cap:ReferenceParcel fid="F0">
   <cap:geometryProperty>
    <gml:Point srsName="EPSG:4326">
     <gml:coordinates>12.970463244,44.292817075000002</gml:coordinates> 
    </gml:Point>
   </cap:geometryProperty>
   <cap:rpID>GH832_A.0305.14/7</cap:rpID> 
   <cap:referenceArea>54.231</cap:referenceArea> 
  </cap:ReferenceParcel>
 </gml:featureMember>
 <gml:featureMember>
  <cap:ReferenceParcel fid="F1">
   <cap:geometryProperty>
    <gml:Point srsName="EPSG:4326">
     <gml:coordinates>13.016643059,44.277870450000002</gml:coordinates> 
    </gml:Point>
   </cap:geometryProperty>
   <cap:rpID>FD832_A.0305.14/3</cap:rpID> 
   <cap:referenceArea>2.9903</cap:referenceArea> 
  </cap:ReferenceParcel>
 </gml:featureMember>
</cap:FeatureCollection>

Figure 2. Example of an LPIS point zero state file.

What are the correct GML attributes?

The INSPIRE Directive, imposes the GML format for the exchange of geospatial data. The following GML Application Schema is defined for LPIS point data LpisPointZeroState.xsd. It must be referenced inside the GML:


 xsi:schemaLocation="http://ec.europa.eu/dgagri/cap ftp://mars.jrc.ec.europa.eu/LPIS/Schemas/4_3_LpisPointZeroState_20101220.xsd"
 xmlns:cap="http://ec.europa.eu/dgagri/cap"

The GML file created by each LPIS custodian should contain, sequentially the following elements:


<gml:featureMember>
 <cap:ReferenceParcel fid="F0">
  <cap:geometryProperty>
   <gml:Point srsName="EPSG:4326">
    <gml:coordinates>12.970463244,44.292817075000002</gml:coordinates> 
   </gml:Point>
  </cap:geometryProperty>
  <cap:rpID>GH832_A.0305.14/7</cap:rpID> 
  <cap:referenceArea>54.231</cap:referenceArea> 
 </cap:ReferenceParcel>
</gml:featureMember>

Where:

  • fid: required by good GML practice. If it is provided, it must be a string that starts with either a letter or the underscore (_) character, followed by printable characters or numbers. fid attribute values must also be unique among all elements in the document.
  • geometryProperty: a point representation of the reference parcel, giving X and Y coordinates (points shall be INSIDE reference parcels)
  • srsName: information on the coordinate reference system, given such as an EPSG code, i.e. EPSG 4326
  • rpID: the unique identification of the reference parcel,
  • referenceArea: the “maximum eligible” area of the reference parcel, given in hectares.

Member States shall provide their reference parcels with the above information in the GML format, after they have performed a standard XML validation process (well-formedness and validity).

Clarifications and explanations of the XML elements in the OrthoimagerySet.xml and OrthoimageryUrl.xml

OrthoimagerySet.xsd

  • zoneID

This field provides the Name of the LPIS control zone (either part of the CwRS or proprietary), as

  • defined by the MS administration in the ApplicableCidZones.xml and ApplicableProprietaryZones.gml (relevant for the year of assessment) and
  • listed in the LIO image acquisition system of CID, JRC (for image acquired and provided in the scope of the annual CwRS program)


NOTE 1: The naming convention of the zones imposes a 4 or 5 character abbreviation.

NOTE 2: For orthomagery provided through WMS by the MS Administration, each LPIS control zone ID should have its correspondent individual entry (layerName) in the OrthoimageryUrl.xml.

  • imageAcquisitionDate

This entry holds the acquisition date of the source imagery. In case of a LPIS QA control zone covered with multiple acquisitions, the following rules apply:

  • VHR spaceborne data: each acquisition is reported separately; a zone name can occur more than once in the XML, if it is associated with several acquisition dates.
  • airborne data: each acquisition is reported separately if the acquisition covering the zone is not spread over more than 5 calendar days. This means that a zone name (providing it is associated with several acquisition dates) can occur up to 5 times in the XML. When there are more than 5 acquisition dates for the zone, the “median” date of the temporal acquisition range is reported for a single XML entry. Information on the multiple acquisition dates should be further provided in non-structured evidence upload.

NOTE 1: The format of the “Acquisition date” field is YYYY-MM-DD. Only orthoimagery acquired in the year of LPIS QA assessment should be given in the OrthoimagerySet.xml.

  • platform

This entry provides the Type (name) of the platform on which the sensor is mounted. For all analogue and digital cameras mounted on an aerial vehicle, the platform is “AERIAL”. No distinction between the camera type is made. For all sensors mounted on a satellite, the name of the sensor should be provided, as follows:

  • "IKONOS": image acquired by IKONOS
  • "GEOEYE1": image acquired by GeoEye-1
  • "QUICKBIRD": image acquired by Quickbird
  • "WORLDVIEW2": image acquired by Worldview-2
  • pixelsizeM

This entry provides the Size of the orthoimage pixel given in meters (rounded to one cm).

NOTE: For all digital sensors, the ratio of the orthoimage pixel size to the ground sampling distance (GSD) should be 1:1, for film cameras it should be at least 1.2:1. For further information see Orthoguidelines_v3

  • imageType

This is a complex XML entry providing information of the type of orthoimage product obtained. It contains (is restricted to) the following values:

  • "PSH": for VHR satellite orthoimage; pan-sharpened; 3 channels only; False Colour Composite
  • "PSH-RGB": for VHR satellite orthoimage; pan-sharpened; 3 channels only; Natural Colour Composite
  • "PSH-FCC-RGB": for VHR satellite orthoimage; pan-sharpened; full spectral range
  • "PAN-MUL-FCC": for VHR satellite orthoimage; bundle (pan+ms); 3 channels only; False Colour Composite
  • "PAN-MUL-RGB": for VHR satellite orthoimage; bundle (pan+ms); 3 channels only; Natural Colour Composite
  • "PAN-MUL-FCC-RGB": for VHR satellite orthoimage; bundle (pan+ms); full spectral range
  • "AERIAL": aerial orthoimagery

OrthoimageryUrl.xsd

  • imageryUploadedToCid

A "TRUE" entry acknowledges that the orthoimagery has been delivered to CID action of the JRC.

For aerial and other imagery that was independently acquired by the Member State for use in the LPIS QA, the field for imageryUploadedToCid should be set to “FALSE”, indicating that orthoimagery was provided by either WMS (preferred), or by ftp Upload (see instructions in Non-CwRS imagery upload)

  • wmsAccessInformation

This complex XML element provides information on the WMS access details, as follows:

  • url: holds the URI of the WMS. Please leave this field blank, if no WMS is provided.
  • accessCredentials: username and password for accessing the WMS. Please leave these fields blank if there are no credentials required to access the WMS (no username or password required).
  • layerName: Name of the layer in the WMS containing the orthoimage set. Each LPIS QA zone should correspond to a separate layer. This means that there can be more than one layer name entry associated with the “wmsAccessInformation” field. The layer name should match the entry from the “zoneID” field of the OrthoimagerySet.xml.
  • ogcFilter: optional field, leave blank if not appliccable. For further information please refer to http://www.opengeospatial.org/standards/filter


NOTE: By default, all VHR orthoimagery produced from CwRS and LPIS QA images provided by the Commission Services (JRC) should be delivered to CID. We encourage MS to also provide the EC with WMS access for the screening operator, guaranteeing equal environments for inspection and screening.



Go forward to LPISQA portal.

Go backwards to table of contents.