All posts by Paul

GSTAR web services; now with added GeoJSON

Archaeogeomancy: Digital Heritage Specialists – archaeological geomatics – the majick of spatial data in archaeology – archaeological information systems for the digital age:

"A boundary is an array of an array of an array of arrays" by Paul Downey

“A boundary is an array of an array of an array of arrays” by Paul Downey

Following on from the last update concerning the GSTAR web services, the final pieces of infrastructure for the case studies and demonstrator are nearly complete. Building on the API, a GeoJSON output format has been added so that results from GeoSPARQL queries can a) be accessed via a simple URL as with all other outputs and b) visualised using a web map or indeed any platform which can consume GeoJSON. 

I had assumed this last element would be straightforward, after all, plotting results on a map is just one of those things one would expect from some geospatial resource. But a couple of hurdles presented themselves.

1. Well Known Text

Geospatial data modelled using the GeoSPARQL ontology and stored in a triple store such as Parliament typically makes use of either Well Known Text (WKT) or Geographic Markup Language (GML) to store geometries. Importantly, as per the GeoSPARQL standard there is the ability to include a URI for a Coordinate Reference System (CRS) within a geo:wktLiteral node:

Req 10: All RDFS Literals of type geo:wktLiteral shall consist of an optional URI identifying the coordinate reference system followed by Simple Features Well Known Text (WKT) describing a geometric value. Valid geo:wktLiterals are formed by concatenating a valid, absolute URI as defined in [RFC 2396], one or more spaces (Unicode U+0020 character) as a separator, and a WKT string as defined in Simple
Features [ISO 19125-1].

If using any of the standard outputs from Parliament via Jena using a SPARQL endpoint, the output includes the data as stored and (as was politely pointed out to me by a couple of eminent GeoJSON folks) embedding WKT within JSON structures is not really the done thing:

Consider me slapped. But the triple store is doing exactly as asked, outputting a JSON version of the query results, whatever form they may be in. So I needed an extra step to produce a proper GeoJSON output in which geometries are represented as JSON arrays rather than WKT.

This inclusion of a CRS URI element in a geo:wktLiteral node is problematic for systems expecting a plain old WKT string in which the first element is the geometry type. The overall node structure of a feature geometry when represented in RDF is as follows, comprising the three elements: CRS, WKT + Datatype.

"<> Point(33.95 -83.38)"^^<>

The final element here is the datatype, recorded using the standard ^^<datatype> notation, as generally used for typed literals; this is handled easily and becomes a datatype node in XML and JSON output formats. As such, this element is not a problem when appended to a WKT string.

{ "datatype": "" , "type": "typed-literal", "value" : "<> Point(33.95 -83.38)"}

More generally speaking, it has been noted by the research community (for example by a number of contributors to the Linked Geosptial Data event I attended) that this compound approach is unhelpful and somewhat at odds with the usual approach to semantic structures in which each assertion is represented as a discrete triple. But currently, this is what we have, so that’s that.

Anyway, the solution was actually rather low tech. One of the only Triple Stores to natively support GeoJSON output is Strabon, but the Triple Store selected for use in the GSTAR project is Parliament. So it was necessary to investigate suitable approaches using the platforms in use. A quick look at the Strabon source code (available online under the Mozilla Public License v2.0) revealed an elegantly simple solution. Taking the same example used above, it is clear that there is a pattern to the structure which, with some judicious use of basic Java string functions, can be used to clean up the string and extract the individual components.

Firstly, the EPSG code for the CRS can be extracted by first checking to see if the string begins with a < character, indicating there is a URI present. If so, the EPSG code will be the last element of the URI, appended to the base URI for the namespace. Secondly, the WKT serialisation itself will follow the closing > for the CRS URI tag. These marker elements used to get indices used for substrings are shown in green, the EPSG code shown in blue and the WKT geometry shown in red.

<> Point(33.95 -83.38)

So this gives a nice plain WKT string (the red bit) which can be processed using GeoTools with the added benefit of a Proj4 Coordinate Reference System (derived from the blue bit) which can be used to control any transformations. If there is no CRS element, the input string can be safely returned as is.

2. Generating GeoJSON output

Having resolved the WKT issue, the only remaining hurdle was to take the processed WKT geometries and output JSON coordinate arrays within a GeoJSON structure, as per the GeoJSON standard. This was accomplished using Jackson to manipulate the JSON data combined with geojson-jackson to fuse the JSON and GeoJSON elements. This made it relatively easy to take the Query Results Json Format data and transform it into GeoJSON.

This extra output format has now been added to the Sparqlr API so GeoJSON can be returned for the results of any query containing spatial data.


A final check was to ensure the GeoJSON data is well formed using a suitable viewer, in this case QGIS which has good support for a wide range of OGC standard formats including GeoJSON.

GeoJSON output from Parliament via the Sparqlr API, viewed in QGIS

GeoJSON output from Parliament via the Sparqlr API, viewed in QGIS

Next steps

The final pieces of the puzzle are the demonstrator interface and case study queries showing the capabilities of such resources. The former comprises a web based interface including some Seneschal widgets (for selecting controlled vocabulary items to be used in queries) combined with a web map for visualising results and making spatial selections. The latter comprises some interesting prebuilt GeoSPARQL queries based on real world archaeological research questions, visualised using the same web based UI.

Then just need to finish off that thesis…

The post GSTAR web services; now with added GeoJSON appeared first on Archaeogeomancy: Digital Heritage Specialists.

GSTAR at Salisbury Museum

Archaeogeomancy: Digital Heritage Specialists – archaeological geomatics – the majick of spatial data in archaeology – archaeological information systems for the digital age:

Beaker Pots by Wessex Archaeology

Beaker Pots by Wessex Archaeology

The final batch of source data has now safely received and is being processed for inclusion in my GSTAR project, kindly provided by the good people at Salisbury Museum. Thanks in particular are due to David Balston for assisting me and Adrian Green for giving the necessary permissions to use the data. This final batch of museum collection data will augment that already supplied by Wiltshire Museum to provide a more complete coverage across the study area. Importantly, this dataset includes much of the material from excavations undertaken by Wessex Archaeology who have also kindly made their archives available to me. Data was extracted in two ways; firstly using the Places node in the collections management system to look for parish names within the study area and secondly using the People/Organisations node to look for projects undertaken by Wessex Archaeology. This provided a good coverage of data for inclusion in GSTAR of around 8000 detailed records.

So the triumvirate of data is complete: data from archaeological fieldwork, data from museums collections and data from the Historic Environment Record. The next step is to use the tools already developed for museums collection data to process this source material and generate more Linked Geospatial Data for analysis and to support the technology demonstrators being constructed. Thankfully, Salisbury Museum, like Wiltshire Museum, use Modes for their collections management so the data provided as an xml export uses the same structure; as such, the xlst transforms and Stellar templates already built can be applied to this new data also without having to develop another processing pipeline.

Amesbury Archer - gold hair ornaments by Wessex Archaeology

Amesbury Archer – gold hair ornaments by Wessex Archaeology

It was also very nice to see the impressive new Wessex Gallery at the museum whilst visiting. Impossible to resist having a quick look around and I will be taking my girls there as soon as possible to see the incredible range of artefacts and have a go with the hands on archaeology activities. Of particular interest for me was the Amesbury Archer display, a chap excavated by Wessex Archaeology up at Boscombe Down and someone who I have been privileged to see close up whilst he was being looked after by WA post excavation and prior to being deposited with the museum.

The post GSTAR at Salisbury Museum appeared first on Archaeogeomancy: Digital Heritage Specialists.

Day of Archaeology 2015

Archaeogeomancy: Digital Heritage Specialists – archaeological geomatics – the majick of spatial data in archaeology – archaeological information systems for the digital age:

Dr. Henry Walton "Indiana" Jones, Jr.

Dr. Henry Walton “Indiana” Jones, Jr.

Yes, it’s that time of year again: Time for the annual Day of Archaeology. And once again, my day does not involve any temples in remote jungles, crystal skulls or raiding any tombs. Indeed, as has become the norm, it does not even involve any digging of holes, artefacts or suchlike.

Yep, archaeology involves a much broader range of activities than many folk believe, many of which are lab based and/or computer based with the result that some archaeologists (myself included) rarely get to see daylight let alone travel to distant far off lands in search of ancient peoples. And this is one reason why I love the Day of Archaeology so much as the range of posts each year covers just about every aspect of archaeology and cultural heritage and goes a long way towards showing what we, as professional archaeologists, really get up to, shattering stereotypes perpetuated by the likes of Indiana Jones and Lara Croft.

Anyway, here’s my post for this year which focusses on the usual range of geospatial and geosemantic stuff and not being chased by angry tribespeople or making dramatic and implausible escapes from imminent danger and almost certain death (although I did get a small electric shock off a laptop power supply this morning…)

The post Day of Archaeology 2015 appeared first on Archaeogeomancy: Digital Heritage Specialists.