With all the source data prepped and ready to go, the next step is to build some demonstrators to show how such geosemantic resources can be used in practice. Whilst very powerful, a Sparql endpoint is not the most friendly way of interacting with data resources, especially from within a web based application where options for programming are a bit limited. There is still quite some debate on this topic which will be covered in more detail in the thesis (watch this space; still on track for submission 1st/2nd quarter 2016!) but the approach I have opted for is an API using web services to provide a range of outputs via a combination of URLs and parameters.
API vs Endpoint
SPARQLr – the GSTAR web service
Sparklers by Derek Key
The Sparqlr web service which implements the API is a RESTful service and is being built using Jersey which is a great platform for such tasks. This talks to the Parliament triple store via Jena with some Geotools components added to handle spatial data. As the service runs as a Java application on a GlassFish web server, it is possible to make use of the full range of Java tools out there without being limited to what can be achieved within a web browser. And thankfully, much of the code produced previously for the GSTAR Pilot is being recycled! As usual, all development is being undertaken using Eclipse+Maven.
Various queries can be performed in this way, some using basic URL syntax eg http://gstar:8080/sparqlr/api/features to return records about excavated features from archaeological archives as an RDF graph (default) or http://gstar:8080/sparqlr/api/artefacts/ntriple to return records about archaeological objects from museum collections as N-Triples. Parameters are also implemented which can be used to return particular records (eg http://gstar:8080/sparqlr/api/sites?MonUID=MWI11909 to return records about some pits near Stonehenge).
On the geosemantic front, parameters are also being used to pass in geometries as UTF-8 encoded WKT strings to facilitate spatial searches with the incoming WKT geometries used within the web service to add Geosparql filters to Sparql queries (eg http://gstar:8080/sparqlr/api/sites/within?polygon=POLYGON+%28%28569186.11565982+169502.18457639%2C+569186.02731245+169502.23116132%2C+569185.82348375+169502.25234642%2C+569185.70491406+169502.19113299%2C+569185.57672168+169502.04343594%2C+569185.54491719+169501.88381892%2C+569185.64107717+169501.66842781%2C+569185.82308162+169501.55315212%2C+569186.01894577+169501.56512577%2C+569186.19385893+169501.68723228%2C+569186.29291775+169501.91384472%2C+569186.25804717+169502.07848985%2C+569186.11565982+169502.18457639%29%29 to return all sites/monuments within a specified region such as a user generated polygon drawn on a web map).
The bulk of this is now up and running with the next step being to build the web application. This will involve the construction of a web map using OpenLayers to present results and facilitate user input (eg to capture polygonal search areas or points with buffer distances; the kind of spatial searches typically used within web based GIS).
One of the outputs from the Pilot Study was an approach to working with geospatial data within the broader framework provided by the CIDOC CRM ontology and the CRMEH archaeological extension. Whilst there is ongoing work by myself and others to add archaeological and spatio-temporal components directly to the CIDOC CRM, for the purposes of the GSTAR project, a lightweight approach has been developed and deployed to suit the needs of the project; CRMEH already adds archaeological excavation capabilities and the spatial extension presented here gives a range of geospatial capabilities, as provided by a mapping to GeoSPARQL.
For the purposes of the GSTAR project, there is a need to be able to incorporate into semantic resources rich geospatial data representing depictions of archaeological features, sites and monuments, also boundaries of activities and events plus locations where objects were discovered. Whilst the CRMEH was developed with spatial information at its core, this has not, to date, been formally expressed. This is now possible using GeoSPARQL.
During the early stages of the GSTAR project, related work became apparent, notably two extensions to the CIDOC CRM (of which CRMEH is itself an extension) pertaining to spatio-temporal information (CRMgeo) and archaeological excavation information (CRMarchaeo). These will ultimately offer greater research potential but the mapping presented here can be seen as an example of a simple, lightweight solution targeting the ‘low hanging fruit’ so often talked about with respect to ontologies and Linked Data; a mapping which meets the needs of the GSTAR project, retains compatibility with the CIDOC CRM and GeoSPARQL standards and provides core geospatial functionality for CRMEH albeit without the reasoning power (and associated complexity) of the two aforementioned extensions.
Application within GSTAR
The semantic resource will be used through the Case Studies planned for the GSTAR project to investigate the use of geosemantic tools for archaeological research. Two of these are focussing on the integration aspect, looking at what I have defined as ‘horizontal’ and ‘vertical’ integration using the spatial components of source data. Horizontal integration refers to linkages between inventories, ie from site finds inventories to museum object inventories to sites and monuments inventories. Vertical integration refers to linkages between primary and derived data, from fieldwork databases containing records of features and finds up to inventories of higher order data objects containing records derived from these primary observations.
Related Work I – CRMgeo
Work is ongoing to produce a spatio-temporal model through integration of GeoSPARQL and CIDOC CRM: the CRMgeo extension, currently in draft form. This promised to be an incredibly powerful resource capable of advanced spatio-temporal description and reasoning.
The work is described as follows:
CRMgeo is an extension for the CIDOC CRM to provide an “articulation” (linkage) between the standards of the geospatial and the cultural heritage community in particular between GeoSPARQL and CIDOC CRM. The model was developed from the analysis of the epistemological processes of defining, using and determining places. This means that we analyzed how a question, such as “is this the place of the Varus Battle” or “is this the place where Lord Nelson died”, can be verified or falsified, including geometric specifications. Consequently, we reached at a detailed model which seems to give a complete account of all practical components necessary to verify such a question, in agreement with the laws of physics, the practice of geometric measurement and archaeological reasoning. This model indeed appears to have the capability to link both ontologies and shows the way how to correctly reconcile data at any scale and time – not by inventing precision or truth that cannot be acquired, but by quantifying or delimiting the inherent indeterminacies, as it is good practice in natural sciences. This model aims at being a comprehensive theory from which mutually compatible simplification can be derived for implementations in more constraint environment, such at those lacking moving frames.
Related Work II – CRMarchaeo
Similarly, work is ongoing to produce an archaeological excavation model: the CRMarchaeo extension, currently in draft form. This promises to support description of and reasoning about archaeological excavation information from a range of recording methodologies.
This project is described as follows:
CRMarchaeo is an ontology and RDF Schema to encode metadata about the archaeological excavation process.
The goal of this model is to provide the means to document excavations in such a way that the following functionality is supported:
Maximize interpretation capability after excavation or to continue excavation Reason of excavation (goals). What is the archaeological question?
Possibility of knowledge revision after excavation
Comparing previous excavations on same site (space)
All kinds of comprehensive statistical studies (“collective behavior”)
My contribution to CRMarchaeo is running in parallel to my work on the CRMEH. Whilst ultimately there will need to be some decisions as to which extension to use for new projects and resources, there is currently a fair amount of data out in the wild which uses CRMEH and at least until CRMarchaeo is finalised and probably longer, there will be some co-existence of these two complimentary models. After all, the two models are very much related and oversight has been maintained to ensure a good degree of compatibility between them.
A lightweight mapping
A decision was made to create a lightweight mapping of CRMEH to GeoSPARQL rather than implement a combination of CRMarchaeo and CRMgeo for three main reasons:
Firstly, these extensions are centred on the core CIDOC CRM ontology rather than the CRMEH extension. As CRMEH is being used for the GSTAR project, their use would have required a mapping process anyway to ensure compliance.
Secondly, both of these ‘emerging’ standards are currently in draft form, in the process of being finalised and formally adopted. As such, they are not fixed yet and subject to review, improvement and change; Some components in particular still require more work to completion.
Finally, the degree to which the advanced features offered by these extensions could be made use of through the GSTAR project is uncertain. A lightweight mapping can be seen as an 80% or 90% solution, covering most eventualities and avoiding the overheads associated with the rather more complex extensions. But retaining overall compatibility.
The key spatial components needed are already present in CRMEH. There are two main components covering excavation data: the Context (aka Stratigraphic Unit; the atomic unit of archaeological recording) and the ContextDepiction (a depiction of the Stratigraphic Unit, typically a polygon shown in plan view). A Context is related to a ContextDepiction through the property Depicts / Is Depicted By with a Context being depicted by one or more depictions.
These extend from the core CIDOC CRM: the CRMEH class Context (EH0007) is a subclass of Place (E57) whilst ContextDepiction (EH0022) is a subclass of Place Appellation (E44). In GeoSPARQL, there are also two classes to describe spatial information with Features having some representation in the form of Geometry. There is a good alignment here between the CRMEH classes (or indeed the parent CIDOC CRM classes) and the GeoSPARQL classes, allowing the ontologies to be linked as described in the GeoSPARQL User Guide written by Dave Kolas and Robert Battle.
This is illustrated in the following diagram:
Alignment of CRMEH classes and properties with GeoSPARQL classes and properties
As shown in the diagram, the rdfs:subClassOf, rdfs:subPropertyOf and rdfs:isA relationships can be used to link the two ontologies. This maps the necessary classes and also allows instances of Context Depictions to behave as Simple Features as used within GeoSPARQL.
From mapping to RDF
This mapping allows Contexts to be depicted by one or more pieces of geometry, each instance of a ContextDepiction making use of an OGC Simple Features type (Point, Line, Polygon, etc) and represented using one of the standard formats, in this case WKT.
The mapping can also be applied at the broader CIDOC CRM level and inherited by the CRM EH (and other) classes and properties if this is advantageous.
With respect to data, resources can be created very simply by adding the class inheritance relationships once to a given resource then creating appropriate assertions relating to the ContextDepiction. This means in practice, GIS data can be converted very easily using a variety of tools (eg StringTemplate, Java(script), Python, VB or even Microsoft Excel) to produce suitable RDF of whatever flavour (ntriples, turtle, etc) ready for ingestion into a triple store.
An example of this for a single ContextDepiction is shown below:
In the example (above), namespaces are shown in blue, subproperty/subclass relationships to be defined once in greenand the block of RDF to be used for each ContextDepiction in red. NB The GSTAR namespace houses a WISSKI installation, currently not configured and acting as a placeholder only; the URIs do not resolve.
A working system
The system used for the Pilot Study took GIS data from the ADS archives, processed it as above and loaded it alongside the CRMEH RDF encoding and Erlangen CIDOC CRM RDF encoding. This was then successfully tested using a range of OGC spatial operators, SPARQL and GeoSPARQL queries.
For a fuller account of this, please see the Transfer Report when that is published more widely. Or wait a while longer for the thesis (due 2016).
In summary, whilst emerging standards building on the CIDOC CRM covering geospatial and archaeological excavation are forthcoming, a simple, lightweight approach can also be deployed for this use case to give a good range of functionality without the complexity, albeit sacrificing some semantic richness.
The mapping described here could also be applied directly to the parent CIDOC CRM classes/properties (rather than the child CRMEH classes/properties used here) to give a more generic linkage to GeoSPARQL suitable for use in a broader range of cases.
After a longer than anticipated gestation, my Transfer Report has left my hands and is working its way through the administrative system to be externally examined. Fingers crossed, this is one of my last posts as an MPhil student and I will soon (post viva) be a PhD student proper.
Time for some celebratory fireworks!
The Transfer Report included a condensed form of the literature review and also a detailed report on Pilot Study. This Pilot Study was designed to lay sound foundations for the PhD research and involved implementing a system using geosemantic technologies, primarily to investigate ways in which semantic and geospatial data can work together but also to help me get to grips with the subject area and technologies available.
The full report will be made available in due course, once it has been examined (viva scheduled for end of November) and any corrections completed, but for now here is an update on some of the key findings of the Pilot Study and conclusions drawn.
Conclusion one: Oracle is really complicated
Sean D. Tucker and the Oracle Challenger by Nathan Rupert
I started off with the idea that using Oracle for the research would be a really good idea. It is available for use under license for research purposes (OTN Developer License) and is the don of the database world. Furthermore, it does everything I required, all in one platform; no need to string bits of open source software together I thought with their undocumented ‘features’ and sparse documentation. After all, Oracle is a commercial system, an enterprise level system which supports all the relevant standards for geospatial, semantic and geosemantic data. It is capable of functioning as a triple store, a geospatial database and integrates with the Jena framework by means of a dedicated connector. The latest version (12c) has also been significantly redesigned and improved with respect to the Spatial and Graph components.
This is true, but being an enterprise level application, it also comes with considerable baggage. Notably, it is really, really complicated and much of this complexity is totally unnecessary for the likes of me undertaking a research project.
Now I don’t want to be unnecessarily critical of the platform but there are some real issues with using it for a research project such as GSTAR. Installation and configuration for starters is necessarily complex as it supports some seriously powerful tools such as security, distributed/pluggable databases, user/group roles and permissions not to mention Extended Data Types (essential for handling big data such as WKT geometries) and Indexing thereof. For a research project, components such as the enterprise level security are quite simply a hindrance rather than a help not to mention indexing. More critically, I found working with the Jena Connector and GeoSPARQL to be fraught with the (copious) documentation for the new version being a bit lacking; forums and blogs were of enormous help in fixing problems where the documentation wasn’t quite as helpful as it might have been for working with this latest version. No doubt this will bed down given time but being at the bleeding edge of such technology was not an ideal place to be.
Given I’m no longer using the Spatial and Graph components, the use of Oracle as the spatial database is no longer useful. Indeed, I won’t be using a spatial database as such with all data being prepared as Linked Geospatial Data within the triple store.
So, it’s been an experience but goodbye Oracle. Thanks for all the fish.
Conclusion two: Open Source software can be really good
‘If you want a culture of collaboration, you need to accept the LOLCats too’ by opensource.com
Still smarting from my Oracle experiences and quite a long way down the road with less than I had hoped to show for my troubles, I returned to my initial review of triple stores, looking for a suitable alternative. My requirements are quite specific: The platform needs to support big data, be responsive, support inferencing/reasoning and, crucially, provide good support for GeoSPARQL. I recalled various papers from my literature review extolling the virtues of Parliament, other folk having used it on similar research projects. It also has a thoroughbred pedigree, originating from research initially undertaken by DARPA through the DARPA Agent Markup Language (DAML) Program and is now used as the base for applications in a range of tough, testing environments by Raytheon BBN Technologies. So an impressive pedigree.
My concern regarding documentation, having worked with various Free and Open Source Software (FOSS) platforms over the years, was still niggling, but it had to be worth some testing. After all, quantity does not necessarily equate with quality, as the Oracle experience demonstrated. And there certainly isn’t quantity: the manual (a single rather short document) is smaller than the document for the Oracle Extended Data Types functionality! The key difference is Parliament is a one trick pony, and it does that trick very well; It does not try and be all things to all users. Installation and configuration was simple as pie, with the user guide providing all the key information without excess baggage. True, some of the latter sections of the user guide are yet to be written (almost a prerequisite for a FOSS application, a bit like for web 2.0 apps where permanent beta status is a badge of honour) but these focused on highly specific aspects of deployment irrelevant to my research.
So within a day, I had gone from review of my systems review to a working system.
Conclusion three: geosemantic applications using GeoSPARQL can really fly!
‘LOL Potential – Now LOL at Warp speed’ by Jen
One aspect to the Pilot Study was an investigation into different ways of integrating semantic and geospatial data. Without going into too much detail (I’ll post a version of the Transfer Report once it’s been examined, I’ve had my viva and everything is finalised), I had a suspicion that working with geospatial data using semantic tools and verbose, text based formats such as GML and WKT would be lacking in the performance department. Especially given some of the criticisms levelled at the performance of some SPARQL implementations and compared with the highly tuned geospatial tools found in GIS and dedicated geospatial platforms. So I wrote a Java application to test this hypothesis, comparing a ‘hybrid’ SPARQL+WFS system with a pure geosemantic system based around GeoSPARQL. The results of this showed very little difference in performance between the two approaches, potentially as any benefits of the optimised geospatial components appeared to be outweighed by overheads associated with additional middleware to process geosemantic queries for the GIS and then handle the WFS outputs to produce RDF. Given this lack of any significant benefits combined with the need for more complex systems architecture, I have opted for a ‘pure’ geosemantic basis for my next stages, based around Parliament, Jena, Joseki/Fuseki and GeoSPARQL, cutting out the need for any RDBMS, GIS and associated web servers.
The Grinder by Kalle Gustafsson
So, a big chunk of my research project is now complete and all being well, I should have my transfer from MPhil to PhD all signed and sealed in the near future. The Pilot Study has provided the groundwork for the next phases of work as detailed above and work on the next Case Studies is already well underway. I have the first tranche of data from Wiltshire Historic Environment Record in hand which is currently being processed to produce a geosemantic resource in Parliament; other data from archaeological units and museums is being sourced with the aim of completing this integration and preparation phase by Christmas.
The Case Studies will then look at the integration of these datasets using inferencing/reasoning on the spatial and other facets, moving from fieldwork data up to heritage asset inventories and across to museum collections, specifically how such linked resources can be used to undertake archaeological research based on current archaeological research questions and also including the use of RDF mapping libraries and query mediation using (spatial) ontologies.
I now have a draft chapter outline agreed for my thesis and already have tens of thousands of words to edit into it pertaining to the Literature Review, Pilot Study, introductory and methodology chapters. In other words, full steam ahead!
Last Friday was the Day of Archaeology and judging by the number and quality of posts, this year’s event looked to be one of if not the most successful yet. Massive congratulations and thanks to the organising team who do all the hard work, so much of it in their own time!
On Thursday 24th April, I gave a presentation on my PhD research project (GSTAR) to the 2014 Computer Applications and Quantitative Methods in Archaeology conference, Paris, France. The presentation formed part of the session S07 Ontologies and standards for improving interoperability of archaeological data: from models towards practical experiences in various contexts organised by Anne-Violaine Szabados, Katell Briatte, Maria Emilia Masci, and Christophe Tufféry. Reinhard Foertsch and Sebastian Rahtz chaired the session.
The abstract describes the talk, which covered work to date in the first year of the project:
Much work has been undertaken over the past decade relating to the application of semantic approaches to archaeological data resources, notably by English Heritage and the University of South Wales. These two organisations, over the course of a number of projects, developed an archaeological extension to the CIDOC CRM ontology through the Ontological Modelling Project (Cripps & May, 2010), then applied this to a number of archaeological resources through the subsequent STAR project (May, Binding and Tudhope, 2011), implementing tools to facilitate integration of other resources through the STELLAR project (May, Binding, Tudhope, & Jeffrey, 2012), and now, in partnership with the Bespoke HER User Group, RCAHMS, RCAHMW and Wessex Archaeology, are implementing SKOS based vocabularies and associated tools to enable the augmentation of these semantic resources through the SENESCHAL project.
From the outset, it was observed that the spatial component of archaeological data would be a key element, archaeological data being inherently spatial in nature. To date, most current applications of spatial semantics in the heritage sector have focussed on place names and named locations for sites and monuments and object provenances. The GSTAR project aims to extend semantic approaches to archaeological data fully into the geospatial domain and is instead focussing on the detailed spatial data emerging from archaeological excavation and survey work and is investigating approaches for the creation, use, management and dissemination of such spatial data within a geosemantic framework, building on the CIDOC CRM, with particular reference to sharing and integration of disparate resources.
This paper will present work to date in the first year of the GSTAR project. This has been centred on the identification of suitable platforms and methods for the integration of semantic and geospatial data including comparisons of different approaches emerging from the Semantic Web and Geospatial research communities. Testing and prototyping has been accomplished using sample data from the Archaeology Data Service, making use of available geospatial and (geo)semantic tools, both FOSS and commercial.
Cripps, P. and K. May 2010. To OO or not to OO? Revelations from Ontological Modelling of an Archaeological Information System, in: Nicolucci, F. and S. Hermon (eds.), Beyond the Artifact. Digital Interpretation of the Past. Proceedings of CAA2004, Prato 13–17 April 2004. Archaeolingua, Budapest, pp. 59-63.
May, K., C. Binding and D. Tudhope 2011. A STAR is Born: Some Emerging Semantic Technologies for Archaeological Resources, in: Jerem, E., F. Redő and V. Szeverényi (eds.), On the Road to Reconstructing the Past. Computer Applications and Quantitative Methods in Archaeology (CAA). Proceedings of the 36th International Conference. Budapest, April 2-6, 2008. Archeaeolingua, Budapest, pp. 111-116 (CD-ROM 402-408).
May, K., C. Binding, D. Tudhope and S. Jeffrey 2012. Semantic Technologies Enhancing Links and Linked Data for Archaeological Resources, in: Zhou, M., I. Romanowska, Z. Wu, P. Xu and P. Verhagen (eds.), Revive the Past. Computer Applications and Quantitative Methods in Archaeology (CAA). Proceedings of the 39th International Conference, Beijing, April 12-16.. Pallas Publications, Amsterdam, pp. 261-272.
The presentation is available on Slideshare:
The presentation also prompted some positive comments on Twitter, which was lovely:
Next it's @pauljcripps on 'Geosemantic Tools for Archaeological Research' (GeoSPARQL). Looking forward to this! #caa2014