Klokan Petr Pridal, the creator of the wonderful Old Maps Online and MapTiler, has been using Unlock Places in some collaborative project work with the National Library of Scotland. He had some technical questions for us, and some questions about the intended usage future of the service, so I thought it worth-while republishing the answers here on the Unlock blog.
First, I like a lot the API… It is well documented, with examples. Easy to use. [Thanks!]
It is a bit confusing that you use “name” parameter instead of “q” (according the OpenSearch.org), but otherwise it is very nice. I was testing it with the Google Closure UI.AutoComplete, which is using JSONP and the callback function – it is similar to the jQuery module.
Right, our use of the “name” parameter for query is a legacy thing – it comes from Unlock’s predecessor, GeoCrossWalk. There’s been a lot of development in OpenSearch Geo since then, it would be worth our while to support it. However, I see OpenSearch as mainly for collections of geo-referenced things (datasets or documents) – not for the georeferences themselves – though of course it could be used to do both.
There’s also the quicklinks API which was a thought experiment. It looks a lot more like the new MapIt API, which we’re also thinking about implementing in front of Unlock.
It is great that you have bbox for the results and external link for the detailed footprint. The API gives anybody access to your combined geonames database with other source of data like OSM or OS. Geometry or at least bounding box is something I horribly miss at GeoNames API – and you have solved this problem!
Ordnance Survey have solved this problem for us with Open Data by releasing sources of shapes that can be used outwith academic publications! (We’ve always had this in the academic-use-only version of Unlock, formerly GeoCrossWalk). We’re now looking at adding OpenStreetmap data to derive the same kind of bounding box and optional detailed shape, for Europe rather than just mainland UK.
In this moment we are especially interested in usage of your Gazetteer via the “nameAndFeatureSearch” for “populated places” database. I am considering to link EDINA Unlock API from our Georeferencer.org service, instead of GeoNames.org API, which was planned originally. We can’t use Google Maps GeoCoding API because of the TOS. I expected that if we use your service and save the coordinates from GeoNames in our database, it is legal, same as if we would use
For geonames data, “This work is licensed under a Creative Commons Attribution 3.0 License“. We preserve the attribution in our search results. If you’re republishing the coordinates then you should ideally keep the source data and make the attribution too – that goes for all the different data source attributions we make.
BTW Georeferencer.org is going to be used also on the National Library of Scotland maps later this year…
I’m really looking forward to seeing this, and I’m hoping to see more use made of the NLS Maps API in projects here at EDINA.
I have a couple of questions related to the API:
- – Is utf-8 input supported? I was not able to find records for “Nürnberg” or “Paříž” while query like “Nurnberg” or “Pariz” gives correct results. Is utf-8 encoded query passed automatically (urlencoded) to your service or are there any special parameters necessary?
I passed this question on to Joe and Lasma; they went into a huddle, and a couple of hours later, Lasma sent this:
Indeed, it was only doing ascii search. Joe just deployed a fix.
Now you can do utf8 search.
So utf8 search should now be behaving correctly as you would have expected it to. Thanks for pointing this out and helping us to improve the search service.
– Is a combined query with country or administrative area possible? Something like “London, USA” or “Leith, Edinburgh”?
Currently, if you do this sort of query – a comma-separated list of names – you see all Londons, and all USAs – as in this query: http://unlock.edina.ac.uk/features/London,USA?format=json
The various Londons that are, in fact, in the USA, will be marked with a country element ‘United States’.
But, i think what you’re asking for isn’t this – you’d like the Unlock Places search to pick out the Londons-contained-by-USA and just return those. We could do this, but don’t expose this sort of query via the API. We could change the meaning of comma-separated lists of names to do this, but that might break other peoples’ worlds. So the best answer I can give you is, we’ll think about how best to implement it and look at the access logs to see if we can reasonably change the meaning of the current API function.
– What are the Terms and Conditions of the online service? Is it completely free for anybody or are there set already some limits on the number of requests, usage from website which are behind password, commercial web services, derived data, etc?
So there are two versions of the Unlock Places gazetteer search service. One is completely open, built on various open data sources, and can be used by anyone for any purpose. We don’t have throttling or quotas on the API.
If persistent or demented-looking requests ever become a problem, we’ll think about throttling requests from particular hosts. I like the approach that OpenStreetmap’s Nominatim search service takes here – to say, “if you’re planning really heavy traffic, please talk to us first, we can schedule it at a quiet time or you can install your own instance of Nominatim”.
In the past I’ve fired off a million requests without any pause, to search through the 1881 census microdata placenames for UKDA, and this happily didn’t affect the performance of the service.
The second gazetteer search service is limited to UK academic institutions that subscribe to the Digimap Ordnance Survey Collection, and the ways in which the data can be re-used are limited to academic services.
– Do you plan to release (make available for download) your Gazetteer database? If not, would you be willing to submit (later on?) at least the database with GeoNames.org IDs and the bboxes back the Mark Wick of GeoNames.org, so the great work you did is preserved also in the official free GeoNames database. You have much more to offer then bbox, but at least that would be excellent for the community.
I feel that release of the database is important for sustainability…
Right, everything in the open data side of Unlock is built from publicly available sources which are open licensed. One thing we could try is putting together a data package – using Open Knowledge Foundation’s datapkg project, for example – that would automate the process of rebuilding a database that looks like Unlock’s, from these different sources.
– Are you going to support the service in the future?
Unlock (Places, and Text) is a service supported by JISC, which manages technology funding for research and innovation in the UK. It’s hosted at the EDINA National Datacentre at the University of Edinburgh, which is also mostly supported directly by JISC.
So EDINA has a service level agreement with JISC to maintain Unlock with maximum 10 hours of downtime in a year – I think we’re close to that.
Our current agreement with JISC to support and develop the Unlock service at EDINA runs until July 2011. Its ongoing existence after that depends whether we, and JISC, can convincingly make the case that Unlock is creating “impact and value” in academia and beyond (museums, libraries and archives nearest by).
One of the best ways we can make the case is to get more feedback from people like you, Petr – what you like about the service, what you wish it did, what it’s offering to research that commercial or government services cannot reach. Some more thoughts about that are at the bottom of my last post discussing MySociety’s MapIt service.
Thank you a lot for you online service!
Thank you a lot for your long email, Petr, and I hope it helps encourage others to write.