On the Ambiguity and Relevance of Place Names in Scientific Text

How hard is it to systematically identify and disambiguate place names in scientific text? In order to address this question, we applied MapAffil, a toponymic search interface, on a random sample of 500 place name sentences from PubMed abstracts. The algorithm correctly identified and disambiguated 39.2% of the place names in sentences. An error analysis revealed six unique challenges: Biological terms (14.2%), Method terms (11.6%), Acronyms (10%), References (6%), Other entity names (4.2%), and Other errors (2.2%). Interestingly, a large portion of the correctly identified place names appeared irrelevant to the subject matter. Many of these errors can be fixed easily, but irrelevance is much harder to address, for it depends on semantics and purpose. To study the role of place in scientific text, it is not sufficient to disambiguate accurately, but it is also necessary to be able to assess the degree of relevance.