Names can refer to сlasses of humans/locations/organizations

[vikmary]

Thank you for publishing the preprocessing pipeline! Successfully launched it on linux. It is convenient that the preprocessed entries are dumped during the process so that you can monitor it in progress.

After looking at some resulting entries in the combined/ folder, I see that the terms "PERSON", "ORGANIZATION" and "LOCATION" are more loose than I expected it to be (my misunderstanding here).
Besides taking instances of "Q5"(persons), "Q82794" (locations) and "Q43229" (organizations), the paranames/io/wikidata_dump_transliterations.py is taking instances of their subclasses as well. This leads to "Hispanic and Latino-American teenage boys" and "Government secretaries of Policies for Women of the State of Bahia" be classified as a person along with ["Samuel Hamington"[(https://www.wikidata.org/wiki/Q111165240), while I expected only "Samuel Hamington" to be included.

Maybe it would be good to have data samples in the git repo or the original paper =)

[j0ma]

Thanks for bringing up this issue and sorry it's taken me so long to respond.

I think the PER entities you brought up are definitely quite weird -- the reason they get assigned PER has to do with the inheritance hierarchy in Wikidata: a lot of entities seem to be instances of something that is a subclass of Q5 (human), which leads to them being classified as PER even if they are not directly instances of Q5.

Even though there will likely never be a perfect solution to this, in future releases we will make this requirement stricter and require all PER entities to directly be instances of Q5. Interestingly, for LOC/ORG, most of the entities our approach detects seem to be through the use of subclasses and it will probably not make as much sense to require those entities to also be instances of Q82794/Q82794, or else we will lose most of the LOC/ORG entities.

I recently analyzed this a bit in a notebook in another branch, feel free to take a look there.

[vikmary]

@j0ma The analysis is really insightful, thank you
From my point of view, the entities that are added when taking instances of subclasses of Q5 (PER) should be considered to be removed, but we can't do the same for LOCs and ORGs, because there are so many of them