Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Tussenvoegsels / family name prefixes support #132

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
patvdleer wants to merge 4 commits into derek73:master
base: master
Choose a base branch
Loading
from patvdleer:master
Draft
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Refs #130 - tussenvoegsels/affix rather than family prefix
  • Loading branch information
patvdleer committed Jan 31, 2022
commit 80d4e5550b859a3b258e30db8f52d347ad2484c5
6 changes: 4 additions & 2 deletions nameparser/config/__init__.py
View file Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@
"""
from __future__ import unicode_literals
import sys

try:
# Python 3.3+
from collections.abc import Set
Expand All @@ -46,6 +47,7 @@
from nameparser.config.titles import TITLES
from nameparser.config.titles import FIRST_NAME_TITLES
from nameparser.config.regexes import REGEXES
from nameparser.config.affixes import AFFIXES

DEFAULT_ENCODING = 'UTF-8'

Expand Down Expand Up @@ -235,7 +237,7 @@ class Constants(object):

def __init__(self,
prefixes=PREFIXES,
family_prefixes=PREFIXES,
family_affixes=AFFIXES,
suffix_acronyms=SUFFIX_ACRONYMS,
suffix_not_acronyms=SUFFIX_NOT_ACRONYMS,
titles=TITLES,
Expand All @@ -245,7 +247,7 @@ def __init__(self,
regexes=REGEXES
):
self.prefixes = SetManager(prefixes)
self.family_prefixes = SetManager(prefixes)
self.family_affixes = SetManager(family_affixes)
self.suffix_acronyms = SetManager(suffix_acronyms)
self.suffix_not_acronyms = SetManager(suffix_not_acronyms)
self.titles = SetManager(titles)
Expand Down
122 changes: 122 additions & 0 deletions nameparser/config/affixes.py
View file Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
# -*- coding: utf-8 -*-
from __future__ import unicode_literals

# https://en.wikipedia.org/wiki/List_of_family_name_affixes

AFFIXES = set([
'a',
'ab',
'af',
'av',
'ap',
'abu',
'ait',
'aït',
'alam',
'at',
'ath',
'aust',
'austre',
'bar',
'bat',
'bath',
'ben',
'bin',
'ibn',
'bert',
'bet',
'bint',
'da',
'das',
'de',
'degli',
Copy link
Owner

@derek73 derek73 Feb 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Many of these affixes are already defined in prefixes.py.

It seems like mainly what you want is the parts that come before a last name and currently get added to it (wether we call them prefixes or affixes or tussenvoegsels) separately from the last name.

'del',
'dele',
'della',
'der',
'di',
'dos',
'du',
'e',
'el',
'fetch',
'vetch',
'fitz',
'i',
'kil',
'gil',
'la',
'le',
'lille',
'lu',
'm\'',
'mc',
'mac',
'mck',
'mhic',
'mic',
'mala',
'mellom',
'myljom',
'na',
'ned',
'nedre',
'neder',
'nic',
'ni',
'ní',
'nin',
'nord',
'norr',
'nord',
'nordre',
'ny',
'o',
'ua',
'ua',
'ui',
'uí',
'opp',
'upp',
'ofver',
'ost',
'oster',
'over',
'ovste',
'ovre',
'oz',
'pour',
'putra',
'putera',
'putri',
'putera',
'setia',
'setya',
'stor',
'soder',
'sor',
'sonder',
'syd',
'sondre',
'syndre',
'sore',
'ter',
'\'t',
'tre',
'van',
'het',
'de',
'vast',
'väst',
'vaster',
'väster',
'verch',
'erch',
'vest',
'vestre',
'vesle',
'vetle',
'von',
'war',
'zu',
])
32 changes: 15 additions & 17 deletions nameparser/parser.py
View file Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -307,14 +307,10 @@ def family(self):
"""
The person's family name.
"""
return " ".join(self.family_list) or self.C.empty_attribute_default

@property
def family_prefix(self):
"""
The person's family name prefix(es).
"""
return " ".join(self.family_prefix_list) or self.C.empty_attribute_default
s = ""
for affix, family in self.family_list:
s += " ".join([*affix, *family]) or self.C.empty_attribute_default
return s

@property
def suffix(self):
Expand Down Expand Up @@ -415,17 +411,17 @@ def is_prefix(self, piece):
else:
return lc(piece) in self.C.prefixes

def is_family_prefix(self, piece):
def is_family_affix(self, piece):
"""
Lowercase and no periods version of piece is in the
:py:data:`~nameparser.config.family_prefixes.PREFIXES` set.
:py:data:`~nameparser.config.family_affixes.AFFIXES` set.
"""
if isinstance(piece, list):
for item in piece:
if self.is_family_prefix(item):
if self.is_family_affix(item):
return True
else:
return lc(piece) in self.C.family_prefixes
return lc(piece) in self.C.family_affixes


def is_roman_numeral(self, value):
Expand Down Expand Up @@ -593,7 +589,6 @@ def parse_full_name(self):
self.middle_list = []
self.last_list = []
self.family_list = []
self.family_prefix_list = []
self.suffix_list = []
self.nickname_list = []
self.unparsable = True
Expand Down Expand Up @@ -732,13 +727,16 @@ def parse_full_name(self):

for last in self.last_list:
if " " in last:
affix = []
family = []
for part in last.split(" "):
if self.is_family_prefix(part):
self.family_prefix_list.append(part)
if self.is_family_affix(part):
affix.append(part)
else:
self.family_list.append(part)
family.append(part)
self.family_list.append([affix, family])
else:
self.family_list.append(last)
self.family_list.append([[], [last]])

if len(self) < 0:
log.info("Unparsable: \"%s\" ", self.original)
Expand Down
18 changes: 12 additions & 6 deletions tests.py
View file Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -187,13 +187,19 @@ def test_prefix_names(self):
self.m(hn.first, "vai", hn)
self.m(hn.last, "la", hn)

def test_family_name(self):
def test_family_name_and_prefix(self):
hn = HumanName("Vincent van Gogh")
self.m(hn.family, "Gogh", hn)

def test_family_name_prefix(self):
hn = HumanName("Vincent van Gogh")
self.m(hn.family_prefix, "van", hn)
self.m(hn.family, "van Gogh", hn)
self.assertEqual(hn.family_list, [
[["van"], ["Gogh"]]
])

def test_family_name_and_double_prefix(self):
hn = HumanName("Vincent van der Gogh")
self.m(hn.family, "van der Gogh", hn)
self.assertEqual(hn.family_list, [
[["van", "der"], ["Gogh"]],
Copy link
Owner

@derek73 derek73 Feb 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it makes sense to have these pieces in a separate attribute, but I'd lean more towards having separate attributes for those 2 nested lists. Maybe "affixes" and "family name", which combine together to give "last name", (which, btw, combines with middle names to give "surnames")?

We are kind of running out of names for things. And Wikipedia thinks that Surname, Last name, and family name are all the same thing, so there's that. 😄

Copy link
Owner

@derek73 derek73 Feb 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, Google translate says the English translation of tussenvoegsels is "infixes". Really? There's yet another one? I did not know that was a word. (It's a good day when I can learn a new English word, so thanks!) But I guess if we're desperate, we could use that.

Copy link
Owner

@derek73 derek73 Feb 4, 2022
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kinda makes me wonder if we could do something more useful with slices, maybe something like each bucket gets an index value and you can slice off the parts you want/don't want?

[0:titles, 1:first, 2:middle, 3:prefixes, 4:last, 5:suffixes]

Copy link
Owner

@derek73 derek73 Feb 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, that is already how slice works. Been so long since I implemented it that I forgot. It returns the concatenated string of whichever members you slice.

So, if we added prefixes in there you could get just the last name with a slice, ex: hn[4:4].

_members = ['title', 'first', 'middle', 'prefix', 'last', 'suffix', 'nickname']

Mostly what we need is just to have the prefixes in their own bucket, then we can be more flexible when we display things. Probably means changing the parse tree to add prefix in there.

Copy link
Author

@patvdleer patvdleer Feb 6, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The more I dive into the surname/last name/family name, the more confused I get... There doesn't seem to be a clear definition of any of these things.

I am missing some entries in the prefixes hence the reason I made a separate list not to interfere with your work. I was looking into possible locales but that seems to be a road I really don't want to go down...

Mostly what we need is just to have the prefixes in their own bucket, then we can be more flexible when we display things. Probably means changing the parse tree to add prefix in there.

I tried Vincent van Gogh van Beethoven which gave me van Gogh van as a middle name. I dropped that attempt for now but that is the reason I created a nested list with pairs, prefix - family name. Otherwise it would give me simply van twice, and (possibly) format it as van van Gogh Beethoven.

Copy link
Owner

@derek73 derek73 Feb 6, 2022
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There used to be similar conversations re: "title", and "suffix" because people got tripped up on the semantic meaning of a title vs a suffix, ex "Dr" and "MD" are kinda the same but they appear in different places in the name. I think the best strategy for the parser is to focus on the position of words (name parts) in the string, and special words that join with words before/after/around them in certain conditions/parts of the name. In the future I should probably choose names that focus on that positional information and avoid semantic meaning.

We can add some of those things from your affixes constants to prefixes. The only danger is ones that could also be first names, like "fitz, "mala" and "ned". I think the current handling for prefixes skips the first name though, so it could be fine to include those.

I tried Vincent van Gogh van Beethoven which gave me van Gogh van as a middle name

👎 That's annoying. Doesn't seem like that was my intention:

# join everything after the prefix until the next prefix or suffix
try:
if i == 0 and total_length >= 1:
# If it's the first piece and there are more than 1 rootnames, assume it's a first name
continue
next_prefix = next(iter(filter(self.is_prefix, pieces[i + 1:])))
j = pieces.index(next_prefix)
if j == i + 1:
# if there are two prefixes in sequence, join to the following piece
j += 1
new_piece = ' '.join(pieces[i:j])
pieces = pieces[:i] + [new_piece] + pieces[j:]
except StopIteration:
try:
# if there are no more prefixes, look for a suffix to stop at
stop_at = next(iter(filter(self.is_suffix, pieces[i + 1:])))
j = pieces.index(stop_at)
new_piece = ' '.join(pieces[i:j])
pieces = pieces[:i] + [new_piece] + pieces[j:]
except StopIteration:
# if there were no suffixes, nothing to stop at so join all
# remaining pieces
new_piece = ' '.join(pieces[i:])
pieces = pieces[:i] + [new_piece]

])

def test_blank_name(self):
hn = HumanName()
Expand Down

AltStyle によって変換されたページ (->オリジナル) /