homepage

This issue tracker has been migrated to GitHub , and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author mrabarnett
Recipients benjamin.peterson, cvrebert, dpk, ezio.melotti, lemburg, loewis, mrabarnett
Date 2013年07月09日.17:25:36
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1373390736.33.0.199287811632.issue18406@psf.upfronthosting.co.za>
In-reply-to
Content
This is basically what the regex module does, written in Python:
 def get_grapheme_cluster_break(codepoint):
 """Gets the "Grapheme Cluster Break" property of a codepoint.
 
 The properties defined here:
 
 http://www.unicode.org/Public/UNIDATA/auxiliary/GraphemeBreakProperty.txt
 """
 # The return value is one of:
 #
 # "Other"
 # "CR"
 # "LF"
 # "Control"
 # "Extend"
 # "Prepend"
 # "Regional_Indicator"
 # "SpacingMark"
 # "L"
 # "V"
 # "T"
 # "LV"
 # "LVT"
 ...
 
 def at_grapheme_boundary(string, index):
 """Checks whether the codepoint at 'index' is on a grapheme boundary.
 
 The rules are defined here:
 
 http://www.unicode.org/reports/tr29/#Grapheme_Cluster_Boundaries
 """
 # Break at the start and end of the text.
 if index <= 0 or index >= len(string):
 return True
 
 prop = get_grapheme_cluster_break(string[index])
 prop_m1 = get_grapheme_cluster_break(string[index - 1])
 
 # Don't break within CRLF.
 if prop_m1 == "CR" and prop == "LF":
 return False
 
 # Otherwise break before and after controls (including CR and LF).
 if prop_m1 in ("Control", "CR", "LF") or prop in ("Control", "CR", "LF"):
 return True
 
 # Don't break Hangul syllable sequences.
 if prop_m1 == "L" and prop in ("L", "V", "LV", "LVT"):
 return False
 if prop_m1 in ("LV", "V") and prop in ("V", "T"):
 return False
 if prop_m1 in ("LVT", "T") and prop == "T":
 return False
 
 # Don't break between regional indicator symbols.
 if (prop_m1 == "REGIONALINDICATOR" and prop ==
 "REGIONALINDICATOR"):
 return False
 
 # Don't break just before Extend characters.
 if prop == "Extend":
 return False
 
 # Don't break before SpacingMarks, or after Prepend characters.
 if prop == "SpacingMark":
 return False
 
 if prop_m1 == "Prepend":
 return False
 
 # Otherwise, break everywhere.
 return True
History
Date User Action Args
2013年07月09日 17:25:36mrabarnettsetrecipients: + mrabarnett, lemburg, loewis, benjamin.peterson, ezio.melotti, cvrebert, dpk
2013年07月09日 17:25:36mrabarnettsetmessageid: <1373390736.33.0.199287811632.issue18406@psf.upfronthosting.co.za>
2013年07月09日 17:25:36mrabarnettlinkissue18406 messages
2013年07月09日 17:25:36mrabarnettcreate

AltStyle によって変換されたページ (->オリジナル) /