Since 5.1.0, three additional escape sequences to match generic character types are available when UTF-8 mode is selected. They are:
The property names represented by xx
above are limited
to the Unicode general category properties. Each character has exactly one
such property, specified by a two-letter abbreviation. For compatibility with
Perl, negation can be specified by including a circumflex between the
opening brace and the property name. For example, \p{^Lu}
is the same as \P{Lu}
.
If only one letter is specified with \p
or
\P
, it includes all the properties that start with that
letter. In this case, in the absence of negation, the curly brackets in the
escape sequence are optional; these two examples have the same effect:
\p{L} \pL
Property | Matches | Notes |
---|---|---|
C |
Other | |
Cc |
Control | |
Cf |
Format | |
Cn |
Unassigned | |
Co |
Private use | |
Cs |
Surrogate | |
L |
Letter |
Includes the following properties: Ll ,
Lm , Lo , Lt and
Lu .
|
Ll |
Lower case letter | |
Lm |
Modifier letter | |
Lo |
Other letter | |
Lt |
Title case letter | |
Lu |
Upper case letter | |
M |
Mark | |
Mc |
Spacing mark | |
Me |
Enclosing mark | |
Mn |
Non-spacing mark | |
N |
Number | |
Nd |
Decimal number | |
Nl |
Letter number | |
No |
Other number | |
P |
Punctuation | |
Pc |
Connector punctuation | |
Pd |
Dash punctuation | |
Pe |
Close punctuation | |
Pf |
Final punctuation | |
Pi |
Initial punctuation | |
Po |
Other punctuation | |
Ps |
Open punctuation | |
S |
Symbol | |
Sc |
Currency symbol | |
Sk |
Modifier symbol | |
Sm |
Mathematical symbol | |
So |
Other symbol | Includes emojis |
Z |
Separator | |
Zl |
Line separator | |
Zp |
Paragraph separator | |
Zs |
Space separator |
Extended properties such as InMusicalSymbols
are not
supported by PCRE.
Specifying case-insensitive (caseless) matching does not affect these escape sequences.
For example, \p{Lu}
always matches only upper case letters.
Sets of Unicode characters are defined as belonging to certain scripts. A character from one of these sets can be matched using a script name. For example:
\p{Greek}
\P{Han}
Those that are not part of an identified script are lumped together as
Common
. The current list of scripts is:
Arabic
Armenian
Avestan
Balinese
Bamum
Batak
Bengali
Bopomofo
Brahmi
Braille
Buginese
Buhid
Canadian_Aboriginal
Carian
Chakma
Cham
Cherokee
Common
Coptic
Cuneiform
Cypriot
Cyrillic
Deseret
Devanagari
Egyptian_Hieroglyphs
Ethiopic
Georgian
Glagolitic
Gothic
Greek
Gujarati
Gurmukhi
Han
Hangul
Hanunoo
Hebrew
Hiragana
Imperial_Aramaic
Inherited
Inscriptional_Pahlavi
Inscriptional_Parthian
Javanese
Kaithi
Kannada
Katakana
Kayah_Li
Kharoshthi
Khmer
Lao
Latin
Lepcha
Limbu
Linear_B
Lisu
Lycian
Lydian
Malayalam
Mandaic
Meetei_Mayek
Meroitic_Cursive
Meroitic_Hieroglyphs
Miao
Mongolian
Myanmar
New_Tai_Lue
Nko
Ogham
Old_Italic
Old_Persian
Old_South_Arabian
Old_Turkic
Ol_Chiki
Oriya
Osmanya
Phags_Pa
Phoenician
Rejang
Runic
Samaritan
Saurashtra
Sharada
Shavian
Sinhala
Sora_Sompeng
Sundanese
Syloti_Nagri
Syriac
Tagalog
Tagbanwa
Tai_Le
Tai_Tham
Tai_Viet
Takri
Tamil
Telugu
Thaana
Thai
Tibetan
Tifinagh
Ugaritic
Vai
Yi
The \X
escape matches a Unicode extended grapheme
cluster. An extended grapheme cluster is one or more Unicode characters
that combine to form a single glyph. In effect, this can be thought of as
the Unicode equivalent of .
as it will match one
composed character, regardless of how many individual characters are
actually used to render it.
In versions of PCRE older than 8.32 (which corresponds to PHP versions
before 5.4.14 when using the bundled PCRE library), \X
is equivalent to (?>\PM\pM*)
. That is, it matches a
character without the "mark" property, followed by zero or more characters
with the "mark" property, and treats the sequence as an atomic group (see
below). Characters with the "mark" property are typically accents that
affect the preceding character.
Matching characters by Unicode property is not fast, because PCRE has
to search a structure that contains data for over fifteen thousand
characters. That is why the traditional escape sequences such as
\d
and \w
do not use Unicode properties
in PCRE.
To select UTF-8 mode for the additional escape sequences (\p{xx}, \P{xx}, and \X) , use the "u" modifier (see http://php.net/manual/en/reference.pcre.pattern.modifiers.php).
I wondered why a German sharp S (ß) was marked as a control character by \p{Cc} and it took me a while to properly read the first sentence: "Since 5.1.0, three additional escape sequences to match generic character types are available when UTF-8 mode is selected. " :-$ and then to find out how to do so.
Examples are always useful! See https://unicodeplus.com/category for more.
C Other
Cc Control (Unicode code points in the ranges U+0000-U+001F and U+007F-U+009F)
Cf Format (Soft hyphen (U+00AD), zero width space (U+200B), etc.)
Cn Unassigned (Any code point that is not in the Unicode table)
Co Private use
Cs Surrogate (Characters in the range U+D800 to U+DFFF, which are invalid in utf-8)
L Letter
Ll Lower case letter (a-z, μßàáâãäåæçèéêëìíîïðñòóôõöøùúûüýþÿ and more)
Lm Modifier letter (Letter-like characters that are usually combined with others, but here they stand alone:
hɦjrɹɻʁwyʹʺʻʼʽʾʿˀˁˆˇˈˉˊˋˌˍˎˏːˑɣlsxʕˬˮʹ ͅՙ and more)
Lo Other letter (aoƻǀǁǂǃʔ and many more ideographs and letters from unicase alphabets)
Lt Title case letter (DžLjNjDzᾈᾉᾊᾋᾌᾍᾎᾏᾘᾙᾚᾛᾜᾝᾞᾟᾨᾩᾪᾫᾬᾭᾮᾯᾼῌῼ)
Lu Upper case letter (A-Z, ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖØÙÚÛÜÝÞ and more)
L& Ordinary letter (Any character that has the Lu, Ll, or Lt property)
M Mark
Mc Spacing mark (None in latin scripts)
Me Enclosing mark (Combining enclosing square (U+20DE) like in a⃞ , combining enclosing circle backslash (U+20E0) like in a⃠)
Mn Non-spacing mark (Combining diacritical marks U+0300-U+036f, like the accents on this letter a: áâãāa̅ăȧäảåa̋ǎa̍a̎ȁa̐ȃ)
N Number
Nd Decimal number (0123456789, ٠١٢٣٤٥٦٧٨٩ and digits in many other scripts.)
Nl Letter number (IIIIIIIVVVIVIIVIIIIXXXIXIILCDMiiiiiiivvviviiviiiixxxixiilcdm and some more)
No Other number (0123456789 0123456789 1⁄21⁄32⁄31⁄43⁄41⁄52⁄53⁄54⁄51⁄65⁄61⁄71⁄83⁄85⁄87⁄81⁄91⁄10 1234567891011121314151617181920, etc.)
P Punctuation
Pc Connector punctuation (_ underscore (U+005F), ‿ undertie U+203F, ⁀ character tie (U+2040), etc.)
Pd Dash punctuation (- hyphen-minus (U+002D), ‐ hyphen (U+2010), ‐ non-breaking hyphen (U+2011), ‒ figure dash (U+2012),
– en dash (U+2013), — em dash (U+2014), ― horizontal bar (U+2015), etc.)
Pe Close punctuation (right parenthesis, bracket, or brace: `)` (U+0029), `]` (U+005D), `}` (U+007D), etc.)
Pf Final punctuation (right quotation marks: » (U+00BB), ’ (U+2019), " (U+201D), etc.)
Pi Initial punctuation (left quotation marks: « (U+00AB), ‘ (U+2018), " (U+201C), etc.)
Po Other punctuation (!"#%&'*,./:;?@\¡§¶·¿)
Ps Open punctuation (left parenthesis, bracket, or brace: `(` (U+0028), `[` (U+005B), `{` (U+007B), etc.)
S Symbol
Sc Currency symbol ($¢£¤,円 ₠ ₡ ₢ ₣ ₤ ₥ ₦ ₧ Rs ₩ ₪ ₫ € ₭ ₮ ₯ ₰ ₱ ₲ ₳ ₴ ₵ ₶ ₷ ₸ ₹ ₺ ₻ ₼ ₽ ₾ ₿ (U+20A0-U+20BF), etc.)
Sk Modifier symbol (Symbol-like characters that are usually combined with others, but here they stand alone:
^` ̈ ̄ ́ ̧ and more)
Sm Mathematical symbol (+<=&g×ばつ÷϶ and many more)
So Other symbol (¦ broken bar (U+00A6), © copyright sign (U+00A9), ® registered sign (U+00AE), ° degree sign (U+00B0);
arrows, signs, emojis and many many more)
Z Separator
Zl Line separator (line separator (U+2028))
Zp Paragraph separator (paragraph separator (U+2029))
Zs Space separator (space, no-break space, en quad, em quad, en space, em space, figure space, thin space, hair space, etc.)
An excellent article explaining all these properties can be found here: http://www.regular-expressions.info/unicode.html
My country, Vietnam, have our own alphabet table:
http://en.wikipedia.org/wiki/Vietnamese_alphabet
I hope PHP will support better than in Vietnamese.
For those who wonder: 'letter_titlecase' applies to digraphs/trigraphs, where capitalization involves only the first letter.
For example, there are three codepoints for the "LJ" digraph in Unicode:
(*) uppercase "LJ": U+01C7
(*) titlecase "Lj": U+01C8
(*) lowercase "lj": U+01C9
Not made clear in the top of page explanation, but these escaped character classes can be included within square brackets to make a broader character class. For example:
<?php preg_match( '/[\p{N}\p{L}]+/', $data ) ?>
Will match any combination of letters and numbers.
these properties are usualy only available if PCRE is compiled with "--enable-unicode-properties"
if you want to match any word but want to provide a fallback, you can do something like that:
<?php
if(@preg_match_all('/\p{L}+/u', $str, $arr) {
// fallback goes here
// for example just '/\w+/u' for a less acurate match
}
?>