正则表达式库-PCRE
2012-12-10 09:59:34 
阿炯
PCRE (Perl Compatible Regular Expressions) 包括 Perl 兼容的正规表达式库，这些在执行正规表达式模式匹配时用与Perl 5同样的语法和语义是很有用的。采用C语言开发编写并在PCRE2 LICENCE(类BSD)协议下授权使用。其是一个广泛使用的正则表达式库，它提供了与Perl 5相同的正则表达式功能；最初由Philip Hazel开发，为了在C语言中提供Perl风格的正则表达式支持。自1997年发布以来，PCRE已经成为许多编程语言和工具的标准组件，包括PHP、Apache、Nginx等。The PCRE library is a set of functions that implement regular expression pattern matching using the same syntax and semantics as Perl 5. PCRE has its own native API, as well as a set of wrapper functions that correspond to the POSIX regular expression API. The PCRE library is free, even for building proprietary software.
The PCRE2 library is a set of C functions that implement regular expression pattern matching using the same syntax and semantics as Perl 5. PCRE2 has its own native API, as well as a set of wrapper functions that correspond to the POSIX regular expression API. The PCRE2 library is free, even for building proprietary software. It comes in three forms, for processing 8-bit, 16-bit, or 32-bit code units, in either literal or UTF encoding.
PCRE2 was first released in 2015 to replace the API in the original PCRE library, which is now obsolete and no longer maintained. As well as a more flexible API, the code of PCRE2 has been much improved since the fork.
PCRE was originally written for the Exim MTA, but is now used by many high-profile open source projects, including Apache, PHP, KDE, Postfix, Analog, and Nmap. PCRE has also found its way into some well known commercial products, like Apple Safari. Some other interesting projects using PCRE include Chicken, Ferite, Onyx, Hypermail, Leafnode, Askemos, and Wenlin.
主要特点
Perl兼容性：支持Perl 5的正则表达式语法，这意味着你可以使用Perl中的大多数正则表达式特性，如捕获组、非捕获组、零宽断言等。
高效性能：经过优化，提供了高效的正则表达式匹配和搜索功能。它使用了一种称为“非回溯”（non-backtracking）的算法，这使得它在处理复杂的正则表达式时更加高效。
跨平台支持：可以在多种操作系统和平台上使用，包括Windows、Linux、macOS等。
丰富的功能：支持多种正则表达式功能，如Unicode字符处理、递归模式、条件模式等。
灵活的配置：提供了多种配置选项，允许用户根据需要调整正则表达式的行为，如忽略大小写、多行模式等。
版本(Versions)
PCRE库有两个主要版本。目前的版本PCRE2于2015年发布，目前为10.39版本。较旧但仍然广泛部署的PCRE库最初于1997年发布，版本为8.45；这一版本的PCRE现在已经寿终正寝，不再积极维护。8.45版本预计将是旧的PCRE库的最终版本，新项目应该使用PCRE2。下面将分别讲解它们的发展与使用情况。
使用详解
主要内容如下：
什么是PCRE
BRE 与 ERE
BRE与ERE详细语法对比
PCRE 与 PCRE2
PCRE 与 PCRE2 语法
什么是PCRE
Perl Compatible Regular Expressions（PCRE）是一个用 C 语言编写的正则表达式库，其设计灵感源自 Perl 编程语言的强大正则功能。该库由 Philip Hazel 于 1997 年夏季开始开发。其语法远比 POSIX 正则表达式的两种风格（BRE 和 ERE）以及其他许多正则库更为强大和灵活。
BRE 与 ERE
BRE 是保守的、向后兼容的选择；ERE 是现代的、功能丰富的选择。
POSIX（Portable Operating System Interface）是 IEEE 制定的一套 Unix 操作系统标准，其中定义了两种正则表达式语法规范：
BRE（Basic Regular Expressions，基本正则表达式）
ERE（Extended Regular Expressions，扩展正则表达式）
这两种风格在元字符处理、转义规则和功能支持上有显著差异，深刻影响着 Linux/Unix 工具的行为。
| 特性 | BRE（基本正则） | ERE（扩展正则） |
| 元字符集合 | ^ $ . [ ] * | 在 BRE 基础上增加 ( ) { } ? + | |
| 特殊功能启用 | 需要反斜杠转义才能启用扩展功能 | 直接使用扩展元字符 |
| 交替（或）操作 | 不支持 | 支持 |（BRE）vs ` |
| 分组 | \(...\) | (...) |
| 量词 | \{m,n\} | {m,n} |
| 典型工具 | grep、sed、awk（默认） | egrep、grep -E |
BRE与ERE详细语法对比
元字符处理
BRE的哲学：只有少数字符具有特殊含义，其他字符要获得特殊功能必须用反斜杠 \"激活"。
ERE的哲学：更多字符天生具有特殊含义，要匹配字面量反而需要转义。
量词（Quantifiers）
| 功能 | BRE 写法 | ERE 写法 |
| 匹配 0 次或多次 | * | * |
| 匹配 1 次或多次 | \+ | + |
| 匹配 0 次或 1 次 | \? | ? |
| 匹配精确次数 | \{n\} | {n} |
| 匹配范围次数 | \{m,n\} | {m,n} |
示例：
匹配 "color" 或 "colour"：
BRE: colou\?r
ERE: colou?r
分组与捕获
| 功能 | BRE | ERE |
| 分组 | \(...\) | (...) |
| 反向引用 | \1, \2... | \1, \2...（但部分工具不支持） |
示例：
匹配重复单词：
BRE: \(word\)\1
ERE: (word)\1
交替（OR 操作）
这是最显著的差异！
BRE：不原生支持交替。某些实现（如 GNU grep）通过 \| 提供，但这不是 POSIX 标准的一部分。
ERE：原生支持 | 操作符。
示例：
匹配 "cat" 或 "dog"：
BRE: 通常需要写两次模式，或依赖特定实现的 \|
ERE: cat|dog
通用元字符（BRE 和 ERE 共享）
这些元字符在两种风格中行为一致：
.：匹配任意单个字符（除换行符）
^：匹配行首
$：匹配行尾
[abc]：字符类，匹配括号内任一字符
[^abc]：否定字符类
[a-z]：范围字符类
工具支持情况
BRE 支持的工具（默认模式）
grep（基本模式）、sed、awk、vi/vim
ERE 支持的工具
egrep（等同于 grep -E）
awk（某些版本支持，但主要用 BRE）
grep -E
实用示例：
# BRE：匹配以数字开头的行（注意 \+ 的使用）
grep '^[0-9]\+' file.txt
# ERE：同样的功能，语法更简洁
grep -E '^[0-9]+' file.txt
# ERE：使用交替匹配多种文件扩展名
grep -E '\.(txt|log|conf)$' file_list.txt
# BRE：同样的功能（GNU grep 支持 \|，但非标准）
grep '\.\(txt\|log\|conf\)$' file_list.txt
POSIX 字符类（两种风格都支持）
为了解决 locale（区域设置）问题，POSIX 定义了标准化的字符类：
[[:digit:]] ≡ [0-9]
[[:alpha:]] ≡ [a-zA-Z]
[[:alnum:]] ≡ [a-zA-Z0-9]
[[:space:]] ≡ 空白字符
[[:punct:]] ≡ 标点符号
推荐使用：在跨平台脚本中优先使用 POSIX 字符类，而非 Perl 风格的 \d、\w（这些在 BRE/ERE 中不被支持）。
PCRE 与 PCRE2
PCRE（第一代）
首次发布：1997 年
最终版本：8.45（已进入生命周期终结（EOL）阶段）
状态：不再主动维护，仅修复严重安全问题
特点：API 简单但不够灵活，内存管理较原始
PCRE2（第二代）
首次发布：2015 年
当前版本：10.47+（截至 2026 年仍在活跃开发）
状态：官方推荐新项目使用
特点：全新设计的 API，更安全、更高效、功能更强
核心区别
| 特性 | PCRE（v1） | PCRE2（v2） |
| API 设计 | 单一函数接口（如pcre_compile, pcre_exec） | 模块化、上下文驱动（使用 pcre2_code, pcre2_match_data 等结构体） |
| 内存管理 | 手动分配/释放，易出错 | 支持自定义内存分配器，更安全 |
| 线程安全 | 部分函数非线程安全 | 完全线程安全（每个操作使用独立上下文） |
| 错误处理 | 返回错误码，需查表 | 返回详细错误信息，支持错误消息字符串 |
| JIT 支持 | 有，但集成较弱 | 深度集成 JIT（即时编译），性能提升 5–50 倍 |
| Unicode 支持 | 基础支持 | 更完善的 UTF-8/16/32 支持，符合现代标准 |
| 选项控制 | 通过整数位掩码 | 使用结构体配置（如pcre2_compile_context） |
| 匹配结果 | 使用 ovector 整数数组 | 使用专用match_data 对象，可复用 |
PCRE1（8.xx 系列）：已停止开发，8.45 为最终版
PCRE2（10.xx 系列）：当前活跃开发版本，是未来方向
绝大多数 PCRE1 的正则模式可直接在 PCRE2 中使用。
PCRE 与 PCRE2 语法
基础语法元素
1. 字面量与转义
普通字符：a, 1, @ 等直接匹配自身
转义序列：
\n：换行符
\t：制表符
\d：数字 [0-9]
\D：非数字
\w：单词字符 [a-zA-Z0-9_]
\W：非单词字符
\s：空白字符（空格、制表符、换行等）
\S：非空白字符
2. 锚点（Anchors）
^：行首（在多行模式下为每行开头）
$：行尾（在多行模式下为每行结尾）
\A：字符串绝对开头
\Z：字符串绝对结尾（忽略末尾换行符）
\z：字符串绝对结尾（严格匹配）
3. 字符类（Character Classes）
[abc]：匹配 a、b 或 c
[^abc]：匹配除 a、b、c 外的任意字符
[a-z]：范围匹配
POSIX 字符类：[[:digit:]], [[:alpha:]] 等
量词（Quantifiers）
| 量词 | 含义 | 示例 |
| * | 0 次或多次（贪婪） | a* |
| + | 1 次或多次（贪婪） | a+ |
| ? | 0 次或 1 次（贪婪） | a? |
| {n} | 精确 n 次 | a{3} |
| {n,} | 至少 n 次 | a{2,} |
| {n,m} | n 到 m 次 | a{2,5} |
惰性量词（在量词后加 ?）：
*?, +?, ??, {n,m}? 等，尽可能少匹配
分组与捕获
1. 基本分组
(pattern)：捕获分组，可被反向引用
(?:pattern)：非捕获分组，仅用于分组不捕获
2. 命名捕获组
(?P<name>pattern) # Python/PCRE 风格
(?<name>pattern) # .NET/PCRE 风格
(?'name'pattern) # 另一种 PCRE 风格
命名捕获组的命名规则
| 版本 | 规则 | 示例 |
| PCRE1（< 8.34） | 允许数字开头的组名 | (?P<123>...) 合法 |
| PCRE1 ≥ 8.34 & PCRE2 | 禁止数字开头，必须符合标识符规则 | (?P<group1>...)合法，(?P<1group>...) 非法 |
3. 反向引用
\1, \2, ...：引用第 1、2... 个捕获组
\g{name}：引用命名捕获组
\k<name>：另一种命名引用方式
断言（Assertions）
1. 前瞻断言（Lookahead）
(?=pattern)：正向前瞻（后面必须匹配）
(?!pattern)：负向前瞻（后面不能匹配）
2. 后顾断言（Lookbehind）
(?<=pattern)：正向后顾（前面必须匹配）
(?<!pattern)：负向后顾（前面不能匹配）
注意：PCRE 要求后顾断言中的每个分支必须是固定长度
模式修饰符（Flags）
可在模式开头或结尾指定：
| 修饰符 | 作用 | 内联形式 |
| i | 忽略大小写 | (?i) |
| m | 多行模式（^ $ 匹配每行） | (?m) |
| s | 单行模式（. 匹配换行符） | (?s) |
| x | 扩展模式（忽略空白和注释） | (?x) |
| U | 非贪婪模式（所有量词变惰性） | (?U) |
接下来再来深入讲解一些变化与使用特性
从 PCRE 到 PCRE2：一次现代化重构
API 使用对比（简单示例）
为什么推荐使用 PCRE2？
核心特性详解
虽然 PCRE 最初的目标是实现与 Perl 功能对等，但两者在实现上并不完全一致。在 PCRE 7.x 与 Perl 5.9.x 时期，两个项目曾协同开发，互相移植新特性。自 Perl 5.10 起，用户还可通过 re::engine::PCRE 模块，将 PCRE 作为 Perl 默认正则引擎的替代方案。
该库支持 Unix、Windows 及多种其他平台。PCRE2 发行包中包含：
一个 POSIX C 封装层
多个测试程序
实用工具 pcre2grep（与库同步构建）
从 PCRE 到 PCRE2：一次现代化重构
2015 年，PCRE 项目发布了一个重大更新——PCRE2，它采用了全新设计的应用程序接口（API）。
1、原始版本（现称 PCRE1，即 1.xx–8.xx 系列）目前已停止功能开发，仅修复严重安全问题。8.45 版本很可能是最后一个正式版，虽仍在部分遗留系统中使用，但已不再积极维护。
2、PCRE2（10.xx 系列）则引入了多项扩展与代码改进，是当前唯一活跃开发的主线版本。
API 使用对比（简单示例）
PCRE（旧版）风格：
pcre *re = pcre_compile(pattern, 0, &error, &erroffset, NULL);
int ovector[30];
int rc = pcre_exec(re, NULL, subject, strlen(subject), 0, 0, ovector, 30);
// 手动管理 re 和 ovector
pcre_free(re);
PCRE2（新版）风格：
pcre2_code *re;
pcre2_match_data *match_data;
int errornumber;
PCRE2_SIZE erroroffset;
re = pcre2_compile((PCRE2_SPTR)pattern, PCRE2_ZERO_TERMINATED, 0, &errornumber, &erroroffset, NULL);
match_data = pcre2_match_data_create_from_pattern(re, NULL);
int rc = pcre2_match(re, (PCRE2_SPTR)subject, strlen(subject),
0, 0, match_data, NULL);
// 自动化程度高，资源管理清晰
pcre2_match_data_free(match_data);
pcre2_code_free(re);
注意：PCRE2 的 API 虽然略显冗长，但更健壮、可扩展、不易出错。
为什么推荐使用 PCRE2？
PCRE 采用 BSD 许可证，商业软件也可自由使用。
持续维护：PCRE1 已停止功能更新；
性能更强：JIT 编译支持显著加速复杂正则；
安全性更高：避免缓冲区溢出等常见问题；
功能更全：支持更多 Perl 特性（如 \K, (*SKIP) 等）；
现代兼容性：更好支持 Unicode、多字节字符；
被广泛采用：PHP 7.3+、Apache httpd、Nginx（可选）、Rust 的 regex 库底层等均转向或支持 PCRE2。
核心特性
1. 即时编译器（JIT）支持
构建 PCRE2 时可启用 JIT 支持。对于重复执行的复杂模式，性能提升可达数倍甚至数十倍。此功能由 Zoltan Herczeg 开发，在 POSIX 封装层中不可用。
2. 灵活的内存管理
PCRE1 使用系统栈进行回溯，易导致栈溢出。PCRE2 自 10.30 版本（2017 年）起改用堆内存，并可限制总用量，彻底解决了这一问题。
3. 一致的转义规则
与 Perl 类似，PCRE2 中：
任何非字母数字字符前加反斜杠 \ 即表示字面量（如 \. 匹配点号）；
字母数字字符前加 \ 通常赋予特殊含义（如 \d 表示数字）；
若某转义序列未定义为特殊含义，则直接报错（Perl 仅在警告模式下报错，而 PCRE2 无警告模式）。
相比之下，POSIX 基本正则表达式中，\ 有时用于转义（如 \.），有时用于引入特殊功能（如 \(\)），规则不统一。
4. 扩展字符类
除 POSIX 长名称（如 [[:digit:]]）外，PCRE2 还支持单字母简写：
\d ≡ [[:digit:]]
\w ≡ [[:alnum:]_]
\s ≡ 空白字符等
5. 最小匹配（非贪婪匹配）
在任何量词后加 ? 可启用最小匹配：
a.*?b 在 "ababab" 中匹配首个 "ab"
a.*b 则匹配整个字符串
若设置了 U 标志，则默认为非贪婪模式，此时 ? 反而表示贪婪。
6. Unicode 字符属性
PCRE2 支持匹配 Unicode 定义的字符属性。例如：
\p{Ps}.*?\p{Pe}
可匹配以任意“开标点”开头、以“闭标点”结尾的字符串（如 [abc]）。
通过编译选项 PCRE2_UCP（或在模式开头写 (*UCP)），可让以下元字符的行为基于 Unicode 属性：
\w、\W、\d、\D、\s、\S、\b、\B
部分 POSIX 字符类
注意：启用 UCP 会降低匹配速度（因需查 Unicode 表），且要求库在编译时启用 Unicode 支持（PCRE2 默认开启）。
编码支持演进：
PCRE1 早期仅支持 ASCII
后续增加 UTF-8 支持
8.30 版加入 UTF-16
8.32 版加入 UTF-32
PCRE2 从诞生起即全面支持 UTF-8/16/32
7. 多行匹配
^ 和 $ 可配置为：
仅匹配整个字符串的首尾
或匹配每行的首尾（需启用多行模式）
8. 换行符/行断点选项
PCRE 编译时会设定默认换行符类型，影响：
^ / $ 的匹配位置（多行模式下）
. 能否匹配换行符（除非启用 (?s) dotall 模式）
匹配失败时的重试策略（自 7.0 起，若模式在换行序列开头失败，PCRE 会跳过整个换行序列再重试）
自 8.10 起，\N 始终匹配非换行字符（行为等同于未启用 (?s) 时的 .）。可在模式开头指定换行符类型：
(*LF)：仅 \n
(*CR)：仅 \r
(*CRLF)：\r\n
(*ANYCRLF)：上述任意一种
(*ANY)：包括 Unicode 行分隔符（U+2028 LS、U+2029 PS）
注意：在非 UTF-8 模式下，(*ANY) 还包含 \x0B（垂直制表）、\f（换页）、\x85（NEL）等。
9. \R 的匹配行为
\R 是通用换行符匹配符。其默认行为（ANYCRLF 或 ANY）可在编译时设定，也可在模式开头覆盖：
(*BSR_UNICODE)：匹配所有 Unicode 换行符
(*BSR_ANYCRLF)：仅匹配 CR、LF、CRLF
可与换行符选项组合使用，如：
(*BSR_UNICODE)(*ANY)rest-of-pattern
10. 模式开头选项
除上述外，还可设置：
(*UTF)：启用 UTF 模式（需库支持）
(*UCP)：启用 Unicode 属性匹配
11. 反向引用
模式可引用先前捕获的内容。例如：
(a|b)c\1
匹配 "aca" 或 "bcb"，但不匹配 "acb"。
12. 命名子模式
子模式可用 (?P<name>...) 命名（源自 Python）。Perl 后来也采纳了此特性，并扩展支持 (?<name>...) 和 (?'name'...)。
命名组可通过以下方式反向引用：
(?P=name)（Python 风格）
\k'name'（Perl 风格）
13. 子例程（Subroutines）
与反向引用不同，子例程复用子模式的定义而非匹配结果：
(a.c)(?1) 可匹配 "aacabc"（因 (?1) 相当于再次匹配 a.c）
而 (a.c)\1 要求两次匹配完全相同（如 "aacaac"）
PCRE 还支持 Oniguruma 风格的子例程：\g<1> 或 \g<name>。
14. 原子分组
原子分组可禁止回溯。例如：
a++bc
会尽可能多地匹配 "a"，且永不回退尝试更短匹配。
15. 前瞻与后顾断言
| 类型 | 后顾 | 前瞻 |
| 肯定 | (?<=pattern) | (?=pattern) |
| 否定 | (?<!pattern) | (?!pattern) |
这些是零宽断言，不消耗字符。例如：
\w+(?=\t)
匹配后跟制表符的单词，但不包含制表符本身。
注意：PCRE 要求后顾断言中每个分支长度固定（但不同分支长度可不同），而 Perl 要求所有分支长度一致。
此外，\K 可重置当前匹配的起始位置，提供比后顾断言更灵活的替代方案（因 \K 左侧部分无需固定长度）。
16. 零宽断言的简写
\b：单词边界，等价于 (?<=\W)(?=\w)|(?<=\w)(?=\W)|^|$
17. 注释
注释以 (?# 开始，到下一个 ) 结束。
18. 递归模式
模式可递归调用自身或子模式。例如：
\((a*|(?R))*\)
可匹配任意嵌套的括号与 "a" 的组合（如 "((a))"）。
19. 通用回调（Callouts）
通过 (?Cn) 可在匹配过程中调用外部用户函数，实现任意逻辑嵌入。
PCRE/PCRE2 是连接 C 语言世界与 Perl 强大正则能力的桥梁。PCRE2 作为现代化版本，在安全性、性能、Unicode 支持和 API 设计上全面超越旧版，是新项目的不二之选。
Regular Expressions (Regex)
Regular Expression, or regex or regexp in short, is extremely and amazingly powerful in searching and manipulating text strings, particularly in processing text files. One line of regex can easily replace several dozen lines of programming codes.
Regex is supported in all the scripting languages (such as Perl, Python, PHP, and JavaScript); as well as general purpose programming languages such as Java; and even word processors such as Word for searching texts. Getting started with regex may not be easy due to its geeky syntax, but it is certainly worth the investment of your time.
1. Regex By Examples
This section is meant for those who need to refresh their memory. For novices, go to the next section to learn the syntax, before looking at these examples.
1.1 Regex Syntax Summary
Character: All characters, except those having special meaning in regex, matches themselves. E.g., the regex x matches substring "x"; regex 9 matches "9"; regex = matches "="; and regex @ matches "@".
Special Regex Characters: These characters have special meaning in regex (to be discussed below): ., +, *, ?, ^, $, (, ), [, ], {, }, |, \.
Escape Sequences (\char):
To match a character having special meaning in regex, you need to use a escape sequence prefix with a backslash (\). E.g., \. matches "."; regex \+ matches "+"; and regex \( matches "(".
You also need to use regex \\ to match "\" (back-slash).
Regex recognizes common escape sequences such as \n for newline, \t for tab, \r for carriage-return, \nnn for a up to 3-digit octal number, \xhh for a two-digit hex code, \uhhhh for a 4-digit Unicode, \uhhhhhhhh for a 8-digit Unicode.
A Sequence of Characters (or String): Strings can be matched via combining a sequence of characters (called sub-expressions). E.g., the regex Saturday matches "Saturday". The matching, by default, is case-sensitive, but can be set to case-insensitive via modifier.
OR Operator (|): E.g., the regex four|4 accepts strings "four" or "4".
Character class (or Bracket List):
[...]: Accept ANY ONE of the character within the square bracket, e.g., [aeiou] matches "a", "e", "i", "o" or "u".
[.-.] (Range Expression): Accept ANY ONE of the character in the range, e.g., [0-9] matches any digit; [A-Za-z] matches any uppercase or lowercase letters.
[^...]: NOT ONE of the character, e.g., [^0-9] matches any non-digit.
Only these four characters require escape sequence inside the bracket list: ^, -, ], \.
Occurrence Indicators (or Repetition Operators):
+: one or more (1+), e.g., [0-9]+ matches one or more digits such as '123', '000'.
*: zero or more (0+), e.g., [0-9]* matches zero or more digits. It accepts all those in [0-9]+ plus the empty string.
?: zero or one (optional), e.g., [+-]? matches an optional "+", "-", or an empty string.
{m,n}: m to n (both inclusive)
{m}: exactly m times
{m,}: m or more (m+)
Metacharacters: matches a character
. (dot): ANY ONE character except newline. Same as [^\n]
\d, \D: ANY ONE digit/non-digit character. Digits are [0-9]
\w, \W: ANY ONE word/non-word character. For ASCII, word characters are [a-zA-Z0-9_]
\s, \S: ANY ONE space/non-space character. For ASCII, whitespace characters are [ \n\r\t\f]
Position Anchors: does not match character, but position such as start-of-line, end-of-line, start-of-word and end-of-word.
^, $: start-of-line and end-of-line respectively. E.g., ^[0-9]$ matches a numeric string.
\b: boundary of word, i.e., start-of-word or end-of-word. E.g., \bcat\b matches the word "cat" in the input string.
\B: Inverse of \b, i.e., non-start-of-word or non-end-of-word.
\<, \>: start-of-word and end-of-word respectively, similar to \b. E.g., \<cat\> matches the word "cat" in the input string.
\A, \Z: start-of-input and end-of-input respectively.
Parenthesized Back References:
Use parentheses ( ) to create a back reference.
Use $1, $2, ... (Java, Perl, JavaScript) or \1, \2, ... (Python) to retreive the back references in sequential order.
Laziness (Curb Greediness for Repetition Operators): *?, +?, ??, {m,n}?, {m,}?
1.2 Example: Numbers [0-9]+ or \d+
A regex (regular expression) consists of a sequence of sub-expressions. In this example, [0-9] and +.
The [...], known as character class (or bracket list), encloses a list of characters. It matches any SINGLE character in the list. In this example, [0-9] matches any SINGLE character between 0 and 9 (i.e., a digit), where dash (-) denotes the range.
The +, known as occurrence indicator (or repetition operator), indicates one or more occurrences (1+) of the previous sub-expression. In this case, [0-9]+ matches one or more digits.
A regex may match a portion of the input (i.e., substring) or the entire input. In fact, it could match zero or more substrings of the input (with global modifier).
This regex matches any numeric substring (of digits 0 to 9) of the input. For examples:
If the input is "abc123xyz", it matches substring "123".
If the input is "abcxyz", it matches nothing.
If the input is "abc00123xyz456_0", it matches substrings "00123", "456" and "0" (three matches).
Take note that this regex matches number with leading zeros, such as "000", "0123" and "0001", which may not be desirable.
You can also write \d+, where \d is known as a metacharacter that matches any digit (same as [0-9]). There are more than one ways to write a regex! Take note that many programming languages (C, Java, JavaScript, Python) use backslash \ as the prefix for escape sequences (e.g., \n for newline), and you need to write "\\d+" instead.
1.3 Code Examples (Python, Java, JavaScript, Perl, PHP)
Code Example in Python
See "Python's re module for Regular Expression" for full coverage.
Python supports Regex via module re. Python also uses backslash (\) for escape sequences (i.e., you need to write \\ for \, \\d for \d), but it supports raw string in the form of r'...', which ignore the interpretation of escape sequences - great for writing regex.
# Test under the Python Command-Line Interpreter
$ python3
......
>>> import re # Need module 're' for regular expression
# Try find: re.findall(regexStr, inStr) -> matchedSubstringsList
# r'...' denotes raw strings which ignore escape code, i.e., r'\n' is '\'+'n'
>>> re.findall(r'[0-9]+', 'abc123xyz')
['123'] # Return a list of matched substrings
>>> re.findall(r'[0-9]+', 'abcxyz')
[]
>>> re.findall(r'[0-9]+', 'abc00123xyz456_0')
['00123', '456', '0']
>>> re.findall(r'\d+', 'abc00123xyz456_0')
['00123', '456', '0']
# Try substitute: re.sub(regexStr, replacementStr, inStr) -> outStr
>>> re.sub(r'[0-9]+', r'*', 'abc00123xyz456_0')
'abc*xyz*_*'
# Try substitute with count: re.subn(regexStr, replacementStr, inStr) -> (outStr, count)
>>> re.subn(r'[0-9]+', r'*', 'abc00123xyz456_0')
('abc*xyz*_*', 3) # Return a tuple of output string and count
Code Example in Java
See "Regular Expressions (Regex) in Java" for full coverage.
Java supports Regex in package java.util.regex.
Code Example in Perl
See "Regular Expression (Regex) in Perl" for full coverage.
Perl makes extensive use of regular expressions with many built-in syntaxes and operators. In Perl (and JavaScript), a regex is delimited by a pair of forward slashes (default), in the form of /regex/. You can use built-in operators:
m/regex/modifier or /regex/modifier: Match against the regex. m is optional.
s/regex/replacement/modifier: Substitute matched substring(s) by the replacement.
In Perl, you can use single-quoted non-interpolating string '....' to write regex to disable interpretation of backslash (\) by Perl.
#!/usr/bin/env perl
use strict;
use warnings;
my $inStr = 'abc00123xyz456_0'; # input string
my $regex = '[0-9]+'; # regex pattern string in non-interpolating string
# Try match /regex/modifiers (or m/regex/modifiers)
my @matches = ($inStr =~ /$regex/g); # Match $inStr with regex with global modifier Store all matches in an array
print "@matches\n"; # Output: 00123 456 0
while ($inStr =~ /$regex/g) {
# The built-in array variables @- and @+ keep the start and end positions
# of the matches, where $-[0] and $+[0] is the full match, and
# $-[n] and $+[n] for back references $1, $2, etc.
print substr($inStr, $-[0], $+[0] - $-[0]), ', '; # Output: 00123, 456, 0,
}
print "\n";
# Try substitute s/regex/replacement/modifiers
$inStr =~ s/$regex/**/g; # with global modifier
print "$inStr\n"; # Output: abc**xyz**_**
Code Example in JavaScript
See "Regular Expression in JavaScript" for full coverage.
In JavaScript (and Perl), a regex is delimited by a pair of forward slashes, in the form of /.../. There are two sets of methods, issue via a RegEx object or a String object.
<!DOCTYPE html>
<!-- JSRegexNumbers.html -->
<html lang="en">
<head>
<meta charset="utf-8">
<title>JavaScript Example: Regex</title>
<script>
var inStr = "abc123xyz456_7_00";
// Use RegExp.test(inStr) to check if inStr contains the pattern
console.log(/[0-9]+/.test(inStr)); // true
// Use String.search(regex) to check if the string contains the pattern
// Returns the start position of the matched substring or -1 if there is no match
console.log(inStr.search(/[0-9]+/)); // 3
// Use String.match() or RegExp.exec() to find the matched substring,
// back references, and string index
console.log(inStr.match(/[0-9]+/)); // ["123", input:"abc123xyz456_7_00", index:3, length:"1"]
console.log(/[0-9]+/.exec(inStr)); // ["123", input:"abc123xyz456_7_00", index:3, length:"1"]
// With g (global) option
console.log(inStr.match(/[0-9]+/g)); // ["123", "456", "7", "00", length:4]
// RegExp.exec() with g flag can be issued repeatedly.
// Search resumes after the last-found position (maintained in property RegExp.lastIndex).
var pattern = /[0-9]+/g;
var result;
while (result = pattern.exec(inStr)) {
console.log(result);
console.log(pattern.lastIndex);
// ["123"], 6
// ["456"], 12
// ["7"], 14
// ["00"], 17
}
// String.replace(regex, replacement):
console.log(inStr.replace(/\d+/, "**")); // abc**xyz456_7_00
console.log(inStr.replace(/\d+/g, "**")); // abc**xyz**_**_**
</script>
</head>
<body>
<h1>Hello,</h1>
</body>
</html>
1.4 Example: Full Numeric Strings ^[0-9]+$ or ^\d+$
The leading ^ and the trailing $ are known as position anchors, which match the start and end positions of the line, respectively. As the result, the entire input string shall be matched fully, instead of a portion of the input string (substring).
This regex matches any non-empty numeric strings (comprising of digits 0 to 9), e.g., "0" and "12345". It does not match with "" (empty string), "abc", "a123", "abc123xyz", etc. However, it also matches "000", "0123" and "0001" with leading zeros.
1.5 Example: Positive Integer Literals [1-9][0-9]*|0 or [1-9]\d*|0
[1-9] matches any character between 1 to 9; [0-9]* matches zero or more digits. The * is an occurrence indicator representing zero or more occurrences. Together, [1-9][0-9]* matches any numbers without a leading zero.
| represents the OR operator; which is used to include the number 0.
This expression matches "0" and "123"; but does not match "000" and "0123" (but see below).
You can replace [0-9] by metacharacter \d, but not [1-9].
We did not use position anchors ^ and $ in this regex. Hence, it can match any parts of the input string. For examples:
If the input string is "abc123xyz", it matches the substring "123".
If the input string is "abcxyz", it matches nothing.
If the input string is "abc123xyz456_0", it matches substrings "123", "456" and "0" (three matches).
If the input string is "0012300", it matches substrings: "0", "0" and "12300" (three matches)!!!
1.6 Example: Full Integer Literals ^[+-]?[1-9][0-9]*|0$ or ^[+-]?[1-9]\d*|0$
This regex match an Integer literal (for entire string with the position anchors), both positive, negative and zero.
[+-] matches either + or - sign. ? is an occurrence indicator denoting 0 or 1 occurrence, i.e. optional. Hence, [+-]? matches an optional leading + or - sign.
We have covered three occurrence indicators: + for one or more, * for zero or more, and ? for zero or one.
1.7 Example: Identifiers (or Names) [a-zA-Z_][0-9a-zA-Z_]* or [a-zA-Z_]\w*
Begin with one letters or underscore, followed by zero or more digits, letters and underscore.
You can use metacharacter \w for a word character [a-zA-Z0-9_]. Recall that metacharacter \d can be used for a digit [0-9].
1.8 Example: Image Filenames ^\w+\.(gif|png|jpg|jpeg)$
The position anchors ^ and $ match the beginning and the ending of the input string, respectively. That is, this regex shall match the entire input string, instead of a part of the input string (substring).
\w+ matches one or more word characters (same as [a-zA-Z0-9_]+).
\. matches the dot (.) character. We need to use \. to represent . as . has special meaning in regex. The \ is known as the escape code, which restore the original literal meaning of the following character. Similarly, *, +, ? (occurrence indicators), ^, $ (position anchors) have special meaning in regex. You need to use an escape code to match with these characters.
(gif|png|jpg|jpeg) matches either "gif", "png", "jpg" or "jpeg". The | denotes "OR" operator. The parentheses are used for grouping the selections.
The modifier i after the regex specifies case-insensitive matching (applicable to some languages like Perl and JavaScript only). That is, it accepts "test.GIF" and "TesT.Gif".
1.9 Example: Email Addresses ^\w+([.-]?\w+)*@\w+([.-]?\w+)*(\.\w{2,3})+$
The position anchors ^ and $ match the beginning and the ending of the input string, respectively. That is, this regex shall match the entire input string, instead of a part of the input string (substring).
\w+ matches 1 or more word characters (same as [a-zA-Z0-9_]+).
[.-]? matches an optional character . or -. Although dot (.) has special meaning in regex, in a character class (square brackets) any characters except ^, -, ] or \ is a literal, and do not require escape sequence.
([.-]?\w+)* matches 0 or more occurrences of [.-]?\w+.
The sub-expression \w+([.-]?\w+)* is used to match the username in the email, before the @ sign. It begins with at least one word character [a-zA-Z0-9_], followed by more word characters or . or -. However, a . or - must follow by a word character [a-zA-Z0-9_]. That is, the input string cannot begin with . or -; and cannot contain "..", "--", ".-" or "-.". Example of valid string are "a.1-2-3".
The @ matches itself. In regex, all characters other than those having special meanings matches itself, e.g., a matches a, b matches b, and etc.
Again, the sub-expression \w+([.-]?\w+)* is used to match the email domain name, with the same pattern as the username described above.
The sub-expression \.\w{2,3} matches a . followed by two or three word characters, e.g., ".com", ".edu", ".us", ".uk", ".co".
(\.\w{2,3})+ specifies that the above sub-expression could occur one or more times, e.g., ".com", ".co.uk", ".edu.sg" etc.
Exercise: Interpret this regex, which provide another representation of email address: ^[\w\-\.\+]+\@[a-zA-Z0-9\.\-]+\.[a-zA-z0-9]{2,4}$.
1.10 Example: Swapping Words using Parenthesized Back-References ^(\S+)\s+(\S+)$ and $2 $1
The ^ and $ match the beginning and ending of the input string, respectively.
The \s (lowercase s) matches a whitespace (blank, tab \t, and newline \r or \n). On the other hand, the \S+ (uppercase S) matches anything that is NOT matched by \s, i.e., non-whitespace. In regex, the uppercase metacharacter denotes the inverse of the lowercase counterpart, for example, \w for word character and \W for non-word character; \d for digit and \D or non-digit.
The above regex matches two words (without white spaces) separated by one or more whitespaces.
Parentheses () have two meanings in regex:
to group sub-expressions, e.g., (abc)*
to provide a so-called back-reference for capturing and extracting matches.
The parentheses in (\S+), called parenthesized back-reference, is used to extract the matched substring from the input string. In this regex, there are two (\S+), match the first two words, separated by one or more whitespaces \s+. The two matched words are extracted from the input string and typically kept in special variables $1 and $2 (or \1 and \2 in Python), respectively.
To swap the two words, you can access the special variables, and print "$2 $1" (via a programming language); or substitute operator "s/(\S+)\s+(\S+)/$2 $1/" (in Perl).
Code Example in Python
Python keeps the parenthesized back references in \1, \2, .... Also, \0 keeps the entire match.
$ python3
>>> re.findall(r'^(\S+)\s+(\S+)$', 'apple orange')
[('apple', 'orange')] # A list of tuples if the pattern has more than one back references
# Back references are kept in \1, \2, \3, etc.
>>> re.sub(r'^(\S+)\s+(\S+)$', r'\2 \1', 'apple orange') # Prefix r for raw string which ignores escape
'orange apple'
>>> re.sub(r'^(\S+)\s+(\S+)$', '\\2 \\1', 'apple orange') # Need to use \\ for \ for regular string
'orange apple'
1.11 Example: HTTP Addresses ^http:\/\/\S+(\/\S+)*(\/)?$
Begin with http://. Take note that you may need to write / as \/ with an escape code in some languages (JavaScript, Perl).
Followed by \S+, one or more non-whitespaces, for the domain name.
Followed by (\/\S+)*, zero or more "/...", for the sub-directories.
Followed by (\/)?, an optional (0 or 1) trailing /, for directory request.
1.12 Example: Regex Patterns in AngularJS
1.13 Example: Sample Regex in Perl
s/^\s+// # Remove leading whitespaces (substitute with empty string)
s/\s+$// # Remove trailing whitespaces
s/^\s+.*\s+$// # Remove leading and trailing whitespaces
2. Regular Expression (Regex) Syntax
A Regular Expression (or Regex) is a pattern (or filter) that describes a set of strings that matches the pattern. In other words, a regex accepts a certain set of strings and rejects the rest.
A regex consists of a sequence of characters, metacharacters (such as ., \d, \D, \s, \S, \w, \W) and operators (such as +, *, ?, |, ^). They are constructed by combining many smaller sub-expressions.
2.1 Matching a Single Character
The fundamental building blocks of a regex are patterns that match a single character. Most characters, including all letters (a-z and A-Z) and digits (0-9), match itself. For example, the regex x matches substring "x"; z matches "z"; and 9 matches "9".
Non-alphanumeric characters without special meaning in regex also matches itself. For example, = matches "="; @ matches "@".
2.2 Regex Special Characters and Escape Sequences
Regex's Special Characters
These characters have special meaning in regex (I will discuss in detail in the later sections):
metacharacter: dot (.)
bracket list: [ ]
position anchors: ^, $
occurrence indicators: +, *, ?, { }
parentheses: ( )
or: |
escape and metacharacter: backslash (\)
Escape Sequences
The characters listed above have special meanings in regex. To match these characters, we need to prepend it with a backslash (\), known as escape sequence. For examples, \+ matches "+"; \[ matches "["; and \. matches ".".
Regex also recognizes common escape sequences such as \n for newline, \t for tab, \r for carriage-return, \nnn for a up to 3-digit octal number, \xhh for a two-digit hex code, \uhhhh for a 4-digit Unicode, \uhhhhhhhh for a 8-digit Unicode.
Code Example in Python
$ python3
>>> import re # Need module 're' for regular expression
# Try find: re.findall(regexStr, inStr) -> matchedStrList
# r'...' denotes raw strings which ignore escape code, i.e., r'\n' is '\'+'n'
>>> re.findall(r'a', 'abcabc')
['a', 'a']
>>> re.findall(r'=', 'abc=abc') # '=' is not a special regex character
['=']
>>> re.findall(r'\.', 'abc.com') # '.' is a special regex character, need regex escape sequence
['.']
>>> re.findall('\\.', 'abc.com') # You need to write \\ for \ in regular Python string
['.']
Code Example in JavaScript
Code Example in Java
2.3 Matching a Sequence of Characters (String or Text)
Sub-Expressions
A regex is constructed by combining many smaller sub-expressions or atoms. For example, the regex Friday matches the string "Friday". The matching, by default, is case-sensitive, but can be set to case-insensitive via modifier.
2.4 OR (|) Operator
You can provide alternatives using the "OR" operator, denoted by a vertical bar '|'. For example, the regex four|for|floor|4 accepts strings "four", "for", "floor" or "4".
2.5 Bracket List (Character Class) [...], [^...], [.-.]
A bracket expression is a list of characters enclosed by [ ], also called character class. It matches ANY ONE character in the list. However, if the first character of the list is the caret (^), then it matches ANY ONE character NOT in the list. For example, the regex [02468] matches a single digit 0, 2, 4, 6, or 8; the regex [^02468] matches any single character other than 0, 2, 4, 6, or 8.
Instead of listing all characters, you could use a range expression inside the bracket. A range expression consists of two characters separated by a hyphen (-). It matches any single character that sorts between the two characters, inclusive. For example, [a-d] is the same as [abcd]. You could include a caret (^) in front of the range to invert the matching. For example, [^a-d] is equivalent to [^abcd].
Most of the special regex characters lose their meaning inside bracket list, and can be used as they are; except ^, -, ] or \.
To include a ], place it first in the list, or use escape \].
To include a ^, place it anywhere but first, or use escape \^.
To include a - place it last, or use escape \-.
To include a \, use escape \\.
No escape needed for the other characters such as ., +, *, ?, (, ), {, }, and etc, inside the bracket list
You can also include metacharacters (to be explained in the next section), such as \w, \W, \d, \D, \s, \S inside the bracket list.
Name Character Classes in Bracket List (For Perl Only?)
Named (POSIX) classes of characters are pre-defined within bracket expressions. They are:
[:alnum:], [:alpha:], [:digit:]: letters+digits, letters, digits.
[:xdigit:]: hexadecimal digits.
[:lower:], [:upper:]: lowercase/uppercase letters.
[:cntrl:]: Control characters
[:graph:]: printable characters, except space.
[:print:]: printable characters, include space.
[:punct:]: printable characters, excluding letters and digits.
[:space:]: whitespace
For example, [[:alnum:]] means [0-9A-Za-z]. (Note that the square brackets in these class names are part of the symbolic names, and must be included in addition to the square brackets delimiting the bracket list.)
2.6 Metacharacters ., \w, \W, \d, \D, \s, \S
A metacharacter is a symbol with a special meaning inside a regex.
The metacharacter dot (.) matches any single character except newline \n (same as [^\n]). For example, ... matches any 3 characters (including alphabets, numbers, whitespaces, but except newline); the.. matches "there", "these", "the ", and so on.
\w (word character) matches any single letter, number or underscore (same as [a-zA-Z0-9_]). The uppercase counterpart \W (non-word-character) matches any single character that doesn't match by \w (same as [^a-zA-Z0-9_]).
In regex, the uppercase metacharacter is always the inverse of the lowercase counterpart.
\d (digit) matches any single digit (same as [0-9]). The uppercase counterpart \D (non-digit) matches any single character that is not a digit (same as [^0-9]).
\s (space) matches any single whitespace (same as [ \t\n\r\f], blank, tab, newline, carriage-return and form-feed). The uppercase counterpart \S (non-space) matches any single character that doesn't match by \s (same as [^ \t\n\r\f]).
Examples:
\s\s # Matches two spaces
\S\S\s # Two non-spaces followed by a space
\s+ # One or more spaces
\S+\s\S+ # Two words (non-spaces) separated by a space
2.7 Backslash (\) and Regex Escape Sequences
Regex uses backslash (\) for two purposes:
for metacharacters such as \d (digit), \D (non-digit), \s (space), \S (non-space), \w (word), \W (non-word).
to escape special regex characters, e.g., \. for ., \+ for +, \* for *, \? for ?. You also need to write \\ for \ in regex to avoid ambiguity.
Regex also recognizes \n for newline, \t for tab, etc.
Take note that in many programming languages (C, Java, Python), backslash (\) is also used for escape sequences in string, e.g., "\n" for newline, "\t" for tab, and you also need to write "\\" for \. Consequently, to write regex pattern \\ (which matches one \) in these languages, you need to write "\\\\" (two levels of escape!!!). Similarly, you need to write "\\d" for regex metacharacter \d. This is cumbersome and error-prone!!!
2.8 Occurrence Indicators (Repetition Operators): +, *, ?, {m}, {m,n}, {m,}
A regex sub-expression may be followed by an occurrence indicator (aka repetition operator):
?: The preceding item is optional and matched at most once (i.e., occurs 0 or 1 times or optional).
*: The preceding item will be matched zero or more times, i.e., 0+
+: The preceding item will be matched one or more times, i.e., 1+
{m}: The preceding item is matched exactly m times.
{m,}: The preceding item is matched m or more times, i.e., m+
{m,n}: The preceding item is matched at least m times, but not more than n times.
For example: The regex xy{2,4} accepts "xyy", "xyyy" and "xyyyy".
2.9 Modifiers
You can apply modifiers to a regex to tailor its behavior, such as global, case-insensitive, multiline, etc. The ways to apply modifiers differ among languages.
In Perl, you can attach modifiers after a regex, in the form of /.../modifiers. For examples:
m/abc/i # case-insensitive matching
m/abc/g # global (Match ALL instead of match first)
In Java, you apply modifiers when compiling the regex Pattern. For example:
Pattern p1 = Pattern.compile(regex, Pattern.CASE_INSENSITIVE); // for case-insensitive matching
Pattern p2 = Pattern.compile(regex, Pattern.MULTILINE); // for multiline input string
Pattern p3 = Pattern.compile(regex, Pattern.DOTALL); // Dot (.) matches all characters including newline
The commonly-used modifer modes are:
Case-Insensitive mode (or i): case-insensitive matching for letters.
Global (or g): match All instead of first match.
Multiline mode (or m): affect ^, $, \A and \Z. In multiline mode, ^ matches start-of-line or start-of-input; $ matches end-of-line or end-of-input, \A matches start-of-input; \Z matches end-of-input.
Single-line mode (or s): Dot (.) will match all characters, including newline.
Comment mode (or x): allow and ignore embedded comment starting with # till end-of-line (EOL).
more...
2.10 Greediness, Laziness and Backtracking for Repetition Operators
Greediness of Repetition Operators *, +, ?, {m,n}: The repetition operators are greedy operators, and by default grasp as many characters as possible for a match. For example, the regex xy{2,4} try to match for "xyyyy", then "xyyy", and then "xyy".
Lazy Quantifiers *?, +?, ??, {m,n}?, {m,}?, : You can put an extra ? after the repetition operators to curb its greediness (i.e., stop at the shortest match). For example,
input = "The <code>first</code> and <code>second</code> instances"
regex = <code>.*</code> matches "<code>first</code> and <code>second</code>"
But
regex = <code>.*?</code> produces two matches: "<code>first</code>" and "<code>second</code>"
Backtracking: If a regex reaches a state where a match cannot be completed, it backtracks by unwinding one character from the greedy match. For example, if the regex z*zzz is matched against the string "zzzz", the z* first matches "zzzz"; unwinds to match "zzz"; unwinds to match "zz"; and finally unwinds to match "z", such that the rest of the patterns can find a match.
Possessive Quantifiers *+, ++, ?+, {m,n}+, {m,}+: You can put an extra + to the repetition operators to disable backtracking, even it may result in match failure. e.g, z++z will not match "zzzz". This feature might not be supported in some languages.
2.11 Position Anchors ^, $, \b, \B, \<, \>, \A, \Z
Positional anchors DO NOT match actual character, but matches position in a string, such as start-of-line, end-of-line, start-of-word, and end-of-word.
^ and $: The ^ matches the start-of-line. The $ matches the end-of-line excluding newline, or end-of-input (for input not ending with newline). These are the most commonly-used position anchors. For examples:
ing$ # ending with 'ing'
^testing 123$ # Matches only one pattern. Should use equality comparison instead.
^[0-9]+$ # Numeric string
\b and \B: The \b matches the boundary of a word (i.e., start-of-word or end-of-word); and \B matches inverse of \b, or non-word-boundary. For examples,
\bcat\b # matches the word "cat" in input string "This is a cat." but does not match input "This is a catalog."
\< and \>: The \< and \> match the start-of-word and end-of-word, respectively (compared with \b, which can match both the start and end of a word).
\A and \Z: The \A matches the start of the input. The \Z matches the end of the input.
They are different from ^ and $ when it comes to matching input with multiple lines. ^ matches at the start of the string and after each line break, while \A only matches at the start of the string. $ matches at the end of the string and before each line break, while \Z only matches at the end of the string. For examples,
$ python3
# Using ^ and $ in multiline mode
>>> p1 = re.compile(r'^.+$', re.MULTILINE) # . for any character except newline
>>> p1.findall('testing\ntesting')
['testing', 'testing']
>>> p1.findall('testing\ntesting\n')
['testing', 'testing']
# ^ matches start-of-input or after each line break at start-of-line
# $ matches end-of-input or before line break at end-of-line
# newlines are NOT included in the matches
# Using \A and \Z in multiline mode
>>> p2 = re.compile(r'\A.+\Z', re.MULTILINE)
>>> p2.findall('testing\ntesting')
[] # This pattern does not match the internal \n
>>> p3 = re.compile(r'\A.+\n.+\Z', re.MULTILINE) # to match the internal \n
>>> p3.findall('testing\ntesting')
['testing\ntesting']
>>> p3.findall('testing\ntesting\n')
[] # This pattern does not match the trailing \n
# \A matches start-of-input and \Z matches end-of-input
2.12 Capturing Matches via Parenthesized Back-References & Matched Variables $1, $2, ...
Parentheses () serve two purposes in regex:
Firstly, parentheses () can be used to group sub-expressions for overriding the precedence or applying a repetition operator. For example, (abc)+ (accepts abc, abcabc, abcabcabc, ...) is different from abc+ (accepts abc, abcc, abccc, ...).
Secondly, parentheses are used to provide the so called back-references (or capturing groups). A back-reference contains the matched substring. For examples, the regex (\S+) creates one back-reference (\S+), which contains the first word (consecutive non-spaces) of the input string; the regex (\S+)\s+(\S+) creates two back-references: (\S+) and another (\S+), containing the first two words, separated by one or more spaces \s+.
These back-references (or capturing groups) are stored in special variables $1, $2, … (or \1, \2, ... in Python), where $1 contains the substring matched the first pair of parentheses, and so on. For example, (\S+)\s+(\S+) creates two back-references which matched with the first two words. The matched words are stored in $1 and $2 (or \1 and \2), respectively.
Back-references are important to manipulate the string. Back-references can be used in the substitution string as well as the pattern. For examples:
# Swap the first and second words separated by one space
s/(\S+) (\S+)/$2 $1/; # Perl
re.sub(r'(\S+) (\S+)', r'\2 \1', inStr) # Python
# Remove duplicate word
s/(\w+) $1/$1/; # Perl
re.sub(r'(\w+) \1', r'\1', inStr) # Python
2.13 (Advanced) Lookahead/Lookbehind, Groupings and Conditional
These feature might not be supported in some languages.
Positive Lookahead (?=pattern)
The (?=pattern) is known as positive lookahead. It performs the match, but does not capture the match, returning only the result: match or no match. It is also called assertion as it does not consume any characters in matching.
Negative Lookahead (?!pattern)
Inverse of (?=pattern). Match if pattern is missing. For example, a(?=b) matches 'a' in 'abc' (not consuming 'b'); but not 'acc'. Whereas a(?!b) matches 'a' in 'acc', but not abc.
Positive Lookbehind (?<=pattern)
Negative Lookbehind (?<!pattern)
Non-Capturing Group (?:pattern)
Recall that you can use Parenthesized Back-References to capture the matches. To disable capturing, use ?: inside the parentheses in the form of (?:pattern). In other words, ?: disables the creation of a capturing group, so as not to create an unnecessary capturing group.
Example:
Named Capturing Group (?<name>pattern)
The capture group can be referenced later by name.
Atomic Grouping (>pattern)
Disable backtracking, even if this may lead to match failure.
Conditional (?(Cond)then|else)
2.14 Unicode
The metacharacters \w, \W, (word and non-word character), \b, \B (word and non-word boundary) recongize Unicode characters.
使用示例
以下是一个简单的C语言示例，展示了如何使用PCRE库进行正则表达式匹配：
#include <stdio.h>
#include <pcre.h>
int main() {
const char *pattern = "freeoa";
const char *subject = "hello freeoa.net";
const char *error;
int erroffset;
pcre *re;
int ovector[30];
int rc;
re = pcre_compile(pattern, 0, &error, &erroffset, NULL);
if (re == NULL) {
printf("PCRE compilation failed at offset %d: %s\n", erroffset, error);
return 1;
}
rc = pcre_exec(re, NULL, subject, strlen(subject), 0, 0, ovector, 30);
if (rc < 0) {
switch(rc) {
case PCRE_ERROR_NOMATCH: printf("No match\n"); break;
default: printf("Matching error %d\n", rc); break;
}
pcre_free(re);
return 1;
}
printf("Match succeeded\n");
pcre_free(re);
return 0;
}
在此示例中首先使用 pcre_compile 函数编译正则表达式模式，然后使用 pcre_exec 函数进行匹配；如果匹配成功，程序将输出“Match succeeded”。
PCRE库是一个强大且灵活的正则表达式库，它提供了与Perl兼容的正则表达式功能，并且具有高效的性能和跨平台支持。无论是开发Web应用、系统工具还是其他类型的软件，它都是一个非常有用的工具。
最新版本：8.3
该版本支持 32 位的字符串和 UTF-32，提升了 \X Unicode 兼容性，Unicode 表升级到 6.2.0，修复了一些 bug 等。
官方主页：http://www.pcre.org/