|
REAL
Regular Expression Algorithmic Library — constexpr C++20 regex
|
Compiles an ast into a dynamic_program (NFA bytecode). More...
#include <compiler.hpp>
Public Member Functions | |
| constexpr | compiler (const ast &tree, flags compile_flags) |
| Binds the compiler to a parsed pattern and its flags. | |
| constexpr dynamic_program | compile () |
| Emits the full NFA program for the bound AST. | |
Static Public Member Functions | |
| static constexpr bool | inner_literal_starts_at (const dynamic_program &prog, std::size_t pc) |
True when the full inner literal starts at pc as consecutive byte ops. | |
| static constexpr bool | opens_on_continuation (const char_class &first_bytes) |
Whether a match can open on a UTF-8 continuation byte (10xxxxxx). | |
| static constexpr std::uint16_t | intern_cp_class (dynamic_program &prog, const class_def &cd) |
Interns cd into prog.cp_classes/prog.cp_ranges (deduplicating), returning its index. | |
Private Member Functions | |
| constexpr void | emit_any_codepoint_class (dynamic_program &prog, const char_class &ascii) const |
| Emits "one codepoint matching \p ascii, or any non-ASCII codepoint". | |
| constexpr void | emit_byte_sequences (dynamic_program &prog, const std::vector< std::vector< char_class > > &branches) const |
Emits an alternation of byte-range sequences as split/jump; each branch is a chain of klass steps and the leftmost matching branch wins. | |
| constexpr void | emit_class_codepoints (dynamic_program &prog, const char_class &ascii, const std::vector< code_range > &ranges) const |
| Emits a code-point class: the ASCII bitmap (if any) OR the canonical UTF-8 byte sequences of each code-point range. | |
| constexpr class_def | effective_class (const ast_node &node) const |
The class a node_kind::klass node effectively accepts, after negation, icase folding and the bytes/code-point split. The one source for both emit_node and l_max_bytes, so the emission and its measured width cannot disagree. | |
| constexpr class_def | finish_class (const ast_node &node, class_def folded) const |
| Applies negation (and its mode-dependent complement) to an already-folded class. | |
| constexpr void | emit_node (dynamic_program &prog, std::int32_t index, bool capture_free=false) const |
Emits the bytecode for the AST node at index (recursively). | |
| constexpr assert_kind | assert_kind_for (anchor_kind anchor, flags node_flags) const |
| Maps an AST anchor_kind to the runtime assert_kind. | |
| constexpr bool | fuse_single_atom_alternation (const ast_node &node, class_def &out) const |
| The class an alternation of single ATOMS is, when it is one. | |
| constexpr void | emit_effective_class (dynamic_program &prog, const class_def &eff) const |
| Emits an already-materialised class the one way this compiler emits classes. | |
| constexpr void | emit_alternation (dynamic_program &prog, const ast_node &node, bool capture_free) const |
| Emits an alternation: branches chained with leftmost-preferred splits. | |
| constexpr void | emit_unbounded_body (dynamic_program &prog, std::int32_t child, bool capture_free) const |
| Emits an UNBOUNDED quantifier's body, promoting a bare literal byte to a one-member byte class so the shape routes can see it. | |
| constexpr void | emit_repeat (dynamic_program &prog, const ast_node &node, bool capture_free) const |
| Emits a quantifier (Thompson construction). | |
| constexpr void | emit_lookaround (dynamic_program &prog, const ast_node &node, bool capture_free) const |
Emits a bounded lookaround: an assert_lookaround whose sub-program is a capture-free region the main flow jumps over. | |
| constexpr std::int32_t | l_max_bytes (std::int32_t index) const |
Upper bound, in bytes, on what the sub-AST at index can consume; -1 if unbounded (a *, + or {n,} repeat) or if it nests a lookaround. | |
| constexpr bool | is_single_atom (std::int32_t index) const |
Is index a bare, unwrapped single atom (a literal byte, a character class, or .)? | |
| constexpr bool | is_tier1_body (std::int32_t index) const |
Tier 1 eligibility: is index a bare single atom, or an ordinary (non-atomic) capturing group wrapping exactly one (X*+, (a)*+, (?>X*), …)? | |
| constexpr std::int32_t | tier1_atom (std::int32_t index) const |
The bare atom Tier 1 should test: index itself, or its single captured child when index is a capturing-group wrapper. is_tier1_body must hold. | |
| constexpr std::int32_t | tier1_capture_group (std::int32_t index) const |
| The capture group number Tier 1 should wrap the loop in, or -1 for none. is_tier1_body must hold. | |
| constexpr bool | is_deterministic (std::int32_t index) const |
Whether compiling the sub-AST at index emits no split reachable from the outer flow. | |
| constexpr std::int32_t | emit_tier1_atom_test (dynamic_program &prog, std::int32_t atom, std::int32_t capture_start_slot) const |
Emits a Tier 1 atom test (byte_loop_possessive / klass_loop_possessive / klass_cp_loop_possessive): secondary_target is a placeholder for the no-match exit, primary_target the capture start slot (-1 for none; the end slot is start + 1). | |
| constexpr void | emit_tier1_loop (dynamic_program &prog, std::int32_t atom, std::int32_t min, std::int32_t max, std::int32_t capture_group, bool capture_free) const |
| Emits a Tier 1 possessive loop over a single atom, optionally wrapped in one capturing group. | |
| constexpr void | emit_possessive_repeat (dynamic_program &prog, std::int32_t body, std::int32_t min, std::int32_t max, bool capture_free) const |
| Dispatches a possessive quantifier body to Tier 1 or a clean rejection; shared by emit_repeat and emit_atomic_group. | |
| constexpr void | emit_atomic_group (dynamic_program &prog, const ast_node &node, bool capture_free) const |
Emits an atomic group (?>...). | |
Static Private Member Functions | |
| static constexpr std::int32_t | here (const dynamic_program &prog) |
| Returns the index of the next instruction. | |
| static constexpr void | emit (dynamic_program &prog, instr instruction) |
| Appends one instruction, enforcing the program-size cap. | |
| static constexpr std::int32_t | emit_split (dynamic_program &prog) |
Emits a split with placeholder targets. | |
| static constexpr std::int32_t | emit_jump (dynamic_program &prog) |
Emits a jump with a placeholder target. | |
| static constexpr void | patch_primary (dynamic_program &prog, std::int32_t pc, std::int32_t target) |
Sets the primary branch target of the instruction at pc. | |
| static constexpr void | patch_secondary (dynamic_program &prog, std::int32_t pc, std::int32_t target) |
Sets the secondary branch target of the split at pc. | |
| static constexpr std::uint16_t | intern_class (dynamic_program &prog, const char_class &klass) |
Interns klass into prog.classes (deduplicating), returning its index. | |
| static constexpr void | emit_klass (dynamic_program &prog, const char_class &klass) |
Emits a klass instruction, interning klass through intern_class. | |
| static constexpr void | emit_klass_cp (dynamic_program &prog, const class_def &cd) |
Emits a code-point class in text mode: klass_cp, then three klass utf8_cont slots. klass_cp decodes one code point and, on membership, enters the chain at a computed skip. cd is already effective (fold and negation applied), so membership is a positive test. | |
| static constexpr bool | try_emit_fixed_width_class (dynamic_program &prog, const class_def &eff, bool probe_only=false) |
| Emits a small non-ASCII class as fixed-width bytes when every member encodes to the same length and they differ in exactly one byte position. Returns false otherwise. | |
Private Attributes | |
| std::array< std::int32_t, fold_cache_ways > | fold_key_ {-1, -1, -1, -1} |
| Cache tag per way: the (class index, fold mode, negated) key, or -1 for empty. | |
| std::array< class_def, fold_cache_ways > | fold_val_ {} |
| Finished (folded, coalesced, negated) class per way. | |
| const ast & | tree_ |
| The AST being compiled. | |
| flags | flags_ {flags::none} |
| Effective compilation flags. | |
Static Private Attributes | |
| static constexpr std::size_t | fold_cache_ways {4} |
| Ways of the effective_class fold cache; a miss only folds again. | |
| static constexpr std::size_t | fixed_width_class_max_members {8} |
| Most members a try_emit_fixed_width_class candidate may have. | |
Compiles an ast into a dynamic_program (NFA bytecode).
|
inlineconstexpr |
Binds the compiler to a parsed pattern and its flags.
| [in] | tree | The AST to compile (borrowed, must outlive the compiler). |
| [in] | compile_flags | The effective compilation flags. |
|
inlineconstexprprivate |
Maps an AST anchor_kind to the runtime assert_kind.
^ and $ depend on multiline (from the anchor's scope) and the global ecma flag; everything else maps one-to-one.
| [in] | anchor | The AST anchor kind. |
| [in] | node_flags | The flags in force at this anchor's scope. |
|
inlineconstexpr |
Emits the full NFA program for the bound AST.
| real::regex_error | if the program exceeds max_program_size. |
|
inlineconstexprprivate |
The class a node_kind::klass node effectively accepts, after negation, icase folding and the bytes/code-point split. The one source for both emit_node and l_max_bytes, so the emission and its measured width cannot disagree.
| [in] | node | The node_kind::klass node. |
|
inlinestaticconstexprprivate |
Appends one instruction, enforcing the program-size cap.
The check lives here so it fires during a large unroll, before the vector reaches the bad size: the defense against nested bounded quantifiers expanding to hundreds of millions of instructions. Exceeding the cap fails compilation of a static_regex, or throws at run time.
| [in,out] | prog | The program being built. |
| [in] | instruction | The instruction to append. |
| real::regex_error | when max_program_size would be exceeded. |
|
inlineconstexprprivate |
Emits an alternation: branches chained with leftmost-preferred splits.
Every branch but the last jumps to a shared exit, patched once at the end.
| [in,out] | prog | The program being built. |
| [in] | node | The node_kind::alternation node. |
| [in] | capture_free | Propagated to each branch (see emit_node). |
|
inlineconstexprprivate |
Emits "one codepoint matching \p ascii, or any non-ASCII codepoint".
The non-ASCII part goes through emit_class_codepoints, whose canonical splitting narrows the first continuation byte after 0xE0, 0xED, 0xF0 and 0xF4, so overlong and surrogate encodings never read as a code point. Flat lead/continuation classes would accept them.
| [in,out] | prog | The program being built. |
| [in] | ascii | The accepted ASCII bytes (non-ASCII is always included). |
|
inlineconstexprprivate |
Emits an atomic group (?>...).
repeat over a Tier 1 body ((?>X*), (?>(a)+)) becomes possessive whatever its own flag, tested before any width check so (?>[^"]*) compiles; same restrictions as emit_possessive_repeat.(?>ab)) never gives back, so ordinary emission is exact, even inside a lookaround.(?>ab|a)) inline emission would let a give-back reach the inner split: rejected.| [in,out] | prog | The program being built. |
| [in] | node | The node_kind::group node (possessive == true). |
| [in] | capture_free | Propagated to the body. |
|
inlineconstexprprivate |
Emits an alternation of byte-range sequences as split/jump; each branch is a chain of klass steps and the leftmost matching branch wins.
| [in,out] | prog | The program being built. |
| [in] | branches | One byte-range chain per branch, tried in order. |
|
inlineconstexprprivate |
Emits a code-point class: the ASCII bitmap (if any) OR the canonical UTF-8 byte sequences of each code-point range.
| [in,out] | prog | The program being built. |
| [in] | ascii | The class's ASCII bitmap; skipped when empty. |
| [in] | ranges | Its non-ASCII code-point ranges. |
|
inlineconstexprprivate |
Emits an already-materialised class the one way this compiler emits classes.
(?:é|à|è) and [éàè] must become the same program, so fusion emits through here. Do not special-case a standalone non-ASCII class: arm64 prefers klass_cp and x86 the fixed-width form by comparable margins, so no choice wins on both. After a literal both prefer fixed width.
| [in,out] | prog | The program being built. |
| [in] | eff | The effective class (ASCII bitmap + non-ASCII ranges). |
|
inlinestaticconstexprprivate |
Emits a jump with a placeholder target.
| [in,out] | prog | The program being built. |
|
inlinestaticconstexprprivate |
Emits a klass instruction, interning klass through intern_class.
| [in,out] | prog | The program being built. |
| [in] | klass | The class bitmap to match. |
| real::regex_error | if more than 65536 distinct classes are needed. |
|
inlinestaticconstexprprivate |
Emits a code-point class in text mode: klass_cp, then three klass utf8_cont slots. klass_cp decodes one code point and, on membership, enters the chain at a computed skip. cd is already effective (fold and negation applied), so membership is a positive test.
| [in,out] | prog | The program being built. |
| [in] | cd | The effective code-point class (ASCII bitmap + non-ASCII ranges). |
|
inlineconstexprprivate |
Emits a bounded lookaround: an assert_lookaround whose sub-program is a capture-free region the main flow jumps over.
Layout: assert_lookaround sub_id; jump AFTER; [sub-program] match; AFTER: …; only the sub-VM enters the region, at code_offset. The sub-pattern must be bounded (L_max in bytes ≤ max_lookaround_length): the linear-time guarantee.
| [in,out] | prog | The program being built. |
| [in] | node | The node_kind::lookaround node. |
| [in] | capture_free | True only when already inside a lookaround (rejected). |
| real::regex_error | on an unbounded or over-long sub-pattern, or nesting. |
|
inlineconstexprprivate |
Emits the bytecode for the AST node at index (recursively).
| [in,out] | prog | The program being built. |
| [in] | index | Index of the node in ast::nodes. |
| [in] | capture_free | When true, capturing groups emit no save ops — used inside a lookaround sub-program, whose captures do not participate in the overall match. |
|
inlineconstexprprivate |
Dispatches a possessive quantifier body to Tier 1 or a clean rejection; shared by emit_repeat and emit_atomic_group.
A compound body ((?:ab)*+) is out of scope: the thread list keeps one position per round, and a thread that fails inside a compound body dies without reaching an exit, so the loop could not offer its exit only once the body has definitively failed. A single atom fails within its own dispatch: the opcode is its own fail-redirect.
| [in,out] | prog | The program being built. |
| [in] | body | Index of the quantified body. |
| [in] | min | Minimum repetition count. |
| [in] | max | Maximum repetition count (-1 = unbounded). |
| [in] | capture_free | Whether captures are suppressed here (inside a lookaround). |
| real::regex_error | when body is not Tier 1 eligible, or capture_free is true: the lookaround sub-VM's dispatchers know only byte/klass/klass_cp, and would read a klass_cp_loop_possessive's index in the wrong class table. |
|
inlineconstexprprivate |
Emits a quantifier (Thompson construction).
Greedy prefers split.primary_target (enter the body); lazy swaps the branches. Counted forms unroll: min mandatory copies, then a loop (max == -1) or optional copies sharing one exit.
| [in,out] | prog | The program being built. |
| [in] | node | The node_kind::repeat node. |
| [in] | capture_free | Propagated to the body copies (see emit_node). |
|
inlinestaticconstexprprivate |
Emits a split with placeholder targets.
| [in,out] | prog | The program being built. |
|
inlineconstexprprivate |
Emits a Tier 1 atom test (byte_loop_possessive / klass_loop_possessive / klass_cp_loop_possessive): secondary_target is a placeholder for the no-match exit, primary_target the capture start slot (-1 for none; the end slot is start + 1).
The opcode writes both capture slots itself, on a match only: a leading save would fire on the failing attempt a possessive loop always makes last and tear the last good capture. The cp form keeps the three-slot continuation chain of emit_klass_cp.
| [in,out] | prog | The program being built. |
| [in] | atom | Index of the single-atom AST node (byte/klass/any). |
| [in] | capture_start_slot | The capture group's start slot, or -1 for none. |
secondary_target needs patching).
|
inlineconstexprprivate |
Emits a Tier 1 possessive loop over a single atom, optionally wrapped in one capturing group.
Mandatory copies (min) are ordinary emission: a failure there kills the thread. The optional tail is a self-loop (max == -1) or unrolled copies, each one emit_tier1_atom_test whose no-match exit is patched to the shared exit.
| [in,out] | prog | The program being built. |
| [in] | atom | Index of the single-atom body (byte, klass, or any). |
| [in] | min | Minimum repetition count. |
| [in] | max | Maximum repetition count (-1 = unbounded). |
| [in] | capture_group | Capture group number to wrap the loop in, or -1 for none. |
| [in] | capture_free | Whether captures are suppressed here (inside a lookaround). |
|
inlineconstexprprivate |
Emits an UNBOUNDED quantifier's body, promoting a bare literal byte to a one-member byte class so the shape routes can see it.
The class-loop recognizer matches klass only, so a bare a+ would miss every fast route that [a]+ takes (over an order of magnitude). Only for max == -1: a bounded a{3} keeps its bytes, a literal run the literal routes read.
| [in,out] | prog | The program being built. |
| [in] | child | The quantifier's body node. |
| [in] | capture_free | Propagated to emit_node. |
|
inlineconstexprprivate |
Applies negation (and its mode-dependent complement) to an already-folded class.
| [in] | node | The class node being emitted. |
| [in] | folded | Its class after any case fold. |
|
inlineconstexprprivate |
The class an alternation of single ATOMS is, when it is one.
(?:é|à|è) is [éàè], which the shape recognisers see and a split chain hides. Exact: every branch consumes one atom and none captures, so leftmost-first preference is unobservable. Both emit_alternation and emit_unbounded_body ask here; they differ only in how they emit.
| [in] | node | The alternation node. |
| [out] | out | The fused class, valid only when this returns true. |
true if every branch is one atom and there are at least two of them.
|
inlinestaticconstexprprivate |
Returns the index of the next instruction.
| [in] | prog | The program. |
|
inlinestaticconstexpr |
True when the full inner literal starts at pc as consecutive byte ops.
A byte equal to the literal's first byte is not the literal: in (?:a){2}ax the prefix's a would anchor the hint at the wrong distance, a silent false negative wherever the route runs (static_regex at any size, dynamic past the size floor; short dynamic subjects never reach it).
| [in] | prog | The program being built. |
| [in] | pc | Index of the candidate first byte op. |
|
inlinestaticconstexprprivate |
Interns klass into prog.classes (deduplicating), returning its index.
Shared by emit_klass and klass_loop_possessive. Identical bitmaps share one slot.
| [in,out] | prog | The program being built. |
| [in] | klass | The class bitmap to intern. |
prog.classes. | real::regex_error | if more than 65536 distinct classes are needed. |
|
inlinestaticconstexpr |
Interns cd into prog.cp_classes/prog.cp_ranges (deduplicating), returning its index.
Shared by emit_klass_cp and klass_cp_loop_possessive, for any effective class.
| [in,out] | prog | The program being built. |
| [in] | cd | The effective code-point class (ASCII bitmap + non-ASCII ranges). |
prog.cp_classes.
|
inlineconstexprprivate |
Whether compiling the sub-AST at index emits no split reachable from the outer flow.
Asked only for a one-shot atomic group (emit_atomic_group), which then has nothing to give back. Mirrors the emitted shape: an alternation always splits; a non-possessive repeat splits unless min == max; an atomic group (deterministic or rejected) and a lookaround (its own sub-region) are opaque to the outer flow.
| [in] | index | Index of the sub-AST node. |
true if compiling index introduces no split reachable from the outer flow.
|
inlineconstexprprivate |
Is index a bare, unwrapped single atom (a literal byte, a character class, or .)?
| [in] | index | Index of the sub-AST node. |
true if index is byte, klass, or any.
|
inlineconstexprprivate |
Tier 1 eligibility: is index a bare single atom, or an ordinary (non-atomic) capturing group wrapping exactly one (X*+, (a)*+, (?>X*), …)?
Such a loop fails within one opcode dispatch (see emit_possessive_repeat).
| [in] | index | Index of the sub-AST node. |
true if index is Tier 1 eligible.
|
inlineconstexprprivate |
Upper bound, in bytes, on what the sub-AST at index can consume; -1 if unbounded (a *, + or {n,} repeat) or if it nests a lookaround.
. counts 4 bytes outside bytes mode; a class counts its widest encoding; a literal byte, 1.
| [in] | index | Index of the sub-AST node. |
|
inlinestaticconstexpr |
Whether a match can open on a UTF-8 continuation byte (10xxxxxx).
| [in] | first_bytes | The pattern's possible first bytes. |
|
inlinestaticconstexprprivate |
Sets the primary branch target of the instruction at pc.
| [in,out] | prog | The program being built. |
| [in] | pc | Index of the split/jump to patch. |
| [in] | target | Instruction index to branch to. |
|
inlinestaticconstexprprivate |
Sets the secondary branch target of the split at pc.
| [in,out] | prog | The program being built. |
| [in] | pc | Index of the split to patch. |
| [in] | target | Instruction index to branch to. |
|
inlineconstexprprivate |
The bare atom Tier 1 should test: index itself, or its single captured child when index is a capturing-group wrapper. is_tier1_body must hold.
| [in] | index | The loop body's node index. |
|
inlineconstexprprivate |
The capture group number Tier 1 should wrap the loop in, or -1 for none. is_tier1_body must hold.
| [in] | index | The loop body's node index. |
|
inlinestaticconstexprprivate |
Emits a small non-ASCII class as fixed-width bytes when every member encodes to the same length and they differ in exactly one byte position. Returns false otherwise.
An icase accented letter (é/É = C3 A9/C3 89) becomes byte C3 plus a byte class: fixed width, so the prefilter's fixed-offset walk and the literal routes still apply. Deliberately narrow: a class needing mixed lengths ((?i)[a-z] gains 2- and 3-byte members) would need an alternation, slower than klass_cp. One length and one varying position never emit a branch.
| [in,out] | prog | The program being built. |
| [in] | eff | The effective class (ASCII bitmap + non-ASCII ranges). |
| [in] | probe_only | When true, answers whether the shape matches and emits nothing (emit_unbounded_body asks, as this form is wrong under an unbounded quantifier). |
true if the class was emitted here (or matches, when probing); false if the caller must emit it otherwise.
|
mutableprivate |
Cache tag per way: the (class index, fold mode, negated) key, or -1 for empty.
mutable: the emit path reaches effective_class through const members. Written only outside constant evaluation: MSVC's constant evaluator rejects an indeterminate subobject here, and a static_regex gains nothing from the cache (its budget is the fold's step count).