Recursive-descent parser: a pattern string in, an ast out. More...

#include <ast.hpp>

Classes
struct	loose_buf
	A loose-match key (lowercase, no `_`/`-`/space; UAX44-LM3-ish) built into a fixed buffer, so no heap or `<string>` is needed at parse time. A name longer than the buffer simply fails to match. More...

struct	property_table_result
	The result of parse_property_table — the property's code-point ranges, plus whether a leading `\p{^...}` caret was stripped (RE2/Perl negation-by-caret, e.g. `\p{^L}` == `\P{L}`). `caret` is XORed into the caller's own negation flag so the caret composes with `\P` / `[^...]` instead of overriding them. More...

struct	quoted_span
	What parse_quoted_span emitted: a bare pre-chained all-but-last prefix (`head`..`head_tail`, -1 when the span has fewer than two characters) plus the span's final atom `last` (-1 when the span is empty) — the caller's quantifier target. More...

struct	shorthand_spec
	The classification of a `\d \D \w \W \s \S` shorthand: its ASCII bitmap, its Unicode range table, and whether it is the negated (uppercase) form. More...

Public Member Functions
constexpr	parser (std::string_view pattern, flags initial_flags=flags::none)
	Binds the parser to a pattern and the constructor flags.

constexpr ast	parse ()
	Parses the whole pattern.

Private Member Functions
constexpr flags	current_flags () const
	The flag set in force at the current nesting level (the scope-stack top).

constexpr bool	is_verbose () const
	True when verbose mode (`re.X`) is in force here — read from the scope stack, so a scoped `(?x:...)` is honoured without a global flag read.

constexpr bool	is_ecma () const
	True in the ECMAScript grammar. `flags::ecma` is not scopable, so the scope-stack base always carries it; reading it here keeps the flag-scope ratchet's global-read count at its terminal state (no new `ecma_` member reads).

constexpr bool	is_icase () const
	True when icase (`re.I`) is in force at the current scope (a scoped `(?i:...)` honoured).

constexpr bool	is_ascii_mode () const
	True when ascii (`re.A`) is in force at the current scope (a scoped `(?a:...)` honoured).

constexpr void	skip_insignificant ()
	In verbose mode, consumes insignificant whitespace and `#` comments.

template<typename Error = regex_error>
constexpr void	fail (const char *message) const
	Aborts the parse with a real::regex_error at the current offset.

template<typename = void>
constexpr void	fail_unsupported (const char *message) const
	Like fail, but tags the error as `unsupported` (well-formed but beyond REAL's linear engine — a backreference, `\p{…}`, a nested lookaround) so a binding can classify it without matching on the message text. Templated like fail so it stays a valid `constexpr`.

constexpr bool	eof () const
	Returns `true` if the read offset is at or past the end of the pattern.

constexpr char	peek () const
	Returns the current character without consuming it (undefined at eof()).

constexpr bool	accept (char ch)
	Consumes the current character if it equals `ch`.

constexpr std::int32_t	add_node (ast &out, ast_node node)
	Appends `node` to the pool.

constexpr std::int32_t	add_class_node (ast &out, const char_class &klass, bool negated, const std::vector< code_range > &ranges={}, bool codepoint_predicate=false)
	Interns a class bitmap and appends a node_kind::klass node.

constexpr bool	text_shorthand () const
	Whether a shorthand (`\d \w \s`) should be a text-mode Unicode code-point predicate: true in the default text mode, false in bytes mode or under `flags::ascii` (`re.A`).

constexpr void	merge_property (char_class &klass, std::vector< code_range > &ranges, const char_class &prop_ascii, std::span< const code_range > table, bool negated, bool &property_derived) const
	Merges an in-class shorthand (`\w \d \s` or a negated `\W \D \S`) into the class being built: its ASCII bitmap (or the complement, negated) plus, in text mode, its non-ASCII ranges (or their complement). Sets `property_derived` so the class is emitted as a match-time `klass_cp` (text mode only). In bytes / ASCII mode it stays a byte class.

constexpr std::vector< code_range >	shorthand_ranges (std::span< const code_range > table) const
	The non-ASCII part of a shorthand's range table, or nothing in bytes / ASCII mode. Wholly-ASCII ranges are dropped: the bitmap already covers them.

constexpr std::vector< code_range >	resolve_property (std::string_view name) const
	Resolves a `\p{...}` property name to its code-point ranges, or fails with a clear error. An optional `gc=` / `sc=` / `scx=` (or `general_category=` / `script=` / `scriptextensions=`) prefix picks the namespace; a bare name tries General_Category, then Script, then a binary property (`\p{Alphabetic}`, no namespace of its own, same as PCRE2) – `scx=` has no bare-name form (PCRE2: a bare name never means Script_Extensions, the explicit prefix is required). GC ranges come straight from the table; a Script's ranges are collected from the partition; a binary property's or a Script_Extensions' ranges come straight from their own table (both are NOT partitions – a code point can satisfy several). The alias resolvers are the generated, loose-keyed `resolve_gc` / `resolve_script` (shared by `sc=` and `scx=` – same script names, long or short UAX24 code) / `resolve_binprop`.

constexpr property_table_result	parse_property_table ()
	Rejects bytes mode, consumes the `p`/`P` and the `{Name}` (or single letter), strips a leading `^` caret-negation (native dialects only), and resolves the remaining name to the property's code-point ranges. Shared by the out-of-class atom and the in-class merge. On entry `pos_` is on the `p`/`P`; on return it is just past the name (caret and all).

constexpr void	property_ascii_high (const std::vector< code_range > &table, char_class &ascii, std::vector< code_range > &high) const
	Splits a property's ranges into its ASCII bitmap (< 0x80) and its non-ASCII ranges. Unconditional: unlike `\w`, `flags::ascii` (`re.A`) does not restrict a Unicode property, so both parts are always used (bytes mode having already been rejected).

constexpr std::int32_t	parse_unicode_property (ast &out, bool negated)
	Parses `\p{Name}` / `\P{Name}` / `\pX` (outside a class) into a negatable Unicode code-point class (`klass_cp`), reusing the same match-time mechanism as `\w`. Negation is the class-node flag, as for `\W`, XORed with a caret-negation `\p{^Name}` stripped by parse_property_table (so `\P{^L}` negates twice back to `\p{L}`, same as `\P{...}` on an already-negated property would). `pos_` is on the letter after `\`; `negated` distinguishes `\P` from `\p`.

constexpr void	merge_unicode_property (char_class &klass, std::vector< code_range > &ranges, const std::vector< code_range > &table, bool negated, bool &property_derived) const
	Merges a `\p{Name}` / `\P{Name}` property into the character class being built (the in-class form) — the un-gated twin of merge_property — `flags::ascii` never restricts it, so it always uses the property's own non-ASCII ranges. A negated `\P{...}` merges the complement (the inverted ASCII bitmap plus the gaps between the non-ASCII ranges), exactly as `\W` negates in a class; an enclosing `[^...]` then negates the whole class on top (so `[^\P{L}]` == `[\p{L}]`). bytes mode is already rejected by parse_property_table.

constexpr std::int32_t	parse_alternation (ast &out)
	Parses ‘alternation := sequence (’\|' sequence)*`.

constexpr std::int32_t	parse_sequence (ast &out)
	Parses `sequence := (atom quantifier?)*`, stopping at `\|` or `)`.

constexpr quoted_span	parse_quoted_span (ast &out)
	Scans a `\Q...\E` literal span (the `\Q` is already consumed) and emits its characters as literal atoms — the same emission as parse_atom's default (whole code point per atom in text mode, single byte in bytes mode, icase folding via emit_literal_codepoint), libre2-measured semantics:

constexpr std::int32_t	parse_atom (ast &out)
	Parses one atom: a literal, `.`, a class, a group, an anchor or an escape.

constexpr std::int32_t	parse_quantifier (ast &out, std::int32_t atom)
	Wraps `atom` in a repeat node if a quantifier follows.

constexpr bool	try_parse_braces (std::int32_t &min, std::int32_t &max)
	Tries to parse `{n} / {n,} / {,m} / {n,m}` starting at `{`.

constexpr std::int32_t	parse_repeat_count ()
	Reads an optional decimal repeat count.

constexpr void	expect (char ch, const char *message)
	Consumes `ch` or fails.

constexpr bool	parse_global_flags_prefix (ast &out)
	Consumes a leading global-flags group – `(?imsxaU)`, or `(?flags-flags)` with a removal suffix – if present. The accepted letters are `i m s x a U` (is_flag_letter).

constexpr std::int32_t	parse_group (ast &out)
	Parses a group construct.

constexpr std::int32_t	parse_lookaround (ast &out, look_dir direction, std::size_t open_pos)
	Parses a lookaround after `(?=` / `(?!` (ahead) or `(?<=` / `(?<!` (behind) — the `=`/`!` is not yet consumed.

constexpr std::int32_t	parse_atomic_group (ast &out, std::size_t open_pos)
	Parses an atomic group after `(?>` (the `>` is not yet consumed).

constexpr std::int32_t	new_group (ast &out, std::size_t open_pos)
	Allocates the next capture group number.

constexpr void	parse_group_name (ast &out, std::int32_t group)
	Parses ‘name := [A-Za-z_][A-Za-z0-9_]* ’>'` and records it.

constexpr std::int32_t	parse_byte_escape ()
	Parses a single-byte escape (valid inside and outside classes).

constexpr std::int32_t	parse_digit_escape ()
	Parses a `\<digit>` escape via the shared decode_digit_escape().

constexpr std::int32_t	hex_digit ()
	Consumes one hexadecimal digit.

constexpr std::int32_t	parse_unicode_codepoint (bool capital)
	Decodes a `\uHHHH` (4 hex) or `\UHHHHHHHH` (8 hex) code-point escape (str only).

constexpr std::int32_t	parse_braced_hex_scalar ()
	Decodes a braced hex scalar `HHHHHH}` (1–6 hex digits, then the closing `}`) — the code- point reader shared by `\N{U+XXXX}` (after its own `U+` prefix) and `\x{XXXX}` (after its own bytes-mode check, see parse_braced_hex_escape). The opening `{` is already consumed by the caller; this reads the hex digits, the closing `}`, and rejects a surrogate (U+D800–U+DFFF) or a value beyond U+10FFFF — the same code-point range `\u`/`\U` enforce (Python semantics).

constexpr std::int32_t	parse_named_codepoint ()
	Decodes a `\N{U+XXXX}` named-code-point escape (1–6 hex digits) — the same code-point path as `\u`/`\U`, spelled by its U+ scalar value. `re` writes `\N{NAME}` for the name; the Python binding rewrites a name to this `U+XXXX` form before parsing, so the engine only ever sees the scalar. A C++ caller writes `\N{U+XXXX}` directly.

constexpr std::int32_t	parse_braced_hex_escape ()
	Decodes a `\x{XXXX}` braced code-point escape — RE2/Perl syntax (ECMAScript spells this `\u{...}` instead, so every caller gates on `!is_ecma()` before reaching here). Rejected in bytes mode, like `\u`/`\U`/`\N` (no code-point meaning there) — read from the scope stack, not the global `bytes_` member (the flag-scope ratchet: bytes is never scoped, so this equals `bytes_` while keeping the parser's global-read count flat; same precedent as `\C` above). Shares its digit-loop / surrogate / overflow validation with `\N{U+XXXX}` via parse_braced_hex_scalar — this function only adds the bytes-mode check and the opening `{`. The backslash and `x` are already consumed by the caller.

constexpr std::int32_t	emit_codepoint_utf8 (ast &out, std::int32_t cp)
	Emits a code point as its 1–4 UTF-8 bytes — the same byte-level form a literal multi-byte character produces — as a single atom (a byte node, or a concat).

constexpr std::int32_t	emit_literal_codepoint (ast &out, std::int32_t cp)
	Emits a code-point literal (code-point provenance: a raw character or `\\u`/`\\U`).

constexpr std::int32_t	parse_escape (ast &out)
	Parses an escape outside a character class.

constexpr std::int32_t	parse_class_item (char_class &klass, std::vector< code_range > &ranges, bool &property_derived)
	Parses one member inside a character class.

constexpr std::int32_t	parse_class (ast &out)
	Parses a bracketed character class `[...]` or `[^...]`.

Static Private Member Functions
static constexpr bool	is_ascii_alnum (char ch)
	Returns `true` if `ch` is in `[0-9A-Za-z]`.

static constexpr char_class	space_set_text_ascii_component ()
	`\s`'s ASCII-range (< 0x80) component for TEXT mode: `space_set()` (the ASCII-MODE set, `[ \t\n\r\f\v]`) plus `U+001C`-`U+001F` (FS/GS/RS/US). Needed because `shorthand_ranges` deliberately omits any wholly-ASCII range from `.ranges` ("already covered by the bitmap") — so for text mode, where `re`'s own `\s` DOES include FS/GS/RS/US (verified: `re.match(r"\s", "\x1c")` matches; `str.isspace()` agrees) but ASCII-mode `\s` does not (`re.match(r"(?a)\s", "\x1c")` does not match), the bitmap that "already covers" the ASCII range must itself differ by mode — `space_set()` alone is only correct for the ASCII-mode case. Found live by differential fuzzing (ASCII mode wrongly matching FS/GS/ RS/US) and fixed once at `space_set()` itself before this second bug (text mode then losing them entirely) surfaced immediately in `test_classes.cpp`'s own regression suite — the two modes generate genuinely different ASCII bitmaps, not one shared one.

static constexpr shorthand_spec	shorthand_class (char letter, bool text_mode)
	Maps a shorthand letter to its shorthand_spec. The single place the letter -> (set, range table, negation) fact lives; the atom ladder (parse_escape) and the class ladder (parse_class_item) share it, then each consumes the spec its own way (emit a class node vs merge into a class) – the same shared-decode / divergent-use split as decode_digit_escape.

static constexpr loose_buf	loose_key (std::string_view s)
	Loose-matches a property name (UAX44-LM3): drops `_`, `-` and spaces, lowercases the rest.

static constexpr flags	flag_for_letter (char letter)
	Maps a flag letter to its flags value.

static constexpr bool	is_flag_letter (char letter)
	Returns `true` if `letter` is a flag letter (imsaxU).

static constexpr flags	without (flags value, flags bit)
	`value` with `bit` cleared. The intermediate cast matches the enum's `std::uint16_t` underlying type — a `std::uint8_t` here (the pre-widening vestige) would silently drop `flags::ungreedy` (512) from every scope.

static constexpr bool	is_name_start (char ch)
	Returns `true` if `ch` may start a group name.

Private Attributes
std::string_view	pattern_
	The pattern being parsed.

std::size_t	pos_ {}
	Current read offset into pattern_.

std::int32_t	depth_ {}
	Current group nesting (see max_nesting_depth).

std::vector< flags >	flag_scopes_
	Stack of the flag set in force per nesting level; the top is current. Replaces a global `verbose_` read so a scoped `(?x:...)` is honoured (see current_flags).

bool	in_lookaround_ {}
	True while parsing a lookaround sub-pattern (rejects nesting).

bool	bytes_ {}
	In flags::bytes mode, rejects code-point escapes (`\u`/`\U`).

bool	ecma_ {}
	ECMAScript grammar: `\A \Z \< \>` are identity-escape literals, not anchors.

Detailed Description

Recursive-descent parser: a pattern string in, an ast out.

Constructor & Destructor Documentation

◆ parser()

constexpr real::detail::parser::parser	(	std::string_view	pattern,
		flags	initial_flags = `flags::none`
	)

inlineexplicitconstexpr

Binds the parser to a pattern and the constructor flags.

Parameters

[in]	pattern	The pattern text (borrowed, must outlive use).
[in]	initial_flags	Flags from the constructor; only `verbose` affects parsing (a leading `(?x)` can add it too).

Member Function Documentation

◆ accept()

constexpr bool real::detail::parser::accept ( char ch )

inlineconstexprprivate

Consumes the current character if it equals ch.

Parameters

[in] ch The character to match.

Returns: true (and advances) on a match, else false.

◆ add_class_node()

constexpr std::int32_t real::detail::parser::add_class_node	(	ast &	out,
		const char_class &	klass,
		bool	negated,
		const std::vector< code_range > &	ranges = `{}`,
		bool	codepoint_predicate = `false`
	)

inlineconstexprprivate

Interns a class bitmap and appends a node_kind::klass node.

Parameters

[in,out]	out	The AST being built.
[in]	klass	The class bitmap as written (before negation).
[in]	negated	Whether the class was written negated.
[in]	ranges	Non-ASCII code-point ranges of the class (code-point mode; empty otherwise).
[in]	codepoint_predicate	Emit as a match-time `klass_cp` (a text-mode Unicode shorthand), not the byte-NFA.

Returns: The index of the new node.

◆ add_node()

constexpr std::int32_t real::detail::parser::add_node	(	ast &	out,
		ast_node	node
	)

inlineconstexprprivate

Appends node to the pool.

Parameters

[in,out]	out	The AST being built.
[in]	node	The node to append.

Returns: The index of the appended node.

◆ current_flags()

constexpr flags real::detail::parser::current_flags ( ) const

inlineconstexprprivate

The flag set in force at the current nesting level (the scope-stack top).

Returns: The active flags.

◆ emit_codepoint_utf8()

constexpr std::int32_t real::detail::parser::emit_codepoint_utf8	(	ast &	out,
		std::int32_t	cp
	)

inlineconstexprprivate

Emits a code point as its 1–4 UTF-8 bytes — the same byte-level form a literal multi-byte character produces — as a single atom (a byte node, or a concat).

Parameters

[in,out]	out	The AST being built.
[in]	cp	A code point in `[0, 0x10FFFF]`.

Returns: The node index.

◆ emit_literal_codepoint()

constexpr std::int32_t real::detail::parser::emit_literal_codepoint	(	ast &	out,
		std::int32_t	cp
	)

inlineconstexprprivate

Emits a code-point literal (code-point provenance: a raw character or \\u/\\U).

Under icase, a CASED literal is promoted to a foldable singleton class so the compiler folds it to its whole case orbit (k↦{k, K, Kelvin}, é↦{é, É}). An ASCII letter folds in any mode; a non-ASCII code point folds only in text mode (a bytes class carries no ranges). A non-cased literal, or no icase, keeps the zero-overhead byte / UTF-8 path. \\xHH has byte provenance and never routes here, so it is never folded — the deliberate provenance split.

Parameters

[in,out]	out	The AST the node is added to.
[in]	cp	The literal's code point.

Returns: The new node's index: a literal, or a foldable singleton class under icase.

◆ eof()

constexpr bool real::detail::parser::eof ( ) const

inlineconstexprprivate

Returns true if the read offset is at or past the end of the pattern.

Returns: Whether the pattern is exhausted.

◆ expect()

constexpr void real::detail::parser::expect	(	char	ch,
		const char *	message
	)

inlineconstexprprivate

Consumes ch or fails.

Parameters

[in]	ch	The required character.
[in]	message	Error message if `ch` is not present.

Exceptions

real::regex_error when the next character is not ch.

◆ fail()

template<typename Error = regex_error>

constexpr void real::detail::parser::fail ( const char * message ) const

inlineconstexprprivate

Aborts the parse with a real::regex_error at the current offset.

A template so the always-throwing body stays legal inside a constexpr function (the ill-formed, no-diagnostic-required rule does not apply to templates); during constant evaluation the throw fails compilation with message in the diagnostic trace.

Template Parameters

Error The exception type to throw (defaults to regex_error).

Parameters

[in] message The cause, shown in the error and the constexpr trace.

◆ fail_unsupported()

template<typename = void>

constexpr void real::detail::parser::fail_unsupported ( const char * message ) const

inlineconstexprprivate

Like fail, but tags the error as unsupported (well-formed but beyond REAL's linear engine — a backreference, \p{…}, a nested lookaround) so a binding can classify it without matching on the message text. Templated like fail so it stays a valid constexpr.

Parameters

[in] message The diagnostic text, reported at the current read offset.

◆ flag_for_letter()

static constexpr flags real::detail::parser::flag_for_letter ( char letter )

inlinestaticconstexprprivate

Maps a flag letter to its flags value.

Parameters

[in] letter One of 'i', 'm', 's', 'x', 'a', 'U'.

Returns: The flag; flags::none for any unrecognized letter.

◆ hex_digit()

constexpr std::int32_t real::detail::parser::hex_digit ( )

inlineconstexprprivate

Consumes one hexadecimal digit.

Returns: Its value in [0, 15].

Exceptions

real::regex_error if the next character is not a hex digit.

◆ is_ascii_alnum()

static constexpr bool real::detail::parser::is_ascii_alnum ( char ch )

inlinestaticconstexprprivate

Returns true if ch is in [0-9A-Za-z].

Parameters

[in] ch A character.

Returns: true if ch is in [0-9A-Za-z].

◆ is_ascii_mode()

constexpr bool real::detail::parser::is_ascii_mode ( ) const

inlineconstexprprivate

True when ascii (re.A) is in force at the current scope (a scoped (?a:...) honoured).

Returns: Whether flags::ascii is active here.

◆ is_ecma()

constexpr bool real::detail::parser::is_ecma ( ) const

inlineconstexprprivate

True in the ECMAScript grammar. flags::ecma is not scopable, so the scope-stack base always carries it; reading it here keeps the flag-scope ratchet's global-read count at its terminal state (no new ecma_ member reads).

Returns: Whether flags::ecma is active.

◆ is_flag_letter()

static constexpr bool real::detail::parser::is_flag_letter ( char letter )

inlinestaticconstexprprivate

Returns true if letter is a flag letter (imsaxU).

Parameters

[in] letter A character.

Returns: true if letter is a flag letter (imsaxU).

◆ is_icase()

constexpr bool real::detail::parser::is_icase ( ) const

inlineconstexprprivate

True when icase (re.I) is in force at the current scope (a scoped (?i:...) honoured).

Returns: Whether flags::icase is active here.

◆ is_name_start()

static constexpr bool real::detail::parser::is_name_start ( char ch )

inlinestaticconstexprprivate

Returns true if ch may start a group name.

Parameters

[in] ch A character.

Returns: true if ch may start a group name.

◆ is_verbose()

constexpr bool real::detail::parser::is_verbose ( ) const

inlineconstexprprivate

True when verbose mode (re.X) is in force here — read from the scope stack, so a scoped (?x:...) is honoured without a global flag read.

Returns: Whether flags::verbose is active here.

◆ loose_key()

static constexpr loose_buf real::detail::parser::loose_key ( std::string_view s )

inlinestaticconstexprprivate

Loose-matches a property name (UAX44-LM3): drops _, - and spaces, lowercases the rest.

Parameters

[in] s The name as written in the pattern.

Returns: The normalised key, truncated to the buffer's capacity.

◆ merge_property()

constexpr void real::detail::parser::merge_property	(	char_class &	klass,
		std::vector< code_range > &	ranges,
		const char_class &	prop_ascii,
		std::span< const code_range >	table,
		bool	negated,
		bool &	property_derived
	)		const

inlineconstexprprivate

Merges an in-class shorthand (\w \d \s or a negated \W \D \S) into the class being built: its ASCII bitmap (or the complement, negated) plus, in text mode, its non-ASCII ranges (or their complement). Sets property_derived so the class is emitted as a match-time klass_cp (text mode only). In bytes / ASCII mode it stays a byte class.

Parameters

[in,out]	klass	The class being built, receiving the ASCII bitmap.
[in,out]	ranges	The class's non-ASCII ranges, appended to in text mode.
[in]	prop_ascii	The shorthand's ASCII bitmap.
[in]	table	The shorthand's full Unicode range table.
[in]	negated	True for the uppercase form (`\W \D \S`).
[out]	property_derived	Set when the class must be emitted as a `klass_cp`.

◆ merge_unicode_property()

constexpr void real::detail::parser::merge_unicode_property	(	char_class &	klass,
		std::vector< code_range > &	ranges,
		const std::vector< code_range > &	table,
		bool	negated,
		bool &	property_derived
	)		const

inlineconstexprprivate

Merges a \p{Name} / \P{Name} property into the character class being built (the in-class form) — the un-gated twin of merge_property — flags::ascii never restricts it, so it always uses the property's own non-ASCII ranges. A negated \P{...} merges the complement (the inverted ASCII bitmap plus the gaps between the non-ASCII ranges), exactly as \W negates in a class; an enclosing [^...] then negates the whole class on top (so [^\P{L}] == [\p{L}]). bytes mode is already rejected by parse_property_table.

Parameters

[in,out]	klass	The class being built, receiving the ASCII bitmap.
[in,out]	ranges	The class's non-ASCII ranges, appended to.
[in]	table	The property's full range table.
[in]	negated	True for `\P{...}`, merging the complement.
[out]	property_derived	Set so the class is emitted as a `klass_cp`.

◆ new_group()

constexpr std::int32_t real::detail::parser::new_group	(	ast &	out,
		std::size_t	open_pos
	)

inlineconstexprprivate

Allocates the next capture group number.

Parameters

[in,out]	out	The AST being built.
[in]	open_pos	Offset of the group's `(` (for error reporting).

Returns: The new (1-based) capture group number.

Exceptions

real::regex_error beyond max_group_count.

◆ parse()

constexpr ast real::detail::parser::parse ( )

inlineconstexpr

Parses the whole pattern.

Returns: The resulting ast.

Exceptions

real::regex_error on any unsupported or malformed syntax.

◆ parse_alternation()

constexpr std::int32_t real::detail::parser::parse_alternation ( ast & out )

inlineconstexprprivate

Parses ‘alternation := sequence (’|' sequence)*`.

The leftmost branch is preferred (Python / Perl semantics, not longest).

Parameters

[in,out] out The AST being built.

Returns: The index of the resulting node (a branch, or a bare sequence).

◆ parse_atom()

constexpr std::int32_t real::detail::parser::parse_atom ( ast & out )

inlineconstexprprivate

Parses one atom: a literal, ., a class, a group, an anchor or an escape.

Parameters

[in,out] out The AST being built.

Returns: The index of the atom node.

◆ parse_atomic_group()

constexpr std::int32_t real::detail::parser::parse_atomic_group	(	ast &	out,
		std::size_t	open_pos
	)

inlineconstexprprivate

Parses an atomic group after (?> (the > is not yet consumed).

Builds a node_kind::group node with possessive = true and group = -1 (atomic groups are never capturing at their own level, exactly like (?:...); a numbered capture group written inside one still gets its own number and stays visible after the atomic group closes — the parser does not special-case this, since it never restricts capture numbering inside the body). Compile-time linearity/support restrictions (deterministic-body tiers) are enforced later by the compiler, not here — this function only builds the tree.

Parameters

[in,out]	out	The AST being built.
[in]	open_pos	Offset of the group's `(` (for error reporting).

Returns: The index of the atomic group's node_kind::group node.

◆ parse_braced_hex_escape()

constexpr std::int32_t real::detail::parser::parse_braced_hex_escape ( )

inlineconstexprprivate

Decodes a \x{XXXX} braced code-point escape — RE2/Perl syntax (ECMAScript spells this \u{...} instead, so every caller gates on !is_ecma() before reaching here). Rejected in bytes mode, like \u/\U/\N (no code-point meaning there) — read from the scope stack, not the global bytes_ member (the flag-scope ratchet: bytes is never scoped, so this equals bytes_ while keeping the parser's global-read count flat; same precedent as \C above). Shares its digit-loop / surrogate / overflow validation with \N{U+XXXX} via parse_braced_hex_scalar — this function only adds the bytes-mode check and the opening {. The backslash and x are already consumed by the caller.

Returns: The code point in [0, 0x10FFFF] (never a surrogate).

Exceptions

real::regex_error in bytes mode, or (via parse_braced_hex_scalar) on a malformed or unterminated {...}, a surrogate, or a value beyond U+10FFFF.

◆ parse_braced_hex_scalar()

constexpr std::int32_t real::detail::parser::parse_braced_hex_scalar ( )

inlineconstexprprivate

Decodes a braced hex scalar HHHHHH} (1–6 hex digits, then the closing }) — the code- point reader shared by \N{U+XXXX} (after its own U+ prefix) and \x{XXXX} (after its own bytes-mode check, see parse_braced_hex_escape). The opening { is already consumed by the caller; this reads the hex digits, the closing }, and rejects a surrogate (U+D800–U+DFFF) or a value beyond U+10FFFF — the same code-point range \u/\U enforce (Python semantics).

Returns: The code point in [0, 0x10FFFF] (never a surrogate).

Exceptions

real::regex_error on a missing digit run, an unterminated brace, a surrogate, or a value beyond U+10FFFF.

◆ parse_byte_escape()

constexpr std::int32_t real::detail::parser::parse_byte_escape ( )

inlineconstexprprivate

Parses a single-byte escape (valid inside and outside classes).

Handles \n \t \r \f \v \a \0, \xHH and escaped ASCII punctuation.

Returns: The byte value, or -1 when the escape is not a single byte (the caller then handles \d \w \s, etc.).

Exceptions

real::regex_error on a malformed \x escape.

◆ parse_class()

constexpr std::int32_t real::detail::parser::parse_class ( ast & out )

inlineconstexprprivate

Parses a bracketed character class [...] or [^...].

Supports ranges, escapes and the embedded set escapes; a ] right after [ or [^ is a literal, and a trailing - is a literal dash.

Parameters

[in,out] out The AST being built.

Returns: The index of the node_kind::klass node.

Exceptions

real::regex_error on an unterminated class or a bad range.

◆ parse_class_item()

constexpr std::int32_t real::detail::parser::parse_class_item	(	char_class &	klass,
		std::vector< code_range > &	ranges,
		bool &	property_derived
	)

inlineconstexprprivate

Parses one member inside a character class.

Parameters

[in,out]	klass	The class being built; a set member (`\d` etc.) is merged directly into it.
[in,out]	ranges	The class's non-ASCII code-point ranges; a Unicode shorthand (`\d` `\w` `\s`, or a negated one) appends its ranges here in text mode.
[in,out]	property_derived	Set when a Unicode shorthand contributed, so the whole class is emitted as a match-time `klass_cp` (text mode only).

Returns: A single byte (usable as a range endpoint), or -1 when the member was a whole set merged into klass.

Exceptions

real::regex_error on a non-ASCII member or an unsupported escape.

◆ parse_digit_escape()

constexpr std::int32_t real::detail::parser::parse_digit_escape ( )

inlineconstexprprivate

Parses a \<digit> escape via the shared decode_digit_escape().

Octal escapes (\0, \012, a three-octal-digit run) become one byte (value & 0xff, mirroring \xHH). A decimal group number is a back-reference, which REAL does not support (a deliberate, documented limitation).

Returns: The byte value of an octal escape.

Exceptions

real::regex_error on an over-long octal escape or a back-reference.

◆ parse_escape()

constexpr std::int32_t real::detail::parser::parse_escape ( ast & out )

inlineconstexprprivate

Parses an escape outside a character class.

Handles the class escapes \d \D \w \W \s \S, the anchors \A \Z \b \B, and single-byte escapes.

Parameters

[in,out] out The AST being built.

Returns: The index of the resulting node.

Exceptions

real::regex_error on a dangling or unsupported escape.

◆ parse_global_flags_prefix()

constexpr bool real::detail::parser::parse_global_flags_prefix ( ast & out )

inlineconstexprprivate

Consumes a leading global-flags group – (?imsxaU), or (?flags-flags) with a removal suffix – if present. The accepted letters are i m s x a U (is_flag_letter).

Like Python (3.11+), global flags are only legal at the very start of the pattern; later occurrences are rejected in parse_group. RE2 additionally permits an optional -removed suffix (e.g. (?i-s), or a pure (?-s)) that clears flags from the base scope for the rest of the pattern — this mirrors the added/-/removed parse in parse_group's scoped-flags branch ((?flags-flags:...)), minus its trailing : (a global prefix has none).

Parameters

[in,out] out Receives the added letters into ast::inline_flags and the removed ones into ast::inline_removed. Two fields rather than one net set because ast::inline_flags is OR-ed across calls and so cannot carry a removal; the caller applies them in order, adding then clearing.

Returns: true if a flags group was consumed (position advanced), else false (position restored, for parse_group to handle).

◆ parse_group()

constexpr std::int32_t real::detail::parser::parse_group ( ast & out )

inlineconstexprprivate

Parses a group construct.

Grammar:

group := '(' alternation ')'           capturing, numbered by '('
       | '(?:' alternation ')'         non-capturing
       | '(?P<name>' alternation ')'   named (Python style)
       | '(?<name>'  alternation ')'   named (.NET style)

Unsupported extensions (lookaround, backreferences, atomic groups, scoped inline flags) fail with a message naming the feature. Under flags::ecma the native-only constructs (?#...), (?P<name> and the atomic group (?>...) fail as "unknown extension" — the ECMAScript grammar has no such groups (possessive quantifiers are gated the same way at their parse site). Nesting beyond max_nesting_depth is rejected.

Parameters

[in,out] out The AST being built.

Returns: The index of the node_kind::group node.

Exceptions

real::regex_error on an unterminated or unsupported group.

< A (?flags:...) group pushed a scope to pop after the body.

◆ parse_group_name()

constexpr void real::detail::parser::parse_group_name	(	ast &	out,
		std::int32_t	group
	)

inlineconstexprprivate

Parses ‘name := [A-Za-z_][A-Za-z0-9_]* ’>'` and records it.

Parameters

[in,out]	out	The AST; the name is appended to ast::names.
[in]	group	The capture number this name refers to.

Exceptions

real::regex_error on a bad character or a duplicate name.

◆ parse_lookaround()

constexpr std::int32_t real::detail::parser::parse_lookaround	(	ast &	out,
		look_dir	direction,
		std::size_t	open_pos
	)

inlineconstexprprivate

Parses a lookaround after (?= / (?! (ahead) or (?<= / (?<! (behind) — the =/! is not yet consumed.

Builds a node_kind::lookaround node. The sub-pattern is a full alternation; its capture groups advance the global group counter (so outer group numbers stay consistent) but are compiled capture-free (V1 limitation, documented). Nesting a lookaround inside a lookaround is rejected. Boundedness and the byte L_max are enforced later by the compiler.

Parameters

[in,out]	out	The AST being built.
[in]	direction	Ahead or behind.
[in]	open_pos	Offset of the group's `(` (for error reporting).

Returns: The index of the lookaround node.

◆ parse_named_codepoint()

constexpr std::int32_t real::detail::parser::parse_named_codepoint ( )

inlineconstexprprivate

Decodes a \N{U+XXXX} named-code-point escape (1–6 hex digits) — the same code-point path as \u/\U, spelled by its U+ scalar value. re writes \N{NAME} for the name; the Python binding rewrites a name to this U+XXXX form before parsing, so the engine only ever sees the scalar. A C++ caller writes \N{U+XXXX} directly.

Rejected with clear messages: byte mode (no code-point meaning ≡ re's bad escape \N), a missing or malformed {U+…}; parse_braced_hex_scalar rejects a surrogate or a value beyond U+10FFFF. The backslash and N are already consumed.

Returns: The code point in [0, 0x10FFFF] (never a surrogate).

◆ parse_property_table()

constexpr property_table_result real::detail::parser::parse_property_table ( )

inlineconstexprprivate

Rejects bytes mode, consumes the p/P and the {Name} (or single letter), strips a leading ^ caret-negation (native dialects only), and resolves the remaining name to the property's code-point ranges. Shared by the out-of-class atom and the in-class merge. On entry pos_ is on the p/P; on return it is just past the name (caret and all).

Returns: The resolved ranges and whether a caret-negation was stripped.

◆ parse_quantifier()

constexpr std::int32_t real::detail::parser::parse_quantifier	(	ast &	out,
		std::int32_t	atom
	)

inlineconstexprprivate

Wraps atom in a repeat node if a quantifier follows.

Grammar: ‘quantifier := (’*' | '+' | '?' | '{n}' | '{n,}' | '{,m}' | '{n,m}') '?'?. An invalid{...}is not a quantifier at all and stays literal text, exactly like Python (e.g.a{,a{2,3x, a{,}` all match literally). A bare anchor cannot be repeated.

Parameters

[in,out]	out	The AST being built.
[in]	atom	Index of the atom the quantifier would apply to.

Returns: The repeat node index, or atom unchanged if no quantifier.

◆ parse_quoted_span()

constexpr quoted_span real::detail::parser::parse_quoted_span ( ast & out )

inlineconstexprprivate

Scans a \Q...\E literal span (the \Q is already consumed) and emits its characters as literal atoms — the same emission as parse_atom's default (whole code point per atom in text mode, single byte in bytes mode, icase folding via emit_literal_codepoint), libre2-measured semantics:

The span ends at the exact two-character \E (consumed) or at the end of the pattern (an unterminated \Q quotes to the end).
Everything inside is literal — metacharacters, |, ), whitespace (even in verbose mode; RE2 has no (?x) so this is REAL's own call: a quoted span protects its spaces), and a backslash NOT followed by E (so \Qa\Qb\E is the literal a\Qb and a trailing \Qa\ is the literal a\ — the "dumb scan": no escape processing, no nesting).
All atoms except the last are chained bare; the last is left unchained so the caller can apply a following quantifier to it alone (\Qab\E+ == ab+).

Parameters

[in,out] out The AST being built.

Returns: A quoted_span (all members -1 for an empty \Q\E).

Exceptions

real::regex_error on an invalid UTF-8 byte inside the span (text mode).

◆ parse_repeat_count()

constexpr std::int32_t real::detail::parser::parse_repeat_count ( )

inlineconstexprprivate

Reads an optional decimal repeat count.

Returns: The count, or -1 when no digits are present.

Exceptions

real::regex_error if the count exceeds max_repeat_count (counted repetitions are compiled by unrolling, so they are capped).

◆ parse_sequence()

constexpr std::int32_t real::detail::parser::parse_sequence ( ast & out )

inlineconstexprprivate

Parses sequence := (atom quantifier?)*, stopping at | or ).

Also intercepts \Q...\E literal quoting here (RE2/Perl syntax, !is_ecma() only — under ecma \Q keeps falling through to the rejected unknown escape): the span emits a SEQUENCE of literal atoms, not one atom, so it cannot live in parse_atom. A quantifier after \E binds to the span's LAST character (\Qab\E+ == ab+, libre2-measured): all-but-last chain bare and the last atom re-enters the loop's normal quantifier path. An empty \Q\E is grammar-invisible (libre2-measured a\Q\E+ == a+): a quantifier after it re-binds to the PREVIOUS atom — the prev tracker exists to re-chain that re-quantified atom — and with no previous atom the next iteration fails ("nothing to repeat"), matching RE2's "no argument for repetition operator".

Parameters

[in,out] out The AST being built.

Returns: The index of a concat node, a single atom, or an empty node.

◆ parse_unicode_codepoint()

constexpr std::int32_t real::detail::parser::parse_unicode_codepoint ( bool capital )

inlineconstexprprivate

Decodes a \uHHHH (4 hex) or \UHHHHHHHH (8 hex) code-point escape (str only).

Rejected with clear messages: byte mode (no code-point meaning), a surrogate (U+D800–U+DFFF), beyond U+10FFFF, or incomplete hex. The backslash and u/U are already consumed; this reads the hex digits.

Parameters

[in] capital True for \U (8 digits), false for \u (4 digits).

Returns: The code point in [0, 0x10FFFF] (never a surrogate).

◆ parse_unicode_property()

constexpr std::int32_t real::detail::parser::parse_unicode_property	(	ast &	out,
		bool	negated
	)

inlineconstexprprivate

Parses \p{Name} / \P{Name} / \pX (outside a class) into a negatable Unicode code-point class (klass_cp), reusing the same match-time mechanism as \w. Negation is the class-node flag, as for \W, XORed with a caret-negation \p{^Name} stripped by parse_property_table (so \P{^L} negates twice back to \p{L}, same as \P{...} on an already-negated property would). pos_ is on the letter after \; negated distinguishes \P from \p.

Parameters

[in,out]	out	The AST the class node is added to.
[in]	negated	True for `\P`, false for `\p`.

Returns: The new node's index.

◆ peek()

constexpr char real::detail::parser::peek ( ) const

inlineconstexprprivate

Returns the current character without consuming it (undefined at eof()).

Returns: The character at the read offset.

◆ property_ascii_high()

constexpr void real::detail::parser::property_ascii_high	(	const std::vector< code_range > &	table,
		char_class &	ascii,
		std::vector< code_range > &	high
	)		const

inlineconstexprprivate

Splits a property's ranges into its ASCII bitmap (< 0x80) and its non-ASCII ranges. Unconditional: unlike \w, flags::ascii (re.A) does not restrict a Unicode property, so both parts are always used (bytes mode having already been rejected).

Parameters

[in]	table	The property's full range table.
[out]	ascii	Bitmap receiving its members below 0x80.
[out]	high	Ranges receiving its members at or above 0x80.

◆ resolve_property()

constexpr std::vector< code_range > real::detail::parser::resolve_property ( std::string_view name ) const

inlineconstexprprivate

Resolves a \p{...} property name to its code-point ranges, or fails with a clear error. An optional gc= / sc= / scx= (or general_category= / script= / scriptextensions=) prefix picks the namespace; a bare name tries General_Category, then Script, then a binary property (\p{Alphabetic}, no namespace of its own, same as PCRE2) – scx= has no bare-name form (PCRE2: a bare name never means Script_Extensions, the explicit prefix is required). GC ranges come straight from the table; a Script's ranges are collected from the partition; a binary property's or a Script_Extensions' ranges come straight from their own table (both are NOT partitions – a code point can satisfy several). The alias resolvers are the generated, loose-keyed resolve_gc / resolve_script (shared by sc= and scx= – same script names, long or short UAX24 code) / resolve_binprop.

Parameters

[in] name The property name as written, prefix and all.

Returns: Its code-point ranges.

◆ shorthand_class()

static constexpr shorthand_spec real::detail::parser::shorthand_class	(	char	letter,
		bool	text_mode
	)

inlinestaticconstexprprivate

Maps a shorthand letter to its shorthand_spec. The single place the letter -> (set, range table, negation) fact lives; the atom ladder (parse_escape) and the class ladder (parse_class_item) share it, then each consumes the spec its own way (emit a class node vs merge into a class) – the same shared-decode / divergent-use split as decode_digit_escape.

Parameters

[in]	letter	The shorthand letter (`d D w W s S`).
[in]	text_mode	Whether this shorthand compiles as a text-mode code-point predicate (the caller's own text_shorthand) — only `\s`/`\S` need it: the ASCII-range component of `\s` legitimately differs between ASCII mode (`[ \t\n\r\f\v]`) and text mode (the same set plus `U+001C`-`U+001F`); `\w`/`\d` do not have this divergence, so they ignore the parameter.

Returns: The shorthand's bitmap, range table and negation flag.

◆ shorthand_ranges()

constexpr std::vector< code_range > real::detail::parser::shorthand_ranges ( std::span< const code_range > table ) const

inlineconstexprprivate

The non-ASCII part of a shorthand's range table, or nothing in bytes / ASCII mode. Wholly-ASCII ranges are dropped: the bitmap already covers them.

Parameters

[in] table The shorthand's full Unicode range table.

Returns: Its ranges clipped to >= 0x80; empty when the shorthand stays ASCII-only.

◆ skip_insignificant()

constexpr void real::detail::parser::skip_insignificant ( )

inlineconstexprprivate

In verbose mode, consumes insignificant whitespace and # comments.

No-op unless is_verbose. Called only between tokens outside character classes; escaped whitespace (\) is read as a literal by the escape parser, never reaching here.

◆ space_set_text_ascii_component()

static constexpr char_class real::detail::parser::space_set_text_ascii_component ( )

inlinestaticconstexprprivate

\s's ASCII-range (< 0x80) component for TEXT mode: space_set() (the ASCII-MODE set, [ \t\n\r\f\v]) plus U+001C-U+001F (FS/GS/RS/US). Needed because shorthand_ranges deliberately omits any wholly-ASCII range from .ranges ("already covered by the bitmap") — so for text mode, where re's own \s DOES include FS/GS/RS/US (verified: re.match(r"\s", "\x1c") matches; str.isspace() agrees) but ASCII-mode \s does not (re.match(r"(?a)\s", "\x1c") does not match), the bitmap that "already covers" the ASCII range must itself differ by mode — space_set() alone is only correct for the ASCII-mode case. Found live by differential fuzzing (ASCII mode wrongly matching FS/GS/ RS/US) and fixed once at space_set() itself before this second bug (text mode then losing them entirely) surfaced immediately in test_classes.cpp's own regression suite — the two modes generate genuinely different ASCII bitmaps, not one shared one.

Returns: The text-mode ASCII bitmap for \s.

◆ text_shorthand()

constexpr bool real::detail::parser::text_shorthand ( ) const

inlineconstexprprivate

Whether a shorthand (\d \w \s) should be a text-mode Unicode code-point predicate: true in the default text mode, false in bytes mode or under flags::ascii (re.A).

Returns: Whether the shorthand compiles as a code-point predicate rather than a byte class.

◆ try_parse_braces()

constexpr bool real::detail::parser::try_parse_braces	(	std::int32_t &	min,
		std::int32_t &	max
	)

inlineconstexprprivate

Tries to parse {n} / {n,} / {,m} / {n,m} starting at {.

Parameters

[out]	min	Lower bound on success.
[out]	max	Upper bound on success (-1 for unbounded).

Returns: true on a valid quantifier (position advanced); false if the braces are not a quantifier (position restored — literal text).

Exceptions

real::regex_error when the bounds are impossible (min > max).

◆ without()

static constexpr flags real::detail::parser::without	(	flags	value,
		flags	bit
	)

inlinestaticconstexprprivate

value with bit cleared. The intermediate cast matches the enum's std::uint16_t underlying type — a std::uint8_t here (the pre-widening vestige) would silently drop flags::ungreedy (512) from every scope.

Parameters

[in]	value	The flag set to clear from.
[in]	bit	The flag to clear.

Returns: value without bit.

The documentation for this class was generated from the following file:

include/real/frontend/ast.hpp

Classes

Public Member Functions

Private Member Functions

Static Private Member Functions

Private Attributes

Detailed Description

Constructor & Destructor Documentation

◆ parser()

Member Function Documentation

◆ accept()

◆ add_class_node()

◆ add_node()

◆ current_flags()

◆ emit_codepoint_utf8()

◆ emit_literal_codepoint()

◆ eof()

◆ expect()

◆ fail()

◆ fail_unsupported()

◆ flag_for_letter()

◆ hex_digit()

◆ is_ascii_alnum()

◆ is_ascii_mode()

◆ is_ecma()

◆ is_flag_letter()

◆ is_icase()

◆ is_name_start()

◆ is_verbose()

◆ loose_key()

◆ merge_property()

◆ merge_unicode_property()

◆ new_group()

◆ parse()

◆ parse_alternation()

◆ parse_atom()

◆ parse_atomic_group()

◆ parse_braced_hex_escape()

◆ parse_braced_hex_scalar()

◆ parse_byte_escape()

◆ parse_class()

◆ parse_class_item()

◆ parse_digit_escape()

◆ parse_escape()

◆ parse_global_flags_prefix()

◆ parse_group()

◆ parse_group_name()

◆ parse_lookaround()

◆ parse_named_codepoint()

◆ parse_property_table()

◆ parse_quantifier()

◆ parse_quoted_span()

◆ parse_repeat_count()

◆ parse_sequence()

◆ parse_unicode_codepoint()

◆ parse_unicode_property()

◆ peek()

◆ property_ascii_high()

◆ resolve_property()

◆ shorthand_class()

◆ shorthand_ranges()

◆ skip_insignificant()

◆ space_set_text_ascii_component()

◆ text_shorthand()

◆ try_parse_braces()

◆ without()