Generating Regular Expressions With LLMs in October 2026: Write the Test Cases First, Guard Against Catastrophic Backtracking, and Pin the Regex Flavor

Asking a language model for a regular expression is one of the most common small jobs developers hand to AI. It is also one of the easiest places to ship a quiet bug. A regex that looks right, passes the one example you tried, and then either misses half your real inputs or freezes a server thread on a hostile string is a very ordinary outcome. The fix is not to stop using models for this. It is to treat the generated pattern like any other untrusted code: specify it with examples, test it in the engine you actually run, and check it for performance traps before it reaches production.

Photo: Red jigsaw puzzle piece on green pieces by Profpcde, CC0, via Wikimedia Commons

This guide walks through a workflow that keeps the speed of AI-generated patterns while removing most of the risk.

Why generated regexes fail in practice

Models are good at producing patterns that match the examples in the prompt. The problems usually come from what the prompt left out:

• Unstated negatives. You asked for "a pattern that matches order IDs like ORD-12345" but never said it must not match ORD-12345-X or xORD-12345. The model has no reason to add anchors or word boundaries unless you ask.
• Wrong flavor. JavaScript, Python, PCRE, Go (RE2), Java, .NET, and POSIX tools like grep -E differ on lookbehind support, named group syntax, Unicode classes, and what \d means. A pattern that works on a website tester may not compile, or may behave differently, in your runtime.
• Unicode surprises. In Python 3, \d and \w match Unicode digits and letters by default, so \d accepts Arabic-Indic digits. In JavaScript without the u flag, \w is ASCII only. If you mean ASCII, say so and use [0-9].
• Catastrophic backtracking. Nested quantifiers such as (a+)+ or overlapping alternations like (\w|\d)* can take exponential time on inputs that almost match. On a web server this becomes a denial-of-service bug, commonly called ReDoS.
• Overreach. Models often reach for a single heroic regex for tasks that should be a parser, such as full email validation, HTML, nested JSON, or dates with real calendar rules.

Step 1: Write the test cases before you ask

The most useful thing you can give a model is a small table of strings that must match and strings that must not. Ten to twenty of each is plenty for most patterns. Include the boring cases and the edge cases you already know about.

MUST MATCH:
ORD-10000
ORD-99999
ord-12345        (case-insensitive is fine)

MUST NOT MATCH:
ORD-1234         (too short)
ORD-123456       (too long)
ORD-12345-X      (trailing suffix)
xORD-12345       (prefix)
ORD 12345        (space instead of dash)
ORD-１２３４５    (full-width digits)

Writing the negatives forces you to decide what you actually want. Often this step alone reveals that the requirement was fuzzy, for example whether lowercase input is allowed or whether the ID can appear inside a longer sentence.

Step 2: Prompt with the flavor, the context, and the output shape

A good prompt states four things: the regex engine, how the pattern will be used (full-string validation or searching inside text), the test table, and the format you want back. Asking for an explanation of each part is worth the extra tokens because it makes review much faster.

Write a regular expression for Python 3.12's re module.
It will be used with re.fullmatch() to validate a single field.
It must pass every case below. Use ASCII digits only ([0-9]), not \d.
Use named groups where it helps readability.
Return: the pattern as a raw string, then a one-line explanation of each part,
then any input you think is ambiguous.

[test table here]

The "full-string validation or search" detail matters. For validation, anchors (^...$ or fullmatch) are essential. For searching inside a log line, you usually want word boundaries or explicit separators instead. Note that in Python, $ also matches just before a trailing newline, so re.fullmatch or \Z is the stricter choice for validation.

Step 3: Run the tests in your real engine

Do not trust an online tester set to a different flavor, and do not trust the model's claim that the pattern passes. Paste the pattern and the table into a tiny script in the language you will deploy, and run it. Here is a minimal Python version:

import re

PATTERN = re.compile(r"(?i)ORD-(?P<num>[0-9]{5})")

must_match = ["ORD-10000", "ORD-99999", "ord-12345"]
must_not = ["ORD-1234", "ORD-123456", "ORD-12345-X",
            "xORD-12345", "ORD 12345", "ORD-\uff11\uff12\uff13\uff14\uff15"]

failures = []
for s in must_match:
    if not PATTERN.fullmatch(s):
        failures.append(("should match", s))
for s in must_not:
    if PATTERN.fullmatch(s):
        failures.append(("should NOT match", s))

print("OK" if not failures else failures)

If anything fails, paste the exact failing strings back to the model and ask for a corrected pattern. This loop usually converges in one or two rounds. Once it passes, keep the script: it becomes a unit test that lives next to the pattern in your codebase, so a future edit cannot silently break a case you already decided.

For extra confidence, ask the model to generate twenty additional tricky inputs, then label each one yourself as match or no-match. The model is often better at inventing adversarial strings than at getting every one of them right, so your labels, not its labels, decide the expected result.

Step 4: Check for catastrophic backtracking

Most mainstream engines (Python re, JavaScript, Java, PCRE, .NET by default) use backtracking. A pattern can pass every functional test and still hang on a long string that nearly matches. Before shipping, review the pattern for these warning signs:

• A quantified group that itself contains a quantifier, such as (\w+)* or (\d+,?)+.
• Alternatives that can match the same characters, such as (a|a)*, (\w|\d)+, or (.*,)*.
• Two adjacent unbounded parts that can both consume the same text, like \s*.*\s*.

Then test it directly. Build an input that matches most of the pattern and then fails at the very end, and time it at increasing lengths:

import re, time

pat = re.compile(r"^(\w+\s?)*$")   # a classic risky shape
for n in (10, 15, 20, 25):
    s = "a" * n + "!"
    t = time.perf_counter()
    pat.match(s)
    print(n, round(time.perf_counter() - t, 4))

If the time roughly doubles with each small increase in length, the pattern is exponential and must be rewritten. Common fixes are removing the nested quantifier, making alternatives mutually exclusive, or adding explicit bounds like {1,64}. Python 3.11 and later support atomic groups (?>...) and possessive quantifiers like \w++, which stop the engine from retrying failed paths. You can also ask the model directly: "Is this pattern vulnerable to catastrophic backtracking? Show an input that triggers it." Then verify its answer with the timing script, because models can be wrong about this in both directions.

Photo: Standardgraph 2522 2.5–7 mm lettering guides by Lucasbosch, CC BY-SA 3.0, via Wikimedia Commons

Step 5: Add runtime limits where input is untrusted

Review catches most problems, but any regex that runs on user-supplied text should also have a safety net:

• Cap input length before matching. A product code field never needs 10,000 characters.
• Use a linear-time engine where you can. Go's standard regexp package and Rust's regex crate are built on automata and guarantee linear-time matching. Google's RE2 library offers the same for C++ and has bindings for other languages. The tradeoff is that they do not support backreferences or lookarounds.
• Set a match timeout where the platform supports it. .NET's Regex constructor accepts a timeout, and Python's third-party regex module accepts a timeout argument on its matching functions.

Step 6: Make the pattern maintainable

A regex that nobody can read will eventually be "fixed" by someone who breaks it. Ask the model for a verbose version with comments, which Python supports with the re.VERBOSE flag:

ORDER_ID = re.compile(r"""
    (?i)            # case-insensitive
    ORD             # fixed prefix
    -               # literal dash separator
    (?P<num>[0-9]{5})  # exactly five ASCII digits
""", re.VERBOSE)

Keep the test table beside it, and add a short comment pointing to the test file. When requirements change, update the table first, then ask the model to update the pattern against the new table. That keeps the examples, not someone's memory, as the specification.

When not to use a regex at all

Part of using an LLM well is asking it whether a regex is the right tool. Some cases where a dedicated parser or library is the better answer:

• Email addresses: a simple sanity check is fine, but real validation means sending a confirmation message.
• URLs: use your language's URL parser, such as urllib.parse in Python or the URL class in JavaScript.
• Dates and times: parse them with a date library so leap years and month lengths are handled for you.
• HTML, XML, and JSON: use a real parser. Nested structures are exactly what regular expressions handle badly.
• Phone numbers: libraries such as Google's libphonenumber encode country rules that a hand-written pattern will miss.

A good prompt habit is to add one line: "If a regular expression is the wrong tool for this, say so and suggest the alternative." Models will usually take that opening when it is offered.

A quick checklist

1. Write must-match and must-not-match examples first.
2. Tell the model the exact engine and whether you are validating or searching.
3. Run the tests in your real runtime and feed failures back.
4. Time near-miss inputs to catch catastrophic backtracking.
5. Cap input length, and prefer a linear-time engine or a timeout for untrusted text.
6. Keep a commented version and the test table in the repository.
7. Ask whether a parser or library would be the better tool.

Generated regular expressions save real time, especially for the fiddly patterns most of us look up every time. The time savings only hold if the pattern is pinned down by tests you wrote and checked in the engine you run. With that small amount of discipline, the model writes the pattern, and your examples decide whether it is correct.