Where regular expressions break
Validation on electronic Case Report Forms (eCRF) for multi-cohort drug trials involves arithmetic formulas that depend on each other, consistency checks across forms, and comparisons with earlier visits. Teams often try to patch these rules with complicated regular expressions, and soon run into catastrophic backtracking, patterns nobody can maintain, and checks that quietly let bad values through.
A regex reads characters. It knows nothing about the rule's syntax, scope or operator precedence. When an investigator writes an expression such as LAB.CREATININE > 1.5 AND (VISITA.DOSE != NULL OR LAB.ALT > 3 * UPPER_LIMIT), a regular expression can't build an evaluation context, check types or promise to terminate, and it risks catastrophic backtracking (ReDoS) that freezes the investigator's browser tab.
A domain-specific AST evaluator
Our clinical schema engine turns rule expressions into Abstract Syntax Trees (ASTs) instead of matching strings. A deterministic scanner tokenizes each rule, and the parser builds a recursive tree of nodes:
type ASTNode =
| { type: "Literal"; value: string | number | boolean }
| { type: "Identifier"; fieldRef: string }
| { type: "BinaryExpression"; operator: "+" | "-" | "*" | "/" | ">" | "<" | "=="; left: ASTNode; right: ASTNode }
| { type: "LogicalExpression"; operator: "AND" | "OR"; left: ASTNode; right: ASTNode }
| { type: "FunctionCall"; name: string; args: ASTNode[] };
Recursive descent parsing
A hand-written recursive-descent parser enforces operator precedence (arithmetic before relational, relational before logical) without eval() or new Function(). Variable references bind to an immutable trial dataset scope, so evaluation is a pure fold over the syntax tree.
Bounded time and a record of every result
Validation runs in the coordinator's browser during a live subject visit, so every rule has to finish in bounded time. The AST traversal allows no unbounded recursion and no circular loops:
- Before anything runs, the rule graph is checked for cyclic dependencies between form fields.
- Tree depth is capped at 32 levels, so evaluation finishes in under 1 millisecond per form item.
- Every evaluation produces a derivation tree you can inspect, showing why a check failed: the kind of traceability a Part 11 reviewer looks for.
No ReDoS, and a derivation for every warning
A regular expression can backtrack exponentially on hostile or unusual input. Recursive AST evaluation is linear, $O(N)$ in the number of nodes in the syntax tree. If an FDA audit asks about an edit check warning from eighteen months earlier, the system can rebuild the exact derivation tree that raised it.
Trying it
Moving from regular expressions to AST compilation removed hundreds of silent validation escapes across our clinical form pipelines. The same compiler runs in the client-side form wizards and in the server-side batch ETL validation, so a rule evaluates bit for bit the same in both.
You can design eCRF forms and watch the AST rules evaluate as you type in CRF Studio. The CRF-XL Case Study covers AST compilation in clinical pipelines in more depth, and Proof Studio has the formal proofs.