Don’t Fire Your Regex Yet: The Hidden Risks of AI-Only WAFs

Background

Over the past few months, I have had the opportunity to evaluate several Web Application Firewalls and API security solutions. Among all the wonderful innovations these solutions are trying to achieve, one thing that stood out was how some vendors want to move away from traditional regex-based systems to fully AI-based solutions.

At first glance, this seems like progress. After all, AI can learn new patterns and thus protect websites against modern attack vectors or payloads. But in practice, fully replacing your existing logic with AI models introduces new risks and can make your ecosystem less secure.

Why does everyone want to adopt AI?

AI-driven systems can analyze complex payloads and traffic patterns that static regex rules might miss. They can:

• Adapt faster to new or obfuscated attack vectors.
• Reduce manual rule updates and tuning overhead.
• Flag subtle anomalies across APIs and endpoints.

Used correctly, this can enhance visibility and reduce false negatives. But when AI becomes the only layer of decision-making, things get dangerous.

The Problem: Determinism vs. Probabilism

Regex rules and pattern-based detections are deterministic. If we give the same input, they’ll always produce the same output. This makes them predictable, testable, and debuggable. AI scoring models, on the other hand, are probabilistic. They assign a “risk score” or “malicious likelihood” based on patterns learned from training data.

What could go wrong with AI scoring systems?

• They can hallucinate, assigning incorrect classifications.
• They rely on the availability of AI inference endpoints, which can experience latency or downtime.
• They often lack transparent logic, making it hard to explain or reproduce a decision.

Imagine a situation where the WAF allows a dangerous SQLi or an XSS payload to go through because its AI score fell below the scoring criteria. You might think this may not happen, but we actually observed a case with one vendor where they were completely dependent on the AI scoring system.

Case Study of a Real XSS Bypass

For one of the AI security vendors, we observed that a simple XSS payload <img src=dd onerror=confirm(1)> was blocked in the URL but went through when the payload was injected into a JSON body parameter. This happened because the AI deduced that no one would inject a payload into a parameter named label. Hence, it reduced its confidence score.

When we tried to block payloads by reducing the confidence threshold, the AI started blocking real traffic because it was already operating at a low score.

Overall, there was no easy way via which we could have blocked the XSS payload. If it would have been a regex based system, the payload would have failed.

Final Thoughts - AI Should Complement, Not Replace

The best security architectures blend deterministic and adaptive approaches:

• Keep your regex and rule-based detections for known signatures and high-confidence patterns.
• Use AI and anomaly scoring as additional signals to flag suspicious patterns, cluster new attack types, or prioritize human review.
• Introduce confidence thresholds and fallback logic so a temporary AI outage doesn’t affect the security of the ecosystem.

AI is a powerful ally, but security solutions should aim for reliability and accountability.