Writing Apache Regex Redirects for Bulk URL Changes

Problem Statement

Bulk URL migrations on Apache trigger redirect loops, TTFB degradation, and equity loss when regex patterns lack strict anchoring or are dropped into .htaccess instead of a virtual-host config. The symptoms are 500 Internal Server Error spikes, broken UTM parameters, and crawl-budget exhaustion from chained hops. The fix is anchored PCRE patterns, terminal flags that stop rule processing, loop-prevention conditions, and validation before production. This page sits under the Regex Redirect Rules section.

Anatomy of an Apache RewriteRule A labelled breakdown of an anchored RewriteRule showing the pattern, capture group, substitution backreference, and terminal flags. Anatomy of a RewriteRule ^old/(.*)$ /new/$1 [R=301,L] anchored pattern substitution + $1 terminal flags ^ and $ prevent partial matches; L stops processing to avoid loops $1 carries the captured tail from the pattern into the target
An anchored pattern feeds a captured tail into the substitution; the L flag halts processing so the request cannot loop.

When to Use This Approach

  • You have hundreds or thousands of path changes that follow predictable patterns rather than one-off pairs.
  • Source URLs share a common prefix or structure that a single capture group can transform.
  • You need query-string handling, conditional logic, or case-insensitive matching that mod_alias cannot express.
  • You control the virtual-host config and can avoid the per-request filesystem cost of .htaccess.
  • You are generating rules from a mapping file and want them anchored and loop-safe before deployment.

Step-by-Step Instructions

1. Pre-Process the Mapping CSV

Generate anchored rules from a two-column CSV so every pattern is escaped and bounded. Keep the mapping under version control alongside the rest of your CSV Mapping Workflows artefacts.

# Escape regex metacharacters in the source column before building rules
sed 's/[.[\*^$()+?{|]/\\&/g' source_paths.txt

# Generate anchored RewriteRule lines from a two-column CSV (source,destination)
sed -E 's|^([^,]+),(.+)$|RewriteRule ^\1$ \2 [R=301,L]|' mapping.csv \
  >> /etc/apache2/sites-available/migration.conf

Pre-process the mapping outside Apache rather than expressing normalisation as rules. Case folding, trailing-slash handling and parameter stripping are all cheaper and clearer as a transformation of the data than as additional RewriteRule lines, and every rule you avoid emitting is one fewer thing evaluated on every request and one fewer opportunity for an ordering surprise.

2. Construct Anchored Rules With Loop Prevention

Anchor every pattern with ^ and $, append L to stop processing, and add a RewriteCond so a request already on the new path is skipped — the most common cause of 500 loops.

RewriteEngine On
RewriteBase /

# NC = case-insensitive, L = last, QSA = preserve query string
# Skip the rule if the request is already on the new path (loop guard)
RewriteCond %{REQUEST_URI} !^/new-pattern/
RewriteRule ^old-pattern/(.*)$ /new-pattern/$1 [R=301,L,NC,QSA]

3. Choose the Status Code and Flatten Hops

Use R=301 for permanent moves so PageRank transfers; reserve R=302 for staging or A/B windows. Map each source straight to its final destination so no rule points at another redirected source — flatten any survivors with Redirect Chain Elimination. When matching in a condition, remember the backreference is %1, not $1.

RewriteEngine On
# A capture from RewriteCond is referenced as %1 (RewriteRule captures use $1)
RewriteCond %{REQUEST_URI} ^/legacy/(.*)$
RewriteRule ^ /new/%1 [R=301,L]

Choose the status code per row from the inventory rather than defaulting it in the rule. A RewriteRule with [R=301] hard-coded across a generated block silently promotes every provisional mapping to permanent, which is the one redirect mistake with no practical reversal. Carry the code as data and emit it per rule, so a row marked temporary in the inventory produces a temporary redirect in the config.

4. Deploy in the vhost, Not .htaccess

Place rules in sites-available, not .htaccess, to avoid 15–30% TTFB degradation from per-request filesystem scans. Escape literal dots so a pattern does not match unintended TLDs, and validate before reload.

# Validate config, then smoke-test one path before going live
apachectl configtest \
  && curl -sI -o /dev/null -w '%{http_code}\n' https://target-domain.com/old-pattern/x
Where Apache redirect rules should live vhost configuration compared with .htaccess on rule evaluation cost, when changes take effect, and reviewability. The convenient location is the expensive one Property vhost config .htaccess Read once at startup on every request, per directory Changes take effect on reload immediately In version control normally yes often not Cost at 5 000 rules negligible measurable on every hit The immediacy of .htaccess is a genuine convenience during development and a permanent tax in production.
Develop wherever is convenient; deploy the bulk map in the vhost, where it is parsed once rather than per request.

Worked Example

A publisher consolidates /old-pattern/2019/case-study into /new-pattern/2019/case-study across 3,200 archive URLs with one anchored rule. The vhost contains:

RewriteCond %{REQUEST_URI} !^/new-pattern/
RewriteRule ^old-pattern/(.*)$ /new-pattern/$1 [R=301,L,NC]

A request to a legacy archive path resolves in exactly one hop:

GET /old-pattern/2019/case-study HTTP/1.1
Host: target-domain.com

HTTP/1.1 301 Moved Permanently
Location: https://target-domain.com/new-pattern/2019/case-study

The RewriteCond %{REQUEST_URI} !^/new-pattern/ line is what makes this safe at scale: without it, a request that lands on /new-pattern/... would re-enter the rule and loop until Apache returns 500. With 3,200 sources collapsed into a single capture-group rule, evaluation stays cheap and there is one rule to audit instead of thousands.

Loop prevention is not optional in Apache, because mod_rewrite re-processes the rewritten URL through the rule set by default. A rule whose output can match its own input will keep matching until the server aborts with a 500, and the pattern that does this is easy to write by accident — most commonly a rule that adds or removes a trailing element without anchoring against the result. The [L] flag stops further processing for that pass, and a RewriteCond excluding the destination shape stops the rule matching its own output.

Test each rule against its own output explicitly. Take the destination the rule produces, feed it back in as a source, and confirm nothing matches. It is a two-line check per rule and it catches the entire class of loop before it reaches an environment where the symptom is a 500 rather than a failed assertion.

Flatten before deploying rather than after. A generated block that contains both /a → /b and /b → /c will happily serve a two-hop chain, and no amount of testing individual rules reveals it because each rule is correct in isolation. The resolution belongs in the data — every source resolved to its terminal destination before emission — and the generator should refuse to write a file where any source also appears as a destination.

Verification

Validate config syntax, confirm a single Location header, and watch live 301 volume. For staging dry-runs enable rewrite tracing (LogLevel alert rewrite:trace3 on Apache 2.4; the old RewriteLog directives are removed and will fail a 2.4 parse). To roll back without deleting rules, flip RewriteEngine On to Off and reload, or restore a timestamped backup of the vhost.

# Syntax check before any reload
apachectl configtest          # or: apache2ctl configtest

# Expect exactly one Location header (single hop)
curl -sI -L https://target-domain.com/old-pattern/2019/case-study \
  | grep -E 'HTTP/|Location:'

# Live 301 volume by path
tail -f /var/log/apache2/access.log \
  | grep '" 301 ' | awk '{print $7}' | sort | uniq -c | sort -nr

Validation checklist:

Flags that change what a RewriteRule does The four mod_rewrite flags that most affect redirect behaviour — R, L, QSA and NC — with what each does and the consequence of omitting it. Four flags, and three of them fail quietly when missing Flag Does Omit it and [R=301] issues the status code rewrites internally, no redirect [L] stops further processing output re-enters the rule set — loops [QSA] appends the original query campaign parameters silently dropped [NC] matches case-insensitively mixed-case requests miss the rule Only the first has a visible symptom when missing; the other three produce a working site that behaves subtly wrongly.
Emit these from the inventory per row rather than hard-coding them across a generated block.

FAQ

How do I preserve query strings with Apache regex redirects? Apache 2.4 appends the original query string automatically unless the substitution URL contains a literal ?. To be explicit, add [QSA]: RewriteRule ^/old/(.*)$ /new/$1 [R=301,L,QSA].

What is the performance impact of 10,000+ regex rules in one vhost? Rules evaluate sequentially per request — O(n). Convert linear evaluation into O(1) lookups with a RewriteMap backed by txt: (plain-text hash) or dbm: (binary hash), which sharply reduces CPU overhead and latency.

Why does configtest pass but the server returns 500 on redirect? apachectl configtest checks directive structure, not runtime logic. A 500 usually means an infinite loop from a missing RewriteCond exclusion, a conflicting mod_alias Redirect, or a backreference like $1 in a rule whose pattern has no capture group.

When should RedirectMatch be preferred over RewriteRule? For straightforward pattern-to-pattern redirects with no conditions, RedirectMatch is simpler, cheaper and much harder to get subtly wrong — it does one thing and cannot loop. Reach for mod_rewrite when the rule needs a condition on something other than the path: a host, a header, a query parameter, an environment variable. Mixing both in one file is fine provided you are deliberate about which handles what, since they are evaluated by different modules at different points and their interaction is not obvious.

How do I keep several thousand rules maintainable? Move to RewriteMap with an external map file rather than expressing every mapping as its own rule. A map turns lookup cost from linear in the number of rules to effectively constant, keeps the mapping data separate from the routing logic, and lets the map file be generated from the inventory while the rule that consumes it stays a single line. The threshold where this becomes worthwhile arrives sooner than most teams expect — a few hundred rules rather than a few thousand.

Does rule order matter as much in Apache as in Nginx? More, if anything, because mod_rewrite re-processes the rewritten URL through the rule set unless told otherwise. That means a broad rule placed above a specific one both shadows it and can feed its own output back in, which turns an ordering mistake into a loop rather than merely a wrong destination. Emit exact matches first, patterns next, catch-alls last, terminate every rule with [L], and have the generator enforce the ordering rather than trusting the assembly.

What is the practical limit before RewriteMap is required? A few hundred rules is where linear evaluation starts to show on a busy site, and a few thousand is where it becomes the dominant cost of serving a request that does not even redirect. A map turns that into a constant-time lookup and separates the mapping data from the routing logic, which has the secondary benefit of letting the map be regenerated without touching the config. If the estate is large enough that you are asking the question, the answer is almost certainly yes.

Related

← Back to Regex Redirect Rules