# ModSecurity public-page lab

Pack 1.0 · 17 September 2026

Run twelve controlled requests in each of four real nginx / ModSecurity / CRS configurations: before, after, late-exclusion and rollback. The fictional HTML tutorial is served only on loopback. The kit does not contact your site, modify a system service, reload a production firewall or verify a real crawler.

## Requirements and run

Use Ubuntu 24.04 amd64-compatible Linux libraries, Python 3.12 or newer, dpkg-deb, and internet access to Ubuntu Snapshot and GitHub releases. On another operating system, use a suitable disposable Linux VM. Extract into a writable directory without whitespace. No repository clone, Python package installation, root privileges or system nginx installation is needed.

From the extracted kit directory:

```sh
sha256sum -c SHA256SUMS.txt
python3 run-lab.py > observed-run.json
python3 -m json.tool observed-run.json > /dev/null
```

The first command verifies every bundled file except the checksum list itself. Run the Python command only after it succeeds. The runner downloads nine original checksum-pinned Ubuntu packages from snapshot 20260915T000000Z, then the original CRS 4.29.0 release archive. It extracts them under test/fixtures/modsecurity_delivery/.runtime and runs isolated local processes. An existing archive cache is checksum-verified. The original runtime.json retains its historical archive URLs; run-lab.py prepares the snapshot cache before calling the unchanged installer. Start with run-lab.py rather than calling the historical runner directly on an empty cache.

A successful run exits zero and writes a fresh executedAt, versions, twelve cases, 48 results and 48 correlated JSON audit observations. A nonzero exit is a failed run; do not treat an empty or partial output file as a receipt. A missing download, checksum mismatch, incompatible native library, failed configuration test or failed control stops execution. Preserve the pins if a mirror is temporarily unavailable, then retry later. The timestamps and request IDs in historical-observations.json are from 13 September 2026; do not replace that file with a new run.

## Inspect the result

Only the four intended GET/HEAD requests become 200 in the after configuration. Unknown user-agent text receives the same public-page treatment. Write method, other host/path and path suffix requests remain 403. The SQL-injection control still triggers rule 942100. The private origin remains 401, ordinary public content remains 200, late placement fails, and rollback matches the before results. A HEAD response is empty; successful GET bodies match the fictional tutorial. Read GUIDE.md for the complete guide, all source links and an editable operator record.

The tests assert these outcomes; they do not certify a production exception. Keep SecRuleEngine On, confirm the actual public content and matched target, and have the responsible operator approve the exact host, method, path, rule ID, placement and rollback before adapting anything. User-agent strings in this exercise are controlled labels, not verified provider identities. Production DNS/TLS, crawler arrival, indexing, citations, traffic and ranking effects are outside this lab.

## Files and cleanup

- run-lab.py: snapshot bootstrap and preserved runner entry point.
- scripts/prepare-firewall-runtime.py: exact checksum-pinned snapshot recovery script.
- test/fixtures/modsecurity_delivery/: exact setup.py, run.py, runtime.json, exclusion.conf and example.json.
- historical-observations.json: unchanged original 13 September execution receipt.
- GUIDE.md: complete original article, exclusion and FAQ, plus this quick start and a blank operator worksheet.
- SHA256SUMS.txt: hashes for all other bundled files.

The runner shuts down its local processes and removes their temporary configurations after each run. After all processes have exited, removing this extracted directory removes the cached native runtime. Keep your new observed-run.json and completed worksheet separately if you need them. Dependencies are downloaded from their original projects; their licenses accompany the extracted packages and CRS archive.

## Your operator worksheet

Complete this record for your own evidence. Leave unresolved items explicitly unresolved; the fictional lab is not approval for a live change.

```text
Site and environment:
Public URL and exact Host header:
Responsible operator and change approver:
Public-content verification and evidence:
Authentication boundary and private control URL:
Native engine, connector and CRS versions:
Observed failing request and timestamp:
Correlated request / audit transaction ID:
Matched rule ID, phase, target and audit evidence:
Exact permitted methods and path:
Unique local exclusion rule ID:
Exclusion placement and config-test result:
Before / after receipt paths and timestamps:
Negative-control and late-placement results:
Rollback procedure, trigger and evidence:
Change window, decision and approver record:
Published-page verification and owner:
Later crawler / search observation window and limits:
```

Record observed responses and the matching audit evidence; expected results below apply only to the bundled fixture.

| Control | Fixture expectation | Your observation / evidence |
| --- | --- | --- |
| Intended GET and HEAD | After 200; GET tutorial body, HEAD empty | [Complete] |
| Unknown agent | Same public-page result; identity unverified | [Complete] |
| POST, other host/path, plain and encoded suffix | Remain 403 | [Complete] |
| SQL-injection control | Remain 403; rule 942100 retained | [Complete] |
| Private origin and ordinary public page | Remain 401 and 200 respectively | [Complete] |
| Late exclusion | Intended requests remain 403 | [Complete] |
| Rollback | All twelve results match before | [Complete] |

| Handoff | Owner and evidence | Decision / unresolved work |
| --- | --- | --- |
| Public-content and authentication review | [Complete] | [Complete] |
| Exact scope, rule collision and placement review | [Complete] | [Complete] |
| Configuration test, controls and rollback | [Complete] | [Complete] |
| Approval and deployed-page verification | [Complete] | [Complete] |
| Later crawler or search observations | [Complete] | [Complete] |

RankEcho's bounded page-change workflow is separate from native firewall administration. Inspect a [sample fix](https://rankecho.io/sample-fix) or the [Fix Engine](https://rankecho.io/product/fix-engine) for a reviewed owned-page change. Record later observations separately; a successful local exclusion does not establish a live crawler visit or a search gain.

## Complete original guide

ModSecurity 403: Fix One Public-Page False Positive

Find the rule and matched request field before changing access. In this tested nginx example, CRS rule 930130 blocks a public HTML tutorial because its URL contains package.json. A conditional exclusion restores that one host and GET/HEAD path while keeping the firewall active. The same test preserves other paths, write methods, SQL-injection detection and the origin's private-route response.

Guide: https://rankecho.io/resources/fix/cdn-waf-ai-crawler-access

The original article, sources and executed example are dated 13 September 2026. This pack was assembled on 17 September. The repository-relative commands in the original guide below are preserved; use the quick start above to bootstrap an extracted kit. Primary documentation was rechecked for this pack; that review is not a fresh production or provider observation.

### Copy the tested ModSecurity exclusion

This example applies only to the fictional public tutorial at docs.example/docs/package.json. Confirm the actual endpoint is intended for public use and contains no private application files. Save as exclusion.conf and load it before CRS; use a unique local rule ID in your installation.

```apacheconf
# Fictional public tutorial. Confirm the real page contains no private files.
# Load before crs-setup.conf and rules/*.conf in this lab.
# Keep SecRuleEngine On. This grants no bot identity or authentication privilege.
SecRule REQUEST_HEADERS:Host "@streq docs.example" \
    "id:10010,phase:1,pass,nolog,t:none,chain"
    SecRule REQUEST_METHOD "@rx ^(?:GET|HEAD)$" "t:none,chain"
    SecRule REQUEST_FILENAME "@streq /docs/package.json" \
        "t:none,ctl:ruleRemoveTargetById=930130;REQUEST_FILENAME"

```

This removes one filename target from rule 930130 for the matching transaction. It does not authenticate a crawler, disable the engine or grant access to private files. Test your real host, normalization, methods and application contract before deploying.

### What caused the tested 403?

The fictional origin serves an approved public HTML tutorial at /docs/package.json. That route returns explanatory text, not a package file read from disk. A direct origin request can succeed while the same request through the firewall is denied. The endpoint's public status is part of this teaching fixture; it is not a reason to expose a real application's manifests, credentials or source files.

The nginx access response and ModSecurity audit record identify one transaction. Rule 930130 reports a restricted-filename match against REQUEST_FILENAME. Rule 949110 then reports that the inbound anomaly score reached the blocking threshold. These are different roles: disabling the final threshold check would affect far more than the original filename match.

Start a real investigation with the final URL, method, host, response and corresponding event. Record the matching rule ID, variable and value, plus the blocking decision. Check whether the route is intentionally restricted. A search console 403 label, an allowed robots file or a missing citation does not independently identify ModSecurity as the cause.


- [Inspect the tested CRS filename rule](https://github.com/coreruleset/coreruleset/blob/v4.29.0/rules/REQUEST-930-APPLICATION-ATTACK-LFI.conf)
- [Review anomaly scoring](https://coreruleset.org/docs/2-how-crs-works/2-1-anomaly_scoring/)
- [Use the Cloudflare diagnosis when that is the acting layer](https://rankecho.io/learn/why-ai-cant-see-your-cloudflare-site)

### What does the exclusion change?

The copied chain matches docs.example, GET or HEAD, and the exact /docs/package.json filename. Only after all three conditions match does it exclude REQUEST_FILENAME from rule 930130 for that transaction. The host is a scope condition, not an identity credential. A production host with a port, alias or different path normalization needs its own tested configuration.

The browser, Googlebot-labeled and unknown-agent requests all receive the same public tutorial after the correction. There is no User-Agent allowlist in the rule. Someone copying a crawler header gains no authentication privilege: this exact content was approved for any public reader. Separate bot-use preferences and other access controls still belong to their existing owners.

Keep the original CRS files unchanged. The small local exclusion is easier to inspect and remove than a forked vendor rule. Leave the engine in blocking mode and retain the existing threshold. This example is a specific response to a verified false positive, not a generic recommendation to exclude rule 930130 wherever package.json appears.


- [Review conditional target exclusions](https://coreruleset.org/docs/2-how-crs-works/2-3-false-positives-and-tuning/)
- [Verify real Google crawler identity separately](https://developers.google.com/crawling/docs/crawlers-fetchers/verify-google-requests)
- [Keep crawl preferences separate](https://rankecho.io/resources/fix/robots-txt-for-ai-crawlers)

### Why does placement matter?

Our tested rule runs in phase 1, so its exclusion must take effect before that rule executes. In the lab, exclusion.conf is included before crs-setup.conf and rules/*.conf. The nginx configuration keeps modsecurity on and loads the complete ModSecurity rules file. Module enablement and the engine's blocking setting are separate configuration controls.

Moving the identical exclusion below the CRS includes did not repair the request. The filename rule had already contributed its score. The later exclusion did not undo that earlier contribution. This failed placement is included in the executed matrix so the handoff specifies order as well as the copied text.

The local rule ID 10010 is an example. Check for an existing ID before adapting it. Run the server's configuration test, then use the deployment and reload procedure appropriate to your installation. The lab starts isolated processes for each variant; it does not test your production reload mechanism or hosting control panel.


- [Check runtime exclusion placement](https://coreruleset.org/docs/2-how-crs-works/2-3-false-positives-and-tuning/)
- [Review nginx connector directives](https://github.com/owasp-modsecurity/ModSecurity-nginx)

### What did the real firewall tests show?

On September 13, 2026, we executed 12 request cases in each of four configurations: before, corrected, late exclusion and rollback. The fixture used nginx 1.24.0, ModSecurity 3.0.12, connector 1.0.3 and the full CRS 4.29.0 ruleset. nginx and the fictional origin listened only on loopback; requests actually passed through the firewall engine.

Each response was correlated with its JSON audit transaction. Successful GET cases had to return the exact tutorial body, while HEAD had to return no body. The SQL-like query still triggered rule 942100 after the filename exception, with no 930130 match in that corrected transaction. The origin's private endpoint retained its 401 response.

Rollback removed the exclusion and reproduced the before-state status, body checks and rule IDs for all 12 cases. That is an executed local restoration check. It does not establish a production rollback, authentic crawler visit, DNS/TLS behavior, indexing or improved traffic.

| Request group | Before | Corrected | Late / rollback |
| --- | --- | --- | --- |
| Public tutorial GET: browser, Googlebot label, unknown agent | 403 | 200; exact HTML | 403 / 403 |
| Public tutorial HEAD: OAI-SearchBot label | 403 | 200; empty body | 403 / 403 |
| POST, other host/path, plain or encoded path suffix | 403 | 403 | 403 / 403 |
| SQL-like query on the public tutorial | 403 | 403; 942100 retained | 403 / 403 |
| Private origin endpoint | 401 | 401 | 401 / 401 |
| Ordinary public start page | 200 | 200 | 200 / 200 |

- [Inspect the complete runnable fixture](https://github.com/abioty/rankecho/tree/main/test/fixtures/modsecurity_delivery)

### How do you reproduce and adapt the example?

The repository fixture includes exclusion.conf, the pinned runtime manifest, setup.py and run.py. On an Ubuntu 24.04 amd64-compatible Linux system with Python 3.12 and dpkg-deb, run python3 test/fixtures/modsecurity_delivery/run.py from a checkout. The setup downloads checksum-pinned packages into the fixture directory; it does not install or start a system nginx service. The runner prints all observations and closes its temporary servers.

The first run requires network access to the listed Ubuntu and CRS release URLs. The exact package revisions and archive hashes are retained with the results. This is a reproducible teaching environment, not a recommendation to freeze a production server indefinitely at those versions. Recheck compatible supported packages and repeat the controls when upgrading.

For the real issue, replace the fictional host and route only after confirming the endpoint's public contract and its matched variable. Inspect path aliases, encoded forms and trailing segments. Keep unrelated requests in the negative set. If your logs identify another rule or enforcement service, write and test that different correction instead of adapting this exclusion by guesswork.


- [Inspect the exact dependency pins](https://github.com/abioty/rankecho/blob/main/test/fixtures/modsecurity_delivery/runtime.json)
- [Use the Cloudflare-specific fix when appropriate](https://rankecho.io/resources/fix/cloudflare-ai-bot-rules)

### What belongs in the operator handoff?

Completed fictional handoff: retain the public tutorial, add the one chained exclusion before CRS, and leave the engine and remaining rules active. Attach the original 930130/949110 event, exact proposed configuration, successful body check, protected controls, failed late-placement example and restoration result. Private endpoints and unrelated filenames remain outside the exception.

For production, name the hosting operator, approver, deployment window and rollback trigger. Save the before configuration and verify the served response after deployment. Stop or restore the prior version if the intended body remains unavailable, the matched scope widens or a negative control changes. A real package or credential file that should remain private must remain blocked.

Use the remediation ticket to assign firewall work to the hosting team. RankEcho's Fix Engine supports the reviewed page-change workflow when the investigation also identifies a supported owned-page correction. The sample fix previews that handoff; RankEcho is not presented here as a native ModSecurity administrator. Record later crawler and search observations separately from the access repair.


- [Prepare the operator ticket](https://rankecho.io/templates/ai-visibility-remediation-ticket)
- [Preview a reviewed page fix](https://rankecho.io/sample-fix)
- [Explore the Fix Engine](https://rankecho.io/product/fix-engine)
- [Keep later answer observations separate](https://rankecho.io/resources/prove/fixed-prompt-retest)

### Should I disable ModSecurity to fix a crawler 403?

Identify the actual rule first. This example restored one approved public route while retaining the engine, scoring threshold and other rules. It does not justify a global disable or an exception for private files.

### Did Googlebot or OpenAI visit the test site?

No. The client supplied those User-Agent strings over loopback. The test establishes firewall behavior for controlled requests; genuine provider identity and arrivals remain separate evidence.

### Why did the same exclusion fail below CRS?

In the executed configuration, the phase-1 filename rule had already added its score. Moving the exclusion after it did not undo that contribution. The tested working placement precedes the CRS includes.

### Will this fix improve rankings or AI citations?

The local tests establish access behavior, preserved controls and restoration. They do not establish indexing, rankings, citations, referrals or a causal traffic effect.

## Snapshot source

The bootstrap uses the [Ubuntu Snapshot service](https://snapshot.ubuntu.com/) and preserves all original dependency versions and SHA-256 pins. See runtime.json and scripts/prepare-firewall-runtime.py for the exact sources.