Sites behind Cloudflare: CrowdSec and the web application firewall
This guide is for a WebPortico site whose name is proxied by Cloudflare (the orange cloud) or served through a Cloudflare Tunnel. The examples use a XenForo forum. The same steps apply to WordPress, phpBB and the other applications the firewall knows.
Which address WebPortico sees
Behind Cloudflare, every request reaches the NAS from one of Cloudflare's addresses. Cloudflare names the visitor in the CF-Connecting-IP header. WebPortico's front (nginx) takes that header only from Cloudflare's published address ranges. WebPortico ships that list, and the agent checks Cloudflare's list once a day and updates the file when it changes. A request from anywhere else keeps its own address, whatever header it sends.
So the front's logs, CrowdSec, the firewall's log and PHP's REMOTE_ADDR all see the visitor's address. A site served through the Cloudflare Tunnel gets the same result: its listener takes the header from cloudflared on the NAS itself.
A block never hits Cloudflare's own addresses
- The front looks up the visitor's address in CrowdSec's block list, never the address of the Cloudflare server the request came through. If a Cloudflare address were on the list, the visitors behind it would still get in.
- When the logs do show a Cloudflare address (a request that came through Cloudflare without the header), CrowdSec does not block it: the hub's
crowdsecurity/cdn-whitelist, which WebPortico ships, leaves Cloudflare's addresses unblocked. Loopback addresses and the private IPv4 ranges (your LAN) get no alert at all. - A request sent to the NAS directly, past Cloudflare, counts under its own address. A forged
CF-Connecting-IPheader does not help a blocked address.
WebPortico's tests check the first and the third point on a real nginx.
The order: what to turn on first
| Step | Where | What happens to visitors | How long |
|---|---|---|---|
| 1. Firewall in Detect only, Application set | the site's Firewall tab | nothing is blocked | 7 to 14 days |
| 2. CrowdSec on for the site | the CrowdSec page | blocked addresses get error 403 | check daily for the first 2 days, then weekly |
| 3. Firewall in Block | the site's Firewall tab | requests that look like attacks get error 403 | once the list shows only attacks for a week |
CrowdSec judges what an address does over time: many requests for pages that do not exist, known exploit paths, repeated failed logins. A forum member rarely does any of that. The firewall judges the content of each request, and forum posts often contain code, shell commands and HTML. So the firewall needs a trial period, and CrowdSec mostly does not.
A site added since WebPortico 0.32.0, and a site the Plesk import brought in, starts with the firewall in Detect only. CrowdSec starts off for every site.
Step 1: the firewall in Detect only
- Open the site's Firewall tab. Check that Mode is Detect only.
- Set Application to XenForo (or the application the site runs). This turns on the rule exclusions made for that application, such as the ones for writing a post in the editor.
- Click Apply changes.
- Use the site normally for a week or two, including the things few people do: posting code, uploading attachments, editing styles and templates in the admin panel, searching.
- Open What the firewall caught. A row marked Would be blocked is a request the firewall would refuse in Block mode. If it came from normal use, click Allow on this path (or Allow on whole site for a rule that fires everywhere), then Apply changes.
What to expect on a XenForo forum, even with the XenForo exclusions on:
- Posts and conversations that quote shell commands or system paths (
cat /etc/passwd,rm -rf,wget … | sh). These match CRS's remote command execution and file access rules. On a forum about servers or NAS units this is the most common case. - Posts that quote SQL or HTML with scripts, for example in a support thread.
- Searches for the same kind of text.
- The admin panel's style, template and add-on pages, which send HTML and code by design.
- Attachments: the firewall reads the first 4 MiB of a request body. A larger upload is checked in that part only and is never refused for its size.
Step 2: CrowdSec on
- Open the CrowdSec page and turn on the switch for the site. Click Apply changes.
- Blocking starts at once. WebPortico has no trial mode for CrowdSec, so watch it closely at the start.
- For the first two days, look at Recent alerts and Blocked addresses once or twice a day. An alert names the scenario, the site and the paths that set it off.
- If a real visitor was blocked, enter the address under Unblock an address and click Unblock. The address goes to Allowed addresses, and CrowdSec does not block it again until you remove it. This works for addresses on the community blocklist too.
What to look for: a scenario such as http-probing or http-crawl-non_statics set off by an address you recognize (a monitoring service, your own office), or one address that many members share (a company's or a mobile network's gateway).
Turning CrowdSec on also sends the addresses it blocks, with the reason, to CrowdSec's central API, and brings in CrowdSec's community blocklist.
Step 3: the firewall in Block
When What the firewall caught has shown only attacks for a week (requests for .env, wp-login.php on a forum, SQL injection in a URL), set Mode to Block and click Apply changes. Keep checking the list for a few days after the switch.
Cloudflare's own firewall
WebPortico does not send its blocks to Cloudflare's edge. The front refuses a blocked address with one lookup in a file it already has in memory, so the gain would be small. A Cloudflare IP list on the Free, Pro and Business plans holds at most 10,000 items across all lists. The CrowdSec page shows how many addresses the community blocklist holds on your NAS (Blocked by the community blocklist): compare it with that limit. Cloudflare's own security settings (its WAF rules, Bot Fight Mode, rate limiting) work in front of WebPortico and do not need any change in it.
What to check now and then
| Where | What it tells you |
|---|---|
| CrowdSec page, Status | the service runs, the community blocklist is connected, when the block list last changed |
| CrowdSec page, Recent alerts | which addresses set off which scenarios on which site |
| the site's Firewall tab, What the firewall caught | rules hit, paths, how many requests, blocked or would be blocked |
logs/waf/<site>.audit.log | the firewall's full log (rule messages, never request bodies or cookies) |
logs/nginx/<site>.access.log | the visitor's address on every request |