CrowdSec on CloudPanel: Mind the Log Format

[drop_cap]I[/drop_cap]’ve written before about the background radiation of the internet: the endless automated noise crawling over every reachable IP. Besides creative filtering on the edge, my preferred tooling answer to that noise is now CrowdSec: an open-source security engine that reads your logs, detects bad patterns, and drops bans straight into your firewall. Fail2ban’s bigger sibling, with a crowd-sourced blocklist to boot.

It works great on most systems, but there are a few things to know when deploying it on CloudPanel. The two important ones are:

1. CloudPanel’s nginx access-log format is not the standard nginx combined format. This matters especially when traffic passes through Cloudflare.
2. On the CrowdSec v1.8.1 Debian pragmatic arm64 build tested here, the usual message parser field was empty. Custom parsers need to apply directly to Line.Raw.

Here’s the full walkthrough of how you can get CrowdSec working on your CloudPanel box:

Install CrowdSec

Add the CrowdSec repository and install the agent:

curl -s https://install.crowdsec.net | sudo sh
apt install crowdsec

Some CrowdSec versions offer a guided cscli setup step during or after installation. For this tutorial, leave unrelated sources such as syslog, SSH, SQL, Docker, and application logs unchecked. The nginx collection and acquisition configuration are installed explicitly below so the setup stays small and predictable.

Fixing Port Conflicts

On many CloudPanel installs, port 8080 may already be occupied, so you can avoid that conflict by moving to a different one, such as 127.0.0.1:8081.

In /etc/crowdsec/config.yaml:

listen_uri: 127.0.0.1:8081

The same port needs to appear in:

/etc/crowdsec/local_api_credentials.yaml
/etc/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml

The bouncer’s API URL should be:

api_url: http://127.0.0.1:8081/

Restart the agent after changing the configuration:

systemctl restart crowdsec
systemctl is-active crowdsec
cscli lapi status

At this point the agent should be running and the Local API should authenticate successfully. Next, we are going to setup Nginx parsing.

Install the Nginx Collection

Install the collection that provides the stock nginx parser and HTTP scenarios:

cscli collections install crowdsecurity/nginx

The custom parser below handles CloudPanel’s format; the collection still supplies the scenarios that decide when parsed requests should be banned.

Acquire CloudPanel Logs

CloudPanel doesn’t keep nginx in the default /var/log/nginx/. It keeps each site’s nginx logs under its home directory. To make CrowdSec aware of your site-specific logs add this to /etc/crowdsec/acquis.yaml:

filenames:
    - /home/*/logs/nginx/access.log
labels:
    type: nginx

This covers the NGINX bit only. You can add syslog and others, but for this tutorial we focus only on nginx since this is where the difficult parts are with CloudPanel.

The CloudPanel Parser

Stock CrowdSec’s nginx parser expects a normal combined log line:

1.2.3.4 - - [07/Sep/2026:10:00:00 +0300] "GET / HTTP/1.1" 200 123 "-" "curl/8.0"

CloudPanel appends $http_x_forwarded_for:

1.2.3.4 - - [07/Sep/2026:10:00:00 +0300] "GET / HTTP/1.1" 200 123 "-" "curl/8.0" "1.2.3.4"

That final field is enough to make the stock parser choke. A strict regex that anchors both ends of the line sees one extra field and drops it.

The solution? Create a local parser at:

/etc/crowdsec/parsers/s01-parse/cloudpanel-nginx.yaml

The important parts are the trailing XFF field and apply_on: Line.Raw. For this tutorial, we assume all traffic is proxied through Cloudflare, so the parser uses the Cloudflare-aware source_ip expression directly. Give the local parser a name: field. CrowdSec may ignore an unnamed parser file.

name: example/cloudpanel-nginx
onsuccess: next_stage

pattern_syntax:
  CP_NOTDQUOTE: '[^"]*'
  CP_USER: '[a-zA-Z0-9.@-+_%]+'

nodes:
  - grok:
      apply_on: Line.Raw
      pattern: '(%{IPORHOST:target_fqdn}(:%{INT:port})? )?%{IPORHOST:remote_addr} - (%{CP_USER:remote_user} )?[%{HTTPDATE:time_local}] "%{WORD:verb} %{DATA:request} HTTP/%{NUMBER:http_version}" %{NUMBER:status} %{NUMBER:body_bytes_sent} "%{CP_NOTDQUOTE:http_referer}" "%{CP_NOTDQUOTE:http_user_agent}" "%{CP_NOTDQUOTE:http_x_forwarded_for}"'
      statics:
        - meta: log_type
          value: http_access-log
        - target: evt.StrTime
          expression: evt.Parsed.time_local

statics:
  - meta: service
    value: http
  - meta: source_ip
    expression: >-
      (IpInRange(evt.Parsed.remote_addr, '173.245.48.0/20') ||
       IpInRange(evt.Parsed.remote_addr, '103.21.244.0/22') ||
       IpInRange(evt.Parsed.remote_addr, '103.22.200.0/22') ||
       IpInRange(evt.Parsed.remote_addr, '103.31.4.0/22') ||
       IpInRange(evt.Parsed.remote_addr, '141.101.64.0/18') ||
       IpInRange(evt.Parsed.remote_addr, '108.162.192.0/18') ||
       IpInRange(evt.Parsed.remote_addr, '190.93.240.0/20') ||
       IpInRange(evt.Parsed.remote_addr, '188.114.96.0/20') ||
       IpInRange(evt.Parsed.remote_addr, '197.234.240.0/22') ||
       IpInRange(evt.Parsed.remote_addr, '198.41.128.0/17') ||
       IpInRange(evt.Parsed.remote_addr, '162.158.0.0/15') ||
       IpInRange(evt.Parsed.remote_addr, '104.16.0.0/13') ||
       IpInRange(evt.Parsed.remote_addr, '104.24.0.0/14') ||
       IpInRange(evt.Parsed.remote_addr, '172.64.0.0/13') ||
       IpInRange(evt.Parsed.remote_addr, '131.0.72.0/22') ||
       IpInRange(evt.Parsed.remote_addr, '2400:cb00::/32') ||
       IpInRange(evt.Parsed.remote_addr, '2606:4700::/32') ||
       IpInRange(evt.Parsed.remote_addr, '2803:f800::/32') ||
       IpInRange(evt.Parsed.remote_addr, '2405:b500::/32') ||
       IpInRange(evt.Parsed.remote_addr, '2405:8100::/32') ||
       IpInRange(evt.Parsed.remote_addr, '2a06:98c0::/29') ||
       IpInRange(evt.Parsed.remote_addr, '2c0f:f248::/32')) &&
      evt.Parsed.http_x_forwarded_for matches '^[0-9a-fA-F.:]{7,45}
  
  




 ?
      evt.Parsed.http_x_forwarded_for : evt.Parsed.remote_addr
  - meta: http_status
    expression: "evt.Parsed.status"
  - meta: http_path
    expression: "evt.Parsed.request"
  - meta: http_verb
    expression: "evt.Parsed.verb"
  - meta: http_user_agent
    expression: "evt.Parsed.http_user_agent"

When traffic is proxied through Cloudflare, $remote_addr is a Cloudflare edge address and the real client is in $http_x_forwarded_for. The parser should check the remote address against Cloudflare’s published ranges and use XFF only when it is a clean IP.

The expression uses CrowdSec’s IpInRange helper. It trusts XFF only when the connecting address belongs to Cloudflare and the XFF value is a clean IP. Otherwise, it keeps the connecting address. Keep the CIDR list synchronized with Cloudflare’s current IPv4 and IPv6 ranges from their published IP list.

Keep the CIDR list synchronized with CloudPanel’s /etc/nginx/cloudflare/ips:

clpctl cloudflare:update:ips

This refreshes CloudPanel’s nginx include file. It does not automatically rewrite the CIDR list inside the CrowdSec parser, so update that list if Cloudflare publishes new ranges. You can set up a cronjob to keep them in sync but that’s out of scope for this tutorial.

After creating or editing parser files, restart CrowdSec:

systemctl restart crowdsec

Whitelist Trusted IPs

To avoid accidentally blocking yourself while testing and ironing out false positives, create a whitelist before enabling remediation:

/etc/crowdsec/parsers/s02-enrich/example-whitelist.yaml

Example contents:

name: example/whitelists
description: "Trusted administrator addresses"
whitelist:
  reason: "trusted administrator IP"
  ip:
    - "192.0.2.44"
    - "2001:db8::44"

Replace the example addresses above with your own trusted public IPv4 and IPv6 addresses, then reload CrowdSec:

systemctl restart crowdsec

Add Remediation

The CrowdSec agent only detects and creates decisions. A bouncer enforces them. If your CloudPanel host uses nftables:

apt install crowdsec-firewall-bouncer-nftables
systemctl enable --now crowdsec-firewall-bouncer

Check that the bouncer is authenticated and that nftables has its tables:

cscli bouncers list
nft list table ip crowdsec
nft list table ip6 crowdsec6

The community blocklist is pulled through the Central API and is also delivered to the bouncer. No GUI or hosted CrowdSec console is required:

cscli capi status

Verify Before Trusting It

The most important command is:

cscli metrics show acquisition

The healthy state is:

• Lines parsed equals Lines read
• Lines unparsed displays -
• Lines poured to bucket eventually becomes non-zero

If Lines unparsed contains numbers, the engine is reading logs but not evaluating them. That is not a quiet server. It is a blind one.

Other useful commands:

# Active decisions, including origin and alert ID
cscli decisions list

# Full history where retained
cscli decisions list --all

# The story behind a decision
cscli alerts inspect 

# Installed collections and parsers
cscli hub list
cscli collections list
cscli scenarios list

Every decision has an Alert ID. Treat cscli decisions list as the index and cscli alerts inspect as the story: exact paths, user agents, timestamps, and source metadata.

The One-Line Summary

CloudPanel’s nginx format needs a custom parser to work well with CrowdSec. On the v1.8.1 Debian pragmatic arm64 build tested here, that parser must use apply_on: Line.Raw, not message. Verify with acquisition metrics to make sure everything is set up correctly.

And yes, before the comments section gets clever: there is probably a CloudPanel module, a clpctl one-liner, or a checkbox somewhere that fixes all of this natively. I looked, but couldn’t find any, so I wired up this custom parser for CloudPanel. Took me a bit to get the grok syntax right, but works pretty well.

The post appeared first on André Klein Dot Net.