Protecting Your Custom Application with Fail2Ban

Tutorials
1

Ever deployed a simple web service only to find your logs filling up with automated scanning attempts? Or watched as the same IP hammers your endpoint over and over? That’s where Fail2Ban comes in - a tool that monitors your log files and automatically bans offending source IP addresses.

This tutorial walks you through integrating Fail2Ban with a custom Python application. Rather than just configuring Fail2Ban for standard services like SSH, you’ll learn how to protect your own applications by teaching Fail2Ban to understand your specific log format.

Prerequisites

This tutorial assumes you have:

  • A Linux system with fail2ban installed (apt install fail2ban or the equivalent of your distribution)
  • Python 3.x (using only built-in libraries)
  • Basic familiarity with log files and regular expressions
  • Root or sudo access for fail2ban configuration

The Example Application

To demonstrate Fail2Ban integration, we’ll use a minimal HTTP server that does one thing only: logging every request in a structured format. Here’s the complete application:

#!/usr/bin/env python3
import logging
from http.server import HTTPServer, BaseHTTPRequestHandler
from datetime import datetime

# Configure logging for fail2ban
logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - IP: %(message)s',
    datefmt='%Y-%m-%d %H:%M:%S',
    handlers=[
        logging.FileHandler('/tmp/fail.log'),
        logging.StreamHandler()
    ]
)

logger = logging.getLogger(__name__)

class FailRequestHandler(BaseHTTPRequestHandler):
    """Handle HTTP requests and log them for fail2ban."""

    def do_GET(self):
        """Handle GET requests."""
        # Extract client IP (check for proxy headers first)
        client_ip = self.headers.get('X-Forwarded-For')
        if client_ip:
            client_ip = client_ip.split(',')[0].strip()
        else:
            client_ip = self.client_address[0]

        # Extract User-Agent
        user_agent = self.headers.get('User-Agent', 'Unknown')

        # Get current timestamp
        timestamp = datetime.now().strftime('%Y-%m-%d %H:%M:%S')

        # Log the request (fail2ban will parse this)
        logger.info(f"{client_ip} - User-Agent: {user_agent}")

        # Generate response
        html_content = f"""FAIL - Request Information

IP Address:  {client_ip}
User-Agent:  {user_agent}
Time:        {timestamp}
Path:        {self.path}
"""

        # Send response
        self.send_response(200)
        self.send_header('Content-type', 'text/plain; charset=utf-8')
        self.end_headers()
        self.wfile.write(html_content.encode('utf-8'))

    def log_message(self, format, *args):
        """Override to suppress default HTTP server logging."""
        pass

def run_server(host='0.0.0.0', port=8000):
    """Start the HTTP server."""
    server_address = (host, port)
    httpd = HTTPServer(server_address, FailRequestHandler)
    print(f"[*] Server running on http://{host}:{port}")
    print(f"[*] Logging requests to /tmp/fail.log")
    print(f"[*] Press Ctrl+C to stop")
    try:
        httpd.serve_forever()
    except KeyboardInterrupt:
        print("\n[*] Server stopped")
        httpd.server_close()

if __name__ == '__main__':
    run_server()

The key here is the log format: YYYY-MM-DD HH:MM:SS - IP: <ip_address> - User-Agent: <user_agent>. This format makes it easy for Fail2Ban to extract the information it needs.

Start the server and test it:

python fail.py

In another terminal:

curl http://localhost:8000

Check the log:

tail /tmp/fail.log

You should see entries like:

2025-10-08 14:23:15 - IP: 127.0.0.1 - User-Agent: curl/7.81.0

Understanding Fail2Ban’s Architecture

Fail2Ban operates with two main components that work together:

Filters define what to look for in your logs. They use regular expressions to extract IP addresses from log lines. Think of filters as the pattern-matching engine.

Jails define what to do when a pattern matches. They specify which log file to monitor, how many matches constitute misbehavior (maxretry), the time window to consider (findtime), and how long to ban the offender (bantime).

This separation means you can reuse the same filter with different jail configurations, or apply multiple jails to the same log file with different thresholds.

Creating the Filter

Create /etc/fail2ban/filter.d/fail.conf:

# Fail2ban filter for custom application
# Log format: YYYY-MM-DD HH:MM:SS - IP: <ip_address> - User-Agent: <user_agent>

[Definition]

# Failregex pattern to match IP addresses in the log
failregex = ^.* - IP: <HOST> - User-Agent:.*$

# Ignoreregex - lines to ignore (none for now)
ignoreregex =

Let’s break down that regex:

  • ^.* - Match any characters at the start (timestamp)
  • - IP: <HOST> - Match our IP field. <HOST> is a Fail2Ban special token that captures IP addresses (both IPv4 and IPv6)
  • - User-Agent:.*$ - Match the rest of the line

<HOST> is a Fail2Ban special token that handles the complexity of IP address matching for you - no need to write separate patterns for IPv4 vs IPv6.

Test your filter before deploying:

# Generate a few log entries first
for i in {1..3}; do curl http://localhost:8000; sleep 1; done

# Test the filter
fail2ban-regex /tmp/fail.log /etc/fail2ban/filter.d/fail.conf

You should see output indicating successful matches:

Lines: 3 lines, 0 ignored, 3 matched, 0 missed

If you see misses, double-check your log format matches the regex pattern exactly.

Creating the Jail

Create /etc/fail2ban/jail.d/fail.conf:

# Fail2ban jail configuration for custom application

[fail]

# Enable this jail
enabled = true

# Port to ban (must match application port!)
port = 8000

# Filter to use (must match filename in filter.d/)
filter = fail

# Path to the log file to monitor
logpath = /tmp/fail.log

# Maximum number of retries before banning
maxretry = 5

# Time window (in seconds) to count retries - 10 minutes
findtime = 600

# Ban duration (in seconds) - 1 minute for demo, use 3600+ in production
bantime = 60

# Action to take when banning
action = %(action_)s

The parameters here define your security policy:

  • maxretry = 5: Ban after 5 requests
  • findtime = 600: Count requests within a 10-minute window
  • bantime = 60: Ban for 1 minute

So this reads as: “If the same IP makes 5 requests within 10 minutes, ban them for 1 minute.” Thats obviously not exactly useful in a production setting, but makes banning and unbanning visible during testing. Once you’re happy, increase bantime to 3600 or more.

The action = %(action_)s uses Fail2Ban’s default action, which typically means iptables rules. You can customize this to send emails, update cloud firewall rules, or trigger webhooks.

Critical note: The port value must match your application’s port. Fail2Ban blocks the port, not just the IP, so a mismatch means your ban won’t affect traffic to your application.

Deploying and Testing

Copy your configurations and restart Fail2Ban:

sudo cp filter.d/fail.conf /etc/fail2ban/filter.d/
sudo cp jail.d/fail.conf /etc/fail2ban/jail.d/
sudo systemctl restart fail2ban

Verify your jail is active:

sudo fail2ban-client status fail

You should see:

Status for the jail: fail
|- Filter
|  |- Currently failed: 0
|  |- Total failed:     0
|  `- File list:        /tmp/fail.log
`- Actions
   |- Currently banned: 0
   |- Total banned:     0
   `- Banned IP list:

Now trigger a ban by making rapid requests.

This requires using different hosts for running the application and connecting, as Fail2Ban usually is configured to avoid self-banning. We’ll use a host to run the application on the 192.168.1.32 address, and 192.168.1.33 to connect to it.

for i in {1..6}; do curl http://192.168.1.32:8000; sleep 1; done

Check the jail status again:

sudo fail2ban-client status fail

Bingo! You should see your IP in the banned list:

`- Banned IP list:   192.168.1.33

Try accessing the service:

curl http://192.168.1.32:8000

The connection should fail or timeout. That’s Fail2Ban’s iptables rule blocking your traffic.

Unban yourself:

sudo fail2ban-client set fail unbanip 192.168.1.33

What’s Actually Happening

When you trigger a ban, here’s the sequence:

  1. Your application writes to /tmp/fail.log
  2. Fail2Ban’s jail watches this file using the filter pattern
  3. Each matching line increments a counter for that IP
  4. When the counter exceeds maxretry within findtime, Fail2Ban executes the ban action
  5. The default action adds an iptables rule: iptables -I INPUT -s 192.168.1.33 -p tcp --dport 8000 -j REJECT
  6. After bantime seconds, Fail2Ban removes the iptables rule

You can watch this in real-time:

# Terminal 1: Watch the log
tail -f /tmp/fail.log

# Terminal 2: Watch Fail2Ban's activity
sudo tail -f /var/log/fail2ban.log

# Terminal 3: Watch iptables rules
watch -n 1 'sudo iptables -L -n | grep -A 5 fail2ban'

Adapting for Your Application

The pattern shown here works for any custom application. The steps are:

  1. Structured logging: Log in a consistent format with the IP address clearly delimited
  2. Create a filter: Write a regex that captures <HOST> from your log format
  3. Test the filter: Use fail2ban-regex to validate before deploying
  4. Configure a jail: Set appropriate thresholds for your use case
  5. Monitor: Watch the logs to verify bans are triggered correctly

For production deployments, consider:

  • Increasing bantime to 3600 (1 hour) or 86400 (1 day)
  • Adjusting maxretry based on legitimate traffic patterns
  • Setting up ignoreip in your jail for trusted IPs
  • Using action_mwl to send email notifications with log excerpts
  • Ensuring log rotation doesn’t interfere with Fail2Ban monitoring

The official Fail2Ban documentation covers advanced configurations, including custom actions and complex filter patterns. For regex testing, Regex101 works well - just remember to use Python regex flavor.

Conclusion

You’ve now integrated Fail2Ban with a custom application, learning the core concepts of filters and jails in the process. This same approach works whether you’re protecting a REST API, an M2M communication protocol, an IoT device’s admin interface… basically any application that is able to write structured logs.