skip to content

System: fail2ban and iptables

Around the beginning of 2005 we saw an increase in brute-force ssh attacks - people or robots trying different combinations of username and password to log into remote servers. A quick search on this topic returns many references to iptables and ipchains but noone really explained how they work.

Having just gone through this learning curve myself, and found a satisfactory solution in the fail2ban package, I'm going to try and explain how to achieve the simple goal of banning IP addresses that make repeated failed ssh login attempts.

If you want more technical information regarding firewalls and iptables in particular, see the References section at the bottom of this page.

The following relates to the default config file provided with fail2ban 0.6.0 in the Debian unstable distribution and should be used for information purposes only. We take no responsibility for any consequences of following the instructions on this page.

What is/are iptables?

iptables is the firewall administration program for the Netfilter firewall mechanism which is built into the Linux kernel.

To view your current 'firewall chains' you need to run the following command as root:

# iptables -L

This will show a list of 'chains' called INPUT, FORWARD and OUTPUT - these are always present - followed by any custom chains. The default (blank) settings will look something like:

Chain INPUT (policy ACCEPT) target prot opt source destination Chain FORWARD (policy ACCEPT) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination

If that's what you're seeing then there are no existing firewall rules and it's probably safe to proceed. Otherwise you may want to check with the sysadmin who installed any existing firewall rules before making changes.

You can also use iptables -L INPUT to list just the INPUT chain. Other useful options are to add -n to display values in numerical format or -v for verbose mode. Finally iptables -h will display all command-line options.

Installing and configuring fail2ban

The fail2ban package is available under Debian/unstable and also as a download for other Linux systems. See the Fail2Ban website linked under Resources at the bottom of the page for details. To install on Debian:

# apt-get -t unstable install fail2ban

If you run this command then fail2ban will be installed and already running as a daemon. However you might want to edit the configuration file and stop/start the daemon to get it running how you want. The configuration file can be found at /etc/fail2ban.conf.

The configuration file is broken up into sections. Most entries don't need to be changed but there are a few that you might want to edit. The DEFAULT settings apply to all sections:

[DEFAULT]

maxfailures = 3 bantime = 900 findtime = 600

This says that after three failed access attempts are detected from a single IP address within 600 seconds or 10 minutes (findtime), then that address will be automatically blocked for 900 seconds (bantime). In most cases you won't see them again as the bots will move on to another server.

[MAIL]

enabled = true to = root@localhost

Needs to be set to true for emails to be sent - to the root user in this case. Only do this if you're actually going to check the root mailbox otherwise you're just wasting disc space.

[Apache]

enabled = false

For now we're not going to worry about monitoring the Apache log files.

[SSH]

enabled = true logfile = /var/log/auth.log

fail2ban will monitor the auth.log file for failed access attempts. As soon as the daemon is running your ssh port (22) will be protected from brute-force attacks - preventing more than a small number of attempts at one time.

To check if it's running:

# /etc/init.d/fail2ban status Status of fail2ban: fail2ban is running.

These commands will differ for different flavours of Linux.

To stop/start the daemon after making configuration changes:

# /etc/init.d/fail2ban stop Stopping fail2ban: .done # /etc/init.d/fail2ban start Starting fail2ban: .done

Actions taken by the daemon are logged by default in /var/log/fail2ban.log and you can change the verbosity in the conf file to one of: 0 - WARN, 1 - INFO or 2 - DEBUG.

What does fail2ban do with iptables?

This code runs when the daemon is started and adds new firewall rules using iptables:

fwstart = iptables -N fail2ban-ssh iptables -A fail2ban-ssh -j RETURN iptables -I INPUT -p tcp --dport ssh -j fail2ban-ssh

Translation:

  • create a NEW chain called fail2ban-ssh;
  • APPEND a RETURN command to the end of the fail2ban-ssh chain (same as return in PHP or other languages);
  • INSERT a rule at the start of the built-in INPUT chain that redirects SSH packets to the fail2ban-ssh chain.

This adds the following to the output of iptables -L:

Chain INPUT (policy ACCEPT) target prot opt source destination fail2ban-ssh tcp -- anywhere anywhere tcp dpt:ssh Chain fail2ban-ssh (1 references) target prot opt source destination RETURN all -- anywhere anywhere

Because there are no actual rules in the fail2ban-ssh chain, connection attempts using SSH are simply redirected from the INPUT chain to the fail2ban-ssh chain and then sent straight back. What it does mean however is that we (or the fail2ban daemon to be precise) can insert new rules at any time and they will be applied to all incoming SSH traffic.

The fwend commands simply reverse this process, removing the JUMP command and and then FLUSHing and DELETEing the fail2ban-ssh chain:

fwend = iptables -D INPUT -p tcp --dport ssh -j fail2ban-ssh iptables -F fail2ban-ssh iptables -X fail2ban-ssh

You should always stop fail2ban before editing the config file - so that it cleans up existing rules./p>

To ban an IP address the following command is run, with <ip> replaced by the actual IP address or hostname captured by the failregex regular expression (see below):

fwban = iptables -I fail2ban-ssh 1 -s <ip> -j DROP

Translation:

  • Insert a rule as line 1 of the fail2ban-ssh chain to DROP all packets from the listed IP address or hostname.

When a fwban event is triggered, the output of iptables -L fail2ban-ssh --line-numbers becomes:

Chain fail2ban-ssh (1 references) num target prot opt source destination 1 DROP all -- <host> anywhere 2 RETURN all -- anywhere anywhere

where <host> is the problem ip address or domain. While this rule is in effect all ssh requests from <host> will be silently dropped. If a second fwban even occurs while the first is in place it will appear as line 1 and the other commands moved down the chain.

The fwunban command simply removes the DROP command from the chain. It's called after 10 minutes or whatever time you have set for the bantime in fail2ban.conf:

fwunban = iptables -D fail2ban-ssh -s <ip> -j DROP

The failregex regular expression

The most complicated part of the configuration file is the regular expression that is applied to auth.log to determine what to count as an access failure.

failregex = : (?:(?:Authentication failure|Failed [-/\w+]+) for(?: illegal user)?|Illegal user|Did not receive identification) .* from (?P<host>\S*)

The regular expression used here has been optimised for the auth.log in Debian Linux (Sarge) and may need to be modified slightly for other platforms.

In short, this matches any/all of the following auth.log entries and triggers a fwban event with the ip address or domain (host) that's been captured by the regular expression:

Authentication failure for username from <host> Failed password for illegal user username from <host> Failed password for username from <host> Failed keyboard-interactive/pam for username from <host> Illegal user username from <host> Did not receive identification string from <host>

For anyone trying to reverse-engineer the regular expression:

  • (?:.*) is a non-grouping version of regular parentheses; and
  • (?P<name>.*) is similar to regular parentheses, but the substring matched by the group is accessible via the symbolic group name name.

The single matched variable <host> will be added to the fail2ban-ssh chain if maxfailures or more matches occur within findtime. It's that simple.

You will see a lot of different (and usually simpler) failregex patterns that don't contain the <host> capture. The reason for having the named capture in the regexp is to counter a bug that allows malicious users to trick your server into blocking non-hostile addresses:

"fail2ban's approach to identifying an IP address in a login failure line is to scan the line for all IP addresses.

Since it is possible to generate false logins from accounts such as 10.2.28.2, it is possible to force fail2ban to block access to addresses which are not attempting to connect to the system. For each IP address available to the attacker, a desired ip address may be blocked."

Customising the Config file

From reading more about iptables in the Linux Firewalls book (very informative but tough reading) and the online resources listed below it appears that some of the iptables commands contain options that are redundant:

  • the RETURN command in fwstart is not necessary as a user-defined chain will automatically return to the parent chain after processing all commands (unless another JUMP is encountered). The only advantage I see in including the RETURN command is that iptables with the -v option can then show you both the number of packets/bytes sent to and returned from fail2ban-ssh - the difference telling you how many were dropped.
  • the rule number 1 in fwban is also redundant as commands will by default be INSERTed at the top of the chain.

The fwstart command then becomes just:

fwstart = iptables -N fail2ban-ssh iptables -I INPUT -p tcp --dport ssh -j fail2ban-ssh

It's also possible to use the LOG feature in iptables rather than relying on the fail2ban program. This gives the advantage of recording the details of packets that are DROPped while a ban rule is in place.

This involves a change to the fwban command:

fwban = iptables -A fail2ban-ssh -s <ip> -j LOG --log-prefix "Fail2Ban: " --log-ip-options --log-tcp-sequence --log-tcp-options iptables -A fail2ban-ssh -s <ip> -j DROP

With the RETURN command removed we can use APPEND instead of INSERT here.

Translation:

  • add a DROP command for <ip> at the start of the chain;
  • add a LOG command with all options enabled at the start of the chain (before the DROP command).

and to fwunban:

fwunban = iptables -D fail2ban-ssh -s <ip> -j DROP iptables -D fail2ban-ssh -s <ip> -j LOG --log-prefix "Fail2Ban: " --log-ip-options --log-tcp-sequence --log-tcp-options

Note: DROP commands must exactly match the INSERT command syntax (or specify a rule number). The order isn't really important but you'll notice that we add the LOG command first and remove it last.

Now your logfiles will record all details of the dropped packages - typically in kern.log or syslog depending on your server configuration. Be aware that this can cause those files to grow quite quickly so only add the LOG commands if you're going to use the data collected.

Does it work?

Does it ever!?!

Here you can see the results for a single server - you can tell when fail2ban was installed and why it was necessary:

$ zcat /var/log/auth.log* | grep 'Failed password' | grep sshd | awk '{print $1,$2}' | sort -k 1,1M -k 2n | uniq -c 408 Dec 11 77 Dec 12 12153 Dec 13 9390 Dec 14 1033 Dec 18 3743 Dec 19 4167 Dec 20 2789 Dec 25 9200 Dec 26 15742 Dec 27 281 Dec 28 7 Dec 29 17 Dec 30 3 Dec 31 4 Jan 1 5 Jan 2 4 Jan 4 9 Jan 5 11 Jan 6 9 Jan 7

There's word of some big changes in the fail2ban package including 'better configuration files' and 'templates for common services' among other things but it's not clear when that might make it into package form.

If you just want to rate-limit SSH connections without installing new software then you can use the recent module but be aware that this can use a lot of system resources on busy networks as it has to store state information for all connections.

References

< System

User Comments

Post your comment or question

23 December, 2014

This was a most informative article. I am learning more and more about security on the Linux/Unix side of the house and this filled in many holes for me.

13 October, 2014

Thanks, this was very informative. I love the way you explained each little part for inquisitive minds like mine!

18 June, 2014

In Ubuntu 12.04 LTS I had to change the apache-auth.conf regex filter in /etc/fail2ban/filters.d/apache-auth.conf from "password mismatch" (lowercase) to "Password Mismatch" (Proper Case) in order to get the regex to work. Just thought I'd share.

Thanks. And yes, it's important to check your log files for the exact syntax you want to ban, and then to test your jails by running fail2ban-regex over some live logs.

22 May, 2012

Hi, first of all thank you for such a good article. I use fail2ban and it seems to work quite well, I use shorewall action filter. Now everything works well and it almost fullfill my goals. Trying to understand how it works and my little knowledge of programming, I have come across, that I would like to ban who appear as DROP on my shorewall log, the jail will set the number of DROP required in a length of time. My problem is that I don't know how to make a proper failregex for the shorewall log. This is a sample of line I would like to capture:

May 22 12:20:04 isp kernel: Shorewall:net2fw.DROP:IN=eth1 OUT= MAC=00:0c:76:ad:5b:15:00:23:48:c6:7a:ec:08:00 SRC=85.105.227.144 DST=192.168.1.254 LEN=60 TOS=0x00 PREC=0x00 TTL=46 ID=24146 DF PROTO=TCP SPT=55248 DPT=23 WINDOW=5840 RES=0x00 SYN URGP=0

I would appreciate it you would point me out to the right place, thank you.

It should be something like this:

failregex = Shorewall:net2fw.DROP.*SRC=<HOST>

You can test it first on the command-line:

# fail2ban-regex /var/log/kern.log "Shorewall:net2fw.DROP.*SRC=<HOST>"

26 February, 2011

Hi, I've created a filter for apache-error-ban which aims to block repeated script abuse, however it is banning all missing files (including images), so for instance normal website users and search engine spiders are being blocked when an image is missing. The filter is: failregex = [[]client <HOST>[]] File does not exist:

Can you think of a way I can allow images (.jpg) using this?

Thanks

There's a built-in filter apache-noscript in the latest version of Fail2Ban, which includes the following:

failregex = [[]client <HOST>[]] (File does not exist|script not found or unable to stat): /\S*(\.php|\.asp|\.exe|\.pl)

9 October, 2010

Nice and very informative tutorial.Thank you very much. Your work is deeply appreciated.

25 September, 2010

I get:

fail2ban.actions.action: ERROR iptables -N fail2ban-apache

using centos python2.6

31 August, 2010

Can I block an entire IP range?

I installed Fail2Ban, and it just works.
But, the it's easy to the blocked user to change ip address.

Can Fail2Ban block the entire XX.XXX.* range?

Fail2Ban won't do that automatically, but you can manually block IP ranges manually in iptables using a netmask/subnet. For more on this checkout our articles on Fighting Form Spammers and Calculating Subnets

27 August, 2010

I modified the command in section 6 to have a better ordered result:

zcat /var/log/auth.log* | grep 'Failed password' | grep sshd | awk '{print $1,$2}' | sort -k 1,1M -k 2n | uniq -c

30 April, 2008

Um, your article helped me A LOT. Unfortunately, i have some bad guys in my apache log, and i think, my regex is wrong

I want to block everybody who tries to scan for nonexistent files/folders, because my sites are static HTML, so i know that all files i link do exist.

Also, is it possible, to make 2 regex in one filterset?

I think the regex you need is:

failregex = [[]client <HOST>[]] File does not exist:

And no, you can't have two regex's in one filterset. However in the latest version of Fail2Ban you can have as many rules running as you want which can have the same effect.

See also the example in this article which is similar, but restricted just to .php requests.

12 July, 2006

Great HOWTO. Just a note: for Fedora Core, the RPM works great. Edit the .conf file as noted above, then use 'chkconfig fail2ban on' to activate fail2ban on reboot, and 'service fail2ban start' to start it up immediately.

21 February, 2006

Super HOWTO. Exactly what I was looking for. Thank you

31 January, 2006

Woouuaaah... What a great howto Excellent work! Really! Could I put a link to your page on Fail2ban website?
Thank you

top