Optimizing a FreeBSD system for large numbers of users

From the makers of InspIRCd.

This tutorial will provide you with the basic information needed to run InspIRCd on a FreeBSD system and fit as many users onto your ircd as is possible, within the limits of your RAM and processor.

This tutorial is mercilessly ripped from a similar document in the ircu source tarball -- i give them credit here, and i hope they don't mind that i've heavily based this tutorial upon their writings. Most of the information they gave was not specific to ircu but to tuning any ircd which will run under FreeBSD, so therefore, it is useful.

This tutorial also applies to NetBSD 2.0, OpenBSD 3.4 (or later), and DragonFly.

When running on FreeBSD, you may compile InspIRCd to support the kqueue() event engine, which results in much improved performance compared to poll and select engines which are enabled in InspIRCd as default. Unless your copy of FreeBSD is over 6 years old, you are likely to have kqueue support in your kernel. When you run ./configure for InspIRCd, you will be asked if you wish to enable kqueue support if your system supports it:

You are running a BSD operating system.
Would you like to enable kqueue support?

Enable kqueue? [yes] ->

In order to serve many clients simultaneously, you need to increase the maximum allowable number of open files in the system. To do this, add commands such as the following during your system's boot sequence:

sysctl -w kern.maxfiles=16384
sysctl -w kern.maxfilesperproc=16384

You might find you need to put them in /boot/loader.conf.local:

kern.maxfiles=16384
kern.maxfilesperproc=16384

Unless you will be serving thousands of clients simultaneously, you will not need to do the following, unless of course you just can't stand having a system that is not optimized to its limits.

• Build a custom kernel: Make your kernel as lean as possible by removing all drivers and options you will not need. The following parameters will affect performance, they are listed with suggested values only. For more information on what they do exactly, see FreeBSD's documentation. Please note: This information may be invalid, as it is based on a relatively old document. If you can suggest improvements, or these kernel options simply do not work (or break your system) feel free to correct this page.

maxusers        2048
options         NMBCLUSTERS=65535
options         ICMP_BANDLIM

In versions of FreeBSD after 4.x the sysctl variable net.inet.icmp.icmplim is used instead.

• You may wish to set the following at system startup (from /etc/sysctl.conf, or whichever other method you prefer):

net.inet.tcp.rfc1323=1
net.inet.tcp.delayed_ack=0
net.inet.tcp.restrict_rst=1
kern.ipc.maxsockbuf=2097152
kern.ipc.somaxconn=2048

Changing SOMAXCONN allows you to accept more connections at once. The default of 256 should be ok for most users, but if you wish to accept very large numbers of connections in a relatively short space of time, you should increase this.

One final word of warning: When you configure InspIRCd, the maximum number of users you specify is exactly that, a maximum number of users not a maximum number of open file descriptors. Therefore, do not set this to your very highest limit, or you can run your process, or your machine, out of file descriptors and cause yourself all kinds of hassle. When a user connects quite a few more file descriptors (and ram!) may be used by their initial connection, for example by DNS lookups and ident requests.