PureMessage for UNIX: open file limitations on Solaris

This article is relevant if you are running PureMessage on Solaris, and you write your own custom filters that open files or sockets using Perl code. It may also be relevant if you wish to tune the PureMessage run-time parameters.

The Solaris stdio subsystem has a limit of no more than 256 open files in a single process. This is a hard-coded limit in the system's C library that cannot be changed. All file or socket operations in Perl require stdio file handles and are therefore subject to this limit.

In a multi-threaded application such as PureMessage, this means that the concurrency achievable by an individual process in the system is also subject to this limit if PureMessage filters open files or sockets and require that they remain open during the lifetime of the filter application.

What to do

When writing custom filters, take the following precautions:

• Avoid opening files or sockets within PureMessage filters if the operation can be performed without doing so. Often, opening temporary files can be avoided by using in-memory operations.
• File and socket handles must be treated like a precious resource due to the stdio limit. If you are opening files or sockets, open them immediately before reading from or writing to them, and close them as soon as possible. Keeping the file or socket open reduces the concurrency that the system can handle.

In addition to the stdio file limit, the UNIX kernel normally has an adjustable limit on the number of file descriptors that can be opened by a single process. When it starts, PureMessage automatically attempts to adjust this limit to its maximum allowed value (the maximum normally allowed by default is 1024 in most versions of Solaris).

If PureMessage reports errors such as "Bad file descriptor" or "Too many open files" in its logs, try one of the following:

• Ensure that the soft limit for the maximum number of system file descriptors (not the same as the stdio limit described above) in a single process is at least 1024. Running ulimit -n should report this number.

  To change it, add the following two lines to your /etc/system file:

  set rlim_fd_cur = 1024set rlim_fd_max = 4096

  Reboot the system to apply the change.

• Disable the use_threads option in pmx.conf (located in the /opt/pmx/etc/ by default).
• Limit the number of concurrent sendmail connections by setting the MaxDaemonChildren option in sendmail.cf.