How to install SMTPfeed 1. Installation of SMTPfeed 1.1 Do "./configure" If you want to specify directory for installed bind4, use switch: --with-bind4=/usr/local/bind4 If you want to link bind8, use switch: --with-bind8 If you also want to specify directory for installed bind8, use switch: --with-bind8=/usr/local/bind8 If you want to enable IPv6 feature, use switch: --enable-ipv6 Destination directory for installation can be specified with switch: --libexecdir= (default of destination directory is /usr/libexec or /usr/lib (in case /usr/libexec is missing).) If you want to link libraries statically, use switch: --with-static (-static will be added to LDFLAGS if CC is gcc, -Bstatic will be added otherwise.) If you want to make FD_SETSIZE lager than system default, use switch: --with-fds=NN (here, NN is a number for FD_SETSIZE. See select(2) for default of FD_SETSIZE. Use limit(1) to see limit of file descriptors on your system. 1.2 Edit "Makefile" (If you need) 1.3 Do "make all" 1.4 Do "make install" 1.5 Configure "syslog.conf" For example, declare as follows: local5.info /var/log/smtplog Default facility is "local5". facility can be changed with command line switch of smtpfeed. do not forget to touch logfile and to send signal to make syslogd reload (kill -HUP {PID of syslogd}). 1.6 test edit TEST.in and try: % smtpfeed -dA -lstderr < TEST.in switch -dA is to get all debugging messages and switch -lstderr is to see the messages on a terminal. 2. Compilation of sendmail 2.1 Get a copy of sendmail.8.11.x archived package and extract files from the archive. 2.2 Apply 3.3W.patch You may need new version of "GNU patch", because the .patch file includes "diff"s to non-existent files. 2.3 Do "sh Build" Do not forget to configure Makefile around DB or BIND to fit your site. 3. Configuration of sendmail.cf In this section, three ways are described: A. Changing current sendmail.cf directly B. Using m4 technique C. Using CF configuration tool 3.A Changing current sendmail.cf directly Do not forget to save current version of sendmail.cf. 3.A.1 Definition of smtpf mailer Duplicate definition lines on smtp (or esmtp, smtp8 etc.) mailer and make changes as follows: * Change mailer name into "smtpf" * Change value for P= equate from P=[IPC] to P=/usr/libexec/smtpfeed (location smtpfeed is installed) * Append flag z to F= equate (to use LMTP) and verify that flag m is included already. * Change value for A= equate to "smtpfeed" (with command line switches if needed) For example, if SMTPfeed does not recognize FQDN of your hostname, specify -c $j as a command line switch. Note that every host names appeares in right hand side of any MX RRs of DNS must defined with -c or -a switch of SMTPfeed so that loopback detection will be performed certainly. Specify -i$i to make queueID of a message be logged in syslog. When a delivery is splitted into multiple transactions because of large number of recipients, the queueID information will not be passed to 2nd or later transaction (cleared by RSET command). If you want to pass the queueID information to later transaction, specify $?{runqueue}-i$i$|-I$i$. instead (only with sendmail 8.9.1 +3.1W or later). This means -i$i will be passed when in runqueue processing and -I$i will be passed when in immediate delivery processing. You can also specify -E option instead of -i/-I options if you trust "id" portion of Received: header line as a queueID inserted by sendmail. In this case, queueIDs for all transactions in runqueue process will be obtained. * line terminating character (E= equate) may be "\n" or "\r\n". (example) Msmtpf, P=/usr/sbin/smtpfeed, F=mDFMuXz, S=11/31, R=21, E=\n, L=990, T=DNS/RFC822/SMTP, A=smtpfeed 3.A.2 Replace smtp (or esmtp, smtp8) mailer into smtpf mailer (at mailer selection rules) When you change mailer name on rules, the host portion (after $@) of the mailer dispatching triple also be changed to be a constant character string like "LMTP" or something else. This is required to pass all recipients at a time. (example) R$* < @$* > $* $#esmtp $@ $2 $: $1 < @ $2 > $3 must be changed to R$* < @$* > $* $#smtpf $@ LMTP $: $1 < @ $2 > $3 ^^^^^^^^ TAB 3.A.3 disable $[ $] operators Comment out all rules which use $[ $] operators for address completion since process by $[ $] may cause delay. But this is not required to use SMTPfeed. (example) R$* < @ $* $~P > $* $: $1 < @ $[ $2 $3 $] > $4 should be changed to #R$* < @ $* $~P > $* $: $1 < @ $[ $2 $3 $] > $4 ^^^^^^^^^^ TAB 3.B use of m4 technique use the following definitions in .mc file: FEATURE(nocanonify)dnl MAILER_DEFINITIONS Msmtpf, P=/usr/sbin/smtpfeed, F=mDFMuXz, S=11/31, R=21, E=\n, L=990, T=DNS/RFC822/SMTP, A=smtpfeed MAILER(smtp)dnl define(`SMART_HOST', `smtpf:LMTP')dnl If you are using mailertable feature, specify 'smtpf:LMTP' in database. 3.C use of CF configuration tool use the following definitions in .def file: LOCALHACKMAILER='# smtp feed mailer\ Msmtpf, P=/usr/libexec/smtpfeed, F=mDFMuXz, S=11, R=21,\ T=DNS/RFC822/SMTP, E=\n, L=990,\ A=smtpfeed' # specify A=smtpfeed -c $j in case hostname is not a FQDN. DIRECT_DELIVER_DOMAINS=none DEFAULT_RELAY='smtpf:LMTP' Be sure about install path of smtpfeed described at P= equate. If you are using mailertable feature, specify 'smtpf:LMTP' in database. And test as follows with sendmail.cf created by ways of A, B or C: > 3,0 user@domain rewrite: ruleset 3 input: user @ domain rewrite: ruleset 96 input: user < @ domain > rewrite: ruleset 96 returns: user < @ domain > rewrite: ruleset 3 returns: user < @ domain > rewrite: ruleset 0 input: user < @ domain > rewrite: ruleset 88 input: < smtpf : LMTP > . user < @ domain > rewrite: ruleset 88 returns: $# smtpf $@ LMTP $: user < @ domain > rewrite: ruleset 0 returns: $# smtpf $@ LMTP $: user < @ domain > It is OK if smtpf is selected. 4. Farther configuration of sendmail.cf 4.1 Number of recipients par transaction In specification of STMP (RFC822), maximum number of recipients par transaction is defined as 100. There is another limitation in sendmail caused by static buffer size for syslog, so actual maximum number is less than 100. When DYNAMIC_TOBUF feature is used, the maximum is relaxed to 100. But there is still a limitation by the specification. By this specification, parallelism of SMTPfeed will not be fully utilized. This limitation can be more relaxed locally, you can make sendmail pass all recipients at once to gain performance with fully parallelism. If you apply 3.4W.patch to sendmail, r= equate at mailer definition is available in sendmail.cf to specify maximum number of recipients par transaction from sendmail to SMTPfeed. If you specify r= equate like: Msmtpf, P=/usr/sbin/smtpfeed, F=mDFMuXz, S=11/31, R=21, E=\n, L=990, T=DNS/RFC822/SMTP, r=1000, A=smtpfeed in sendmail.cf, you can relax maximum number of recipients par transaction to 1000. Default value is 100. If there is enough memory, the value for r= should be more than number of recipients in every mailing-lists. If ConnectionCacheSize in sendmail.cf is defined with 1 or more, the same smtpfeed process manages multiple transactions. If you want make new smtpfeed process for each transactions, set ConnectionCacheSize to 0. With 3.2W.patch, you can suppress caching feature on specific mailer with mailer flag ';'. 4.2 Suppression of SMTPfeed execution for few recipients If recipient is only one for a message, smtpfeed should not be executed since execution of SMTPfeed requires one more process resource but no speed improvement will be performed. Moreover, SMTPfeed stores all information in core memory, delivery of a huge message is not fit to SMTPfeed. So, execution of smtpfeed should be controlled by number of recipients or size of a message. Here is a sample configuration with MULTI_MAILER feature provided by 3.4W.patch. With this MULTI_MAILER feature, you can specify multiple mailers in mailer dispatching rule sets as follows: R$* < @$* > $* $#smtpf $@ LMTP $: $1 < @ $2 > $3 $#esmtp $@ $2. $: $1 < @ $2 > $3 (Be sure that there is no TAB between $3 at the end of 1st line and $# at the top of 2nd line. There are TABs only between $* and $# in 1st line.) By using this rule, if delivery by 1st mailer is failed with TEMPFAIL, delivery will be re-tried with 2nd mailer. Trial with 1st mailer can be suppressed by settings on least number of recipients (*= equate on smtpf mailer definition line (starts with M) or largest size of a message (M= equate). There are also 2 mailer flags to control the suppression: mailer flag '!' is for suppression on run-queue process, mailer flag '_' is not for use of following mailer if the 1st mailer is used. Sample full specification of the above is: Msmtpf, P=/usr/sbin/smtpfeed, F=mDFMuXz!_, S=11/31, R=21, E=\n, L=990, *=5, T=DNS/RFC822/SMTP, M=50000, A=smtpfeed By this definition, If there is 2nd mailer in dispatching rule and number of recipients is less than 5, delivery with smtpfeed will be skipped (If there is no 2nd mailer, definition *=5 is ignored and smtpfeed will be used). If M= equate is defined, when size of a message is larger than the size specified (50000) and 2nd mailer exists, delivery with smtpfeed will be skipped instead of bouncing the message. If mailer flag '!' is specified, the mailer will be used only for the 1st trial of delivery and will not be used by 2nd or later run-queue process. If mailer flag '_' is specified, when the mailer is used for trial of delivery, following mailer will not be used in spite of result of delivery (this mailer flag should be used with *= or M= equates). Configuration to use MULTI_MAILER feature with CF may be as follows: LOCALHACKMAILER='# smtp feed mailer\ Msmtpf, P=/usr/libexec/smtpfeed, F=mDFMuXz!_, S=11, R=21,\ T=DNS/RFC822/SMTP, E=\n, L=990, M=50000, *=5,\ A=smtpfeed' DIRECT_DELIVER_MAILER=smtpf DIRECT_DELIVER_DOMAINS=all #DEFAULT_RELAY= DSPTLOCALHACK='R.$* $#smtpf$@LMTP$:$2$#smtp$@$1$X$:$2' ^TAB Be sure about install path of smtpfeed described at P= equate. Configuration to use MULTI_MAILER feature with m4 technique may be as follows: FEATURE(nocanonify)dnl MAILER_DEFINITIONS Msmtpf, P=/usr/sbin/smtpfeed, F=mDFMuXz!_, S=11/31, R=21, E=\n, L=990, T=DNS/RFC822/SMTP, M=50000, *=5, A=smtpfeed MAILER(smtp)dnl define(`SMART_HOST', `smtpf:LMTP')dnl LOCAL_CONFIG # These rules should be inserted at the top of S95 S95 R< smtpf : LMTP > $*<@$*>$* $#smtpf$@LMTP$:$1<@$2>$3$#smtp$@$2.$:$1<@$2>$3 R< smtpf : $* > $* $#smtpf$@LMTP$:$2$#smtp$@$1.$:$2 R< smtpf > $*<@$*>$* $#smtpf$@LMTP$:$1<@$2>$3$#smtp$@$2.$:$1<@$2>$3 ^^^^^^^^^^ TAB 4.3 Timeout on LMTP DATA state When sendmail executes SMTPfeed, it waits for termination of SMTPfeed and gets delivery status by response of DATA command of LMTP. If SMTPfeed is not terminated during a period, sendmail will give up to wait the termination and assume deliveries for all recipient were not successful and save the message to queue for next trial. By this reason, if SMTPfeed will not be finished during timeout period, some recipients may be accept copies of the message twice or more. To avoid duplication, SMTPfeed must be finished during timeout period. By default, SMTPfeed will be finished within 57 minutes (If it takes 55 minutes, SMTPfeed switches to termination process (sending QUIT commands) and wait for response of QUIT commands (at most 2 minutes)). Then, SMTPfeed will reports delivery status at the moment. So, only recipients to which deliveries are not completed are queued. Value for this timeout can be set by -t total= command line option of SMTPfeed. If you set 60 minutes, specify -t total=60m (will take 62 minutes at most). Timeout at side of sendmail can be configured by 'O Timeout.datafinal=' option in sendmail.cf. Default is 1 hour (1h). 5. Delivery test with new sendmail and SMTPfeed # newsendmail -C newsendmail.cf addresses < file Please verify logged messages of maillog or smtplog. 6 Replace of sendmail and sendmail.cf, and restart sendmail Good luck! 7 Caution BIND resolver implemented in SMTPfeed will utilize additional information section of answers sent from BIND servers. To avoid problems reported by "CERT Advisory CA-97.22: Topic: BIND - the Berkeley Internet Name Daemon," version of BIND servers which serve for SMTPfeed must be 4.9.6/8.1.1 or later. There are, of couse, other problems reported by "CERT Advisory CA-98.05," you should use 4.9.7/8.1.2 or later. Your mail server will receive many many IDENT(RFC1413) request from SMTP peers at a time since the SMTPfeed will open much many SMTP connections at once. This may introduce your server into heavily loaded condition if "identd" executed from "inetd" with "nowait" mode. In this case, you may want to disable "identd" or execute daemon "identd" to avoid such bad condition. If you want to reduce number of connections requested at a time, use option -n of the SMTPfeed instead of reducing number as a parameter for r= equate of mailer definition in sendmail.cf. Reducing value at r= makes the SMTPfeed less effective.