READMAIL: A "list-server" like SRE-http addon, with automatic storage of messages in an SRE-http message box. ------------------------ Introduction SRE-http provides two "list-server" like facilities: the READMAIL procedure and the MAILIST.CMD stand alone facility. This document discusses both of these. READMAIL (and MAILIST) has two purposes: rebroadcasting messages (the list-server function), and storing e-mail messages in SRE-http message-boxes (where they can be perused using the FORUM add-on). Basic Requirements: 1) ReadMail should be invoked using a GET or POST request. The simplest method is to issue the following command from your browser (given note 4 below): http://your.server.org/ReadMail? When called in this fashion, ReadMail will use "initialization" variables that are included in the source code. 2) ReadMail is pointless if your server does not have read/write access to an "inbox" directory of a mail server. In many cases you can use OS/2's built-in SENDMAIL as an SMTP mail server; and use it's MAIL directory (see below for a discussion of SENDMAIL). 3) For the "list-server" like broadcast features to work, your server requires access to an SMTP Gateway (as set in the SMTP_GATEWAY SRE-http variable). Again, SENDMAIL can be used for this purpose. 4) Only clients with a SUPERUSER privilege are allowed to invoke this procedure. 5) RECOMMENDATION: If you expect a fair amount of incoming mail, or will "broadcast" to a moderately sized list of recipients, you should probably use the MAILLIST.CMD "stand along" version of READMAIL. See below for a short discussion of MAILLIST.CMD. Configuring ReadMail: There are two methods of invoking this procedure: 1) Using request string options to process messages for a single discussion group 2) Setting parameters in the READMAIL.CMD file, which allows you to process messages for multiple discussion groups simultaneously. Method 1 is safer for nervous programmers: you don't have to edit the source code. To demonstrate how one might do this, a simple HTML document, READMAIL.HTM, is included. Method 2 is more powerful, but does require you to "open up the code". However, READMAIL.CMD is extensively commented, and the variables to change are not obscure. Therefore, with reasonable prudence (i.e.; make a backup copy first), this should pose few difficulties to most webmasters. In either case, the same variables can be set. These are: INBOX_DIR -- the incoming mail storage directory. By default, this is the MAILBOX_DIR of SRE-http. FORUM.n -- name of a "discussion group" USERLIST.n -- Allowed posters to the discussion group BROADCASTLIST.n -- List of e-mail addresses to broadcast submissions to. AUTOSUBSCRIBE.n -- Allow SUBSCRIBE/UNSUBSCRIBE. AUTOVERIFY -- Send a "verification" message to posters. MAIL_ARCHIVE -- Where to archive messages. The .n variables (FORUM.n, USERLIST.n, BROADCASTLIST.n, and AUTOSUBSCRIBE.n) are "stem" variables, where n takes integer values from 1.. the number of "discussion groups" you are supporting. Note that when using method 1 (the request string options), you can NOT specify .n -- since only 1 "discussion group" is processed at time (essentially, a value of n=1 is used). ----------------------------------- Short description of user configured variables: INBOX_DIR: Required. The directory your mail server(i.e.; SENDMAIL) uses to store incoming messages. This must be a fully qualified directory name; and must be directly accessible from your server (either locally, or on a LAN). If not specified, the MAILBOX_DIR directory of SRE-http is used. If you intend to use the WRITE_TO_FILE option of the SRE-http message facility, we recommend using this default value (so long as externally generated e-mail is also stored in the MAILBOX_DIR directory). Example: INBOX_DIR='C:\MPTN\ETC\MAIL' (this might be the SENDMAIL "Mail directory when using another mail package" directory) The next several stem variables refer to n=1..N possible "discusson groups". If specified as request string options, drop the ".n". FORUM.n: Required. A name (possibly with / in it). These refer to "message boxes" (as used by SRE-http); the .LOG files in the MESSBOX_DIR directory. Examples, assuming MESSBOX_DIR=D:\GOSERVE\MESSAGE FORUM.1='FORUMX' D:\GOSERVE\MESSAGE\FORUMX.LOG file is used. FORUM.2='HISTORY/GREEK' D:\GOSERVE\MESSAGE\HISTORY\GREEK.LOG file is used. In all cases, the appropriate file MUST exist. That is, you have to create it before messages can be posted to it (typically, you would create a text file with a short description at the top). Note: any leading / is stripped -- the message box MUST be under the MESSBOX_DIR! USERLIST.n: Required. A filename. If not fully qualified, then it is assumed to be relative to the MESSBOX_DIR. A userlist is required for each FORUM.n entry. The file should contain a list of "allowed posters" for "forum.n"; the list should contain E-Mail addresses (with possible wildcards). Examples; USERLIST.1='FORUMX.USR' USERLIST.2='D:\USERLIST\HISTORY.USR' Note that the FORUMX.USR is assumed to in the MESSBOX_DIR (i.e.; D:\GOSERVE\MESSAGE). See the bottom of this document for a sample of a USERLIST file. BROADCASTLIST.n: Optional. A file name. If not fully qualified, then it is assumed to be relative to the MESSBOX_DIR. Assuming the file exists, messages will be rebroadcast to all e-mail addresses in this list. If wildcard entries are found, they are ignored. Examples: BROADCASTLIST.1='FORUMX.BRD' AUTOSUBSCRIBE.n: Optional. A 0, 1, or 2. 0 = Ignore this option 1 = allow automatic subscription; the USERLIST is used as the BROADCASTLIST, and SUBSCRIBE and UNSUBSCRIBE "subjects" are used to modify the USERLIST. 2 = same as 1, except the poster is NOT sent the message. Use of AUTOSUBSCRIBE provides a "list server" like method of allowing self-subscription to the discussion group. If you do not use AUTOSUBSCRIBE, you will need to modify the USERLIST (and the BROADCASTLIST) files directly. In Other Words: When AUTOSUBSCRIBE is on (for a given FORUM.n entry), a message sent to the discussion group, with a subject field of SUBSCRIBE, causes the poster's E-mail address to be added to the appropriate USERLIST. Similarly, UNSUBSCRIBE causes it to be removed. Examples: AUTOSUBSCRIBE.1=0 AUTOSUBSCRIBE.2=2 Note: when using AUTOSUBSCIRBE, the use of wildcard (*) entries in the USERLIST is not recommended. These next two variables are global to all discussion groups. AUTOVERIFY: 0 or 1. Send a verification note to the message sender. O = Do not send a verification message 1 = Send a verification message to the poster. Note: for automatic verification, you must have a valid entry in the SRE-http SMTP_GATEWAY variable. Example: AUTOVERIFY=1 MAIL_ARCHIVE: The fully qualified name of an archive directory to move messages to. Set MAIL_ARCHIVE=0 if you want to delete messages after recording them. Note: the MAIL_ARCHIVE directory MUST be on the same disk as the INBOX_DIR (but not necessarily in the same directory tree). Example: MAIL_ARCHIVE='C:\MPTN\ETC\MAIL\ARCH1' ----------------------------------- Miscellaneous Subjects Short example using request string options: READMAIL?INBOX_DIR=dir_name&FORUM=forum_2&USERLIST=usr_file&AUTOVERIFY=0 In this example, ONLY messages sent to the forum_2 message-box will be procesed (using usr_file as the userlist), with no verification. :::: Sample of a UserList file: ;comment lines can begin with ; DRROBERT@*.*.* DRROBERT@*.* *@*.clink.net *@BIGCOMP.COM SUSAND@METALINK.NET NOT Mailer-Daemon@*.*.* NOT Mailer-Daemon@*.*.*.* Note that DRROBERT@A.B.C does NOT match DRROBERT@*.* hence the need for two entires (one with *.*.*, the other with *.*). If you want a completely open discussion group (anyone can post to it), then your UserFile should have a single asterisk in it. For example: ;let everyone in * Although you can put other e-mail addresses in such a UserList, it has no net effect (the * signals "match all addresses"). Fact to keep in mind when using wildcards: > The * character, by itself, is a general wildcard. It is used to allow free access -- all posters are allowed in. > While a convenience, be aware that in the event of errors (say, incorrect e-mail addresses), this can lead to problems. > In particular, error messages mailed to you by the SMTP server software may end up being interpreted as legitimate messages, with subsequent rebroadcast. > To avoid this, we recommend use of NOT entries -- when an e-mail address matches a NOT entry, the message is NOT processed. Of course, you can also use NOT entries to keep out pests (who may have addresses that you'ld otherwise like to wildcard). > Note that NOT entries "trump" any wildcard match. However, an exact entry can take precedence over a NOT entry (but it must be located earlier in the userlist file). :::: Sample of a broadcastlist file: ; comments are allowed SUSAND@METALINK.NET BIF@COSMO.BELL.EDU GRODY@MAXTER.COM Note that * (wildcard) entries are NOT allowed in the broadcastlist file. :::: Sample of a E-mail message: From DD@econ.ag.gov Thu Oct 24 00:32:31 1996 Received: from MAILBOX.econ.ag.gov by DANIELH.ECON.AG.GOV (IBM OS/2 SENDMAIL VERSION 1.3.14/2.12um) id AA0202; Thu, 24 Oct 96 00:32:31 -0400 Date: Thu, 24 Oct 96 00:32:31 -0400 Message-Id: <9610240432.AA0202@DANIELH.ECON.AG.GOV> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" To: DD discussion group <> From: Message processor Subject: This is a test .... contents of message :::: Sample of file generated by WRITE_TO_FILE option of SRE-http's message facility: Date: Fri, 25 Oct 1996 00:17:08 From: danielh hellerstein To: SAMPLES2@DANIELH.ECON.AG.GOV X-Archive: NO Subject: this is true .... contents of message . Note the X-Archive: NO field. It has special meaning to READMAIL and MAILIST. When such a field occurs, READMAIL and MAILLIST will NOT store the file into a message box. This is used by the MESSAGE facility to transfer the task of broadcasting to MAILIST, without causing multiple entries in the message box file. :::: Using FORUM with READMAIL: The ReadMail, the MESSAGE, and the FORUM facilities of SRE-http are designed to work in a complementary fashion. For example, the "BROADCAST" options of MESSAGE can be used to distribute mail. Similarly, ReadMail stores mail in message boxes that are designed to be read by the FORUM facility. The notion is to provide a flexible combination of "store messages recieved via E-mail or by HTML FORMS", in "messages boxes that can be displayed with the FORUM facility", and with optional distribution (via e-mail) regardless of how the message was entered. There is one minor difference: when messages are recieved via E-mail, they will be stored in the message box as "seperate threads". That is, hierarchical responses to messages can only be entered via the HTML interface. Note that the broadcast option of MESSAGE can be quite time consuming. If you are generating a lot of e-mail due to broadcasting, you should investigate the use of the MAILLIST facility in conjunction with the WRITE_TO_FILE option of MESSAGE (see below for more discussion of MAILLIST and rebroadcasting messages to a forum). ::::: The MAILLIST.CMD "stand alone" variant of READMAIL: If you expect to process a large number of incoming messages, or expect to send a lot of mail (say, you have allowed broadcasting to a large list of recipients), you should probably use the MAILLIST.CMD program instead of READMAIL. MAILLIST is run from an OS/2 command prompt, and is independent of the web server portion of SRE-http. In most other respects, it is identical to READMAIL. The source code for MAILIST contains extensive comments; here we mention some broad issues: 1) Analogously to READMAIL, parameters can be set either by editing the MAILLIST.CMD file, or by including parameters on the command line invocation of MAILLIST. For example: D:\GOSERVE>MAILLIST INBOX_DIR=dir_name FORUM=forum_2 USERLIST=usr_file AUTOVERIFY=0 2) There are several extra parameters that MAILLIST requires (that SRE-http supplies to READMAIL). These include your SMTP Gateway's address, the VERBOSE level, the name of your server, and your message box directory. 3) The biggest advantages of MAILLIST is that it can be run in the background, and it will not be effected by GoServe's "timing out" requests. This can be especially important when mailing to a long list of addresses; if this takes a few minutes, you risk GoServer "killing the inactive thread". Furthermore, the ability to run in the background (say, waking up every 15 minutes to check for incoming mail) means you can put it on automatic pilot and forget about it (assuming that nothing untoward occurs!). 4) Another major advantage of MAILLIST is that it can coexist with ULTIMAIL. See the next section for details. 5) Since MAILLIST uses SRE-http message boxes, MAILLIST provides the same access to a "list-server/news-group" capability as that provided by READMAIL. 6) Since MAILLIST is independent of SRE-http, you MUST specify an INBOX_DIR directory. 7) If you want to use ULTIMAIL (or some other mail package) to process "personal" mail sent to your server, you should be sure that: 1) SENDMAIL put's incoming mail in the MAILLIST (or READMAIL) INBOX_DIR directory. 2) The OTHER_MAILBOX directory is pointing to ULTIMAIL's incoming directory. 3) If you are using ULTIMAIL, set OTHER_ULTIMAIL=1. :::::: Using MAILLIST to rebroadcast messages delivered to a forum. One nice feature of the MAILIST add-on is it's synchronization with the SRE-http "FORUM" facility. In particular, it is relatively easy to both store, and broadcast, message submitted to a forum; where the submission can be either thorugh E-mail or through a web-entered message. The above documentation discussed how one can use MAILLIST (or READMAIL) to transfer e-mail to SRE-http message boxes. In this section, we'll discuss how "web-entered" messages can be broadcast to a list of e-mail addresses. Basically, SRE-http's messages box writer (see SAMPMBOX.HTM for and example) can write "web-entered" messages to SRE-http message boxes (which are simply structured ascii files). The goal is to also to broadcast these messages to a list of e-mail addresses. To do this, you will have to modify the document that contains your "submit to a SRE-http message box" FORM. In particular, you should add the following option (assuming you are using an HTML form): When SRE-http see this, it will create a special "mail-file", which will then be processed by MAILLIST (see the E-mail sample above). Of course, use of this element has no effect on how, or where, the submitted message is stored -- it just writes additional information to your disk. Summarizing: With some attention to detail, messages submitted via e-mail, and messages that are "web-entered", will be treated equivalently. That is, both types of messages can be stored in (the same) SRE-http message box, and both types of messages can be broadcast to (the same) list of e-mail addresses. :::::: A Discussion of SENDMAIL: The OS/2 SENDMAIL program can be used as both an SMTP_GATEWAY and as an SMTP mail server. Configuration of SENDMAIL is best done using the TCPCFG configuration utility; you should see the on-line help for details. Here, a few important facts are mentioned: 1) When used as an SMTP Gateway (a mail sender), be aware that SENDMAIL is somewhat slow, and has limitations when broadcasting messages to multiple (>150) addresses. Of particular concern: if are mailing a number of messages, and you have GoServe's "end client after inactive" value set too low, it is quite likely that you will time out (with unpredictable effects on mail delivery). To prevent this, ReadMail will monitor the remaining time, and stop if time is running out. In such cases, you may need to submit several requests for ReadMail. An even better solution is to use the MAILFILE "stand alone" variant of READMAIL. 2) When used as an SMTP server (end destination), be aware that SENDMAIL can store mail in one of several directories. For example: i) if SENDMAIL is started from an OS/2 command prompt using: C:>START SENDMAIL -bd -q30m then the SENDMAIL.CF is used, with a mail storage directory set by the "When Using another mail package" setting on the "Configuring Sendmail Parameters" screen of TCPCFG. Typically, this is the MAIL subdirectory of the directory pointed to by the ETC environment variable (i.e.; C:\MPTN\ETC\MAIL). ii) if SENDMAIL is AUTOSTARTed (see the AUTOSTART page of TCPCFG), the default configuration file is usually SENDMAIL.UML -- with a mail storage directory set by the "When Using Ultimail .." set on the "Configuring Sendmail Parameters" screen. typically the UMAIL\SERVER\INBOX directory of the TCP/IP directory. 3) If you want to use UltiMail and MAILLIST at the same time, you'll need to make a few modifications to SENDMAIL and MAILLIST.CMD variables. Basically, you can't share a directory with ULTIMAIL, it's too greedy about snarfing up an files it discovers in it's incoming mailbox. Therefore, it's necessry to give MAILLIST the first crack at incoming messages, and then transfer unused messages to the Ultimail mail directory. The following outlines the necessary steps: a) In sendmail.cf, have the "mlocal" parameter point to your SRE-http "mailbox" directory. Since SENDMAIL.CF is a scary file (VERY picky about syntax), the best way to do this is to: i) Run TCPCFG.EXE (from an OS/2 command prompt); ii) open the SENDMAIL option, iii) click on the "When Using Another Mail Package" button, iv) change the MAIL DIRECTORY field to the SRE-http "mailbox" directory v) depending on your LAN & INTERNET environment, you might also need to play with the MAIL-ENVIRONMENT choices under the MAIL option of TCPCFG. a.1) Whilst in the SENDMAIL option of TCPCFG, you should also click on the "When Using Ultimail or Mailing from NewReader/2" button, and record (but do not necessarily modify) the Mail Directory field (a typical value of which might be: c:\tcpip\umail\server\inbox) b) Start sendmail with sendmail.cf as the "configuration" file. You can do this from the command line with: c:>SENDMAIL -bd -Cx:\aaa\bbb\SENDMAIL.CF (x:\aaa\bbb is often C:\MPTN\ETC) Or, if you want SENDMAIL to " auto start" on powerup: i)Run TCPCFG (from an OS/2 command prompt) ii)change the -Cfilename portion of the "autostart - sendmail- parameters" field Note: there may be more then one copy SENDMAIL.CF on your system (i.e.; one in D:\MPTN\ETC, and one in D:\TCPIP\DOS\ETC. Be sure you select the correct one! c) In the MAILLIST.CMD program, set the OTHER_MAILBOX parameter to equal the "mail" directory of your mail program. * If you are using ULTIMAIL, this will be the value you recorded in step a.1 above (i.e.; c:\tcpip\umail\server\inbox). d) If you are using Ultimail, set OTHER_ULTIMAIL=1. If you do not do this, a special .UML (used by Ultimail to identify it's mail) file will not be created in the ultimail mailbox. With these steps in place, MAILLIST.CMD and Ulti-mail should coexist peacefully.