SRE-http

The SRE-http WWW Server

Users Guide For Version 1.3


1) Introduction.

Welcome to SRE-http, version 1.3h (and counting). For those of you who've stumbled upon this document:
SRE-http is an http/1.1 compliant WWW Server for OS/2. SRE-http is written in REXX, and uses the IBM EWS GoServe program.
Narrowing our focus just a bit ...
SRE-http is designed to be a relatively full-featured package for non-cutting edge, small-to-medium load sites being maintained by non-professionals
In fact, SRE-http has many obscure, and useful, features. These include: At this point you are either long gone, or still interested. Assuming the latter, the goal of this document is to outline some of the possibilities of SRE-http, and point you to the relevant documentation. Before we do that, it is de-rigueur to ask you to read the disclaimer. You might also want to peruse the basic terminology section of the introduction (we use some not-necessarily-intuitively-obvious terms in this documentation).
Bored already? You may find the FAQ more informative, the manual more immediately useful, and the list of documentation files a useful index.
Not bored? You can proceed linearly through this documentation, or you can use the table of contents to jump to the desired section.
Please be aware that this guide strikes a balance between brevity and thoroughness. There will be features not mentioned. So repeat after me: the manual, the discussion of initialization parameters, and the list of documentation files are the places to go for more detailed discussion.

/ Are you looking for a quick start? /

Although this guide attempts to be succinct, there are a fair amount of details to bog down the unwary.
So... if you really don't need all those paranoid access controls, and your needs don't require refining, re-refining, and otherwise mangling requests, you probably can skim right through & and over this user guide! That is, if you are reasonably careful during installation, and answer all the questions that the install program asks, you'll be ready to go. I suppose you might want to read the installation section (section 3), but even that's not absolutely necessary (the INSTALL program will cover most of the details).

Let me add one proviso: if you want to use server side includes, you should carefully read the relevant section of the SRE-http manual. They can do a lot for you...

2) Table of Contents ------------
Introductionyou just read it.
Table of contents you're reading it now.
Installationsome installation instructions, and some basic terminology.
Multiple Hostsusing SRE-http to support multiple hosts and IP aliases
Controlling Access controlling access to your server's resources
Resolving requests redirection and interpretation of the client's request
Server side includes and Document Customizationcreating dynamic HTML documents
CGI-BIN, Imagemaps, SRE-http addons the wonderful world of server side processing (FORMS and beyond)
Uploads, HEAD method requests, and other methods some powerful, but more obscure, WWW tricks
Auditing, Pre-filter processing, Pre-reply, and Post-filter processing initializations, anticipations, are record keeping.
Optimization hints a few suggestions on how to improve SRE-http's performance.
The synopsis of features an outline of what SRE-http can do
The disclaimer and acknowledgments the documentation wouldn't be complete without one
...... ...

3) Installation

All good things must start somewhere, and that means you have to install SRE-http before you can use it (or have you already done that).

First, have you installed the most recent version of GoServe (ver 2.52 or above)? I'll assume that you have.
Second do you have the most recent copy of SRE-http? Assuming that you have, you should create (or clear out) a temporary directory, and UNZIP the (possibly several) SRE-http distribution files.

  1. Look at the READ.ME file that you just unzipped -- it outlines the installation steps (more impatient souls can skip this file with little ill effect).
  2. Ready to go? Run the INSTALL program from an OS/2 prompt (be sure you are in the temporary directory, or you might end up running somebody else's installation program!).
  3. INSTALL is self explanatory. It does require that you enter the GoServe working directory and the GoServe data directory (see the GoServe documentation, or the SRE-http manual, for details).
  4. INSTALL also let's you set a few parameters, and will ask if you want to add a SUPERUSER entry to SRE-http's username database. We especially recommend adding the SUPERUSER entry, without it you may have a hard time accessing SRE-http's configurator.
When you are done installing, it's a good idea to review the notes available at the end of the INSTALL program.

Some Basic Terminology

The SRE-http documentation uses some not-necessarily-intuitively-obvious terms. The following lists a few of the more important ones. For a more complete list, see the SRE-http manual.
URL
Universal Resource Locator (URL) is a scheme for specifying Internet resources using a single line of printable ASCII characters. A URL should contain a protocol, domain name, port number (optional), the location of the resource, and an (optional) option list (following a ?).
Examples:
  • http://your.server.net/dir1/sample.htm
  • http://pet.store.com/prices?type=mammals&class=retail
    • Selector (also referred to as the request selector )
    The selector is the location of the resource on the server -- it's the part of the URL after the domain name. Examples (assuming the above URLs):
  • dir1/sample.htm
  • prices?type=mammal&class=retail

    • In many cases, the connection between the selector and a server resource it refers to is simple (such as when the selector is a filename relative to the GoServe data directory). In other cases, it's completely virtual (as when the resource consists of the output of a program that's run using information provided by the client).

    Selector-specific (usually written as SEL-specific )
    SEL-specific refers to SRE-http configuration information that is specific to a selector -- such as resource privileges, or a virtual directory.
    In much the same way that OS/2 extended attributes contain additional information about a file, SRE-http's SEL-specific information contain additional information about a selector ; or, to be bit more precise, about the server resource identified by the selector (needless to say, this information is only used by SRE-http).
    Privileges
    SRE-http uses the concepts of resource and client privileges to control access to server resources. The basic notion is that access can be regulated by requiring that a request for a server resource come from a client with a client privilege that matches a resource privilege. Note that resources privileges are assigned by the webmaster on a selector specific basis, a given selector can be assigned multiple resource privileges. Similarly, client privileges are assigned on a user specific basis, and a given user may possess multiple client privileges.
    Realms
    Realms are a commonly used means of subsetting a server's resources. Typically, when a server first asks a client to provide authorization information, the Realm of this protected resource will be displayed in the username/password box that the client is asked to fill in. Most browsers will then store this username and password on a server and Realm specific basis, and attempt to use them for near-future "authorization requests".
    In a sense, realms and resource privileges are parallel concepts; they both are used to identify a subset of a server's resource.
    Return to table of contents.
    ...... ...

    4) Multiple Hosts

    SRE-http is a multi-host (multi-homing) web server -- it can handle requests for several IP hosts simultaneously. This does require some configuration; in particular, you must define a host nickname for each host. Note that each host can be a unique IP address, or can be an IP alias.

    The easiest way to add (and remove) hosts is through the Multiple Hosts section of the simple-mode configurator. You may also want to examine the description of the HOSTS parameter in INITFILT.DOC, the discussion of the CFGLIST.CFG configuration file, and the discussion of hosts in the SRE-http manual.

    Return to table of contents.
    ...... ...

    5) Controlling Access to your Site

    SRE-http has several levels of access control, including Logon controls, SEL-specific access controls, and file specific access controls. In addition, you can identify resources that are open to the public.
    Type of
    control     		Description Configuration Information?
    LOGON controls At the coarsest level, you can require all clients to LOGON. In most cases, logging on requires that the client provide a username and password, which are then compared against the SRE-http user database. If there is no match, SRE-http will return a simple you are not authorized response. Alternatively, you can construct a customized response.

    Since this logon requirement is a bit of a hassle, you can specify OWNERS and In-house clients; for whom logon requirements are waived. This specification is by IP address, with the possibility of using wildcards to grant access rights to entire domains.

    		The Logon Controls section of the simple-mode configurator can be used to enable this Logon requirement, to add and remove users, and to add and remove In-house clients.

    Advanced User Notes: The following SRE-http parameters may be of interest (see INITFILT.DOC for details).

  • LOGON_LIMITS: limits the number of logon attempts per minute
  • DNS_CHECK: denies access to clients with no IP Name.
  • UNALLOWED_IPS: identify IP addresses that will not be allowed onto your site.
  • THE_REALM: the default REALM name (can be changed with the simple mode configurator)
    • Public areas The basic idea is that before logon or access_controls are checked, SRE-http checks if the request selector points to a PUBLIC area. If so, no logon or access controls are attempted.
    In a sense, the PUBLIC areas are purposely placed outside of the protection of SRE-http's various access controls.
    		 These public areas are specified in the PUBURLS.IN file. You can also use the intermediate configurator to define "public realms", which are identical to PUBLIC_URLS.

    Since many sites will contain a continuum of private and public resources, this coarse level of control will often be inappropriate. SRE-http offers two finer methods of access controls: SEL-specific access controls, and the HTACCESS method.

    SEL-specific access controls Selector-specific access (aka SEL-specific access controls ) are based on a comparison between the request selector (the request selector is part of the http request string that the client sends to the server... ... it's the URL minus http://x.y.z/) and a list of SEL-specific access controls. This list (which may contain wildcarded entries) dictates who can access the various resources (files, etc.) on your site.

    Given a match, SRE-http will then compare the list of resource privileges assigned to the selector to the client privileges granted to the client. In general, the client must have at least one client privilege that's also in the resource privilege list (though you can specify more complicated conditions). If no such matching privilege exists, SRE-http returns an unauthorized access response. This response can be a generic response, a customized-for-your-site response, or a customized-for-this- selector response.

    For a more detailed description of SRE-http's various access control mechanisms, see the the SRE-http FAQ

    		The Access Controls section of the simple-mode configurator lets you enable access controls, as well as choose between the generic and customized unauthorized access response. You can also use the simple-mode configurator to add and remove entries in the list of SEL-specific access controls.

    Note that when you use the simple-mode configurator to create a selector specific access control , you will be asked to provide the selector (with optional wildcards) a list of resource privileges, and an optional permission list.

    You can also use the intermediate mode configurator to define realms, where each realm consists of one or more (possibly wildcarded) selectors. Realms can be assigned:

  • a set of resource privileges
  • an optional permission list.
  • an access-failure response file (for details, see the description of ACCESS_FAIL_FILE in INITFILT.DOC),
  • a advanced options file

    • Client privileges are granted when you set up user entries. You may also want to use the intermediate mode configurator to modify the INHOUSE_PRIVS and PUBLIC_PRIVS parameters. If you want to get fancy, see ADDPRIVS.DOC for details on how you can specify dynamic client privileges.

    The HTACCESS method The HTACCESS method (which is something of a standard) uses special HTACCESS files located in the directory (and in the parent directories) of the requested file. This (or these) HTACCESS files contain information on who has access rights to files in the directory (and in child directories). HTACCESS files can be created using your favorite text-editor. You might want to read some documentation on the HTACCESS method of access control. Alternatively, you can use the intermediate configurator's Modify HTACCESS Controls option to change HTACCESS files.

    The choice between the SEL-specific and HTACCESS methods is a matter of taste and convenience -- controlling directories is a sure way to prevent access to files, but controlling selectors is more flexible (if your physical directory structure changes more frequently then your pages). In fact, these two methods (SEL-specific and directory-specific) can be used jointly, which implies multiple checks. But be careful when using both methods, it is possible to get stuck in authorization loops!

    Since SEL-specific controls are native to SRE-http, they will tend to be faster (and better tested) then the HTACCESS method. Therefore, our recommendation is to use the SEL-specific method.

    Encypting responses and requests

    Ideally, SRE-http would be SSL compliant. Unfortunately, the licensing and registration requirements for the public-key procedures used by SSL are overwhelming (at least in the USA). But given the open nature of the Internet, some form of encryption is sometimes necessary,

    SRE-http provides a solution to this problem, using a shared-secret encryption methodology. Basically, this requires that a client be "registered" with your site, that she have a username and password stored on your server. Although this is not as convenient as SSL, in many cases it will be sufficient.

    Return to table of contents.
    ...... ...

    6) Resolving requests

     Since all kinds of requests might show up on your server's doorstep, SRE-http offers a slew of techniques and tricks for resolving requests. These include specifying your home-page, indicating directory specific documents, establishing virtual directories, redirection to other servers, and revising the request selector .
    The problem Solutions
    A document was not specified You can use the Default Documents section of the simple-mode configurator to:
    • identify your home page -- it is sent in response to empty selectors.
    • identify possible files (including directory specific file names) to use when a request for a directory appears
      .. or, generate a cachable directory listing instead (see DIR.DOC for details).
    • Advanced Users Note: see the description of the NOEXT_TYPE parameter (in INITFILT.DOC) for an alternative approach.
    Document could not be found By default, SRE-http will:
    • send a generic response that mentions the colloquial name of your site
    Alternatively, you can
    • modify the generic response,
    • create a customized document not found response file, or
    • return a "directory specific" document not found response file.
    Use the Site Identification variables section of the simple-mode configurator to modify the colloquial name, the generic response, and the document not found response file.
    A document has been moved to another location SRE-http uses aliases to redirect requests to another URL. This URL need not be on the same server, and it need not have the same name. You can use the Home Directory, virtual directories, and redirection section of the simple-mode configurator to create these redirection aliases.
    You want to catch common mistaakes SRE-http uses aliases to specify local redirections: which involve textual substitutions in the received request selector. Among other advantages, this gives you quite a bit of control over how "SRE-http facilities, and external procedures" perceive the request. The intermediate mode's Modify Redirection Aliases, section can be used to create these local redirection. Alternatively, you can use a redirect: internal entry in a realm-definition.
    You want to specify a ~ directory The home directory is used as a text replacement whenever a ~ is encountered in a request selector. A typical value of this would be "Users/".

    Further refining the use of home directory, you may wish clients to have access to particular subdirectories of your "Users" directories. For example, all "students" may have space on the server machine, some of which is used for web, and some for "personal" purposes. The goal is to give clients direct access to the "web" related sub-directories but not to the "personal" sub-directories.

    To modify these home directory options, see the Home Directory, virtual directories, and redirection section of the simple-mode configurator

    You want to transfer files from outside of GoServe's data directory By default, SRE-http will match a request selector to a file in the GoServe data directory. While a good security feature (files not in or under the GoServe data directory are inaccessible), this can be an inconvenience.

    To remedy this inconvenience, one can define virtual directories

    Basically, SRE-http will compare the starting portion of the request selector to see if it matches one of your virtual directory entries. If it does, the target directory listed in the matching entry is used (instead of the GoServe data directory). Thus, you can make available a wide, but controllable, set of directories (on or LAN accessible from) your server.

    To create virtual directories, see the Home directory, virtual directories, and redirection section of the simple-mode configurator.

    Advanced users notes:

    • Virtual directories can point to "remote" directories on other http servers -- SRE-http will attempt to retrieve the file from the remote server, without using a redirection (see the SRE-http manual for details).
    • You can also use virtual directories with CGI-BIN scripts, file uploads, and SRE-http addons (and you can specify which virtual directories are to be used for each of these several types of requests).
    • Although somewhat obsolete, you can use special transfer alias to transfer selected files from outside the GoServe data directory (see the SRE-http manual for details).

    Return to table of contents.
    ...... ...

    7)Server Side Includes and Document Customization

     SRE-http supports a rich variety of server side include (SSI) features. In addition, SRE-http supports several means of providing client-specific documents.
    The feature Discussion More details
    Caching of documents that contain SSIs. To improve performance, SRE-http will "cache" HTML documents that have server side includes (SSI). This SSI-caching is of the document after the SSIs have been performed. In simple cases SRE-http can reuse this cached file, without having to repeat the actual process of SSI lookups, etc. Needless to say, this can greatly improve server performance. You can use the Server Side Includes section of the simple-mode configurator to enable SSI-caching.

    For a detailed discussion of SSI-Caching, see SSICACHE.HTM.

    Suppressing Server Side Includes You can suppress server side includes for everyone, or for selected HTML documents.

    The NO_INCLUDE parameter can be used to suppress all SSI's. A less drastic approach is to use a NO_SSI access-control permission.

    Alternatively, a YES_SSI permission will force SRE-http to check an HTML document for ssi's -- YES_SSI overrides NO_SSI, SSI_SHTML_ONLY, and NO_INCLUDE.

    		See INITFILT.DOC for a discussion of NO_INCLUDE. The SRE-http manual discusses the various access control permissions.
    Limiting SSIs to a subset of your HTML documents SRE-http can be instructed to attempt server side includes (SSIs) only on a subset of HTML documents; such as those with .SHT or a .SHTML extension. In other words, HTML documents with .HTM or .HTML extension will not be checked for server side includes. This can speed up file transfer (especially when used in conjunction with SSI-caching), but does require more care when naming html files. See the Server Side Includes section of the simple-mode configurator to enable this SSI on SHTML files only feature.

    Advanced users notes:

  • If you wish to use extensions other then .SHT or .SHTML, you can change the SSI_EXTENSIONS parameter (described in INITFILT.DOC).
  • You may also need to add text/html mime type entries to the MEDIATYP.RXX file (see the SRE-http manual for details).
    • Headers and Footers You can include headers and footers in all your HTML documents. They can contain Server Side Include keyphrases (see below). You can set both headers and footers in the Server Side Include section of the simple-mode configurator (or see the SRE-http manual for further details).
    The NSCA HTTPD-style server side include syntax The NSCA HTTPD-style server side include syntax is fully supported by SRE-http. These include:
    • INCLUDEing files
    • Reporting size and creation date of files
    • Displaying selected system variables (such as current date and time, and the servername and client's IP addresses).
    • Including the output of CGI-Bin scripts
    		The SRE-http manual also describes how to use NCSA HTTPD-style server side. You may also want to see TIMEFMT.DOC for a description of time and date display formats.

    SRE-http also supports most of the Apache XSSI extensions to the HTTPD-style SSI syntax.

    The SRE-http syntax The SRE-http syntax overlaps the NCSA HTTPD-style syntax, but there are some important additions (note: you can use both types of SSIs in your documents). Additional features include:
    • INTERPRETing REXX code, either in-line or in external files
    • SELECTive excludes are supported
    • You can specify a large set of static variables to include in your documents
    • Several built-in text counters are available.
      Of particular interest, SRE-http can suppress augmenting the hit counter when:
    • Repetitive hits arrive from the same client
    • Requests arrive from OWNERS
    • OPTIONs passed to the html document can be incorporated into your document
    		The SRE-http manual contains a detailed discussion of the SRE-http SSI syntax, including a discussion of the suppression of hits feature. The SRE-http FAQ contains a discussion of the various "hit" counting mechanisms available to SRE-http, as does COUNTER.DOC.

    You can use the Server Side Include section of the simple-mode configurator to modify most of the static variables.

    Customizing Documents SRE-http provides several tools that allow you to easily customize your documents on a client specific basis. As a simple example, you can specify special "server side includes" that are used only for clients with INHOUSE or SUPERUSER privileges.

    Two very useful tools are

  • the ByCLIENT utility, which can be used to specify a set of files to be INCLUDEd on a client-specific basis.
  • the ByBROWSR utility, which can be used to specify a set of files to include on a browser-specific basis.
  • The SRE-http manual discusses the INHOUSE. and SUPERUSER. "replace" variables.
  • ByBROWSR.DOC discusses the ByBROWSR browser-specific utility
  • ByCLIENT.DOC discusses the ByCLIENT client-specific utility
  • For a highly dynamic (cookie based) means of customizing documents, check out the CUSTOMIZ addon.
    • Content Negotiation SRE-http supports server-side and client-side content negotiation. Content negotiation is used to select one (of several) variants of a resource; with the selection a function of mimetype, language, and a few other client-supplied pieces of information. NEGOTIAT.DOC contains a complete description of SRE-http's support for content negotiation.

    Return to table of contents.
    ...... ...

    8)Server Side Processing: CGI-BIN, imagemaps, SRE-http addons

    SRE-http supports a variety of server side processing features. These include support for server-side imagemaps and for searchable indices, CGI-Bin support, and support for SRE-http addons.
    The feature Discussion
    ImageMaps SRE-http supports NSCA and CERN server-side imagemaps.

    The SRE-http manual contains a detailed discussion of how to specify and configure imagemaps. You can also play with SAMPMAP.HTM (a sample image map document, it uses the SAMPMAP.MAP imagemap file).

    Searchable Indices The DOSEARCH SRE-http addon is a moderately powerful text-file search engine. When used in conjunction with aliases, it is easy to implement searchable indices.

    Alternatively, GoSWISH can be to be used to quickly search an arbitarily large subset of your web site.

    Special Requests SRE-http supports several special requests, all of which start with an exclamation point (!) The current set of special requests are:
  • !authorize !ping !statistics !ssi?option !host+ip.add.res.s !reset !save !variable?varname
    • See the SRE-http manual for details.
    CGI-Bin support SRE-http offers full support for the Common Gateway Interface.

    SRE-http offers a few enhancements, including:

  • an ability to specify alternative directories (within which to find CGI-Bin scripts)
  • an ability to use PERL scripts
    • SRE-http addons As an alternative to the somewhat clunky CGI-Bin interface, SRE-http addons can be used. The STATUS program is a sample of one such addon.

    But that's a trivial example. Far more useful addons are available -- the following lists some of the larger ones, all of which can be obtained from the SRE-http home page.

    Notes:

    Return to table of contents.
    ...... ...

    9)Uploads, HEAD requests, and other methods

    SRE-http supports some of the less frequently used HTTP 1.0 protocols, as well as a few proposed HTTP 1.1 protocols.
    The protocol Discussion
    HEAD requests Although not frequently used, HEAD method requests offer some throughput advantages when searching over a broad range of sites. To fully support such requests, SRE-http can parse the <HEAD> section of HTML documents, and return (in the response header) the contents of LINK, NAME and META-EQUIV elements. In fact, SRE-http can parse the <HEAD> for GET requests too (though this may not be a very useful activity).

    For further discussion, see the description of the AUTO_HEADER parameter in INITFILT.DOC, or see the SRE-http manual.

    File Uploads using HTML FORMS SRE-http's PUT_FILE facility supports the multipart/form-data method of file upload (as supported by NetScape 2.01). This requires the use of an HTML document that contains a special type of <FORM>.

    Notes:

  • For an example of such a document, you can examine UPLOAD.HTM .
  • The SRE-http FAQ contains a discussion of file uploads using HTML forms.
  • For further discussion, see the SRE-http manual.
    • Uploads from other servers Though somewhat obsolete with the advent of multipart/form-data method of file upload, SRE-http's GET_URL facility permits clients to transfer files from different http servers to your server.

    Notes:

  • For an example of the use of GET_URL, you can examine UPLOAD.HTM .
  • The UPLOAD_MINFREE and UPLOAD_MAXSIZE parameters (as described in INITFILT.DOC) can be used to control the size and extent of file upload capabilities.
  • For further discussion, see the SRE-http manual.
    • The PUT and DELETE methods The PUT and DELETE HTTP 1.1 methods allow clients to upload, and delete, files from your server. Most browsers do not support these methods.

    Notes:

    • The DOPUT.CMD and DODELETE.CMD files can be used to PUT and DELETE files from your server.
    • As a security measure, special SEL-specific access-control permissions must be granted before the PUT and DELETE methods are recognized.
    • For further discussion, see the SRE-http manual.
    Byte-range
    retrieval    		 SRE-http supports byte-range retrieval. For further discussion, see the description of the ACCEPT_RANGE parameter in INITFILT.DOC.
    Setting the expiration date of documents To work around NetScape's history bug (that over-refreshes documents that have an immediate expiration date), you can instruct SRE-http to modify GoServe's default temporary file response header. For example, you can change the immediate expiration date to an expire in 2 hours expiration date (but to have a 0 second http/1.1 age).

    The simple mode configurator's Miscellaneous section can be used to select this option. For further discussion, see the description of FIX_EXPIRE in INITFILT.DOC.
    Delta-encoding SRE-http offers partial support for delta encoding. Delta encoding, which is a proposed http/1.1+ standard, can be used by advanced clients to substantially reduce response size. This support can be enabled for all requests, or on a selector specific basis. For further discussion, see DELTA.DOC.

    Return to table of contents.
    ... ...... ...

    10)Auditing, Pre-filter, Pre-Reply, and Post-Filter Processing

    SRE-http supports several forms of auditing, including the common-log format. In addition, SRE-http can do a few things before and after processing the request.
    Action Description
    SRE-http Audit files SRE-http allows you to use several different audit mechanisms.
    The common-log, browser, and referer audit files .
    SRE-http can maintain a log of all requests in a common-log file (the common-log format is something of a WWW standard). In addition, SRE-http can maintain separate logs of browsers, and referrers. You can even archive these log files on a scheduled basis.
    For details, see the SRE-http manual, the description of the WRITE_LOGS parameter in INITFILT.DOC, or see SREFLOGS.DOC for a detailed description.
    The RECORD_ALL_FILE
    SRE-http has a special audit mechanism to record all requests. This recording consist of augmenting a unique counter for every file (or selector ) requested. That is, it does not record who made the request.
    For details on SRE-http's record all requests option, see the SRE-http manual, or see the description of the RECORD_OPTION and RECORD_CACHE_LINES parameters in INITFILT.DOC.
    GoServe's GOAUDIT file.
    GoServe maintains an audit file (typically called GOAUDIT.80). It contains a lot of information, but is a bit difficult to read (see GOSERVE.DOC for further details on how to configure the GOAUDIT file).
    Load Balancing SRE-http is shipped with support for a very simple form of load balancing. Specifically, when the load on your server (measured in number of current transactions) gets too high, SRE-http can redirect the request to (one of several) backup servers.
    For those with more complicated load balancing needs (such as may occur on multi-host servers), you easily upgrade this simple load balancer (check the SRE-http home page for "load balancing" addons).

    For further details, see the description of the BACKUPSERVERLIST, and LOADTHRESHOLD parameters (in INITFILT.DOC), or see the SRE-http manual.

    Special Directives SRE-http understands several special directives. These are prefixes added to request selector that can modify SRE-http's logic (they all start with a ! )
    • !force/sel : Forces a logon (if needed), and suppresses the SSI-Cache
    • !norecord/sel : Do not record this request (in the RECORD_ALL_FILE, the COUNTER.RXX files, or the COUNTER.CNT file)
    • !sendas_mimetype_mimesubtype/URL : Send the request with a content-type: mimetype/mimesubtype response header
    • !delsend/filename.ext : Return a transient file from (from the TEMPFILE_DIR directory), and then delete it.
    See the SRE-http manual for more details on these special directives.
    Pre-filter processing There may be occasions when you want some other filter to process a request before SRE-http get's a shot at it. For example, ambitious programmers may wish to implement a more sophisticated load balancing algorithm; or you might wish to implement an alternative access control mechanism. As a more practical example, SRE-http comes with PREFILTR.80; a pre-filter that passes a request to the GOREMOTE.80 server remote control package.

    The SRE-http manual describes how to enable pre-filters; or you can read the description of the PRE_FILTER and PREFILTER_NAME parameters in INITFILT.DOC.

    Pre-Reply processing After SRE-http has determined the contents to be returned to the client, a final round of custom modifications can be made. For example, you can use a pre-reply procedure to transform code-pages or to add a signature to all text files.
    Post-filter processing After SRE-http has responded to a request (and GoServe has closed the connection), you may wish to perform further actions. For example, you may wish to notify the webmaster of special events (see POSTMAIL.80 for an example), or you may wish to record select requests (see POSTRCRD.80 for an example).

    The SRE-http manual describes how to enable post-filters; or you can read the description of the POST_FILTER and POSTFILTER_NAME parameters in INITFILT.DOC.

    Some advanced-users options
    Mid-Filter requests You can specify, on a SEL-specific basis the execution of "mid-filter" REXX procedures. These "mid filter" procedures are run after resolution of a request (and after access control checks), but before the actual transmission of any information. For details, see the advanced options file
    Custom initialization Using the CUSTOM_INITS parameter (located in the SRE-http file) you can specify a set of REXX procedures to run when SRE-http is first initialized. For example, a PRELOAD procedure (available as an SRE-http addon) can be specified to "preload" commonly called files. For the details, see the description of the CUSTOM_INITS parameter in INITFILT.DOC.
    Scheduled Programs You instruct SRE-http to execute REXX procedures on a scheduled basis (such as hourly, daily, or monthly). For details, see the description of the SCHED. parameter(s) in USE_CFGS.DOC
    Event Monitor The SRE-http event monitor is used to monitor a variety of events -- such as the posting of an event semaphore, or the creation of a transient file. If one of these events occurs, a number of actions can be taken. Examples include shutting down GoServe, suspend/resume acceptance of connections, and resetting parameters. For further details, see EVENTS.DOC

    Return to table of contents.
    ...... ...

    Optimization Hints

    Let's admit it: SRE-http is not a super-fleet web server. The design goal has always been comprehensiveness (without an enormous amount of complication). Within the context of an interpreted language (REXX), that has a price.

    That said, a lot has been done to improve throughput. In particular; the extensive use of macrospace, the use of daemons (independent threads) to handle much of the mundane processing, and the incorporation of an SSI-cache help keep SRE-http's performance respectable. Still, it would be nice if it were faster.

    Well, if you are willing to work on it, there's actually quite a bit you can do to improve it's performance. The following outlines some of these optimization strategies. If you want more specific information, take a look atthe first section of INITFILT.DOC (it contains a number of optimization hints).
    Action Description
    SRE-http and proxy caches One of the most important aspects of the http/1.1 standard is the attempt to encourage the use of proxy caches. The hope is that such caches can respond to a significant number of requests, thereby saving the delay (and bandwidth consumption) of recovering the resource from the "origin server".

    You can optimize SRE-http's communication with proxy caches in several ways:

  • the setting of the PROXY_CACHE variable (in INIT_STA.80)
  • the setting of the FIX_EXPIRE variable (in INITFILT.80)
  • the use of PUBLIC_URLS
  • the use of the NOCACHE, CACHE and CACHE* selector-specific permissions
  • the use of selector-specific advanced options to specify an explicit response header.
    • For more details, please see CACHING.DOC.

    The SREPROXY variant of SRE-http SREPROXY is a small front-end (a "proxy") for SRE-http. SREPROXY will review all requests; and if a "cached" response is available (from an earlier request, to SRE-http, for the same resource). SREPROXY will answer the request (by sending the "cached" response).

    Responses are cached when they are static (do not change on a per request basis) and are publically available (are not subject to access controls). Since checking & sending a cached response is easy to do, throughput can be markedly improved. Moreover, requests that can not be resolved by SREPROXY (for which no cached response exists) are simply transferred to the main SRE-http procedures; hence, no functionality is lost.

    In a sense, SREPROXY replaces the GoServe cache -- which (unfortunately) is not http/1.1 compliant, hence should not be enabled.
    Suppressing unneeded features If you know you aren't going to use one of SRE-http's features; you can and should instruct SRE-http not to use it. In general, there are two ways of doing this: across the board, or on a selector specific basis.

    For example, you can turn off Virtual Directory lookup for everyone (by setting VIRTUAL_FILE=' '), you can use the NO_VIRTUAL permission to suppress virtual directory lookup on a case by case ( selector by selector ) basis, or you can set up a literal PUBLIC_URL.
    Another example is the use of SHTML files to indicate "SSI-capable" documents; normal HTML files will not be subject to SSI compilation (a time consuming action).

    Using GZIP compression Although it takes CPU cycles, the use of on-the-fly GZIP encoding can signficantly reduce the size of responses. For information on how to enable on-the-fly GZIP compression, see the descriptions of the SUPPRESS_GZIP_TE and CE_GZIP parameters in INITFILT.DOC.

    Return to table of contents.
    ...... ...

    Synopsis of Features

    The following summarizes most of SRE-http's features: Return to the table of contents.
    ...... ...

    The Disclaimer

     In a nutshell:
    SRE-http, and related products, are to be used at your own risk; the authors assume no liability for any damage that may be caused by the use or misuse of these software products.
    The nice news is:
    SRE-http is copyright by Daniel Hellerstein, and hereby placed in the public domain (yes, I realize that's somewhat of a legalese contradiction). You may use is it any manner you deem fit, up to and including use of the code in your own software.
    Out of the nutshell: 
    
  Copyright 1996,1997,1998 by Daniel Hellerstein. 
  Permission to use this, and related, programs for any purpose
is hereby 
  granted without fee, provided that the author's name not be
used in
  advertising or publicity pertaining to distribution of the
software 
  without specific written prior permission.
  With some provisos, this includes the right to subset and reuse
the code,
  with proper attribution  The provisos are  several fold:
   1)  Portions of the code are adapted from other authors' work
       (these are noted where appropriate); you'll need to
contact these other
       authors for appropriate permissions.
   2)  SRE-http uses several 3rd party dynamic libraries:
       i) Quercus System's REXXLIB procedure library.  SRE-http's
REXXLIB
          license gives us (the authors)  the right to distribute
REXXLIB
          without charge.  This right may NOT extend to
redistributors.
          Please contact Quercus Systems for details.
      ii) Info-Zip's UNZIPAPI.  Although this is freely
          available software, you may wish to contact Info-Zip
for details.
     iii) Jeff Glatt's FILEREXX, which is freely available
software
          (a 1995 address is 6 Sycamore Drive East, New Hartford,

           NY 13413, 315-735-5350).

  3)  We, the authors of SRE-http and any potentially affiliated
institutions,
      disclaim any and all liability for damages due to the use,
misuse, or
      failure of the product or subsets of the product.

  Furthermore you may also charge a reasonable re-distribution
fee for
  SRE-http; with the understanding that this does not remove the
  work from the public domain and that the above provisos remain
in effect.

    THIS SOFTWARE PACKAGE IS PROVIDED "AS IS" WITHOUT EXPRESS
    OR IMPLIED WARRANTY.
    THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS
SOFTWARE PACKAGE,
    INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND
FITNESS.
    IN NO  EVENT SHALL THE AUTHOR (Daniel Hellerstein) OR ANY
PERSON OR
    INSTITUTION ASSOCIATED WITH THIS PRODUCT BE LIABLE FOR ANY
    SPECIAL,INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER
    RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION
    OF CONTRACT,NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT
OF OR
    IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE
PACKAGE.


   SRE-http was developed on the personal time of Daniel
Hellerstein,
   and is not supported, approved, or in any way an official
product
   of my employer (USDA/ERS).

    Acknowledgments

     Though my wife probably put up with the most grief, the following people all contributed:

    Return to table of contents.
    ...... ...
    Do you have questions, comments, or suggestions relating to SRE-http? Send 'em in ( Daniel Hellerstein.) -- that's DANIELH@crosslink.net if your browser can't deal with MAILTO:.