Sept 1996.

This document summarizes the changes from SRE-FILTER version 1.11 to
version 1.2d. A history of recent changes is contained in CHANGES.DOC
(there is some overlap between this file and CHANGES.DOC).

You may also want to examine the documentation files:
   A "frequently asked questions" file: SREFILTR.FAQ, 
   An outline of how SRE-Filter works: OUTLINE.HTM
   The "all in one file" SRE-Filter documentation: SREFILTR.HTM
   A detailed description of initialization parameters: INITFILT.DOC



1) NEW NAMES, OR REMOVALS, OF SOME PARAMETERS AND FILES **********************

1a)  The NO_GETAFILE_CONTROL parameter is no longer supported.
If you want to limit access to the GETAFILE facility, create
an entry in the access file (for example, GETAFILE*  INHOUSE).

1b) The SSI_ALLOW, SSP_ALLOW, and CHECK_CACHE variables are no longer
used.  The RETAIN_BAD_KEYPHRASES variable is no longer used. 

1c) CHECK_ALIAS no longer accepts a value of HTM -- the only
recognized values for CHECK_ALIAS are YES and NO.  For backward's
compatability, values of HTM (or HTML) are converted into YES values.

1d) The INHOUSE_NAME variable is now called THE_REALM -- and is now
used for transfering "realm" information to clients (when
username/passwords are requested). For backward compatability, if no
THE_REALM variable is specified, the INHOUSE_NAME variable (if specified)
will be used.

1e) The NOT_FOUND_URL variable will now substitue the possibly host
specific value of HOME_NAME when a HOME_NAME substring appears.

1d) GETAFILE.80, DOSEARCH.80, STATUS.80, SRCHINDX.80, and FILTINST.80
 have been renamed to *.CMD.  

        When re-installing SRE-Filter ver 1.2 in the same directory as
        ver 1.11, the INSTALL program will ask you if you want to 
        delete the "old" versions (the .80 files).  You SHOULD answer
        YES -- if you retain these old versions, SRE-Filter will
        use them instead of the (newer) .CMD versions!

1e) The INTERPRET_FILE is supported, but is considered obsolete. Similarly,
use of !TRANSFER in alias definition is supported, but also considered
to be obsolete.


2) SPECIFICATION OF ACCESS RIGHTS -- ADDED SUPPORT FOR HTACCESS FILES ********

2) When access rights (for a "logged in user") are not available, SRE-Filter
will reprompt for a new username/password.  This will continue "forever" (or
until the client gives up).  In contrast, ver 1.11 would issue an error
message (to the client) after the first attempt.

2a) In order to clarify it's use, the PUBLIC_FILES variables are
now called PUBLIC_URLS.  For compatability reasons, both phrases will
be recognized (that is, PUBLIC_FILES.n will be synonymous with PUBLIC_URL.n)

2b) In order to bypass unnecessary processing (especially the need to
wait on the independent threads described below), the PUBLIC_URLS entries
can now include a "LITERAL" (or LITERAL_NORECORD) option, and an "EXACT_NAME".

When the LITERAL option is present, SRE-Filter will return the file
that matches the URL "as is".  Specifically, no aliasing is attempted on
the requested URL, no "auto naming" is attempted, virtual directory support
is not available (the file should be in the data directory),
server side processing can not be requested, auto header creation
will not occur, and no server side includes are attempted.
LITERAL_NORECORD is basically the same as LITERAL, with the additional
instruction of "don't bother recording, or post-filtering, this request".

2c) In lieu of aliasing and virtual directories, SRE-Filter can use the
"EXACT_NAME". When an "exact name" (an fully qualified filename) is present, it is
used as the match for the URL (in other words, the data directory is not used).

2d) Non-"LITERAL" public URL entries will now perform "home directory",
replacement on the ~ character.  They will also now perform 
"auto naming" (of directory only URLS).


3) VIRTUAL DIRECTORIES :: RETRIEVAL FROM ANOTHER SERVER **********************

3a) Virtual files can now be remote.  That is, they can now point
to locations on other http servers -- SRE-Filter will attempt to
retrieve the file (from the remote http server) and transfer it to
the client (possibly with server side includes) -- without using redirection.

3b) Basically, throughout SRE-Filter, you'll have to specify appropriate
virtual directories if you need to get out of the data directory.
The only exceptions to this are when specifying "exact names" in
"literal public_urls", and when using the !TRANSFER alias (and, of
course, any explicit transfers you hand code using the
INTERPRET and SELECT ssi keyphrases, CGI-Bin scripts, or
other types of server side processing).


4) SUPPORT FOR  MULTIPLE HOSTS ***********************************************

4) Multiple hosts are now supported.  Multiple hosts
consist either of multiple IP addresses, or of "aliases" identified
by "host:" request headers.  Regardless of how requests to different domains
arrive at your server; you can specify  most of SRE-Filter's parameters on a
"host specific" basis, and a few you MUST specify on a host-specific basis.

Of course, the standard "single" host setup can be used.

Note that the same set of .CTL, .CNT, and .IN files are used for
all hosts (each entry in these files can now have a "host_nickname" identifier,
along with a URL identifier).

5)  IMPROVEMENTS TO SERVER SIDE INCLUDES ************************************

5a) The INCLUDE AFILE keyphrase will use virtual directory information (by
   default, INCLUDE files are relative to the data directory).

5b) The #INCLUDE, #FLASTMOD, and #FSIZE HTTPD-style server side
includes now understand the difference between VIRTUAL=filename and
FILE=filename.  VIRTUAL looks in the data directory or local virtual
directories.  FILE looks relative to the directory containing the requested
file (the file containing this server side include).  Note that FILE uses 
the path of this "requested file", rather then the URL that lead to this 
requested file.

5c) Note that the "fully qualified directory" specification is NO LONGER
supported by the INCLUDE, INTERPRET FILE , #INCLUDE, #FLASTMOD,
or #FSIZE ssi keyphrases (i.e., INCLUDE C:\AUTOEXEC.BAT won't work).

5d) The #EXEC HTTPD-style server side include now works more like it's
supposed to.  Rather then being equivalent to INTERPRET, it now will
call a cgi-bin script.  Scripts called by #EXEC should SAY (or otherwise
write to standard output) the text to be included in the file, and
should NOT output other information (that is, Content-type, etc. should
not be "output" by the script).  Note that the CGI and CMD "options"
are ignored.

5e)When using the INTERPRET server side include keyphrase:
   In addition to the INTERPRET.RESULTS variable, strings
  "written to the queue" by QUEUE and PUSH statements can
   also be included.
To enable this "#EXEC"-like feature, set USE_STDOUT='YES'


5f) It is possible that the INCLUDE, REPLACE, SELECT, OPTIONS,
and INTERPRET server side include keywords may match the first words in
normal html comments -- which will cause SRE-Filter to try
to use these comments as server side include instructions, with
usually un-noticable but potentially odd effects.
To avoid this problem, you can add a "preface" to these keywords
by setting the KEY_PREFACE variable.  For example,
setting KEY_PREFACE="!" will cause SRE-Filter to look for
!INCLUDE, !REPLACE, etc.  Note that KEY_PREFACE is set in
SREFILTR.80-- it is NOT set in the INITFILT.80 file.
Also note that KEY_PREFACE may NOT be the # character (use of #
will cause conflict with the HTTPD-style server side include keyphrases).

5g) When the NO_PROCESSING option = YES, or when a NO_SSP permission
exists for a given URL, then the INTERPRET, SELECT and #EXEC server side
includes will be ignored (since they all require server side processing).

6) CHANGES IN THE ACCESS_FILE  ***********************************************

6a) SRE-FIlter will always check for access information (ver 1.11 only
checked when ALLOW_ACCESS<>"YES").  To suppress this (thereby saving
some processing time, at the expense of suppressing certain SRE-Filter
features), you should set the ACCESS_FILE variable to "".  Similarly,
virtual file lookup can be suppressed by setting the VIRTUAL_FILE variable
to "" (or through use  of a NO_VIRTUAL permssion).

6b) Several additional pieces of information are now stored in the
access file.  The access file can now include:
        URL   resource_privileges , permission_list , realm_name
(note: eariler documentation refers to "resource privileges" as
       "acceptable privileges")

6c) One can selectively suppress "aliasing", "virtual directory lookup" and
"post filter activity" on a per URL basis, by including NO_ALIAS, NO_VIRTUAL,
and NO_POSTFILTER in the "permission list".


7)  BETTER SUPPORT AND USE OF REALMS  ****************************************

7a) When prompting for username/password, the realm displayed will be:
  i)For "logons", the default realm name (stored in the THE_REALM variable)

  ii)For URL specific access control, a "URL specific realm" is
     displayed if one is available (the "URL specific realm" is
     one of the items stored in the ACCESS_FILE).  If no URL specific
     realm is available, the default realm is displayed.

7b) You can now assign a set of "resource privileges" to REALM names.
    That is, each REALM  can now have an associated set of
    "resource" privileges.
    This means that the client is allowed access to a URL if:
         i)she has a privilege that matches one of the URL-specific
           resource privileges
     or ii)she has a privilege that appears in the REALM-resource-privileges
           list (of the REALM specifed for the URL).

     Note that this use of "realm-specific-resource-privileges" list is just
     a shorthand -- one could always just include these privileges in the 
     the (several) URL-specific resource privileges list(s). Also note that
     if no  "url-specific resource privileges" are mentioned, and no
     REALM is specified for the URL, then access  is always permitted.
     However, if no url-specific resource privileges appear, but
     a REALM is specified, then the client must have a
     privilege in the appropriate  "realm-resource-privileges" list.

8)  FILE UPLOAD CHANGES, IMPLEMENTATION OF THE PUT AND DELETE HTTP METHODS ***

8a) The PUT and DELETE http methods are now supported; but only if PUT and
DELETE, respectively, appear in the requested URL's "permission list" (in the 
ACCESS_FILE).

8b) When specifying a file for the GET_URL or PUT_FILE facilities, you can
now specify a relative path (that is, a path/filename). This will be
taken relative to the UPLOAD_DIR, or to "local" virtual directory.  Note
that if there is a match to a virtual directory, the access file (for this
relative filename) will be checked for PUT permissions (subdirectories
of the UPLOAD_DIR do not require a PUT permission).


9)  MISCELLANOUS CHANGES IN VARIOUS SRE-FILTER OPTIONS/FACILITIES ************

9a) SENDFILE no longer checks for "access rights".  If you want to limit
the use of SENDFILE, we recommend creating an appropriate entry in the
access file; say SENDFILE*  priv_list.  SENDFILE will also wait
until all of the requested file's bytes have been transfered before
it augments the counter (in the SENDFILE_FILE).  Thus, if a client
kills the connection midway through transfer (say, of a large file),
SENDFILE will NOT augment the SENDFILE_FILE.

9b) The { and } characters are no longer used as default keyphrase
delimiters in server side include processing (that is, only the
<!-- and --> are used.  This can be easily changed by creating
DELIM_1.2='{' and DELIM_2.2='}' entries in INITFILT.80.

9c) The PROCESS_RANGE parameter only applies to non-html files.  That is,
byte retrieval is no longer supported for html files.

9d) The "transfer immediately" feature of server side include processing
no longer requires that AUTO_HEADER <>"ALWAYS".  However, when AUTO_HEADER=
"ALWAYS" (when response headers are created for all GET requests to HTML
files), the response header creation will take place BEFORE server
side includes have been performed.

9e) Currently, the following !special requests are recognized:
 !ping !statistics !reset !save !variable.

9f) The !GETMESSBOX and !VIEWMESSBOX "built in message box features"
are now called using GETMESSBOX and VIEWMESSBOX respectively (the leading
! is no longer used).

9g) External procedures (as used to implement server side processing)
should now issue GoServe completion codes, and should return a
simple status message.  For example, a typical return could consist of:
   'FILE type text/html name tempfile '
   return 'external script/procedure completed successfully'
However, SRE-Filter will still recognize "FILE xxx " and "VAR xx" returns,
and process them appropriately.

9h) URLS in .MAP image files will now be treated as potentially
"virtual URLS", resolved using the URL that invoked the .MAP file.
That is, an entry of ANIMALS/TIGER.HTM, in a mapfile invoked using
/ZOO/WHERE.MAP, would yield a URL of /ZOO/ANIMALS/TIGER.HTM.

9i) Output of intermediate results (to the PMPRINTF window)
will occur only if the VERBOSE variable is set to 1.  For
backward compatability, the default value of verbose is 1.


10) AN EXPERIMENTAL FORM OF SERVER SIDE INCLUDE FILE CACHING *****************

10) An EXPERIMENTAL, simple form of explicit caching is now supported
by SRE-FILTER. Basically, you explicitily identify a list of
html/text  files for SRE-FILTER to cache, using the
CACHED_FILES.n  stem variables (as set in INITFILT.80 file).
When a server side include is being attempted, SRE-Filter will
check to see if the needed file is one of these "cached files".

This feature should be viewed as EXPERIMENTAL!
In particular, the performance aspects of SRE-Filter's caching feature
have not been carefully examined.  There may even be cases where
use of this caching will slow down throughput!
For now, we recommend conservative use of this caching;
with the number of files, and total number of bytes stored, kept
small (say, less than 10 files, and less than 20k bytes).

Basically, we recommend using GoServe's caching if possible
(with use of the CACHE option in the ACCESS_FILE, if security is a concern),
with the CACHED_FILES list used as an adjunct.


11) SRE-FILTER CONFIGURATOR GREATLY EXPANDED   ******************************

11) The FILTINST configurator has been greatly expanded.  It now can
be used to change entries in the users file, the alias file, the access
control file, the virtual directories file, and HTACCESS FILES.  
Basically, for most purposes, you never need edit a configuration file 
(though for some advanced options, for multiple host configuration, 
or for large changes, you may need to edit the various .80, .CTL,  
.IN files, and HTACCESS files).

12) MAJOR CHANGES TO SRE-FILTER ALGORITHIMS GREATLY IMPROVE THROUGHPUT  ****

12a)  The biggest changes to SRE-Filter are "under the hood".  In particular
SRE-Filter now uses several "independent threads" for handling several
chores, such as username lookup and recovering "access" information.
By reducing the size of the SRE-Filter "filter" program, this change
has greatly speeded up the performance of SRE-Filter, especially under
heavy loads.

These threads use queues (and semaphores) to communicate with the main
portion of SRE-Filter.  When GoServe is stopped, these queues will
not be closed. To free up the resources associated with these queues,
you should run the FREEUP.CMD program.

12b) Another large change is that macrospace is no longer used to
store parameters.  Hence, the MACROEYE.CMD file is no longer needed.

Instead, parameter information is now stored in the environment, as well
as in the "independent threads".  Ambitious REXX programmers can examine
the environment (local to the GoServe process),
as well as "call" the "independent threads", to extract these
variables (see SREFPRC1.DOC for details on programming with SRE-Filter).

In fact, it is not that difficult to completely replace the (relatively
simple) "independent threads" with more sophisticated algorithims (say,
using a database manager to store username, access, and other information;
instead of flat ascii files stored into memory).

13)  SPECIAL VARIANT OF SRE-FILTER CAN DOUBLE THROUGHPUT *********************

13) In certain environments, where most of one's files can be
expressed as "literal public_urls", the use of  SREFQUIK.80,
a mini-filter supplied with SRE-Filter, can substantially speed up
throughput.  Use of SREFQUIK will cause no change in  functionality --
if SREFQUIK can't deal with a request, it simply calls SREFILTR.80.

Use of SREFQUIK is not recommended if there are few, or no, literal
public urls -- it just adds code without doing anything. Similarly,
in multiple host setups, SREFQUIK should not be used.;

CAUTION: When using SREFQUIK be aware that modifications to SREFILTR will
NOT take effect until GoServe is stopped and restarted (since SREFQUIK calls,
but never upgrades, a "macrospace version" of SREFILTR.80).