If you need to know how to configure your web server to produce user agent, user-identifying "cookie", and/or referrer log information, this section will be of interest to you.

Tips for the Apache server

The free Apache web server, which we highly recommend for use on Unix platforms, can be easily configured to produce user agent and referrer logs. You may have already configured your server to log this information in separate log files, and Wusage can analyze these; see the Command Line Options section.

However, this is not the best way. When your referrer and user agent logs are separate, Wusage cannot display these reports as part of your normal daily, weekly or monthly usage reports. We recommend that you take the following steps:

1. Make sure the mod_log_config and mod_usertrack Apache "modules" are compiled into your server. See the Apache documentation for details.

2. Add the following line to your httpd.conf file in the Apache conf directory:

FOR APACHE 1.2 OR BETTER, WITH THE MOD_USERTRACK MODULE:

LogFormat "%h %l %u %t \"%r\" %s %b %v \"%{Referer}i\" \"%{User-agent}i\" \"%{Cookie}n\""

FOR APACHE WITHOUT THE MOD_USERTRACK MODULE:

LogFormat "%h %l %u %t \"%r\" %s %b %v \"%{Referer}i\" \"%{User-agent}i\""

(Note that referrer must be spelled with only two r's when configuring Apache. For Wusage, we chose to go with the spelling in the dictionary.)

Optional: Viewing Compressed Reports Through Apache

The Apache server can be used in conjunction with the compressed reports option of Wusage 7.0 and above. This option is enabled in Wusage via the optimization settings page, or through the use of the -compressed command line option.

When this option is enabled, Wusage produces HTML documents with a .html.gz suffix. The links within these documents do not include the .gz suffix. The built-in administrative web server of Wusage automatically substitutes the compressed version if the regular version is not found. Apache can do the same thing.

To enable this capability in Apache, make sure you have the MultiViews option turned on in the Options line within the appropriate <Directory> block of your httpd.conf file. In some versions of Apache, Options ALL does NOT enable this feature. Be sure to explicitly turn on MultiViews.

You will also need the following line:

AddEncoding gzip gz

Which is usually already present. If you experience intermittent problems with this feature, make sure Apache is not attempting to parse the raw compressed report files for SSI directives, PGP commands or other special content. If you choose to turn off compressed reports, it is OK to uncompress all of your existing reports, using the gunzip command.

3. When you have done both of these things, restart the Apache server on the fly using the kill -1 Unix command.

The Apache server will now log information in the usual "common log format," with the addition of three new fields, virtual server name, referrer and user agent, and a fourth "cookie field" uniquely identifying the user if you have compiled Apache with the mod_usertrack module installed (please see the Apache documentation for details on this subject).

For Microsoft IIS 4.0, 5.0 and Above

Microsoft IIS 4.0 and above supports the new W3C extended log format. For best results, turn on this logging format with all of the optional fields. By default, IIS 4.0 only logs a very limited set of fields.

For Microsoft IIS 3.0

IIS 3.0 IS VERY OLD. IF YOU HAVE 4.0 OR BETTER, SEE THE PREVIOUS PARAGRAPH. YOU PROBABLY DO NOT NEED TO FOLLOW THESE SPECIAL STEPS WHICH ARE FOR 3.0 ONLY.

Microsoft IIS 3.0 does not have the option of logging user agent and referrer information in its basic form. However, a filter that adds this capability is freely available. The filter was written by Marc Grossman of Microsoft, who is in no way associated with Follow this link to obtain the filter. Instructions for installation are in the HTML document default.htm which is included in the zip file. The filter is distributed by the ISAPI developer's site, a useful service run by Steven Genusa. We are not associated with the ISAPI developer's site in any way. We do not provide technical support for software not written by Boutell.Com. Please see the file default.htm included with the filter software for installation instructions.

Once you have the filter installed, Wusage will recognize the user agent and referrer fields in the resulting log files. Be sure to turn on the Chart Top Referring URLs ^(referrers), Chart Top Referring Sites ^{(referringsites)}, Chart Top Referring URLs by Document ^{(referrersbydocument)}, ^{(referrersbynotfound)}, Chart Top Web Browsers ^(useragents) and Chart Top Operating Systems ^(os) options in your configuration file.

For Other Servers

Most web servers can be configured to log in a variety of different formats. For best results, configure your web server to log in the common log format, with the addition of the referrer and user agent fields. The referrer field should come first, followed by the user agent field. Both fields should be in quotes for best results, especially the user agent field, which can contain spaces. If the user agent field cannot be enclosed in quotes, configure your server to place it at the end of the line after all other fields. Wusage will also recognize a virtual server domain name field, if present, and compare this to the ^(servername) option to determine whether a particular access is relevant.

Below is a sample line of the common log format, extended to include referrer and user agent information. Line break added for legibility only, do not introduce line breaks into individual log file lines.

foo.bar.com - - [20/Apr/1997:16:48:44 -0700] "GET /boutell/index.html HTTP/1.0"

200 5898 "http://www.altavista.digital.com/query" "Mozilla/3.01 (Macintosh; I; PPC)"

Please note: Boutell.Com does not provide web server configuration technical support. This information is provided purely as a supplement to the manuals of your web server software.

---

---

---