Most sites solve this problem by periodically compressing and setting aside old log files. Wusage can analyze log files compressed with the gzip utility automatically. If Wusage is being asked to analyze every file in a directory, care must be taken to remove files which contain the same data Wusage has already seen, but under a new name, such as a compressed verison of a log file wusage has already analyzed. Otherwise, it is possible to get doubled results.

"My server generates a new log file for each day, so I have dozens of log files already and more on the way. How can I analyze these logs with wusage?"

If you have many log files, just specify the location of the log directory rather than the location of a single file when you set the Log Files and Directories ^(logfiles) option.

"What if my log files are in several directories, or on several computers?"

With Wusage 7.0 and above, you can add more than one entry to the Log Files and Directories ^(logfiles) option. Each entry can be a local file path, an HTTP: URL, or an FTP: URL. Each entry can refer to a single log file or to an entire directory of log files.

"Should I keep my log files in the same place forever?" No, this will slow the program down. Although old log data will be correctly ignored, it will be necessary for the program to check each file for new information. To solve this problem, move your log files after they are analyzed. If you wish to keep your log data compressed and not uncompress it for disk space reasons, use the well-known free program gzip to compress your files. (This is not the same as pkzip or zip.)

Wusage can analyze gzip-compressed log files directly without the need to "gunzip" them first.

"I have many mirror sites, so I have a collection of log files that all contain entries from the same period in time. Can wusage cope with this?"

Yes! Just use the Log Files and Directories ^(logfiles) option to tell Wusage where to find the log files you have collected. You can specify more than one location, and HTTP and FTP URLs are accepted. If you have more than 20 log files that cover the same period in time, for instance 50 log files from 50 mirrors of the same site, Wusage may slow down considerably. This does not occur with non-overlapping log files, even if there are hundreds of them. If your needs cannot be met with a limit of 20 overlapping log files, please see the Contacting Boutell.Com, Inc. section for more information.

"This is all well and good, but I already have several old compressed log files that were not compressed with gzip. How do I analyze them with wusage?"

The Unix cat and zcat commands are extremely useful to reconstruct a single log file. Even better, thanks to the -l option of wusage, you can avoid creating an actual combined disk file of your log entirely. Consider the following Unix command line:

cat oldlog1 oldlog2 oldlog3 | wusage -l - -c conffile.conf

The special filename - (a single dash) signals wusage to read its log entries from standard input, which is piped in from the Unix cat program. For more information about cat, try the Unix command man cat.

Important note: it is important to feed log files to cat in ascending order. An older log file should precede a newer one. If this is not possible, be sure to set the Sort Log Files First ^(sortlogs) option first.

Because the log information is arriving at "standard input" as a single stream of data, wusage cannot keep track of its position in each file. Take care not to present the same data more than once to Wusage 6.0 unless you are deliberately regenerating old reports.

"I ran wusage on an old log file, and now I have zero accesses for the last two months! What happened?"

Wusage normally creates reports through the most recent complete day or week, and will not generate those reports again. You can override this behavior using the -b and -e command line options, which are used to force wusage to start re-generating reports at an earlier date, or to stop well before the present date. If you inadvertently produce empty reports for the most recent several weeks or months, just use the -b option to specify a date from which wusage should start re-generating those reports, and specify the more recent logfile as well using the -l option. To protect your reports, Wusage will not attempt to regenerate reports for time periods that are not entirely within the date range specified when -b and -e are in use.