Plans for cronolog

There are a number of things that I would like to do to improve cronolog; many of these are listed in the wish list below.

One of the problems I have is finding much time to work on cronolog. However I hope that creating this new web site for cronolog will give me a structure around which to organize my cronolog activities. So this page list some of the tasks I want to address in the short to medium term:

continue to tidy up the web site
review all the emails I've had about cronolog over the years and update the wish list and FAQ
bring out a new release that incorporates the simpler of the wish list items, for example
- add options --uid, --gid and --umask to cronolog
- add an option to read from a named pipe (FIFO) rather than standard input; this would allow cronolog to be used with programs other than Apache.
finish off the module version
adapt cronolog for Apache 2.0
make cronolog more flexibile (possibly add a configuration file)
... (any ideas)

Andrew Ford, 24 January 2002

cronolog wishlist

These are some of the features that are commonly requested.

cronolog as an Apache module

This is the most frequently requested feature and is something I have been working on. Rewriting cronolog as an Apache module, would avoid the overhead of passing all log messages through external processes, and the overhead of having an external process per log file.

A beta version of the module is now available.

Ability to split on virtual host

This would be quite straightforward to do in an Apache module as the module has access to information about the request, however the standalone cronolog program does not parse the information in the log file entries.

Running external post-processing programs

I would also like to add an option to run an external program (such as gzip or a stats program) on each log file as it is finished with (this was suggested by Juergen Lesny <lesnyj@informatik.tu-muenchen.de>). However when run by Apache there may be multiple cronolog processes running (the external program should probably be only run once when the last cronolog process closes the file). One idea I had to get round this was for each process to read-lock the log file it has open and then try to write-lock it before it closes it. If the attempt to get a write-lock succeeds, then the process then has exclusive access to the file and so can run the external program; otherwise the process should just close the file and leave it to another process to run the program. Cronolog probably also needs a timer to signal that files that have become out-of-date should be closed rather than just detecting the situation when it comes to write the next log entry, which could be several minutes, hours or even days after the end of the period. Suggestions on this aspect are welcome.

Create new log files at rotation time

Currently cronolog waits on its standard input for more data and only creates a new log file when data arrives after the roll-over time. However this could be well after the roll-over time and if entire periods go by with no data then cronolog will not create empty log files.

Does cronolog support large file

There is no explicit support for large files. Cronolog just opens each log file in append mode and carries on writing to it. Some operating systems impose a limit of 2 gigabytes on the size of files. Cronolog ought to detect when it has reached such a limit. One possible enhancement would be to create overflow files by appending a sequence number, i.e. create files named so:

/web/logs/2002-02.log /web/logs/2002-02.log.1 /web/logs/2002-02.log.2 ...

Of course if you are generating more than two gigabytes of log information per rotation period, then the obvious thing to do is to choose a shorter rotation period. You might also want to consider separating out certain categories of file, such as images

# flag requests for image files SetEnvIf Request_URI (gif|png|jpe?g|tiff?)$ IMAGE=1 # separate log entries into image and non-image log files CustomLog "|/usr/sbin/cronolog /web/logs/%Y-%m.log" env=!IMAGE CustomLog "|/usr/sbin/cronolog /web/logs/%Y-%m-images.log" env=IMAGE

Other

There are other suggestions and wish list items in the mailing lists, which I will add here.