------------------------------------------------------------------------
amavis-stats - generate amavis scan statistics from logfiles
------------------------------------------------------------------------

Licence
-------
Copyright (C) 2003, Mark Lawrence (nomad@null.net)

amavis-stats is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.


Author
------
Mark Lawrence <nomad@null.net>. Aussie living in Switzerland.
Comments are welcome. So would somewhere else to host this.


Description
-----------
amavis-stats is a simple amavis statistics generator based on rrdtool. It
produces graphs of clean emails, spam emails and infected emails broken
down by virus, from amavis log entries (such as /var/log/mail.info).

You can see an example of this in action here:

  http://rekudos.net/amavis-stats/

The RRD files are created and updated by a perl script. Since the script
uses the date/time values from the log entries it can be run (ir)regularly
from cron(8) or by hand as desired. Graphs are generated by a php script.


Dependencies
------------
As of version 0.1.3 amavis-stats works with amavisd and amavisd-new on linux
and Solaris. There has also been one confirmed report of this working on 
MacOS 10.3! For amavisd I believe you need to have the log level set at 5.

The perl script uses nothing which probably isn't in the standard perl
installation.

Graphs are generated by php in combination with either the command-line
rrdtool or the php4-rrdtool shared library.


Download
--------
Raw Source:
    http://rekudos.net/download/amavis-stats.tar.gz

Debian Apt (package name "amavis-stats"):
    sources.list: "deb http://rekudos.net/debian ./"
    root# apt-get update; apt-get install amavis-stats

#RPM (EXPERIMENTAL!):
#    http://rekudos.net/amavis-stats/amavis-stats.rpm
#    root# rpm -i amavis-stats.rpm


Install (not debian or rpm)
-------
1. Download and extract the source:

 root# wget http://rekudos.net/download/amavis-stats.tar.gz
 root# tar -xvf amavis-stats.tar.gz
 root# cd amavis-stats-0.X.X/

2. Install the program by typing (as root):

 root# ./configure [options]
 root# make
 root# make -n install (optional)
 root# make install
 root# make installapache

The options to configure (apart from the standard prefix's) can be:

  --prefix=DIR                installation prefix [/usr/local]
  --with-install-prefix=DIR   (not normally used - mostly for Debian)
  --with-var-prefix=DIR       /var prefix
  --with-amavis-user=USER     specify user to run as [amavis]
  --with-amavis-group=GROUP   specify group to run as [amavis]
  --with-web-user=USER        specify user which webserver runs as [www]
  --with-web-group=GROUP      specify group which webserver runs as [www]

The amavis-user,group and web-user,group must already exist - they will
*not* be automatically created.

4. If you do not already have a user "amavis" you should create one (or
decide on another user to run amavis-stats as). You should also check that
the user has permissions to read your particular log file. On debian, this
means that in the .deb I have to add the "amavis" user to the "adm" group.
Otherwise you can run it as root, but no responsibility lies here for that.

5. As your user "amavis" simply run amavis-stats on the log file that
contains the amavis entries. You should see something like the following:

 root# su amavis
 amavis# /usr/sbin/amavis-stats /var/log/mail.info
 ./amavis-stats: First Time Run
 ./amavis-stats: First seen virus #1 (passed) at 1064126856
 ./amavis-stats: First seen virus #2 (infected) at 1064127319
 ./amavis-stats: First seen virus #3 (W32/Swen.A@mm) at 1064127319
 ./amavis-stats: First seen virus #4 (Forten.F@m) at 1064131924
 ./amavis-stats: First seen virus #5 (W32/Lentin.H@mm) at 1064133728
 ./amavis-stats: First seen virus #6 (W32/Bugbear.B@mm) at 1064146880
 ...

You should then see a bunch of rrd files in /var/lib/amavis-stats.

4. If this looks right then you can copy the debian/amavis-stats.cron.d
file to the appropriate location (sometimes /etc/cron.d/amavis-stats)
so that it runs every five minutes. Check that this format of contab
is compatible with whatever version you are running.

5. If all has gone well you should be able to hit
http://localhost/amavis-stats/ with your web-browser and see some nice
looking graphs.

6. You could also run the php script by hand to check for obvious
errors. Afterwards there should be eight "png" files in your directory:

 root# php4 /usr/share/amavis-stats/amavis-stats.php
 X-Powered-By: PHP/4.1.2
 Content-type: text/html


 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
 <html>
 <head>
 ...
 </html>

 root# ls *png
 passed-day.png    passed-week.png  virus-day.png    virus-week.png
 passed-month.png  passed-year.png  virus-month.png  virus-year.png

7. I suggest that you send mail for the "amavis" user to another account on
the system as amavis-stats generates text each time it sees a new virus.

I am also thinking of possibly generating further statistical information
for a days worth of events. Such things as:

- Top 10 recipients of SPAM, BANNED, INFECTED, Passed emails
- Top 10 senders of SPAM, BANNED, INFECTED, Passed emails

If you would find this useful please let me know.


Problems/FAQ
------------
Q. What is the best way to handle rotated (*.gz) logfiles? How do I
get all of the history in the graphs and not just the latest log?

A. amavis-stats doesn't handle .tar or .gz files at the moment. What it
expects is just the name of a single uncompressed file. So when I first
install it on a machine I usually do the following to get the history:

amavis$ ls mail.info*
mail.info  mail.info.0  mail.info.1.gz  mail.info.2.gz  mail.info.3.gz

amavis$ sudo gzip -d mail.info.*.gz
amavis$ /usr/sbin/amavis-stats -r mail.info.3
amavis$ /usr/sbin/amavis-stats -r mail.info.2
amavis$ /usr/sbin/amavis-stats -r mail.info.1
amavis$ /usr/sbin/amavis-stats -r mail.info.0
amavis$ /usr/sbin/amavis-stats -r mail.info
amavis$ sudo gzip mail.info.[1-3]

ie, you have to manually run ammavis-stats (as your "amavis" user) on each
file in the correct chronological order


Q. We restarted the amavisd with another loglevel (the default one: 0) and
now I dont get any spam info in the amavis-stats. Where did it go?

A. Spam information is only generated in some versions of amavis when the
log level is set to a higher value. There is an effort underway to
standardise the log output of all the Amavis, but please be patient, this
will take some time.


Q. The rrd files seem to get generated ok but I when I try and display the
web page I get the error: amavis-stats::error: rrd_graph(): 127

A. I believe that the error code 127 is returned from bash when it can't find
the program to execute. I guess in this case that your web user (www-data
or www or web) is unable to find the "rrdtool" executable. Perhaps it is
not in the standard location, or the path for that user is not set?

You could try the following to see if the tool can be found.

    root# su -c env - www | grep PATH
        root# su -c rrdtool - www


