Calamaris
Version 2

What is it?
-----------

Calamaris is a Perl script, which was first intended as demo for a statistical
software for Squid.  I started it at 13 January 1997 (Version 1.1) as a
rewrite of my old Squid-Analysis-Script weekly.pl (which was in German
language). I announced it (Version 1.16) to the public at 28 Feb 1997.  (see
http://squid.nlanr.net/Mail-Archive/squid-users/9702/archive/0555.html for the
Original-Announcement) Since then it is used by people all around the world,
and i decided to build a new improved version of it.  Calamaris V2 is a nearly
complete rewrite, with changed and more reports.

It works with Native logfiles from Squid V1.1.x and later and with
Squid-style-Logfiles from NetCache. (NetCache-Users should check the
'Bugs&Shortcomings' below)


Where to get it?
----------------

The Calamaris-Home-page is located at http://Calamaris.Cord.de/ 
(If you use an old browser which does not send a 'Host:'-Header in its request
you have to use http://Cord.de/homepages/cord/tools/squid/calamaris/)

There is also an Announcement-Mailing-list. To subscribe send mail with
'subscribe your@mail.adr.es' in the Mail-Body to
Calamaris-announce-request@Cord.de.  Subscribers will get a mail on every new
release, including a list of the changes. --> low traffic.

Kevin Way (kevin@research-inc.com) has build rpms of Calamaris, which can be
found at http://overtone.org/rpms/

Philipp Frauenfelder (pfrauenf@debian.org) has build a Debian Package, which
can be found in the slink (Debian 2.1) tree.

Dima Sivachenko (demon@gpad.ac.ru) made a port for FreeBSD, which can be found
at http://www.freebsd.org/ports/www.html


How to use it?
--------------

* You'll need Perl Version 5 (see http://www.Perl.com/). Calamaris is reported
  to work with Perl 5.001 (maybe you have to remove the '-w' from the first
  line and comment out the 'use vars'-line), but it is highly recommended
  (especially for security of your computer) that you use a recent version
  (>=5.005_02) of it.

* You'll also need Squid (see http://squid.nlanr.net/) Version 1.1.x or later
  (or logfiles derived from it). It has to produce the 'Native logfile format'
  (which is the default since Version 1.1.x).  If Calamaris can't parse the
  access.log, check your logfile format.  It is explained in the Squid FAQ at
  http://squid.nlanr.net/Squid/FAQ/FAQ-6.html.

* Put Calamaris itself into a warm, dry place on your computer (i.e. into the
  Squid-bin-directory, or /usr/local/bin/). Maybe (if your Perl isn't located
  at /usr/bin/perl) you'll have to change the first line of Calamaris to point
  to your copy of Perl.

* There is also a manpage for Calamaris. You should copy it to an appropriate
  place like /usr/local/man/man1, where your man(1) can find it.

* Use it! 

  'cat access.log.1 access.log.0 | calamaris'

  Calamaris by default generates by a brief ASCII report of incoming and
  outgoing requests.

  NOTE: If you pipe more than one log-file into Calamaris, make sure that they
    are chronologically ordered (oldest file first), else some reports can
    return wrong values.

  You can alter Calamaris' behaviour with switches. Start Calamaris with '-h'
  or check the manpage.

  You should also take a look at the EXAMPLES-File, for
  'Real-Life'-usage-examples of Calamaris.


Are there known bugs or other problems?
---------------------------------------

* CONNECT-requests weren't handled correctly? anybody send me some
  logfile-lines which were interpreted wrong?

* NetCache-Logfile-parsing is working now. However: If you encounter problems
  with it, please send me the lines Calamaris complains about. (PLEASE: Don't
  send me loads of lines. It is enough to send me a few lines of the offending
  lines. If I need more i'll ask for it. Thank You.)

* If Calamaris reports an 'unknown log_hier_method' to you, please check if
  you're logfile is NOT corrupted by pointing cache.log and access.log to the
  same location or by Squid's behaviour that it sometimes writes two
  access.log lines at the same time. (This is a known Squid bug.)

* I fixed a bug regarding caching of Performance-Data. This breaks old
  Calamaris-Cache-Files... I put a workaround in, which allows to parse old
  and new cachefiles. But you will loose the 'Cache-Hits'-value from old files
  in the Performance-Report.  It is set to '-' in the output.

* It is possible that you encounter some problems, if you use Squid
  V1.2.x/V2.x with some of the new features enabled. If you find problems,
  please report them to Calamaris-bug@Cord.de. I'll try to fix them ASAP.  I'm
  especially interested how Calamaris works with Squids CARP-Support. So if
  you use it that way, please send a mail to Calamaris@Cord.de, even if it
  works without problem. Thank You.

* If you want to parse more than one logfile (i.e. from the 'logfilerotate')
  or want to use more than one input-cache-file you have to put them in
  chronological sorted order (oldest first), else you get wrong peak values.
  (Is this something that i should fix? Don't think so...)

* However: If you use the caching function the peak-values can be wrong. 

* Squid doesn't log outgoing UDP-Requests, so i can't put them into the
  statistics without parsing squid.conf and the cache.log-file. (Javier Puche
  (Javier.Puche@rediris.es) asked for this), but i don't think that i should
  put this into Calamaris...

* Squid and NetCache also supports some kind of 'Common Logfileformat'.  I
  don't think that i'll put support for that into Calamaris because that
  Common Log is missing some very important data i.e. the requesttime and the
  hierarchie-information.  If you're still stuck with that format, i recommend
  the 'analog'-software by Steven Turner. Other way round:  change logging to
  'native' and convert it to 'common'. There is software for that available,
  i.e. my shrimp.pl. This also applies for the Common-style Logfiles which
  NetCache produces.

* It is written in Perl. Yea, Perl is a great language for something like this
  (also it is the only one I'm able to write something like this in ;-).
  Calamaris was first intended as demo for what i expect from a statistical
  software. (OK, it is fun to write it, and it is even more fun to recognize
  that many people use the script). For my Caches with about 150MB logfile per
  week it is OK, but for those people on a heavy loaded Parent-cache it is
  possibly to slow.


What will happen next?
----------------------

I think that Calamaris v2 is now finished. (except for bugs, that maybe were
not found yet.)

But if you have an idea what is still missing in a software for parsing
proxy logfiles, let me know. --> Calamaris@Cord.de. I'll will build it in, or
add it to the wish-list below :-)

* seperate on/off switching of incoming UDP/TCP-Requests (based on
  UDP-Requests?) (suggested by John Line <webadm@info.cam.ac.uk>)

* rewrite peak-measurement. The new calculation method is very time intensive
  and slows down Calamaris by 30 or more percent, but it is faster than the
  old way. However: i'm not really satisfied with it, so i put it out of the
  -a -option. You'll have to add a '-p (old|new)' -option to get
  old or new peak-statistic. 

  HELP REQUEST: If someone has an idea how to build an efficient AND fast
  method to work it out... let me know!

* try 'use integer'. This can result in a less memory-hungry, but faster
  version of Calamaris. (idea by Gerold Meerkoetter)

* build graphics (hope i remember who suggested this first, the mail must be
  somewhere in my work-mailbox ;-) (This is a thing for Calamaris v3, if i
  ever going to write it. there are nice gd-libs in Perl ;-)

* make Calamaris faster. see above. If someone wants to rewrite Calamaris in a
  faster language: Feel Free! (But respect the GNU-License) It would be nice
  if you drop me a line about it, I'll mention it below. And please please
  please don't use the name 'Calamaris' for it without asking me!
  
* add a n-level requester report. (suggested by Jarkko Saloranta <jjs@kpo.fi>)


Is there anything else?
-----------------------

Ernst Heiri has build a spin-off of my Calamaris V1, which can be found
*where?*

There is also a C++-port of Ernst Heiri's Calamaris available. It is
(according to the author Jens-S. Voeckler (voeckler@rvs.uni-hannover.de)) five
times faster than the Perl-variant.  check
http://statistics.www-cache.dfn.de/Projects/seafood/ for this.

more Squid-logfile-Analysers can be found via the Squid-Home-page at
http://squid.nlanr.net/Squid/Scripts/

A description of the Squid-Logfiles can be found at
http://squid.nlanr.net/Squid/FAQ/FAQ-6.html.

Thank You!
----------

* The developers and contributors of Squid.
* The developers and contributors of Perl.
* The contributors, feature requesters and bug-reporters of Calamaris.
* Gerold 'Nimm Perl' Meerkoetter.


Not happy yet?
--------------

Drop me a line to Calamaris@Cord.de and tell me what is missing or wrong or
not clear or whatever. You are welcome (especially if you read this file that
far :-)


Version of the README
---------------------

$Id: README,v 2.22 1999/09/25 09:47:17 cord Exp $
