Using LogParser With Awstats To Filter AVG Spam

Following on from my post LogParser to the rescue, I’ve now worked out how to integrate logparser into the Awstats update process with very minimal effort.

Note: Awstats is a cross platform web analysis tool, but unfortunately logparser isn’t, this therefore is windows only.

To make life easier, I dropped the logparser files (exe and dll, although I’m not sure you need the dll) directly in to the cgi-bin where Awstats lives on the server. I understand doing this may have security implications, so do this at your own risk.

Open up the config file for your Awstats report (awstats.<config>.conf) and find the LogFile directive

LogFile=”E:/logs/W3SVC2074709632/ex%YY-1%MM-1%DD-1.log”

It’ll be something like the above, assuming you use daily logs on IIS. We need to change it to

LogFile=”logparser -i:iisw3c -o:w3c -rtp:-1 -stats:off file:rem-avg-spam.sql?logfile=E:/logs/W3SVC2074709632/ex%YY-1%MM-1%DD-1.log |”

This tells Awstats to execute logparser setting any necessary options and passing in the path to the log as before, it then grabs the output from the pipe and processes it.

That’s it!

The contents of my rem-avg-spam.sql file is just

select *
from %logfile%
where not (cs(User-Agent)=’Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;1813)’
or cs(User-Agent)=’Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1)’
and cs(Cookie) is null
and cs(Referer) is null)

I’m now using this for some fairly large logs (100mb+) and it works fine.

I hope this helps.

add to del.icio.us :: Bookmark Post in Technorati :: Add to Blinkslist :: add to furl :: Digg it :: add to ma.gnolia :: Stumble It! :: add to simpy :: seed the vine :: :: :: TailRank :: post to facebook :: Bookmark on Google :: Add to Netscape :: Share on Yahoo :: Add this to Live

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: