OSBlues

Warning: This doesn’t seem to work with very large IIS logfiles, I tried with a 750mb file which didn’t error but was unreadable with a disk full error. My mistake, it does work, it’s TextPad that can’t handle it.

Microsoft LogParser may be the answer to our AVG logfile spam woes, I’m been fiddling with it and have come up with a quite simple way pre-processing the logs with logparser to remove the offending spam. You can put the query inline within the logparser commandline, but it’s easier to stick it in a file once it gets a bit longer. So, I have this in my file

select * into c:\logs\ex%log%out.log
from c:\logs\ex%log%.log
where not (cs(User-Agent)=’Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;1813)‘
or cs(User-Agent)=’Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1)’
and cs(Cookie) is null
and cs(Referer) is null)

Which you run with

C:\Program Files\Log Parser 2.2>LogParser -i:iisw3c -o:w3c file:c:\logs\avgspam.
sql?log=080601

Which yields something like

Statistics:
———–
Elements processed: 209607
Elements output: 151434
Execution time: 8.47 seconds

This shows that logparser has removed ~58K rows from our log and created a new log that we can feed to our stats program.

I’ve tested this with a couple of logs from different clients now and it seems to to work.

Let me know if you have improvements or have come up with a different work around for this.

AVG better not start using more UserAgent strings though as this could get very messy.

If you’re new to logparser (as I was), this guide has some good (IIS centric) examples – Analysing IIS logs with LogParser

Update: I’ve now worked out how to use logparser with Awstats to filter out AVG spam. If you’re using Awstats this is a very quick fix.

:: :: :: :: :: :: :: :: :: :: :: :: :: :: :: ::