My name in lights

June 14, 2008 — osblues

… well, not quite.

A week has passed since I email The Register about the AVG logfile spam problems, but, after lots of emails between myself and their Internet Editor, they’re now running the story.

I’m a little annoyed that they don’t acknowledge that I brought this to their attention, but ah well. All I really wanted to do was let the community know that this happening and see if together we can sort this.

I’m really pleased to see that an employee from AVG has stepped up within the comments of the article on El Reg and wants to work with us (webmasters/site owners) to see if we can improve the situation. I’ve already emailed them and await a response.

I’ll keep you posted.

add to del.icio.us :: Bookmark Post in Technorati :: Add to Blinkslist :: add to furl :: Digg it :: add to ma.gnolia :: Stumble It! :: add to simpy :: seed the vine :: :: :: TailRank :: post to facebook :: Bookmark on Google :: Add to Netscape :: Share on Yahoo :: Add this to Live

Posted in Avg, Spam, Technology. Tags: , , , , . 2 Comments »

SPF compliant .net system.net.mail .mailmessage

June 11, 2008 — osblues

As a follow up to my post SPF compliant CDO message, here is the equivalent (well, not quite, but similar) vb.net version that I use –

Public Sub Send(ByVal strTo As String, ByVal strFrom As String, _
   ByVal strSender As String, ByVal strCC As String, _
   ByVal strBCC As String, ByVal strSubject As String, _
   ByVal strBody As String)

Dim MailObj As New System.Net.Mail.MailMessage(strFrom, _
   strTo, strSubject, strBody)

'SPF Stuff
If Not String.IsNullOrEmpty(strSender) And strSender <> strFrom Then
   MailObj.Headers.Add("return-path", strSender)
   MailObj.Headers.Add("reply-to", strFrom)
   MailObj.Sender = New Net.Mail.MailAddress(strSender)
End If

If Not strCC = String.Empty Then
   For Each cc As String In strCC.Split(";")
      MailObj.CC.Add(cc)
   Next
End If

If Not strBCC = String.Empty Then
   For Each bcc As String In strBCC.Split(";")
      MailObj.Bcc.Add(bcc)
   Next
End If

Dim MailClient As New System.Net.Mail.SmtpClient
MailClient.Host = "mailserver"
MailClient.Send(MailObj)

End Sub

Again, as before, this is generally useful for web generated emails (like send a friend forms etc.), simply specify the users address as strFrom and a generic local address (noreply@mydomain.com) as strSender.

I hope it proves useful.

add to del.icio.us :: Bookmark Post in Technorati :: Add to Blinkslist :: add to furl :: Digg it :: add to ma.gnolia :: Stumble It! :: add to simpy :: seed the vine :: :: :: TailRank :: post to facebook :: Bookmark on Google :: Add to Netscape :: Share on Yahoo :: Add this to Live

Posted in .Net, Spam, Technology. Tags: , , , , . Leave a Comment »

Using LogParser With Awstats To Filter AVG Spam

June 5, 2008 — osblues

Following on from my post LogParser to the rescue, I’ve now worked out how to integrate logparser into the Awstats update process with very minimal effort.

Note: Awstats is a cross platform web analysis tool, but unfortunately logparser isn’t, this therefore is windows only.

To make life easier, I dropped the logparser files (exe and dll, although I’m not sure you need the dll) directly in to the cgi-bin where Awstats lives on the server. I understand doing this may have security implications, so do this at your own risk.

Open up the config file for your Awstats report (awstats.<config>.conf) and find the LogFile directive

LogFile=”E:/logs/W3SVC2074709632/ex%YY-1%MM-1%DD-1.log”

It’ll be something like the above, assuming you use daily logs on IIS. We need to change it to

LogFile=”logparser -i:iisw3c -o:w3c -rtp:-1 -stats:off file:rem-avg-spam.sql?logfile=E:/logs/W3SVC2074709632/ex%YY-1%MM-1%DD-1.log |”

This tells Awstats to execute logparser setting any necessary options and passing in the path to the log as before, it then grabs the output from the pipe and processes it.

That’s it!

The contents of my rem-avg-spam.sql file is just

select *
from %logfile%
where not (cs(User-Agent)=’Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;1813)’
or cs(User-Agent)=’Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1)’
and cs(Cookie) is null
and cs(Referer) is null)

I’m now using this for some fairly large logs (100mb+) and it works fine.

I hope this helps.

add to del.icio.us :: Bookmark Post in Technorati :: Add to Blinkslist :: add to furl :: Digg it :: add to ma.gnolia :: Stumble It! :: add to simpy :: seed the vine :: :: :: TailRank :: post to facebook :: Bookmark on Google :: Add to Netscape :: Share on Yahoo :: Add this to Live

Posted in Avg, Spam, Technology. Tags: , , , , . Leave a Comment »

More AVG & LinkScanner Information

June 4, 2008 — osblues

I’m still testing my LogParser fix for AVG log spam and it appears to do a pretty good job. It’s scarey how many visits are being removed from our stats once this crap is cleared out though. I’ve seen one clients stats for a recent day, drop from 14K to 8K so it really is a serious problem, especially if you aren’t even aware it’s happening.

For more information on the user agents used and some background on other similar AV tools, see this LinkScanner, AVG, TrendMicro, 1813 and SV1 post at WebmasterWorld.

add to del.icio.us :: Bookmark Post in Technorati :: Add to Blinkslist :: add to furl :: Digg it :: add to ma.gnolia :: Stumble It! :: add to simpy :: seed the vine :: :: :: TailRank :: post to facebook :: Bookmark on Google :: Add to Netscape :: Share on Yahoo :: Add this to Live

Posted in Avg, Spam, Technology. Tags: , , , , . Leave a Comment »

LogParser To The Rescue

June 3, 2008 — osblues

Warning: This doesn’t seem to work with very large IIS logfiles, I tried with a 750mb file which didn’t error but was unreadable with a disk full error. My mistake, it does work, it’s TextPad that can’t handle it.

Microsoft LogParser may be the answer to our AVG logfile spam woes, I’m been fiddling with it and have come up with a quite simple way pre-processing the logs with logparser to remove the offending spam. You can put the query inline within the logparser commandline, but it’s easier to stick it in a file once it gets a bit longer. So, I have this in my file

select * into c:\logs\ex%log%out.log
from c:\logs\ex%log%.log
where not (cs(User-Agent)=’Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;1813)‘
or cs(User-Agent)=’Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1)’
and cs(Cookie) is null
and cs(Referer) is null)

Which you run with

C:\Program Files\Log Parser 2.2>LogParser -i:iisw3c -o:w3c file:c:\logs\avgspam.
sql?log=080601

Which yields something like

Statistics:
———–
Elements processed: 209607
Elements output: 151434
Execution time: 8.47 seconds

This shows that logparser has removed ~58K rows from our log and created a new log that we can feed to our stats program.

I’ve tested this with a couple of logs from different clients now and it seems to to work.

Let me know if you have improvements or have come up with a different work around for this.

AVG better not start using more UserAgent strings though as this could get very messy.

If you’re new to logparser (as I was), this guide has some good (IIS centric) examples – Analysing IIS logs with LogParser

Update: I’ve now worked out how to use logparser with Awstats to filter out AVG spam. If you’re using Awstats this is a very quick fix.

add to del.icio.us :: Bookmark Post in Technorati :: Add to Blinkslist :: add to furl :: Digg it :: add to ma.gnolia :: Stumble It! :: add to simpy :: seed the vine :: :: :: TailRank :: post to facebook :: Bookmark on Google :: Add to Netscape :: Share on Yahoo :: Add this to Live

Posted in Avg, Spam, Technology. Tags: , , , , , , . Leave a Comment »
« Older posts
Newer posts »